Interactive risk prediction method, device, medium and equipment

By using a three-dimensional probability occupancy grid sequence in an intelligent transportation system to describe the uncertainty of traffic participants’ trajectory, the problem of insufficient accuracy in the prediction of interaction risk in the prior art is solved, and more efficient traffic risk identification and early warning is achieved.

CN118644989BInactive Publication Date: 2024-12-03YANGTZE DELTA REGION INST OF TSINGHUA UNIV ZHEJIANG
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411108415.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-13
Publication Date
2024-12-03
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing intelligent transportation systems have shortcomings in the accuracy of interaction risk prediction. Traditional trajectory prediction methods cannot effectively characterize the uncertainty of traffic participants' trajectory, resulting in low accuracy of interaction risk prediction.

Method used

Using the interactive risk prediction method, by obtaining image information, obtaining the current occupancy grid information and trajectory prediction information of traffic participants, judging the trajectory intersection, and determining the interaction risk based on the probability occupancy grid sequence, and converting the one-dimensional trajectory prediction line into The 3D initial probability occupancy grid is extended to better describe the uncertainty of traffic participants’ trajectory.

Benefits of technology

It improves the accuracy of interactive risk prediction, can more accurately identify potential traffic risks, and reduce the incidence of traffic accidents.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118644989B_ABST
    Figure CN118644989B_ABST
Patent Text Reader

Abstract

A method, device, medium and equipment for predicting interaction risk. The method includes: obtaining the current grid occupation information and trajectory prediction information of each traffic participant in the area to be predicted based on the obtained image information; judging whether the trajectories of each traffic participant in the area to be predicted intersect; for traffic participants with intersecting trajectories, obtaining the probability grid occupation sequence corresponding to each traffic participant with intersecting trajectories based on the current grid occupation information of each traffic participant in the area to be predicted; determining whether there is interaction risk between each traffic participant with intersecting trajectories based on the probability grid occupation sequence corresponding to each traffic participant with intersecting trajectories; wherein the probability grid occupation sequence is a collection of probability grid occupations of the initial probability grid occupation at each time step within a preset time length after expansion on a spatial scale. The above scheme can more accurately predict interaction risk.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent transportation systems, and in particular to an interactive risk prediction method, device, medium and equipment. Background Art

[0002] With the acceleration of urbanization, traffic congestion and accident risks have become increasingly serious problems at intersections with heavy traffic. Traditional traffic monitoring and management methods rely on traffic lights, road signs, and manual intervention, which have limitations in handling complex traffic scenarios and improving traffic safety.

[0003] In order to better improve traffic efficiency and safety, Intelligent Transportation Systems (ITS) have gradually become a research hotspot.

[0004] However, the accuracy of interactive risk prediction in existing intelligent transportation systems still needs to be improved. Summary of the invention

[0005] The problem to be solved by the present invention is to improve the accuracy of interactive risk prediction.

[0006] To solve the above problems, an embodiment of the present invention provides an interaction risk prediction method, which may include: acquiring image information of an area to be predicted; obtaining occupation grid information and trajectory prediction information of each traffic participant in the area to be predicted at a current moment based on the obtained image information; judging whether there is an intersection among the trajectories of each traffic participant in the area to be predicted based on the trajectory prediction information of each traffic participant in the area to be predicted; for traffic participants with intersecting trajectories, obtaining a probability occupation grid sequence corresponding to each traffic participant with intersecting trajectories based on the occupation grid information of each traffic participant in the area to be predicted at a current moment; determining whether there is an interaction risk between each traffic participant with intersecting trajectories based on the probability occupation grid sequence corresponding to each traffic participant with intersecting trajectories; wherein the probability occupation grid sequence is a collection of probability occupation grids expanded on a spatial scale from the initial probability occupation grids of each time step within a preset time length.

[0007] An embodiment of the present invention also provides an interactive risk prediction device, which includes: an image acquisition unit, suitable for acquiring image information of an area to be predicted; an information acquisition unit, suitable for obtaining, based on the obtained image information, the occupation grid information and trajectory prediction information of each traffic participant in the area to be predicted at the current moment; a judgment unit, suitable for judging whether there is an intersection among the trajectories of each traffic participant in the area to be predicted based on the trajectory prediction information of each traffic participant in the area to be predicted; a calculation unit, suitable for obtaining, for traffic participants with intersecting trajectories, a probability occupation grid sequence corresponding to each traffic participant with intersecting trajectories based on the occupation grid information of each traffic participant in the area to be predicted at the current moment; a risk assessment unit, suitable for determining whether there is an interactive risk between each traffic participant with intersecting trajectories based on the probability occupation grid sequence corresponding to each traffic participant with intersecting trajectories; wherein the probability occupation grid sequence is a collection of probability occupation grids expanded on a spatial scale after the initial probability occupation grid of each time step within a preset time length.

[0008] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. The computer program is executed by a processor to implement the steps of any of the above methods.

[0009] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and the processor executes the steps of any of the above methods when running the computer program.

[0010] Compared with the prior art, the technical solution of the embodiment of the present invention has the following advantages:

[0011] By applying the solution of the present invention, after obtaining the grid occupation information and trajectory prediction information of each traffic participant in the area to be predicted at the current moment, for traffic participants with intersecting trajectories, based on the grid occupation information of each traffic participant in the area to be predicted at the current moment, a probability grid occupation sequence corresponding to each traffic participant with intersecting trajectories is obtained, and then based on the probability grid occupation sequence corresponding to each traffic participant with intersecting trajectories, it is determined whether there is an interaction risk between each traffic participant with intersecting trajectories. Since there is uncertainty in the trajectory of traffic participants in actual applications, the probability grid occupation sequence obtained by expanding the initial probability grid occupation at each time step within a preset time length on a spatial scale can describe the trajectory of traffic participants in three-dimensional space, and can better characterize the uncertainty of the trajectory of traffic participants compared to the traditional one-dimensional trajectory prediction line, so that the interaction risk can be predicted more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a flow chart of an interactive risk prediction method in an embodiment of the present invention;

[0013] Figure 2 It is a flow chart of a method for obtaining the current moment occupied grid information and trajectory prediction information of each traffic participant in the area to be predicted in an embodiment of the present invention;

[0014] Figure 3 is a flow chart of a method for obtaining a spatiotemporal sequence of a three-dimensional scene representation corresponding to a region to be predicted in an embodiment of the present invention;

[0015] Figure 4 is a flow chart of a method for obtaining trajectory prediction information of each traffic participant in a to-be-predicted area in an embodiment of the present invention;

[0016] Figure 5 is a flow chart of a method for obtaining a multi-head self-attention output matrix in an embodiment of the present invention;

[0017] Figure 6 is a flow chart of a method for obtaining a probability grid occupancy sequence corresponding to each traffic participant with intersecting trajectories in an embodiment of the present invention;

[0018] Figure 7 It is a schematic diagram of an initial probability occupancy grid of multiple traffic participants at an intersection after expansion in spatial scale in an embodiment of the present invention;

[0019] Figure 8 It is a structural schematic diagram of an interactive risk prediction device in an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In traditional interaction risk prediction schemes, interaction risk is evaluated only based on a one-dimensional trajectory prediction line. However, the one-dimensional trajectory prediction line cannot well characterize the uncertainty of the trajectory of traffic participants, resulting in low accuracy of interaction risk prediction.

[0021] In view of the above problems, the present invention provides an interactive risk prediction method. According to the method, in the process of obtaining a probability occupancy grid sequence, the one-dimensional scale data of the traditional trajectory prediction line is converted into an initial probability occupancy grid in a three-dimensional space, and the initial probability occupancy grid is expanded in space over time. The obtained probability occupancy grid can more accurately describe the uncertainty of traffic participants, thereby more accurately predicting the interactive risk.

[0022] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0023] Reference Figure 1The embodiment of the present invention provides an interactive risk prediction method, which may include the following steps:

[0024] Step 11: Obtain image information of the area to be predicted.

[0025] Step 12: Based on the obtained image information, obtain the current grid occupation information and trajectory prediction information of each traffic participant in the area to be predicted.

[0026] Step 13: Based on the trajectory prediction information of each traffic participant in the to-be-predicted area, it is determined whether the trajectories of each traffic participant in the to-be-predicted area intersect.

[0027] Step 14, for traffic participants with intersecting trajectories, based on the current grid occupancy information of each traffic participant in the to-be-predicted area, a probability grid occupancy sequence corresponding to each traffic participant with intersecting trajectories is obtained.

[0028] The probability occupancy grid sequence is a set of probability occupancy grids expanded in spatial scale from the initial probability occupancy grids of each time step within a preset time length.

[0029] Step 15, based on the probability occupancy grid sequence corresponding to each traffic participant with intersecting trajectories, determining whether there is an interaction risk between each traffic participant with intersecting trajectories.

[0030] Regarding step 11, in a specific implementation, the area to be predicted includes but is not limited to a crossroad. In practical applications, a visual camera can be installed at a suitable location in the area to be predicted, and internal and external parameters can be calibrated to form a visual camera system for the area to be predicted. Traffic participants in the area to be predicted include motor vehicles, non-motor vehicles, pedestrians, etc.

[0031] Specifically, when installing visual cameras in the area to be predicted, you can first conduct a field survey of the area to be predicted to understand the specific conditions such as road layout, traffic flow, lighting conditions, etc., and select a suitable camera installation location based on the survey results. When installing cameras at the selected location, you should ensure the stability of the cameras, adjust the angle and direction of each camera, and ensure that the visual camera system of the entire area to be predicted can cover every corner of the area to be predicted.

[0032] After the camera is installed, you can perform a preliminary calibration on the camera to adjust the camera's focus, exposure, white balance and other parameters to ensure that the captured image is clear and stable. Then, calibrate the internal and external parameters of each camera. The steps of calibrating the internal and external parameters of each camera can include:

[0033] 1) Take multiple images containing a calibration plate at different angles, distances, and directions. The coordinates of the calibration plate are known.

[0034] 2) Use computer vision algorithms (such as the findChessboardCorners function in the opencv library) to detect the chessboard corners in each image and perform precision processing on the detected corners. For example, the cornerSubPix function in the opencv library can be used to perform precision processing on the detected corners;

[0035] 3) Using the corner point data in all images, the calibration algorithm (such as the calibrateCamera function in the opencv library) is used to calculate the intrinsic parameters of the camera (including the intrinsic parameter matrix K and the distortion coefficient D);

[0036] 4) Use the internal calibration results to correct the image distortion using a correction algorithm (such as the undistort function in the opencv library) to verify the correctness of the internal calibration results;

[0037] 5) Use the intrinsic parameters of the camera obtained by the intrinsic calibration, combined with the key point data in the image used, and use the calibration algorithm (such as the solvePnpRansac function in the opencv library) to calculate the extrinsic parameters of the camera (including the rotation matrix R and the translation vector T);

[0038] 6) Use the extrinsic calibration results to project the images taken by the camera into the world coordinate system to ensure the spatial consistency of multi-camera images and verify the correctness of the extrinsic calibration results.

[0039] After calibrating the internal and external parameters of each camera, the image information of the area to be predicted can be obtained.

[0040] By adopting the above steps, a more stable and robust visual camera system can be obtained for image acquisition, ensuring that multi-angle images captured in real time can be used in time for the next step of three-dimensional reconstruction.

[0041] Regarding step 12, in one embodiment of the present invention, refer to Figure 2 , based on the obtained image information, obtaining the current grid occupation information and trajectory prediction information of each traffic participant in the area to be predicted, which may include:

[0042] Step 21, based on the obtained image information, reconstructing the three-dimensional scene of the area to be predicted to obtain a spatiotemporal sequence corresponding to the three-dimensional scene representation of the area to be predicted;

[0043] Step 22, based on the spatiotemporal sequence of the three-dimensional scene representation corresponding to the area to be predicted, using a multi-layer perceptron, obtain the grid occupancy information of each traffic participant in the area to be predicted at the current moment;

[0044] Step 23, based on the spatiotemporal sequence represented by the three-dimensional scene corresponding to the area to be predicted, using the Informer trajectory prediction model, obtain trajectory prediction information of each traffic participant in the area to be predicted.

[0045] Regarding step 21, after acquiring image information of the area to be predicted in real time, the multi-angle images of the area to be predicted can be preprocessed and fused first, and a spatiotemporal sequence representing the three-dimensional scene corresponding to the area to be predicted can be obtained.

[0046] In a specific implementation, the occupancy network technology can be used to reconstruct the three-dimensional scene of the area to be predicted. The occupancy network is a scene reconstruction technology based on deep learning. It can generate a three-dimensional scene from a two-dimensional image captured by a camera and provide detailed information about the occupancy of objects in the scene. The occupancy grid can represent the position and movement status of different traffic participants on the road, providing basic data for further analysis.

[0047] Specifically, refer to Figure 3 , step 21 may include the following steps:

[0048] Step 211 , using the internal parameter matrix K and the distortion coefficient D, and utilizing a correction algorithm (such as the undistort function in the opencv library) to perform distortion correction on the multi-angle images of the prediction area.

[0049] Step 212, using the intrinsic parameter matrix K, the distortion coefficient D, the rotation matrix R and the translation vector T, the corrected multi-angle image is projected into the world coordinate system.

[0050] In a specific implementation, when the corrected multi-angle image is projected into the world coordinate system, each pixel point in the image of the area to be predicted taken by the camera can be first converted from the image coordinate system to the normalized camera coordinate system, and then each pixel point in the normalized camera coordinate system can be projected into the world coordinate system to obtain the image of the area to be predicted in the world coordinate system.

[0051] Step 213, performing multi-scale feature fusion on the image of the area to be predicted in the world coordinate system to extract a key feature map.

[0052] In a specific implementation, a reparameterized generalized feature pyramid network (Region Proposal-based Global Feature Pyramid Network, RepGFPN) can be used to perform multi-scale feature fusion on the image of the region to be predicted in the world coordinate system. Specifically, RepGFPN can extract feature maps of different scales from the image of the region to be predicted in the input world coordinate system and perform multi-scale feature fusion to obtain a key feature map.

[0053] For example, the extracted feature maps of different scales include N frames, namely F1, F2, F3, ..., F N . , the key feature map F obtained after multi-scale feature fusion of N images can be expressed as: .

[0054] Step 214, applying a spatial attention mechanism to perform high-dimensional spatial feature extraction on the input multi-scale feature map to obtain a high-dimensional spatial feature map.

[0055] Specifically, the 3D spatial query matrix Q, key matrix K and value matrix V can be calculated for the key feature graph F, where:

[0056] (1)

[0057] (2)

[0058] (3)

[0059] , and is the weight matrix.

[0060] Next, we can calculate the spatial attention weights between any two vectors in the query matrix Q and the key matrix K to obtain the spatial attention weight matrix. Specifically, the i-th vector Q in the query matrix Q i The jth vector K in the sum key matrix K j The spatial attention weight matrix between It can be expressed as:

[0061] (4)

[0062] λ represents the total number of vectors in the key matrix K, and k represents the kth vector in the key vector K.

[0063] Finally, the spatial attention weight matrix is ​​weighted and summed to obtain the weighted high-dimensional spatial feature map: .

[0064] Step 215, performing spatial frame alignment on the high-dimensional spatial feature map obtained at each moment to obtain a high-dimensional spatiotemporal feature map.

[0065] For example, when the weighted high-dimensional spatial feature map obtained at each time t When the spatial frame is aligned, the high-dimensional spatiotemporal feature map can be expressed as: s,t .

[0066] Step 216, upsampling the high-dimensional spatiotemporal feature map step by step through multi-layer deconvolution to obtain a spatiotemporal sequence representing the three-dimensional scene corresponding to the area to be predicted.

[0067] Specifically, the high-dimensional spatiotemporal feature map is upsampled step by step through multi-layer deconvolution until it is restored to the original resolution, and the 3D scene representation of the final area to be predicted is obtained. Assume that at time t, the high-dimensional spatiotemporal feature map is upsampled step by step through L layers of deconvolution, where at time t, l+1 After layer deconvolution samples the high-dimensional spatiotemporal feature map, we can get , then at time t, the high-dimensional spatiotemporal feature map is upsampled step by step through multi-layer deconvolution to obtain the corresponding 3D scene representation In this way, the spatiotemporal sequence obtained by upsampling the high-dimensional spatiotemporal feature map from time tn to time t can be expressed as: .

[0068] Regarding step 22, the spatiotemporal sequence of the three-dimensional scene representation corresponding to the prediction area can be input into a multi-layer perceptron (MLP) to query and obtain the corresponding occupied grid information of each traffic participant.

[0069] Regarding step 23, the Informer trajectory prediction model is a new and efficient long time series prediction model that can process multi-dimensional time series data and provide high-precision future motion trajectory prediction.

[0070] In traffic forecasting, future traffic flow and movement trajectories of participants can be predicted based on historical traffic data, thereby identifying potential traffic risks in advance.

[0071] In one embodiment of the present invention, referring to Figure 4 , the step of obtaining trajectory prediction information of each traffic participant in the area to be predicted by using the Informer trajectory prediction model based on the spatiotemporal sequence represented by the three-dimensional scene corresponding to the area to be predicted may include:

[0072] Step 231, constructing a general information sequence of each traffic participant in the area to be predicted based on the spatiotemporal sequence represented by the three-dimensional scene corresponding to the area to be predicted.

[0073] In a specific implementation, the overall information I(t) of each traffic participant at time t may include: the position information P(t), motion information M(t) and semantic information S(t) of the traffic participant corresponding to the occupied grid, and the environmental information E(t) of the traffic participant. For example, the overall information of the first traffic participant in the predicted area can be expressed as: .

[0074] Among them, the position information P(t), motion information M(t) and semantic information S(t) of the traffic participant corresponding to the occupied grid can be obtained by inputting the spatiotemporal sequence of the three-dimensional scene representation corresponding to the prediction area into a multi-layer perceptron (MLP) for query. The environmental information E(t) of the traffic participant can be obtained based on the position information, motion information and semantic information of the occupied grids of all traffic participants that may interact with the traffic participant. Specifically, the environmental information E1(t) of the first traffic participant in the predicted area = , f represents the fth traffic participant in the predicted area that may interact with the first traffic participant.

[0075] For each traffic participant, construct the overall information sequence from time tn to time t. For example, for the rth traffic participant in the predicted area, the corresponding overall information sequence is = .

[0076] In a specific implementation, for all traffic participants that may interact with the current traffic participant, different types of traffic participants (motor vehicles, non-motor vehicles, pedestrians, and unknown types) can be judged separately. Specifically, different thresholds can be set for traffic conditions in different areas to be predicted. For example, for an intersection with medium traffic flow, the motor vehicle threshold is set to 8, the non-motor vehicle threshold is set to 5, the pedestrian threshold is set to 5, and the unknown class threshold is set to 4. Among the traffic participants of different categories, they are arranged according to the Euclidean distance from the current traffic participant, and the traffic participants within the threshold range are selected as traffic participants that may interact. If the number of traffic participants that may interact within the intersection does not reach the threshold, the remaining environmental information is supplemented with a zero placeholder.

[0077] Step 232: input the overall information sequence of each traffic participant in the area to be predicted into the Informer trajectory prediction model to obtain the trajectory prediction information of each traffic participant in the area to be predicted.

[0078] In a specific implementation, the Informer trajectory prediction model can be pre-trained. When the trajectory prediction information of each traffic participant in the area to be predicted is obtained using the Informer trajectory prediction model, the overall information sequence of each traffic participant in the area to be predicted can be processed based on the multi-head self-attention mechanism to obtain the final multi-head self-attention output matrix of each traffic participant in the area to be predicted, and then the trajectory prediction information of each traffic participant in the area to be predicted can be obtained based on the final multi-head self-attention output matrix of each traffic participant in the area to be predicted.

[0079] In one embodiment of the present invention, referring to Figure 5, based on the multi-head self-attention mechanism, the overall information sequence of each traffic participant in the to-be-predicted area is processed to obtain the final multi-head self-attention output matrix of each traffic participant in the to-be-predicted area, which may specifically include the following steps:

[0080] Step 51, based on the overall information sequence of each traffic participant in the area to be predicted, obtain the attention weight matrix corresponding to each traffic participant in the area to be predicted.

[0081] Specifically, the steps of obtaining the attention weight matrix corresponding to each traffic participant in the area to be predicted are as follows:

[0082] 1) Perform a linear transformation on the overall information sequence of each traffic participant in the prediction area to obtain the corresponding query matrix Q, key matrix K and value matrix V.

[0083] Take the overall information sequence of the rth traffic participant For example, the corresponding query matrix Q r , key matrix K r Sum value matrix V r as follows:

[0084] (5)

[0085] (6)

[0086] (7)

[0087] , and is the weight matrix.

[0088] 2) Divide the query matrix, key matrix and value matrix of each traffic participant into H attention heads respectively, and the dimension of each attention head is d / H.

[0089] For example, the query matrix Q corresponding to the rth traffic participant is r , key matrix K r Sum value matrix V r , we can get:

[0090] (8)

[0091] (9)

[0092] (10)

[0093] 3) Calculate the attention weight matrix corresponding to each traffic participant in the area to be predicted.

[0094] Taking the rth traffic participant as an example, the softmax function can be used to obtain the attention weights corresponding to the rth traffic participant, as shown in formula (11):

[0095] (11)

[0096] Among them, h represents any attention head among the H heads, that is, h∈[1,H].

[0097] All heads correspond to attention weights, forming an attention weight matrix .

[0098] Step 52, based on the attention weight matrix corresponding to each traffic participant in the area, obtain the output matrix of the attention head corresponding to each traffic participant.

[0099] Specifically, the attention weight matrix corresponding to the same traffic participant and the value matrix corresponding to the traffic participant can be weightedly summed to obtain the output matrix of each attention head of the traffic participant.

[0100] Taking the rth traffic participant as an example, the attention weight matrix corresponding to the rth traffic participant is and value matrix V r Perform weighted summation. Among them, the attention head output matrix corresponding to any attention head h of the rth traffic participant is as follows:

[0101] (12)

[0102] Step 53, concatenate the output matrices of each attention head and transform them through a linear layer to obtain the final multi-head self-attention output matrix.

[0103] Specifically, by concatenating the output matrices of each attention head of the same traffic participant, the final multi-head self-attention output matrix of the traffic participant can be obtained.

[0104] Taking the rth traffic participant as an example, the final multi-head self-attention output matrix of the rth traffic participant is as follows:

[0105] (13)

[0106] In one embodiment of the present invention, based on the final multi-head self-attention output matrix of each traffic participant in the area to be predicted, obtaining the trajectory prediction information of each traffic participant in the area to be predicted may specifically include the following steps:

[0107] 1) Performing a nonlinear transformation on the multi-head self-attention output matrix to obtain a nonlinear transformation result of each traffic participant in the area to be predicted.

[0108] In a specific implementation, a series of feedforward neural networks can be used to perform nonlinear transformation on the multi-head self-attention output matrix corresponding to each traffic participant.

[0109] Specifically, taking the multi-head self-attention output matrix of the r-th traffic participant as an example, the multi-head self-attention output matrix of the r-th traffic participant is As input variable x, the nonlinear transformation process is as follows:

[0110] (14)

[0111] in, represents the weight matrix of the first linear transformation of the feedforward neural network, Represents the weight matrix of the second linear transformation of the feedforward neural network, b1 and b2 are the bias vectors of the linear transformation.

[0112] For each time step, a nonlinear transformation is performed on the multi-head self-attention output matrix of the r-th traffic participant to obtain:

[0113] (15)

[0114] in, is the multi-head self-attention output matrix of the rth traffic participant The nonlinear transformation result corresponding to the current moment (that is, moment t).

[0115] 2) Adding a position code to the nonlinear transformation result of each traffic participant in the area to be predicted, to obtain the position information of each traffic participant in the area to be predicted at the current moment.

[0116] Specifically, taking the rth traffic participant as an example, the nonlinear transformation result of the rth traffic participant is Adding position coding, based on formula (16), the position information of the rth traffic participant can be obtained :

[0117] (16)

[0118] in, Indicates the position coding information added at the current moment.

[0119] 3) Performing informer decoding on the current position information of each traffic participant in the area to be predicted, and obtaining the position sequence of each traffic participant in the area to be predicted at each time step within a preset time length.

[0120] Specifically, taking the rth traffic participant as an example, the position information of the rth traffic participant at any time step is By decoding, we can get the position sequence corresponding to each time step of the rth traffic participant within the preset duration. :

[0121] (17)

[0122] 4) Mapping the high-dimensional information of the position sequence of each traffic participant in the area to be predicted at each time step within a preset time length into the position coordinates of the three-dimensional scene corresponding to the area to be predicted, and obtaining the trajectory prediction information of each traffic participant in the area to be predicted.

[0123] Specifically, a linear layer can be used to map the high-dimensional information in the position sequence obtained after decoding into the specific position coordinates of the three-dimensional scene corresponding to the area to be predicted, so as to obtain the final trajectory prediction information of each traffic participant.

[0124] Taking the rth traffic participant as an example, the position sequence corresponding to each time step of the rth traffic participant within the preset duration is By mapping, we can obtain the trajectory prediction information of the rth traffic participant at each time step within the preset duration, as shown in formula (18):

[0125] (18)

[0126] in, , represents the predicted trajectory coordinates of the rth traffic participant in M ​​time steps within the preset duration.

[0127] Regarding step 13, after obtaining the trajectory prediction information of each traffic participant in the area to be predicted, it is determined whether the trajectories of each traffic participant in the area to be predicted intersect, that is, it is determined whether there are traffic participants with the same trajectory prediction coordinates at each time step within the preset time length.

[0128] For example, the trajectory prediction information of the rth traffic participant is: , the trajectory prediction information of the e-th traffic participant is If the predicted trajectory coordinates of the rth traffic participant at a certain time step are the same as the predicted trajectory coordinates of the eth traffic participant at a certain time step, then the two have crossed trajectories. For example, if the predicted trajectory coordinates of the rth traffic participant at the t+1 time step are the same as the predicted trajectory coordinates of the eth traffic participant at the t+3 time step, then the two have crossed trajectories.

[0129] If within the preset time period, there are traffic participants with the same predicted trajectory coordinates, that is, there are traffic participants with intersecting trajectories, then step 14 is executed, otherwise the interactive risk prediction is terminated.

[0130] Regarding step 14, for traffic participants with intersecting trajectories, based on the current grid occupancy information of each traffic participant in the area to be predicted, a probability grid occupancy sequence corresponding to each traffic participant with intersecting trajectories is obtained.

[0131] Specifically, refer to Figure 6 , the following steps can be used to obtain the probability grid sequence corresponding to each traffic participant with intersecting trajectories:

[0132] Step 61 , discretize the occupancy grids of each traffic participant with intersecting trajectories at the current moment into a plurality of grids, and obtain the initial probability occupancy grids of each traffic participant with intersecting trajectories in all corresponding grids.

[0133] In a specific implementation, for each traffic participant with intersecting trajectories, the initial probability occupancy grid of the traffic participant at the current time step can be calculated based on the initial probability occupancy grid of the traffic participant at the previous time step.

[0134] Assume that the trajectories of the rth traffic participant and the eth traffic participant in the predicted area intersect, and the grid occupied by the rth traffic participant at the current moment is G r (t), the grid occupied by the e-th traffic participant at the current moment is G e (t), the grid G r (t) and G e (t) are discretized into smaller three-dimensional grids, each grid corresponding to an occupancy probability.

[0135] Among them, the occupancy probability of the rth traffic participant in each grid can be expressed as p r (x1, y1, z1, t), the occupancy probability of the e-th traffic participant for each grid can be expressed as p e (x2, y2, z2, t), where (x1, y1, z1) represents the grid G r (t) The position coordinates of any grid in the grid, (x2, y2, z2) represents the grid G e(t) is the position coordinate of any grid. At the current moment, the occupancy probability of the rth traffic participant and the eth traffic participant for each grid is 1, that is, p r (x1, y1, z1, t) = p e (x2, y2, z2, t) = 1.

[0136] Then at time t+m, the initial probability of the rth traffic participant occupying a grid can be expressed as:

[0137] (19)

[0138] At time t+m, the initial probability of the e-th traffic participant occupying a grid can be expressed as:

[0139] (20)

[0140] in, The probability attenuation coefficient is a fixed value and can be set through external input.

[0141] Using formulas (19) and (20), we can obtain the initial probability occupancy grids of all grids corresponding to each time step within the preset time length for any traffic participant with intersecting trajectories.

[0142] Step 62, at each time step within the preset duration, use a dilation operation to expand the initial probability occupancy grid corresponding to each grid in the spatial scale to obtain the probability occupancy grid at each time step within the preset duration.

[0143] In a specific implementation, for traffic participants with intersecting trajectories, an expansion coefficient may be applied to expand the initial probability occupancy grid in spatial scale to obtain a probability occupancy grid for each time step within the preset duration.

[0144] For example, the initial probability occupancy grid of the rth traffic participant at time t+m is Expanding on the spatial scale, we can get the probability of the rth traffic participant occupying the grid at time t+m :

[0145] (twenty one)

[0146] The initial probability occupancy grid of the e-th traffic participant at time t+m Expanding on the spatial scale, we can get the probability of the e-th traffic participant occupying the grid at time t+m :

[0147] (twenty two)

[0148] in, , , is the initial expansion coefficient, is the expansion rate. and It can be set according to actual conditions, m is less than or equal to M, and t+m is any time step within the preset duration.

[0149] Step 63, combining the probability occupancy grids of each time step within the preset time length to obtain the probability occupancy grid sequence.

[0150] For the same traffic participant, the combination of its probability occupancy grids at the same time step is called the probability occupancy grid corresponding to the time step, and the combination of the probability occupancy grids corresponding to each time step is called the probability occupancy grid sequence of the traffic participant.

[0151] For example, based on formula (21), the probability of the rth traffic participant occupying the grid at each time step within the preset duration can be obtained, that is, .

[0152] Based on formula (22), the probability of the e-th traffic participant occupying the grid at each time step within the preset duration can be obtained, that is, .

[0153] In practical applications, each traffic participant has a certain volume. Therefore, by expanding the initial probability occupancy grid corresponding to each grid in the spatial scale, compared with the traditional method of predicting the interaction risk by predicting each point corresponding to the trajectory line, the trajectory uncertainty of each traffic participant can be better described, thereby making the interaction risk prediction more accurate.

[0154] Figure 7 FIG. 1 is a schematic diagram of an initial probability occupancy grid of multiple traffic participants at an intersection after expansion in spatial scale in one embodiment of the present invention. Figure 7 , pink vehicle 712 is a schematic diagram of the initial probability occupancy grid of vehicle 711 after expansion in spatial scale. Pink vehicles 722, 723 and 724 are schematic diagrams of the initial probability occupancy grid of vehicle 721 after expansion in spatial scale. Pink vehicle 732 is a schematic diagram of the initial probability occupancy grid of vehicle 731 after expansion in spatial scale. Pink vehicle 742 is a schematic diagram of the initial probability occupancy grid of vehicle 741 after expansion in spatial scale. The pink solid line represents the trajectory line.

[0155] from Figure 7It can be seen that the pink vehicle is the grid occupied by the corresponding vehicle at a certain time step within the preset duration. Taking vehicle 721 as an example, comparing pink vehicle 724 with pink vehicle 723 and pink vehicle 722, it can be seen that as time changes, the grid occupied by vehicle 721 becomes larger and larger, which can better describe the uncertainty of the trajectory of vehicle 721 in the future, so the trajectory prediction is more accurate.

[0156] Regarding step 15, based on the probability occupancy grid sequence corresponding to each traffic participant with intersecting trajectories, it is determined whether there is an interaction risk between the traffic participants with intersecting trajectories.

[0157] Specifically, the probability grid overlap rate of each traffic participant with intersecting trajectories at each time step can be calculated, and then each probability grid overlap rate can be compared with a preset probability grid overlap threshold. Finally, based on the comparison result, it can be determined whether there is an interaction risk between the traffic participants with intersecting trajectories.

[0158] For multiple traffic participants with intersecting trajectories, the probability grid overlap rate between any two traffic participants can be calculated separately, and the calculation result of each time can be compared with the preset probability grid overlap threshold to determine whether there is an interaction risk.

[0159] For example, when there is an interaction risk between the rth traffic participant and the eth traffic participant, the probability grid overlap rate between the rth traffic participant and the eth traffic participant can be calculated using formula (23): :

[0160] (twenty three)

[0161] Get the probability grid overlap rate between the rth traffic participant and the eth traffic participant The probability grid overlap rate can then be Overlap threshold with a pre-set probability grid For comparison, > When there is an interaction risk between the rth traffic participant and the eth traffic participant, it is determined that there is no interaction risk between the rth traffic participant and the eth traffic participant; otherwise, it is determined that there is no interaction risk between the rth traffic participant and the eth traffic participant.

[0162] In one embodiment of the present invention, referring to Figure 1 , the method may further include:

[0163] Step 16: output warning information.

[0164] In a specific implementation, after obtaining the interactive risk prediction result, it can be determined whether to output warning information based on the interactive risk prediction result. When there is an interactive risk, the warning information can be output, otherwise the warning information may not be output.

[0165] In a specific implementation, before outputting the warning information, the location information of the traffic participant who is about to have an interaction risk and the location of the target output object of the warning information can be verified. For example, when there is an interaction risk between the rth traffic participant and the eth traffic participant, the location information P of the rth traffic participant can be r (t) and the position information P of the e-th traffic participant e (t), compared with the GPS data of the target output object of the warning information, r (t) and P e (t) Warning information is output only when the GPS data is consistent with the target output object.

[0166] In specific implementation, the warning information can be output to a mobile phone application (APP) or a navigation APP connected to the Application Programming Interface (API) to achieve intelligent scene visualization and directional hazard prompts, play a warning role, and reduce the occurrence of potential traffic accidents.

[0167] From the above content, it can be seen that the interactive risk prediction method in the embodiment of the present invention can perform three-dimensional reconstruction of the predicted area in real time, and analyze the three-dimensional reconstruction, so as to predict the movement trajectory of each traffic participant in the predicted area in the future period of time, and generate a corresponding probability occupancy grid sequence, and then identify potential dangerous interactions by analyzing whether the generated probability occupancy grids overlap, such as potential collisions between vehicles and vehicles, and vehicles and pedestrians.

[0168] By adopting the solution of the present invention, real-time interactive risk warning is carried out at the infrastructure level, all-round and no-blind-angle monitoring is performed on the waiting prediction area of ​​the intersection with a high accident rate, and accurate prediction is made based on the images obtained by monitoring, which can reduce the incidence of traffic accidents at the intersection.

[0169] In some embodiments, once a potential dangerous interaction is detected, real-time warning information can be sent to the driver through mobile phone APP, vehicle-side APP, open API, etc., so that the driver can notice the potential danger in time and take corresponding risk avoidance measures.

[0170] In order to enable those skilled in the art to better understand and implement the present invention, the device, electronic device and computer-readable storage medium corresponding to the above method are described in detail below.

[0171] Reference Figure 8 The embodiment of the present invention further provides a device 85180, wherein the device 80 may include: an image acquisition unit 81, an information acquisition unit 82, a judgment unit 83, a calculation unit 84, and a risk assessment unit 85. Among them:

[0172] The image acquisition unit 81 is adapted to acquire image information of the area to be predicted;

[0173] The information acquisition unit 82 is adapted to obtain the current grid occupancy information and trajectory prediction information of each traffic participant in the area to be predicted based on the obtained image information;

[0174] The judgment unit 83 is adapted to judge whether the trajectories of the traffic participants in the to-be-predicted area intersect based on the trajectory prediction information of the traffic participants in the to-be-predicted area;

[0175] The calculation unit 84 is adapted to obtain, for the traffic participants with intersecting trajectories, a probability occupancy grid sequence corresponding to each traffic participant with intersecting trajectories based on the occupancy grid information of each traffic participant in the to-be-predicted area at the current moment;

[0176] The risk assessment unit 85 is adapted to determine whether there is an interaction risk between the various traffic participants with intersecting trajectories based on the probability occupancy grid sequence corresponding to the various traffic participants with intersecting trajectories;

[0177] The probability occupancy grid sequence is a set of probability occupancy grids expanded in spatial scale from the initial probability occupancy grids of each time step within a preset time length.

[0178] In one embodiment, the calculation unit 84 may include: a discrete subunit 841, an expansion subunit 842, and a combination subunit 843. Among them:

[0179] The discrete subunit 841 is adapted to discretize the occupied grids of each traffic participant with crossed trajectories at the current moment into a plurality of grids, and obtain the initial probability occupied grids of all grids corresponding to each time step of each traffic participant with crossed trajectories within the preset time length;

[0180] The expansion subunit 842 is adapted to expand the initial probability occupancy grid corresponding to each grid in the spatial scale at each time step within the preset duration using a dilation operation to obtain the probability occupancy grid at each time step within the preset duration;

[0181] The combining subunit 843 is adapted to combine the probability occupancy grids of each time step within the preset time length to obtain the probability occupancy grid sequence.

[0182] In one embodiment of the present invention, the expansion subunit 842 is suitable for applying a dilation coefficient at each time step within the preset time length to expand the initial probability occupancy grid on a spatial scale to obtain a probability occupancy grid at each time step within the preset time length.

[0183] In an embodiment of the present invention, the risk assessment unit 85 may include: a calculation subunit 851, a comparison subunit 852, and a judgment subunit 853. Among them:

[0184] The calculation subunit 851 is adapted to calculate the probability grid overlap rate of each traffic participant with track intersection at each time step;

[0185] The comparison subunit 852 is adapted to compare the probability grid overlap rate with a preset probability grid overlap threshold;

[0186] The judgment subunit 853 is adapted to judge whether there is an interaction risk between the traffic participants whose trajectories intersect based on the comparison result.

[0187] In an embodiment of the present invention, the device may further include: an output unit 86. The output unit 86 is adapted to output warning information when there is an interaction risk between the traffic participants whose trajectories intersect.

[0188] In an embodiment of the present invention, the information acquisition unit 82 may include: a three-dimensional scene reconstruction subunit 821, a first information acquisition subunit 822, and a second information acquisition subunit 823. Wherein:

[0189] The 3D scene reconstruction subunit 821 is adapted to perform 3D scene reconstruction on the area to be predicted based on the obtained image information, and obtain a spatiotemporal sequence of the 3D scene representation corresponding to the area to be predicted;

[0190] The first information acquisition subunit 822 is adapted to obtain the grid occupancy information of each traffic participant in the area to be predicted at the current moment by using a multi-layer perceptron based on the spatiotemporal sequence represented by the three-dimensional scene corresponding to the area to be predicted;

[0191] The second information acquisition subunit 823 is adapted to obtain trajectory prediction information of each traffic participant in the area to be predicted based on the spatiotemporal sequence represented by the three-dimensional scene corresponding to the area to be predicted and using the Informer trajectory prediction model.

[0192] In an embodiment of the present invention, the second information acquisition subunit 823 may include: a construction module 8231 and a prediction module 8232. Wherein:

[0193] The construction module 8231 is adapted to construct a general information sequence of each traffic participant in the area to be predicted based on the spatiotemporal sequence of the three-dimensional scene representation corresponding to the area to be predicted;

[0194] The prediction module 8232 is adapted to input the overall information sequence of each traffic participant in the area to be predicted into the Informer trajectory prediction model to obtain trajectory prediction information of each traffic participant in the area to be predicted.

[0195] Regarding the above-mentioned functional units, they can be implemented specifically by referring to the description of the corresponding steps in the above-mentioned interactive risk prediction method, and will not be repeated here.

[0196] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. The computer program is executed by a processor to implement the steps of any of the above methods.

[0197] In a specific implementation, the computer-readable storage medium may include: ROM, RAM, magnetic disk or optical disk, etc.

[0198] An embodiment of the present invention further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and the processor executes the steps of any of the above methods when running the computer program.

[0199] Regarding the various modules / units included in the various devices and products described in the above embodiments, they can be software modules / units, or hardware modules / units, or they can be partially software modules / units and partially hardware modules / units. For example, for various devices and products applied to or integrated in a chip, the various modules / units included therein can all be implemented in the form of hardware such as circuits, or at least some of the modules / units can be implemented in the form of software programs, which run on a processor integrated inside the chip, and the remaining (if any) modules / units can be implemented in the form of hardware such as circuits; for various devices and products applied to or integrated in a chip module, the various modules / units included therein can all be implemented in the form of hardware such as circuits, and different modules / units can be located in the same component (such as a chip, circuit module, etc.) or different components of the chip module, or at least some of the modules / units can be implemented in the form of hardware such as circuits. The element can be implemented in the form of a software program, which runs on the processor integrated inside the chip module, and the remaining (if any) modules / units can be implemented in the form of hardware such as circuits; for various devices and products applied to or integrated in the terminal, the various modules / units contained therein can be implemented in the form of hardware such as circuits, and different modules / units can be located in the same component (for example, chip, circuit module, etc.) or different components in the terminal, or, at least some modules / units can be implemented in the form of a software program, which runs on the processor integrated inside the terminal, and the remaining (if any) modules / units can be implemented in the form of hardware such as circuits.

[0200] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the scope defined by the claims.

Claims

1. An interactive risk prediction method, characterized in that: include: Obtain image information of the area to be predicted; Based on the obtained image information, the grid information and trajectory prediction information of each traffic participant in the area to be predicted are obtained at the current moment; Based on the trajectory prediction information of each traffic participant in the to-be-predicted area, determining whether the trajectories of each traffic participant in the to-be-predicted area intersect; For traffic participants with intersecting trajectories, based on the grid occupancy information of each traffic participant in the to-be-predicted area at the current moment, a grid occupancy probability sequence corresponding to each traffic participant with intersecting trajectories is obtained; Based on the grid occupancy probability sequence corresponding to each traffic participant with intersecting trajectories, determine whether there is an interaction risk between the traffic participants with intersecting trajectories; Among them, for the traffic participants with intersecting trajectories, based on the occupied grid information of each traffic participant in the predicted area at the current moment, a grid occupation probability sequence corresponding to each traffic participant with intersecting trajectories is obtained, including: discretizing the occupied grids of each traffic participant with intersecting trajectories at the current moment into a plurality of grids, and obtaining the initial grid occupation probability of all grids corresponding to each traffic participant with intersecting trajectories at each time step within a preset time length; at each time step within the preset time length, applying an expansion coefficient to expand the initial grid occupation probability on a spatial scale, obtaining the grid occupation probability of each time step within the preset time length, and obtaining the grid occupation probability of each time step within the preset time length; combining the grid occupation probabilities of each time step within the preset time length to obtain the grid occupation probability sequence; The initial grid occupancy probability of the e-th traffic participant at time t+m Expanding on the spatial scale, we can obtain the grid occupancy probability of the e-th traffic participant at time t+m : ; , , is the initial expansion coefficient, is the expansion rate, and Set according to actual conditions, m is less than or equal to M, and t+m is any time step within the preset duration; Probability decay coefficient.

2. The interactive risk prediction method according to claim 1, characterized in that: The determining whether there is an interaction risk between the traffic participants with intersecting trajectories based on the grid occupancy probability sequence corresponding to the traffic participants with intersecting trajectories includes: Calculate the probability grid overlap rate of each traffic participant with intersecting trajectories at each time step; Comparing each probability grid overlap rate with a preset probability grid overlap threshold; Based on the comparison result, it is determined whether there is an interaction risk between the traffic participants whose trajectories intersect.

3. The interactive risk prediction method according to claim 2, characterized in that: When there is a risk of interaction between traffic participants whose trajectories intersect, it also includes: outputting warning information.

4. The interactive risk prediction method according to claim 1, characterized in that: The method of obtaining the grid occupation information and trajectory prediction information of each traffic participant in the area to be predicted at the current moment based on the obtained image information includes: Based on the obtained image information, a three-dimensional scene is reconstructed for the area to be predicted to obtain a spatiotemporal sequence corresponding to the three-dimensional scene representation of the area to be predicted; Based on the spatiotemporal sequence of the three-dimensional scene representation corresponding to the area to be predicted, using a multi-layer perceptron, obtaining the grid occupancy information of each traffic participant in the area to be predicted at the current moment; Based on the spatiotemporal sequence represented by the three-dimensional scene corresponding to the area to be predicted, the trajectory prediction information of each traffic participant in the area to be predicted is obtained by using the Informer trajectory prediction model.

5. The interactive risk prediction method according to claim 4, characterized in that: The method of obtaining trajectory prediction information of each traffic participant in the area to be predicted by using the Informer trajectory prediction model based on the spatiotemporal sequence represented by the three-dimensional scene corresponding to the area to be predicted includes: Based on the spatiotemporal sequence of the three-dimensional scene representation corresponding to the area to be predicted, constructing the overall information sequence of each traffic participant in the area to be predicted; The overall information sequence of each traffic participant in the area to be predicted is input into the Informer trajectory prediction model to obtain the trajectory prediction information of each traffic participant in the area to be predicted.

6. The interactive risk prediction method according to claim 5, characterized in that: The overall information sequence of each traffic participant in the area to be predicted is input into the Informer trajectory prediction model to obtain trajectory prediction information of each traffic participant in the area to be predicted, including: Based on the multi-head self-attention mechanism, the overall information sequence of each traffic participant in the to-be-predicted area is processed to obtain a final multi-head self-attention output matrix of each traffic participant in the to-be-predicted area; Based on the final multi-head self-attention output matrix of each traffic participant in the area to be predicted, trajectory prediction information of each traffic participant in the area to be predicted is obtained.

7. The interactive risk prediction method according to claim 6, characterized in that: The overall information sequence of each traffic participant in the to-be-predicted area is processed based on the multi-head self-attention mechanism to obtain the final multi-head self-attention output matrix of each traffic participant in the to-be-predicted area, including: Based on the overall information sequence of each traffic participant in the area to be predicted, obtaining an attention weight matrix corresponding to each traffic participant in the area to be predicted; Based on the attention weight matrix corresponding to each traffic participant in the area, an output matrix of the attention head corresponding to each traffic participant is obtained; The output matrices of each attention head are concatenated and transformed through a linear layer to obtain the final multi-head self-attention output matrix.

8. The interactive risk prediction method according to claim 6, characterized in that: The obtaining of trajectory prediction information of each traffic participant in the to-be-predicted area based on the final multi-head self-attention output matrix of each traffic participant in the to-be-predicted area includes: Performing a nonlinear transformation on the multi-head self-attention output matrix to obtain a nonlinear transformation result for each traffic participant in the area to be predicted; Adding a position code to the nonlinear transformation result of each traffic participant in the area to be predicted to obtain the position information of each traffic participant in the area to be predicted at the current moment; Perform Informer decoding on the position information of each traffic participant in the area to be predicted, and obtain the position sequence of each traffic participant in the area to be predicted at each time step within a preset time length; The high-dimensional information in the position sequence of each traffic participant in the area to be predicted at each time step within a preset time length is mapped to the position coordinates of the three-dimensional scene corresponding to the area to be predicted, so as to obtain the trajectory prediction information of each traffic participant in the area to be predicted.

9. An interactive risk prediction device, characterized in that: include: An image acquisition unit, adapted to acquire image information of a region to be predicted; An information acquisition unit, adapted to obtain the current grid occupancy information and trajectory prediction information of each traffic participant in the area to be predicted based on the obtained image information; A judgment unit, adapted to judge whether the trajectories of the traffic participants in the to-be-predicted area intersect based on the trajectory prediction information of the traffic participants in the to-be-predicted area; A calculation unit, adapted to obtain, for traffic participants with intersecting trajectories, a grid occupancy probability sequence corresponding to each traffic participant with intersecting trajectories based on the grid occupancy information of each traffic participant in the to-be-predicted area at a current moment; A risk assessment unit, adapted to determine whether there is an interaction risk between the various traffic participants whose trajectories intersect based on the grid occupancy probability sequence corresponding to the various traffic participants whose trajectories intersect; Wherein, the computing unit comprises: The discrete subunit is adapted to discretize the occupied grids of each traffic participant with intersecting trajectories at the current moment into a plurality of grids, and obtain the initial grid occupancy probabilities of all grids corresponding to each time step of each traffic participant with intersecting trajectories within a preset time length; An expansion subunit, adapted to apply a dilation coefficient at each time step within the preset time length to expand the initial grid occupancy probability in a spatial scale to obtain a grid occupancy probability at each time step within the preset time length; A combining subunit, adapted to combine the grid occupancy probabilities of each time step within the preset time length to obtain the grid occupancy probability sequence; The initial grid occupancy probability of the e-th traffic participant at time t+m Expanding on the spatial scale, we can obtain the grid occupancy probability of the e-th traffic participant at time t+m : ; , , is the initial expansion coefficient, is the expansion rate, and Set according to actual conditions, m is less than or equal to M, and t+m is any time step within the preset duration; Probability decay coefficient.

10. The interactive risk prediction device according to claim 9, characterized in that: The risk assessment unit comprises: A calculation subunit, adapted to calculate the probability grid overlap rate of each traffic participant with trajectory intersection at each time step; A comparison subunit, adapted to compare each of the probability grid overlap rates with a preset probability grid overlap threshold; The judgment subunit is adapted to judge whether there is an interaction risk between the traffic participants whose trajectories intersect based on the comparison result.

11. The interactive risk prediction device according to claim 10, characterized in that: Also includes: The output unit is adapted to output warning information when there is a risk of interaction between various traffic participants whose trajectories intersect.

12. The interactive risk prediction device according to claim 10, characterized in that: The information acquisition unit comprises: A three-dimensional scene reconstruction subunit, adapted to perform three-dimensional scene reconstruction on the area to be predicted based on the obtained image information, and obtain a spatiotemporal sequence of three-dimensional scene representation corresponding to the area to be predicted; The first information acquisition subunit is adapted to obtain the grid occupancy information of each traffic participant in the area to be predicted at the current moment by using a multi-layer perceptron based on the spatiotemporal sequence represented by the three-dimensional scene corresponding to the area to be predicted; The second information acquisition subunit is adapted to obtain trajectory prediction information of each traffic participant in the area to be predicted based on the spatiotemporal sequence represented by the three-dimensional scene corresponding to the area to be predicted and using the Informer trajectory prediction model.

13. The interactive risk prediction device according to claim 12, characterized in that: The second information acquisition subunit includes: A construction module, adapted to construct an overall information sequence of each traffic participant in the area to be predicted based on a spatiotemporal sequence of a three-dimensional scene representation corresponding to the area to be predicted; The prediction module is adapted to input the overall information sequence of each traffic participant in the area to be predicted into the Informer trajectory prediction model to obtain the trajectory prediction information of each traffic participant in the area to be predicted.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program is executed by a processor to implement the steps of the method according to any one of claims 1 to 8.

15. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, wherein:When the processor runs the computer program, the steps of the method according to any one of claims 1 to 8 are performed.

Citation Information

Patent Citations

  • Intelligent traffic refined path planning method based on grid expansion model

    CN109959388A

  • Signal-lamp-free intersection automatic driving vehicle refined path scheduling method based on double learning networks

    CN115713854A

  • Obstacle trajectory prediction method and device, equipment and medium

    CN116309689A

  • Intersection motor vehicle flow multi-mode conflict risk identification method considering queue features

    CN117576948A

  • Computer-implemented method, computer programme, and arrangement for predicting and planning trajectories

    WO2022214414A1