Safety information auxiliary method and system based on express way individual vehicle risk short-term and temporary prediction

Abstract

The invention relates to a safety information auxiliary method and system based on expressway individual vehicle risk short-term and imminent prediction, the method comprises two stages of risk prediction and risk early warning, the risk prediction uses a vehicle risk short-term and imminent prediction model, the risk prediction is carried out based on the preprocessed traffic state data, and the risk early warning is carried out based on the vehicle risk short-term and imminent prediction model. The vehicle risk short-term and temporary prediction model comprises a Resnet module and an LSTM module which are improved based on a plug-and-play attention mechanism and a bidirectional pyramid structure; the traffic state data comprises vehicle track data and road section geometric data; the risk early warning is based on the result of the vehicle risk short-term and temporary prediction model, traffic conflict indexes of all vehicles in a preset range of the target vehicle are calculated in real time, and risk early warning is carried out; the system is used for implementing the method. Compared with the prior art, the two-stage auxiliary driving method provided by the invention can reduce effective information loss and guarantee the accuracy of a risk prediction result.

Description

Technical Field

[0001] The present invention relates to the field of expressway accident risks and vehicle control, and in particular to a safety information assistance method and system based on short-term and imminent prediction of individual vehicle risks on expressways. Background Art

[0002] Existing active safety control can be divided into two aspects: roadside-based and vehicle-mounted traffic safety control. Vehicle-mounted safety control, i.e., driving assistance, is also considered the main focus in the future field of traffic safety improvement due to its strong pertinence. With the rapid development of communication and computer technologies, video and radar monitors can obtain full-sample spatio-temporal continuous trajectory data in real time, effectively overcoming the defects of time aggregation of previous sectional detection data and spatial discontinuity. Video or radar coverage of important expressways and expressways has been basically achieved in many cities; at the same time, vehicle trajectory detection and tracking technologies based on video or radar equipment have made great progress in the academic and industrial circles, which will provide a solid data foundation for the research on the accident risk mechanism of continuous traffic flow. At the same time, driving assistance technologies are currently widely applied and promoted by automobile manufacturers, mainly including lane keeping assistance systems, automatic parking assistance systems, braking assistance systems, reverse assistance systems, and driving assistance systems. Driving assistance technologies can effectively reduce the workload of drivers, provide a better driving experience, and minimize accident risks. Driving assistance technologies will surely exist in the future for a long time, which will provide a reliable platform foundation for individual vehicle safety information assistance strategies. Chinese patent application 《CN119037415A》 provides a multi-modal large model driving risk judgment method, which divides vehicle collision risks into three levels. When it is the first level, the first preset alarm mode is activated; when it is the second level, real-time video is obtained; the real-time video and vehicle bus data are input into the multi-modal first-order large model to obtain a first result; the first result and the out-of-vehicle environment video are input into the multi-modal second-order large model to obtain a second result; it is judged whether the corrected risk probability is greater than the preset risk probability; if it is greater, it is judged whether it is in the preset risk type table; if not, the first preset alarm mode is activated; if so, the vehicle is controlled to decelerate at a preset acceleration and the second preset alarm mode is activated; if it is not greater, the first preset alarm mode is activated; when it is the third level, the vehicle is controlled to travel according to a preset avoidance plan. Although it improves the accuracy of driving risk judgment to a certain extent by combining the real-time state of the driver, it still cannot avoid the following disadvantages: The risk prediction research is limited by the model structure, resulting in the omission of the risk transmission process, that is, there is information attenuation between the first-stage model and the second-stage model. Specifically, the key parameters in the first-stage model are not time-aligned with the environmental video information in the second-stage model, so that effective information will be lost during the transmission process of risk characteristics between the two-stage models, affecting the accuracy of the results output by the model. Summary of the Invention

[0003] The object of the present invention is to overcome the defects of the above-mentioned existing technologies and provide a two-stage safety information assistance method based on short-term and imminent prediction of individual vehicle risks on expressways. An improved Resnet module and an LSTM module are used for risk prediction in the first stage, and then the real-time traffic state data after the driver's risk prediction actions is obtained to determine whether to enter the risk warning in the second stage. The risk warning in the second stage is carried out through the real-time traffic state data, and the improved Resnet is used for risk prediction, reducing the loss of effective information between the assisted driving stages and ensuring the accuracy of the prediction results.

[0004] A two-stage assisted driving system for risk prediction and risk driving is established, providing a directional assisted driving method when the specific source of the risk is clear on the basis of the existing ambiguity strategy, enabling the driver to be aware of the conflict risk earlier and allowing for a more sufficient reaction time.

[0005] The object of the present invention can be achieved by the following technical solutions:

[0006] According to the first aspect of the present invention, a safety information assistance method based on short-term and imminent prediction of individual vehicle risks on expressways is provided. The method includes two stages: risk prediction and risk warning. Among them, the risk prediction uses a vehicle risk short-term and imminent prediction model to perform risk prediction based on the preprocessed traffic state data. The vehicle risk short-term and imminent prediction model includes a Resnet module and an LSTM module improved based on a plug-and-play attention mechanism and a bidirectional pyramid structure; the plug-and-play attention mechanism includes spatial attention and channel attention; the risk warning is based on the results of the vehicle risk short-term and imminent prediction model, and the traffic conflict indicators of all vehicles within the preset range of the target vehicle are calculated in real time for risk warning.

[0007] As a preferred technical solution, the preprocessing includes:

[0008] Missing value filling: Detect the traffic state data and use linear interpolation to fill in the missing values;

[0009] Outlier repair: Use the three-standard deviation principle to detect outliers in the traffic state data and repair them;

[0010] Noise processing: Use a moving average method to smooth the data;

[0011] Data downsampling: Sample the trajectory data of the target vehicle in the form of a downsampling rate.

[0012] As a preferred technical solution, the Resnet module includes an input layer, an initial convolutional layer, a max pooling layer, an improved residual layer, a bidirectional pyramid structure fusion layer, an attention layer, and an output layer;

[0013] The improved residual layer includes a four-stage residual block sequence. Each stage of the residual block sequence includes a plurality of basic blocks, and each basic block in the third and fourth stages embeds a plug-and-play attention mechanism. A skip connection branch is established between the second, third, and fourth stages using a bidirectional pyramid structure.

[0014] As a preferred technical solution, the risk prediction includes:

[0015] Rasterize the preprocessed traffic state data within a preset time and within a preset range of the target vehicle to obtain a raster map;

[0016] Process the raster map through the input layer and then use the initial convolutional layer to obtain a first feature map;

[0017] Use the max pooling layer to reduce the spatial resolution of the first feature map to obtain a second feature map;

[0018] The first-stage residual block sequence performs convolutional processing on the second feature map to obtain a third feature map, and the third feature map has the same size and number of channels as the second feature map;

[0019] The second-stage residual block sequence performs convolutional processing on the third feature map to obtain a fourth feature map, and the size of the fourth feature map is smaller than that of the third feature map, and the number of channels is an integer multiple of that of the third feature map;

[0020] The third-stage residual block sequence receives the fourth feature map and combines the output of the skip connection branch, and performs convolutional processing based on the plug-and-play attention mechanism to obtain a fifth feature map, and the size of the fifth feature map is smaller than that of the fourth feature map, and the number of channels is an integer multiple of that of the fourth feature map;

[0021] The fourth-stage residual block sequence receives the fifth feature map and combines the output of the skip connection branch, and performs convolutional processing based on the plug-and-play attention mechanism to obtain a sixth feature map, and the size of the sixth feature map is smaller than that of the fifth feature map, and the number of channels is an integer multiple of that of the fifth feature map;

[0022] The fusion layer is based on the third, fourth, fifth, and sixth feature maps, and uses dilated convolution for upsampling and downsampling to obtain a fusion feature. After the fusion feature is processed by the attention layer, the output layer outputs an intermediate feature;

[0023] Use the LSTM module to process the intermediate feature, and obtain the predicted traffic conflict index based on the output result of the LSTM module.

[0024] As a preferred technical solution, the method of rasterization processing is:

[0025] Based on the preprocessed traffic state data, obtain the speed, lateral acceleration, and longitudinal acceleration of the bicycle. Construct vehicle microscopic operation state diagrams for speed, lateral acceleration, and longitudinal acceleration, including: taking the position of the bicycle as the center point, selecting a road section within a preset range as the grid base map; dividing the grid base map into multiple cells of a preset size, selecting the cells to be filled based on the vehicle size, and setting the values of the remaining cells to be empty; filling the acceleration, lateral acceleration, or longitudinal acceleration into the cells to be filled;

[0026] Integrate the vehicle microscopic operation state diagrams of speed, lateral acceleration, and longitudinal acceleration to obtain the rasterization result.

[0027] As a preferred technical solution, the spatial attention mechanism is constructed using 1×7 + 7×1 asymmetric convolution and is adaptively adjusted through a learnable gating coefficient.

[0028] As a preferred technical solution, in the second-stage residual block sequence, the third-stage residual block sequence, and the fourth-stage residual block sequence, downsampling is required for the first basic block, and the output of the first basic block is the sum of the output of the previous-stage residual block sequence and the downsampling result.

[0029] As a preferred technical solution, the risk warning includes:

[0030] When the traffic conflict index obtained by risk prediction is less than or equal to the preset value, enter the adjustment period;

[0031] Obtain the current traffic state data affected by the driver's behavior after risk prediction during the adjustment period, and calculate the current traffic conflict index. If the current traffic conflict index is less than or equal to the preset value, enter the risk warning; otherwise, return to the risk prediction stage;

[0032] Obtain real-time traffic state data, where the real-time traffic state data includes real-time vehicle trajectory data and road geometry data;

[0033] Based on the real-time vehicle trajectory data, obtain the spatial distance between the target vehicle and all vehicles within its preset range, the speed of the target vehicle, and the speeds of all vehicles within the preset range;

[0034] Calculate the traffic conflict index between the target vehicle and each vehicle within the preset range based on the spatial distance, the speed of the target vehicle, and the speeds of all vehicles within the preset range;

[0035] Screen potential collision vehicles based on the calculated traffic conflict index and add potential collision vehicle identifiers.

[0036] As a preferred technical solution, the method for calculating the traffic conflict index is:

[0037]

[0038] where x a,b represents the spatial distance between the target vehicle and any vehicle within the prediction range; v a (t) represents the speed of the target vehicle, and v b (t) represents the speed of any vehicle within the prediction range; TTC(t) represents the traffic conflict index.

[0039] According to the second aspect of the present invention, a safety information assistance system based on short-term and imminent prediction of individual vehicle risks on expressways is provided, and the system is used to implement the above method.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] 1), The present invention improves the traditional Resnet structure by using the plug-and-play attention mechanism and the bidirectional pyramid. Through channel attention, the model focuses on key areas, maintaining the advantage of the residual network in alleviating gradient disappearance; through spatial attention, global perception and feature expression ability are realized to improve the accuracy of the model in processing data; the bidirectional pyramid network is used to strengthen the cross-level feature complementarity between residual block sequences to enhance the semantic consistency of multi-scale features; combined with LSTM, the microscopic operating state of the vehicle is fully considered, and the time series characteristics of the traffic state evolution are captured, so as to realize the accurate prediction of vehicle risks; and when performing second-stage assisted driving, the real-time traffic state data after risk prediction is directly obtained, and the second-stage assisted driving is carried out based on the real-time traffic state data, avoiding the loss of effective information between cross-stage assisted driving, and improving the assistance benefit of the information assistance strategy.

[0042] 2), The present invention deeply combines risk prediction with the driving assistance system, takes into account the time requirements of the driver's reaction, and based on the short-term prediction results output by the risk prediction model, uses the predicted risk as the input of the assistance strategy, and designs the form of the safety information assistance strategy in the "risk prediction - early warning" stage, so that the driver can be aware of the conflict risk earlier and reserve more sufficient reaction time. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a schematic diagram of the two-stage safety information assisted driving logic of the present invention;

[0044] Figure 2 is a schematic diagram of the preprocessing logic of the present invention;

[0045] Figure 3 is a schematic diagram of the grid base map of the present invention;

[0046] Figure 4Schematic diagram of the grid map of the present invention;

[0047] Figure 5 Schematic diagram of the network structure of the present invention;

[0048] Figure 6 Flowchart of data processing for the residual sequence block of the present invention;

[0049] Figure 7 Schematic diagram of the ambiguous picture in the risk prediction stage of the present invention;

[0050] Figure 8 Schematic diagram of the ambiguous text in the risk prediction stage of the present invention;

[0051] Figure 9 Schematic diagram of assisted driving with safety information in the risk warning stage of the present invention; (9a) is a schematic diagram of the risk vehicle located on the side of the target vehicle; (9b) represents a schematic diagram of the risk vehicle located in front of the vehicle. Detailed implementation manners

[0052] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0053] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the ordinary meanings understood by those with ordinary skills in the technical field to which this application belongs. The words such as "a", "an", "one", "the" and the like involved in this application do not indicate a limitation in quantity and may represent a singular or plural number. The terms "include", "comprise", "have" and any variations thereof involved in this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may further include unlisted steps or units, or may further include other steps or units inherent to these processes, methods, products or devices. The words such as "connect", "be connected", "couple" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The term "plurality" involved in this application means two or more. "And / or" describes the association relationship of associated objects and indicates that three relationships may exist. For example, "A and / or B" may represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after. The terms "first", "second", "third" and the like involved in this application are only used to distinguish similar objects and do not represent a specific order for the objects.

[0054] In order to reduce the problems of loss of effective information and inaccurate and uncertain risk prediction results during the two-stage blocking driving process, the present invention provides a two-stage assisted driving that combines a vehicle risk short-term prediction model and a real-time calculated traffic conflict index. And in order to enable the driver to be aware of the conflict risk earlier and reserve a more sufficient reaction time, a first stage (risk prediction stage) of establishing an information assistance strategy based on the prediction result 5s after the output of the risk prediction model is set, and on the basis of the ambiguity strategy, the influence of the driver on the traffic state after the first-stage assisted driving strategy is incorporated into the consideration scope of the two-stage assisted driving. A second stage (risk warning stage) of establishing an information assistance strategy based on the real-time calculated traffic conflict index is designed with a directional assistance strategy. By combining two complementary auxiliary stages in time, a two-stage auxiliary system is realized. The detailed process of the two-stage safety information assisted driving is as follows Figure 1 shown, including:

[0055] S1. Obtain traffic state data and perform preprocessing.

[0056] S11. Extract traffic state data including vehicle trajectory and road section geometric data, where the vehicle trajectory data includes vehicle position, speed, acceleration, etc.

[0057] S12. Perform preprocessing on the obtained traffic state data. The detailed preprocessing process is as follows Figure 2As shown in the figure, it includes:

[0058] S121, Missing value filling: Detect the traffic state data and use linear interpolation method for missing value filling.

[0059] S122, Outlier repair: Use the three - standard - deviation principle to detect outliers in the traffic state data and repair them.

[0060] S123, Noise processing: Use the moving average method to smooth the data.

[0061] S124, Data downsampling: Use the downsampling rate form to sample the trajectory data of the target vehicle.

[0062] S2, Based on the pre - processed traffic state data, convert the vehicle trajectory into a grid map form with high integration and easier to identify.

[0063] S21, Extract a grid base map that is 25m long in the longitudinal direction and 5m wide in the transverse direction within 15s with the target vehicle as the center, as Figure 3 shown in the figure, where the target vehicle in the figure is the rectangle where the five - pointed star is located.

[0064] S22, Based on the pre - processed traffic state data, obtain the speed, lateral acceleration, and longitudinal acceleration of a single vehicle, and construct a vehicle microscopic operation state map for each of the speed, lateral acceleration, and longitudinal acceleration.

[0065] S23, Divide the grid base map into multiple cells of 0.2×0.5, as Figure 4 shown in the left figure in the figure, and select the cells to be filled with values based on the vehicle size, and set the values of the remaining cells to be empty.

[0066] S24, Fill the cells to be filled with values with acceleration, lateral acceleration, or longitudinal acceleration to obtain the corresponding vehicle microscopic operation state map, as Figure 4 shown in the right figure in the figure.

[0067] S25, Integrate the vehicle microscopic operation state maps of speed, lateral acceleration, and longitudinal acceleration to obtain the rasterization result.

[0068] S3, Use the vehicle risk short - term prediction model to perform the first - stage safety information - assisted driving - risk prediction based on the grid map obtained in step S2.

[0069] Among them, the vehicle risk short - term prediction model includes a Resnet module, an LSTM module, and multiple fully - connected layers improved based on the plug - and - play attention mechanism and the bidirectional pyramid structure. The model architecture is as Figure 5 shown in the figure.

[0070] In this embodiment, the Resnet module selects the Resnet-18 architecture. The plug-and-play attention mechanism includes channel attention and spatial attention. The Resnet module includes an input layer, an initial convolutional layer, a max pooling layer, an improved residual layer, a bidirectional pyramid structure fusion layer, an attention layer, and an output layer. The improved residual layer includes a sequence of four-stage residual blocks. Each stage of the residual block sequence includes multiple basic blocks. And each basic block in the third and fourth stages embeds the plug-and-play attention mechanism. A skip connection branch is established using the bidirectional pyramid structure between the second, third, and fourth stages.

[0071] Its processing process includes:

[0072] S31. After passing the raster image through the input layer, it is processed by the initial convolutional layer to obtain the first feature map. Specifically, the convolutional kernel size of the initial convolutional layer is 7×7, the set convolutional sliding step is 2. And since the sliding of the convolutional kernel during the convolutional operation will cause the size of the output feature map to decrease, when performing the convolutional operation in the initial convolutional layer, 3 pixels are padded at the edge of the raster image to offset the size reduction, so that the convolutional kernel can completely cover the edge and avoid information loss. At this time, the channel of the output first feature map is 64.

[0073] S32. Use the max pooling layer to reduce the spatial resolution of the first feature map to obtain the second feature map. Specifically, the pooling kernel of the max pooling layer is 3×3, and the set sliding step is 2. The maximum value is taken in each pooling window.

[0074] The second feature map is input into the improved residual layer for processing. Specifically, in the first-stage residual block sequence, there are two basic blocks (BasicBlock). Each basic block is a 3×3 convolution and its channel number changes from 64 to 64; in the second-stage residual block sequence, there are two BasicBlocks. The convolutional sliding step of the first block is 2 and downsampling is required. The channel number change between the two BasicBlocks is from 64 to 128; the third-stage residual block sequence includes two BasicBlocks. The convolutional sliding step of the first block is 2 and downsampling is required. The channel number change between the two BasicBlocks is from 128 to 256; the fourth-stage residual block sequence includes two BasicBlocks. The convolutional sliding step of the first block is 2 and downsampling is required. The channel number change between the two BasicBlocks is from 256 to 512.

[0075] And the plug-and-play attention mechanism (CBAM) is embedded in both the third-stage residual block sequence and the fourth residual block sequence. Among them, in the channel attention of this attention mechanism, grouped global pooling (4 groups) is used to generate weights, that is, the channel dimension of the input feature map is evenly divided into four groups, and each group of channels is processed independently. Global average pooling and global max pooling are respectively performed on each channel group, and the pooling results of each group are passed through a fully connected layer with shared parameters. The generated weight vectors of each group are concatenated and normalized to the [0, 1] interval through the Sigmoid function to form the final channel attention weight map. Through this, local correlations in the channel subspace can be captured by grouped processing, avoiding information loss in global statistics, reducing the number of parameters at the same time, and alleviating the risk of overfitting; the spatial attention is constructed using 1×7 + 7×1 asymmetric convolution, that is, the 1×7 convolution kernel covers 7 pixel units along the vertical direction. When cascaded with the 7×1 horizontal convolution, it constitutes a receptive field equivalent to a 7×7 symmetric kernel, which not only retains the global perception ability but also enhances the feature expression ability, and introduces a learnable gating coefficient (α ∈ [0, 1]) to realize the adaptive adjustment of the attention intensity. Its process is as Figure 6 shown and includes the following steps:

[0076] S33. The first-stage residual block sequence performs convolution processing on the second feature map to obtain a third feature map, and the third feature map has the same size and number of channels as the second feature map, both being 64. Its expression is:

[0077] F(x) = σ(W 2 (σ(W 1 ·x + b 1 )) + b 2 ),

[0078] where x represents the second feature map; W 1 and W 2 represent the convolution kernel weights of the corresponding BasicBlock; b 1 and b 2 represent the biases.

[0079] S34. The second-stage residual block sequence performs convolution processing on the third feature map to obtain a fourth feature map, and the size of the fourth feature map is smaller than that of the third feature map, and the number of channels is increased to 128. The form of its convolution is the same as that of the first-stage residual block sequence. Among them, for the first BasicBlock, its processing process also includes:

[0080] x′ = W s ·F(x),

[0081] F″(x) = F′(x) + x′;

[0082] where F(x) represents the third feature map; W sDenote the downsampling convolution weights as; denote the downsampling result as x'; denote the convolution result as F'(x); denote the fourth feature map as F''(x).

[0083] S35. The third-stage residual block sequence receives the fourth feature map and combines the output of the skip connection branch, performs convolution processing based on the plug-and-play attention mechanism to obtain the fifth feature map, and the size of the fifth feature map is smaller than that of the fourth feature map, with the number of channels being 256.

[0084] Similarly, the convolution form in this stage is the same as that in the first-stage residual block sequence, and the downsampling method of the first BasicBlock is the same as that in the second-stage residual block sequence. The expression of its attention mechanism is:

[0085] M c (y) = σ(MLP(F avg ) + MLP(F max ))

[0086] This formula is the channel attention expression. M c (y) represents the channel attention result. F avg represents global average pooling of the features, and F max represents global max pooling of the features.

[0087]

[0088] This formula is the spatial attention expression. M s (y) represents the spatial attention result. represents average pooling of the features in the spatial dimension. represents max pooling of the features in the spatial dimension.

[0089] y = (α · M s (M c (y)) + (1 - α)) ⊙ F'''(x)

[0090] This formula is the output calculation formula of the third-stage residual block sequence. y represents the final output, α represents the gating coefficient, ⊙ represents element-wise multiplication, M c (·) represents the channel attention operation, M s (·) represents the spatial attention operation, and F'''(x) represents the operation result in the third stage except for the attention mechanism.

[0091] S36. The fourth-stage residual block sequence receives the fifth feature map and combines the output of the skip connection branch, performs convolution processing based on the plug-and-play attention mechanism to obtain the sixth feature map. The processing process is the same as that in step S35, and the size of the sixth feature map is smaller than that of the fifth feature map, with the number of channels being 512.

[0092] S37. The fusion layer is based on the third feature map, the fourth feature map, the fifth feature map, and the sixth feature map, and uses dilated convolution for upsampling and downsampling to obtain fused features, so as to take into account both detailed information and global semantic information. After the fused features are processed by the attention layer, intermediate features are output by the output layer.

[0093] The fusion layer is a bidirectional pyramid structure and embeds dilated convolutions with three different dilation rates (6 / 12 / 18) to expand the receptive field, and realizes soft selection of multi-scale features through differentiable weight calculation. Specifically, between the outputs of the second-stage residual block sequence, the third-stage residual block sequence, and the fourth-stage residual block sequence, skip connections are established using a bidirectional pyramid structure, and channel alignment and spatial dimension matching are performed with adjacent low-level features through the FPN backbone. Detail information is enhanced with high-level features through downsampling, and information is superimposed and fused to avoid feature degradation caused by continuous sampling. By parallelly deploying convolution kernels with different dilation rates, collaborative perception of local details and global semantics can be achieved. Differentiable weights can automatically adjust the feature fusion ratio according to the complexity of the input content, enhance the weight of the small-dilation path for fine-grained targets, and emphasize high-level semantic information for large-scale targets.

[0094] The output of the final attention layer is: F final = M s (M c (y′)) ⊙ y′, where y′ represents the fused features, M c (·) represents channel attention operation, and M s (·) represents spatial attention operation.

[0095] S38. Use the LSTM module to process the intermediate features and obtain the predicted traffic conflict indicators.

[0096] There are also four fully connected layers between the LSTM modules. These four fully connected layers gradually compress the output channels of the Resnet module from 512 to 64, and finally obtain a feature with 64 channels, which is used as the input of the LSTM module for calculation.

[0097] Specifically, the LSTM module has one layer, the number of hidden neurons is 64, and the output is a feature vector with a dimension of 16. This 16-dimensional feature vector is processed by a fully connected layer and then outputs a feature vector with a dimension of 2. Based on this 2-dimensional feature vector, the predicted traffic conflict indicators 5s later are obtained.

[0098] In the above process, by reducing the dimension (512 → 64) through the fully connected layer and the single-layer LSTM design, the balance between model complexity and real-time requirements is achieved, adapting to the hardware limitations of in-vehicle computing units.

[0099] S39. If the predicted traffic conflict index after 5 seconds (set to 3 seconds in this embodiment) is greater than the preset value, then generate a blurred picture and text for risk prediction. The detailed blurred picture is as shown in Figure 7 which prompts the driver in the form of a halo and background markings. The blurred text is as shown in Figure 8 which directly conveys the risk in concise language. The blurred indication process lasts for 2 seconds.

[0100] In addition, since traffic conflicts belong to minority events, there is a serious imbalance between conflict and non-conflict samples. If all samples are included in the construction of the model, according to the optimization mechanism of the loss function, the model parameters will tend to classify all samples as non-conflicts by a large margin, and the prediction accuracy for conflict samples is extremely low, even dropping to 0. To avoid this problem and improve the model performance, the present invention selects the method of undersampling, that is, matching conflict samples and non-conflict samples at a ratio of 1:4 to jointly form the training set of the vehicle risk short-term prediction model, and setting that during the training process, the number of batches is 30, the learning rate is 3×10 -5 , and the dropout rate is 0.3. After the training is completed, the AUC of the model prediction performance on the training set and the test set can reach 0.868 and 0.816 respectively, indicating that on the one hand, the model has good prediction performance, and on the other hand, it can better distinguish positive and negative samples, and has good prediction ability for both conflict samples and non-conflict samples.

[0101] S4. Risk warning.

[0102] S41. If the traffic conflict index obtained by risk prediction is less than or equal to the preset value, enter the adjustment period.

[0103] S42. Obtain the current traffic state data affected by the driver's behavior after risk prediction during the adjustment period, and calculate the current traffic conflict index. If the current traffic conflict index is less than or equal to the preset value, enter the risk warning; otherwise, return to the risk prediction stage.

[0104] S43. Obtain the real-time traffic state data, where the real-time traffic state data includes real-time vehicle trajectory data and road geometry data.

[0105] S44. Based on the real-time vehicle trajectory data, obtain the spatial distance between the target vehicle and all vehicles within its preset range, the speed of the target vehicle, and the speeds of all vehicles within the preset range.

[0106] S45. Calculate the traffic conflict index between the target vehicle and each vehicle within the preset range based on the spatial distance, the speed of the target vehicle, and the speeds of all vehicles within the preset range. The expression is:

[0107]

[0108] Among them, x a,b represents the spatial distance between the target vehicle and any vehicle within the prediction range; v a (t) represents the speed of the target vehicle, and v b (t) represents the speed of any vehicle within the prediction range; TTC(t) represents the traffic conflict indicator.

[0109] S46. According to the calculated TTC value, clearly locate the relative position between the potential collision vehicle and the target vehicle, and mark the potential collision vehicle on the auxiliary interface to help the driver make better driving decisions. The marking form is as shown in Figure 9 (9a) and (9b) below.

[0110] This embodiment also provides a safety information assistance system based on short-term and imminent prediction of individual vehicle risks on expressways. This system is used to implement the above method. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the described modules can refer to the corresponding process in the foregoing method embodiment and will not be elaborated here.

[0111] The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or substitutions within the technical scope disclosed by the present invention. These modifications or substitutions should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A safety information assistance method based on short-term prediction of individual vehicle risks on expressways, characterized in that: The method includes two stages: risk prediction and risk warning. The risk prediction uses a vehicle short-term risk prediction model to predict risks based on preprocessed traffic status data. The vehicle short-term risk prediction model includes a Resnet module and an LSTM module improved based on a plug-and-play attention mechanism and a bidirectional pyramid structure; the plug-and-play attention mechanism includes channel attention and spatial attention; the risk warning is based on the result of the vehicle short-term risk prediction model, and the traffic conflict indicators of all vehicles within a preset range of the target vehicle are calculated in real time to perform risk warning.

2. A safety information assistance method based on short-term prediction of individual vehicle risk on expressway according to claim 1, characterized in that: The pretreatment comprises: Missing value filling: detecting the traffic status data and using linear interpolation to fill in the missing values; Outlier repair: Use the triple standard deviation principle to detect outliers in the traffic status data and perform repairs; Noise processing: The sliding average method is used to smooth the data; Data downsampling: The trajectory data of the target vehicle is sampled in the form of downsampling rate.

3. The safety information assistance method based on the short-term prediction of individual vehicle risk on expressway according to claim 1 is characterized in that: The Resnet module includes an input layer, an initial convolution layer, a maximum pooling layer, an improved residual layer, a bidirectional pyramid structure fusion layer, an attention layer and an output layer; The improved residual layer includes a four-stage residual block sequence, each stage of the residual block sequence includes multiple basic blocks, and each basic block of the third stage and the fourth stage is embedded with a plug-and-play attention mechanism, and a jump connection branch is established between the second stage, the third stage and the fourth stage using a bidirectional pyramid structure.

4. A safety information assistance method based on short-term prediction of individual vehicle risk on expressway according to claim 3, characterized in that: The risk foreseeability mentioned includes: The pre-processed traffic status data within a preset time and within a preset range of the target vehicle is rasterized to obtain a raster map; The grid image is processed by an initial convolutional layer after passing through the input layer to obtain a first feature map; Using the maximum pooling layer to reduce the spatial resolution of the first feature map to obtain a second feature map; The residual block sequence in the first stage performs convolution processing on the second feature map to obtain a third feature map, and the third feature map has the same size and number of channels as the second feature map; The second-stage residual block sequence performs convolution processing on the third feature map to obtain a fourth feature map, and the size of the fourth feature map is smaller than the size of the third feature map, and the number of channels is an integer multiple of the third feature map; In the third stage, the residual block sequence receives the fourth feature map and combines it with the output of the jump connection branch, performs convolution processing based on the plug-and-play attention mechanism, and obtains the fifth feature map, and the size of the fifth feature map is smaller than the size of the fourth feature map, and the number of channels is an integer multiple of the fourth feature map; In the fourth stage, the residual block sequence receives the fifth feature map and combines it with the output of the jump connection branch, performs convolution processing based on the plug-and-play attention mechanism, and obtains the sixth feature map, and the size of the sixth feature map is smaller than the size of the fifth feature map, and the number of channels is an integer multiple of the fifth feature map; The fusion layer uses dilated convolution to perform upsampling and downsampling based on the third feature map, the fourth feature map, the fifth feature map, and the sixth feature map to obtain fusion features. After the fusion features are processed by the attention layer, the output layer outputs the intermediate features. The intermediate features are processed using the LSTM module, and the predicted traffic conflict index is obtained based on the output result of the LSTM module.

5. A safety information assistance method based on short-term prediction of individual vehicle risk on expressway according to claim 4, characterized in that: The rasterization processing method is: Based on the pre-processed traffic status data, the speed, lateral acceleration and longitudinal acceleration of the bicycle are obtained, and a vehicle microscopic operation state diagram is constructed for the speed, lateral acceleration and longitudinal acceleration, including: taking the position of the bicycle as the center point, selecting a road section within a preset range as a grid base map; dividing the grid base map into a plurality of cells of preset sizes, and selecting a cell to be filled in based on the vehicle size, and setting the values ​​of the remaining cells to be empty; filling the acceleration, lateral acceleration or longitudinal acceleration into the cell to be filled in; The vehicle microscopic operation state diagram of the integrated speed, lateral acceleration and longitudinal acceleration is obtained to obtain the rasterized results.

6. According to the safety information auxiliary method based on short-term prediction of individual vehicle risks on expressways described in claim 4, the spatial attention mechanism is constructed using 1×7+7×1 asymmetric convolution and is adaptively adjusted through learnable gating coefficients.

7. A safety information assistance method based on short-term prediction of individual vehicle risk on expressway according to claim 4, characterized in that: In the second-stage residual block sequence, the third-stage residual block sequence and the fourth-stage residual block sequence, the first basic block needs to be downsampled, and the output of the first basic block is the sum of the output of the residual block sequence in the previous stage and the downsampling result.

8. The safety information auxiliary method based on the short-term prediction of individual vehicle risk on expressway according to claim 1 is characterized in that: The risk warnings include: If the traffic conflict index obtained by risk prediction is less than or equal to the preset value, the adjustment period begins; Obtain the current traffic status data affected by the driver's behavior after risk prediction during the adjustment period, and calculate the current traffic conflict index. If the current traffic conflict index is less than or equal to the preset value, enter the risk warning stage, otherwise return to the risk prediction stage; Acquiring real-time traffic status data, wherein the real-time traffic status data includes real-time vehicle trajectory data and road geometry data; Based on the real-time vehicle trajectory data, the spatial distance between the target vehicle and all vehicles within a preset range, the target vehicle speed, and the speeds of all vehicles within the preset range are obtained; Calculate the traffic conflict index between the target vehicle and each vehicle within the preset range based on the spatial distance, the speed of the target vehicle and the speeds of all vehicles within the preset range; Potential collision vehicles are screened based on the calculated traffic conflict index and potential collision vehicle identifiers are added to them.

9. The safety information auxiliary method based on the short-term prediction of individual vehicle risk on expressway according to claim 1 is characterized in that: The method for calculating the traffic conflict index is: Among them, x a,b represents the spatial distance between the target vehicle and any vehicle within the prediction range; v a (t) represents the speed of the target vehicle, v b (t) represents the speed of any vehicle within the prediction range; TTC(t) represents the traffic conflict index.

10. A safety information assistance system based on short-term prediction of individual vehicle risks on expressways, characterized in that: The system is used to implement the method according to any one of claims 1 to 9.

Images