Traffic safety warning system and method for highway

Abstract

Disclosed is a traffic safety warning system and method for highways. First, a real-time highway road image of a road section to be analyzed collected by a drone is obtained, then the real-time highway road image is subjected to data preprocessing to obtain a sequence of real-time highway road image blocks, then local real-time road state features of the sequence of real-time highway road image blocks are extracted to obtain a sequence of real-time highway road state feature matrices, then inter-section road state association features between the sequences of real-time highway road state feature matrices are extracted to obtain an inter-section highway road state association feature map, and finally, based on the inter-section highway road state association feature map, it is determined whether to generate a highway traffic safety warning. In this way, it is possible to automatically determine whether a highway traffic safety warning needs to be generated.

Description

Technical Field

[0001] The present application relates to the field of traffic safety, and more specifically, to a traffic safety warning system and method for highways. Background Art

[0002] The development of expressways has promoted the high-speed operation of passenger flow, logistics and information flow, greatly improved transportation efficiency, and brought significant social and economic benefits. However, expressways also have traffic safety risks. In order to effectively prevent and reduce traffic accidents on expressways, it is necessary to monitor and analyze the road conditions of expressways in real time and provide traffic safety warning information to drivers or traffic management personnel in a timely manner.

[0003] However, the traditional road status monitoring method based on fixed cameras or sensors has problems such as limited coverage and susceptibility to weather and light, which cannot meet the needs of highway traffic safety warning. Therefore, an optimized traffic safety warning method for highways is expected. Summary of the invention

[0004] In view of this, the present application proposes a traffic safety warning system and method for highways, which can automatically determine whether it is necessary to generate a highway traffic safety warning, and promptly inform the driver to adjust the driving route or inform traffic management personnel to take corresponding measures.

[0005] According to one aspect of the present application, a traffic safety warning method for a highway is provided, which includes:

[0006] Acquire real-time highway road images of the road section to be analyzed collected by a drone;

[0007] Performing data preprocessing on the real-time highway road image to obtain a sequence of real-time highway road image blocks;

[0008] Extracting local real-time road state features of the sequence of highway real-time road image blocks to obtain a sequence of highway real-time road state feature matrices;

[0009] Extracting inter-section road state correlation features between the sequences of the highway real-time road state feature matrix to obtain an inter-section road state correlation feature graph of the highway; and

[0010] Based on the road state association feature map between the highway sections, it is determined whether to generate a highway traffic safety warning.

[0011] According to another aspect of the present application, there is provided a traffic safety warning system for a highway, comprising:

[0012] An image acquisition module, used to acquire a real-time highway road image of the road section to be analyzed collected by a drone;

[0013] A data preprocessing module, used for performing data preprocessing on the real-time highway road image to obtain a sequence of real-time highway road image blocks;

[0014] A local real-time road state feature extraction module, used to extract local real-time road state features of the sequence of highway real-time road image blocks to obtain a sequence of highway real-time road state feature matrices;

[0015] an inter-section road state correlation feature extraction module, used to extract inter-section road state correlation features between the sequences of the highway real-time road state feature matrix to obtain an inter-section road state correlation feature map for the highway; and

[0016] The safety warning module is used to determine whether to generate a highway traffic safety warning based on the road state association feature map between the highway sections.

[0017] According to an embodiment of the present application, it first obtains a real-time highway road image of a road section to be analyzed collected by a drone, then performs data preprocessing on the real-time highway road image to obtain a sequence of real-time highway road image blocks, then extracts local real-time road state features of the sequence of real-time highway road image blocks to obtain a sequence of real-time highway road state feature matrices, then extracts inter-section road state association features between the sequence of real-time highway road state feature matrices to obtain an inter-section highway road state association feature map, and finally, based on the inter-section highway road state association feature map, determines whether to generate a highway traffic safety warning. In this way, it can be automatically determined whether a highway traffic safety warning needs to be generated.

[0018] Other features and aspects of the present application will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the present application and, together with the description, serve to explain the principles of the present application.

[0020] Figure 1 A flow chart of a traffic safety warning method for a highway according to an embodiment of the present application is shown.

[0021] Figure 2 A schematic diagram of the architecture of a traffic safety warning method for highways according to an embodiment of the present application is shown.

[0022] Figure 3 A block diagram of a traffic safety warning system for a highway according to an embodiment of the present application is shown.

[0023] Figure 4 An application scenario diagram of a traffic safety warning method for a highway according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work also fall within the scope of protection of the present application.

[0025] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0026] Various exemplary embodiments, features and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0027] In addition, in order to better illustrate the present application, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present application can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present application.

[0028] In response to the above technical problems, the technical concept of this application is to use the flexibility and real-time performance of drones to collect real-time road images of highways, and to perform deep learning-based image analysis on the real-time road images of highways, thereby automatically determining whether it is necessary to generate a highway traffic safety warning, and promptly inform the driver to adjust the driving route or inform traffic management personnel to take corresponding measures.

[0029] Based on this, Figure 1 A flow chart of a traffic safety warning method for a highway according to an embodiment of the present application is shown. Figure 2 FIG. 1 is a schematic diagram showing the architecture of a traffic safety warning method for a highway according to an embodiment of the present application. Figure 1 and Figure 2 As shown, according to the embodiment of the present application, the traffic safety warning method for highways includes the following steps: S110, acquiring a real-time highway road image of a road section to be analyzed collected by a drone; S120, performing data preprocessing on the real-time highway road image to obtain a sequence of real-time highway road image blocks; S130, extracting local real-time road state features of the sequence of real-time highway road image blocks to obtain a sequence of real-time highway road state feature matrices; S140, extracting inter-section road state association features between the sequence of real-time highway road state feature matrices to obtain an inter-section highway road state association feature map; and, S150, determining whether to generate a highway traffic safety warning based on the inter-section highway road state association feature map.

[0030] It should be understood that the purpose of step S110 is to collect highway road images through drones to obtain real-time road conditions. The drone can provide a bird's-eye view and can capture information such as traffic conditions, vehicle density, and road conditions on the road. In step S120, the collected highway road images are preprocessed. The preprocessing may include image denoising, image enhancement, image segmentation, and other operations to extract a sequence of road image blocks. Segmenting the image into blocks can make subsequent feature extraction and analysis more efficient. In step S130, local real-time road state features are extracted from the sequence of highway real-time road image blocks. These features may include information such as vehicle speed, lane deviation, and vehicle density. Extracting these features can help analyze the traffic conditions and safety risks of the road. In step S140, road state association features between sections are extracted from the sequence of highway real-time road state feature matrices. These features may include information such as relative positions between vehicles and speed differences between vehicles. By analyzing the road state association features between sections, the traffic flow and traffic safety risks between different sections can be understood. In step S150, based on the road state correlation feature map between highway sections, a traffic safety warning decision is made. According to the pre-set warning rules and models, it is determined whether the current road state poses a safety risk. If there is a safety risk, a corresponding traffic safety warning is generated to remind relevant departments and drivers to take corresponding measures to ensure traffic safety. These steps combine drone image acquisition, data preprocessing, feature extraction and analysis technologies, which can help monitor and warn traffic safety situations on highways.

[0031] Specifically, in the technical solution of the present application, firstly, a real-time highway road image of the road section to be analyzed collected by a drone is obtained; and the real-time highway road image is segmented along the extension direction of the road section to be analyzed to obtain a sequence of real-time highway road image blocks. Here, the real-time highway road image is decomposed into smaller units by image segmentation, which is convenient for extracting road state features of each unit. In other words, in this way, the subsequent network model can be guided to pay attention to the local road state of the real-time highway road image. In addition, the road usually has a certain continuity in the extension direction, and segmenting the road image according to the extension direction can better retain the geometric shape and structural information of the road. In this way, the road can be divided into multiple smaller image blocks, so that the road features in each image block are clearer and more prominent.

[0032] Accordingly, in step S120, the real-time highway road image is subjected to data preprocessing to obtain a sequence of real-time highway road image blocks, including: segmenting the real-time highway road image along the extension direction of the road section to be analyzed to obtain a sequence of real-time highway road image blocks.

[0033] Then, the sequence of the highway real-time road image blocks is passed through a road state feature extractor using a spatial attention mechanism to obtain a sequence of highway real-time road state feature matrices. That is, the local real-time road state features of the sequence of highway real-time road image blocks are extracted. Among them, the use of the spatial attention mechanism can induce the network model to automatically focus on the most important areas in the image, such as vehicles, obstacles, road conditions and other information.

[0034] Accordingly, in step S130, local real-time road state features of the sequence of highway real-time road image blocks are extracted to obtain a sequence of highway real-time road state feature matrices, including: extracting the sequence of highway real-time road image blocks through a road state feature extractor using a spatial attention mechanism to obtain a sequence of highway real-time road state feature matrices.

[0035] It is worth mentioning that the spatial attention mechanism is a commonly used attention mechanism in deep learning, which is used to enhance the model's attention to different positions in the input data. It can help the model capture and utilize spatial information more effectively when processing spatially structured data (such as images, videos, or speech). In the traditional convolutional neural network (CNN), the convolution layer extracts local features by sliding windows, but the features at different positions are processed in the same way without considering the relationship between positions. The spatial attention mechanism introduces learnable weights, which enables the model to adaptively adjust the weights of features according to different positions of the input data. The spatial attention mechanism usually includes the following steps: 1. Feature extraction: extract features from the input data through a series of operations such as convolutional layers and pooling layers. 2. Attention weight calculation: calculate the attention weight of each position based on the extracted features. This can be achieved through different methods, such as using fully connected layers, convolutional layers, or self-attention mechanisms. 3. Feature weighting: apply attention weights to the extracted features to weight features at different positions. Usually, the features are multiplied element-by-element or weighted summed by the attention weights. 4. Feature fusion: The weighted features are fused to obtain the final representation or feature map. By introducing the spatial attention mechanism, the model can automatically learn the degree of attention to different positions in the input data, so as to better capture the spatial structure and important features. This plays an important role in tasks such as image classification, object detection, and image segmentation, and can improve the performance and generalization ability of the model.

[0036] Specifically, the sequence of the real-time road image blocks of the highway is obtained by a road state feature extractor using a spatial attention mechanism to obtain a sequence of the real-time road state feature matrices of the highway, including: each layer of the road state feature extractor using the spatial attention mechanism performs the following on the input data during the forward transmission process of the layer: convolution processing is performed on the input data to generate a sequence of convolution feature maps; pooling processing is performed on the sequence of convolution feature maps to generate a sequence of pooling feature maps; non-linear activation is performed on the sequence of pooling feature maps to generate a sequence of activation feature maps; the mean of each position of each activation feature map in the sequence of activation feature maps along the channel dimension is calculated to generate a sequence of spatial feature matrices; the Softmax-like function value of each position of each spatial feature matrix in the sequence of spatial feature matrices is calculated to obtain a sequence of spatial score matrices; and the positional point multiplication of the sequence of spatial feature matrices and the sequence of spatial score matrices is calculated to obtain multiple feature matrices; wherein the multiple feature matrices output by the last layer of the road state feature extractor using the spatial attention mechanism are the sequence of real-time road state feature matrices of the highway.

[0037] Next, the sequence of the highway real-time road state feature matrix is ​​aggregated into a highway real-time road state input tensor along the channel dimension, and then the road state correlation feature map between highway sections is obtained by using the road state correlation feature extractor between sections of the channel attention mechanism. That is, the road state correlation features between sections of the sequence of the highway real-time road state feature matrix are extracted. It should be understood that by capturing the road state changes and correlations between different image blocks, such as the changes in traffic volume between image blocks, the safety status and road traffic status of the entire road section to be analyzed can be reflected. In addition, the channel attention mechanism will adaptively enhance or suppress different feature channels, thereby highlighting the feature distribution of image blocks with high road safety risks. Furthermore, the road state correlation feature map between highway sections is passed through a classifier to obtain a classification result, and the classification result is used to indicate whether a highway traffic safety warning is generated.

[0038] Accordingly, in step S140, the inter-section road state association features between the sequence of the highway real-time road state feature matrix are extracted to obtain the inter-section road state association feature map of the highway, including: aggregating the sequence of the highway real-time road state feature matrix into a highway real-time road state input tensor along the channel dimension and then using the inter-section road state association feature extractor using the channel attention mechanism to obtain the inter-section road state association feature map of the highway.

[0039] It is worth mentioning that the channel attention mechanism is a commonly used attention mechanism in deep learning, which is used to enhance the model's attention to different channels in the input data. It can help the model adaptively adjust the weight of the channel when processing features in the channel dimension to extract more useful and important information. In the traditional convolutional neural network (CNN), the convolution layer extracts features in the channel dimension by operating the filter of each channel. However, the importance of features between different channels may be different, and the features of some channels may be more critical for specific tasks. The channel attention mechanism introduces learnable weights so that the model can automatically learn the importance of different channels and adjust the weights of the channels. The channel attention mechanism usually includes the following steps: 1. Feature extraction: extract features from the input data through a series of operations such as convolutional layers and pooling layers. 2. Channel feature statistics: perform statistics on the extracted features and calculate the statistical features of each channel, such as the average value, maximum value, etc. 3. Attention weight calculation: calculate the attention weight of each channel based on the statistical features of the channel. Fully connected layers or convolutional layers are usually used to implement this calculation process. 4. Feature weighting: Apply attention weights to the extracted features and weight features of different channels. Usually, the features are multiplied element by element by the attention weights. 5. Feature fusion: The weighted features are fused to obtain the final representation or feature map. By introducing the channel attention mechanism, the model can adaptively adjust the weights of the channels so that channels that are more useful for specific tasks receive higher weights, thereby improving the performance and generalization ability of the model. The channel attention mechanism plays an important role in tasks such as image classification, object detection, and image segmentation, and can help the model better utilize information in the channel dimension.

[0040] Accordingly, in step S150, based on the road state association feature map between highway sections, it is determined whether to generate a highway traffic safety warning, including: passing the road state association feature map between highway sections through a classifier to obtain a classification result, and the classification result is used to indicate whether to generate a highway traffic safety warning.

[0041] More specifically, the road state association feature map between highway sections is passed through a classifier to obtain a classification result, and the classification result is used to indicate whether a highway traffic safety warning is generated, including: expanding the road state association feature map between highway sections into a classification feature vector according to a row vector or a column vector; using the fully connected layer of the classifier to fully connect encode the classification feature vector to obtain an encoded classification feature vector; and inputting the encoded classification feature vector into the Softmax classification function of the classifier to obtain the classification result.

[0042] That is, in the technical solution of the present application, the labels of the classifier include generating a highway traffic safety warning (first label), and not generating a highway traffic safety warning (second label), wherein the classifier determines which classification label the highway road state association feature map between sections belongs to by means of a soft maximum function. It is worth noting that the first label p1 and the second label p2 here do not contain artificially set concepts. In fact, during the training process, the computer model does not have the concept of "whether to generate a highway traffic safety warning". It only has two classification labels and the probability of the output feature under these two classification labels, that is, the sum of p1 and p2 is one. Therefore, the classification result of whether to generate a highway traffic safety warning is actually converted into a binary classification class probability distribution that conforms to natural laws through the classification label. In essence, the physical meaning of the natural probability distribution of the label is used, rather than the linguistic text meaning of "whether to generate a highway traffic safety warning".

[0043] It should be understood that the role of the classifier is to use the given categories and known training data to learn classification rules and classifiers, and then classify (or predict) unknown data. Logistic regression, SVM, etc. are often used to solve binary classification problems. For multi-class classification problems, logistic regression or SVM can also be used, but multiple binary classifications are required to form a multi-classification, but this is prone to errors and inefficient. Commonly used multi-classification methods include the Softmax classification function.

[0044] Further, in the technical solution of the present application, the traffic safety warning method for highways further includes a training step: training the road state feature extractor using the spatial attention mechanism, the road state association feature extractor between sections using the channel attention mechanism, and the classifier. It should be understood that the role of the training step in the highway traffic safety warning method is to learn and optimize the parameters of the model by using the labeled data, so that the model can better understand and predict the road state, thereby achieving accurate traffic safety warning. Specifically, the training step includes training the road state feature extractor, the road state association feature extractor between sections, and the classifier, wherein the spatial attention mechanism and the channel attention mechanism are used. 1. Training of the road state feature extractor: The road state feature extractor is a model for extracting road state features from real-time road images of highways. By training the road state feature extractor, the model can learn to extract useful features about road conditions from the image. These features can include information such as vehicles, pedestrians, road signs, road markings, etc. During the training process, the labeled road state data is used as a training sample, and the parameters of the road state feature extractor are optimized by minimizing the difference between the predicted results and the true labels. 2. Training of the road state correlation feature extractor between sections: The road state correlation feature extractor between sections is a model used to extract the inter-section correlation features from the road state features. It can help the model capture the relationship between different sections, such as changes in traffic flow, vehicle flow, etc. By training the road state correlation feature extractor between sections, the model can learn to extract useful features about the inter-section correlation from the road state features. During the training process, the labeled road state data and the inter-section correlation information are used as training samples, and the parameters of the road state correlation feature extractor between sections are optimized by minimizing the difference between the predicted results and the true labels. 3. Training of the classifier: The classifier is a model used to map the extracted features to specific traffic safety warning categories. By training the classifier, the model can learn to map the road state features and the inter-section correlation features to the traffic safety warning categories. During the training process, the labeled road state data and the corresponding traffic safety warning categories are used as training samples, and the parameters of the classifier are optimized by minimizing the difference between the predicted results and the true labels. Through the training steps, the model can gradually learn to extract useful features about road conditions and the relationship between road segments from road images, and use these features for accurate traffic safety warnings. Training can improve the performance and generalization ability of the model, enabling it to better adapt to different road conditions and traffic scenarios.

[0045] In one example, the training step includes: obtaining training data, the training data including a training highway real-time road image of a road section to be analyzed collected by a drone, and whether to generate a true value of a highway traffic safety warning; segmenting the training highway real-time road image along the extension direction of the road section to be analyzed to obtain a sequence of training highway real-time road image blocks; passing the sequence of training highway real-time road image blocks through the road state feature extractor using the spatial attention mechanism to obtain a sequence of training highway real-time road state feature matrices; aggregating the sequence of training highway real-time road state feature matrices along the channel dimension into training highway real-time road state feature matrices. After the state input tensor is obtained, the road state association feature extractor between road sections using the channel attention mechanism is passed to obtain a training highway road state association feature map between road sections; the training highway road state association feature map between road sections is passed through a classifier to obtain a classification loss function value; and the classification loss function value is used to train the road state feature extractor using the spatial attention mechanism, the road state association feature extractor between road sections using the channel attention mechanism and the classifier, wherein in each round of iteration of the training, the training highway road state association feature vector obtained after expanding the training highway road state association feature map between road sections is subjected to iterative optimization of the weight matrix in the weight space.

[0046] In the technical solution of the present application, after the sequence of training highway real-time road image blocks is processed by a road state feature extractor using a spatial attention mechanism, each training highway real-time road state feature matrix in the sequence of training highway real-time road state feature matrices expresses the image semantic features of the spatial distribution enhancement of the corresponding training highway real-time road image blocks, and after the sequence of training highway real-time road state feature matrices is aggregated into a training highway real-time road state input tensor along the channel dimension, the road state association feature map between road sections obtained by using a road state association feature extractor between road sections using a channel attention mechanism further enhances the channel distribution enhancement of the image semantic features in the global image semantic space on the basis of the spatial distribution enhancement of the image semantic features in the local image semantic space, so that the training highway inter-section road state association feature map has a dense fusion of the distribution dimensions corresponding to the spatial distribution dimension and the channel distribution dimension corresponding to the local image semantic space and the global image semantic space, and each eigenvalue also has a dense distribution representation of the image semantics enhanced by the spatial distribution and the channel distribution, resulting in that when the training highway inter-section road state association feature map is subjected to classification and regression training by a classifier, the training efficiency of the weight matrix of the classifier is reduced.

[0047] Based on this, the applicant of the present application, when training the road state association feature map between training highway sections for classification regression through a classifier, performs iterative optimization of the weight matrix in the weight space based on the road state association feature vector between training highway sections obtained after expanding the road state association feature map between training highway sections.

[0048] Accordingly, in a specific example, in each round of iteration of the training, the road state association feature vectors between the training highway sections obtained after the road state association feature graph between the training highway sections is expanded are subjected to iterative optimization of the weight matrix in the weight space, including: in each round of iteration of the training, the road state association feature vectors between the training highway sections obtained after the road state association feature graph between the training highway sections is expanded are subjected to iterative optimization of the weight matrix in the weight space according to the following optimization formula; wherein the optimization formula is: in, and are the weight matrices of the last and current iterations, respectively. In the first iteration, different initialization strategies are used to set and ,(For example, Set to the identity matrix and Set to the mean diagonal matrix of the feature vector to be classified), is the road state association feature vector between the training highway sections to be classified, and Represents the eigenvectors and The global mean of is the bias matrix, for example, initially set to the identity matrix, the vector is in column vector form, represents vector multiplication, represents matrix addition, It means point multiplication by position. represents the transpose operation, Represents the maximum value function Represents the optimized weight matrix.

[0049] That is, considering the road state association feature vector between the training highway sections to be classified When performing dense prediction tasks, it is necessary to associate the high-resolution representation of the weight matrix with the road state feature vector between the training highway sections to be classified. The image semantic feature distribution dimension is densely associated with the context for integration. Therefore, by maximizing the distribution boundary of the weight space in the iterative process, progressive integration is achieved based on the iterative association representation resource-awareness, thereby improving the training effect of the weight matrix and improving the overall training efficiency of the model.

[0050] In summary, based on the traffic safety warning method for highways according to the embodiment of the present application, it can automatically determine whether it is necessary to generate a highway traffic safety warning, and promptly inform the driver to adjust the driving route or inform the traffic management personnel to take corresponding measures.

[0051] Figure 3 FIG. 1 is a block diagram of a traffic safety warning system 100 for a highway according to an embodiment of the present application. Figure 3 As shown, according to an embodiment of the present application, a traffic safety warning system 100 for a highway includes: an image acquisition module 110, which is used to obtain a real-time highway road image of a road section to be analyzed collected by a drone; a data preprocessing module 120, which is used to perform data preprocessing on the real-time highway road image to obtain a sequence of real-time highway road image blocks; a local real-time road state feature extraction module 130, which is used to extract local real-time road state features of the sequence of real-time highway road image blocks to obtain a sequence of real-time highway road state feature matrices; an inter-section road state association feature extraction module 140, which is used to extract inter-section road state association features between the sequence of real-time highway road state feature matrices to obtain an inter-section road state association feature map; and a safety warning module 150, which is used to determine whether to generate a highway traffic safety warning based on the inter-section road state association feature map.

[0052] In a possible implementation, the data preprocessing module 120 is used to segment the real-time highway road image along the extension direction of the road section to be analyzed to obtain a sequence of real-time highway road image blocks.

[0053] In a possible implementation, the local real-time road state feature extraction module 130 is used to: extract the sequence of the highway real-time road image blocks through a road state feature extractor using a spatial attention mechanism to obtain a sequence of the highway real-time road state feature matrices.

[0054] Here, those skilled in the art will appreciate that the specific functions and operations of the various units and modules in the above-mentioned traffic safety warning system 100 for highways have been described in detail above. Figure 1 to Figure 2The invention has been described in detail in the description of the traffic safety warning method for highways, and therefore, its repeated description will be omitted.

[0055] As described above, the traffic safety warning system 100 for highways according to the embodiment of the present application can be implemented in various wireless terminals, such as a server with a traffic safety warning algorithm for highways. In a possible implementation, the traffic safety warning system 100 for highways according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the traffic safety warning system 100 for highways can be a software module in the operating system of the wireless terminal, or can be an application developed for the wireless terminal; of course, the traffic safety warning system 100 for highways can also be one of the many hardware modules of the wireless terminal.

[0056] Alternatively, in another example, the traffic safety warning system 100 for highways and the wireless terminal may also be separate devices, and the traffic safety warning system 100 for highways may be connected to the wireless terminal via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0057] Figure 4 FIG. 1 shows an application scenario diagram of a traffic safety warning method for a highway according to an embodiment of the present application. Figure 4 As shown, in this application scenario, first, a real-time highway road image of the road section to be analyzed collected by a drone is obtained (for example, Figure 4 D), then, the real-time highway road image is input to a server (for example, Figure 4 In S) as shown in , the server can use the traffic safety warning algorithm for the highway to process the real-time road image of the highway to obtain a classification result indicating whether to generate a highway traffic safety warning.

[0058] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can also be executed in the opposite order sometimes, depending on the function involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a dedicated hardware-based system that performs the function or action of the specification, or can be realized by a combination of special-purpose hardware and computer instructions.

[0059] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A traffic safety early warning method for expressways, characterized in that: include: Acquire real-time highway road images of the road section to be analyzed collected by a drone; Performing data preprocessing on the real-time highway road image to obtain a sequence of real-time highway road image blocks; Extracting local real-time road state features of the sequence of real-time highway road image blocks to obtain a sequence of real-time highway road state feature matrices; Extracting inter-section road state correlation features between the sequences of the highway real-time road state feature matrix to obtain an inter-section road state correlation feature graph of the highway; as well as Determining whether to generate a highway traffic safety warning based on the highway road status correlation feature map between sections; The step of extracting local real-time road state features of the sequence of highway real-time road image blocks to obtain a sequence of highway real-time road state feature matrices includes: The sequence of the highway real-time road image blocks is extracted by a road state feature extractor using a spatial attention mechanism to obtain a sequence of the highway real-time road state feature matrices; The step of extracting the inter-section road state correlation features between the sequences of the highway real-time road state feature matrix to obtain the inter-section road state correlation feature graph of the highway includes: After aggregating the sequence of the highway real-time road state feature matrix along the channel dimension into a highway real-time road state input tensor, a road state correlation feature map between highway sections is obtained by using a road state correlation feature extractor between road sections using a channel attention mechanism; Wherein, determining whether to generate a highway traffic safety warning based on the highway road state correlation feature map between sections includes: Passing the highway road state correlation feature map between sections through a classifier to obtain a classification result, wherein the classification result is used to indicate whether to generate a highway traffic safety warning; The method further includes a training step of: training the road state feature extractor using the spatial attention mechanism, the road state association feature extractor between road sections using the channel attention mechanism, and the classifier; Wherein, the training step includes: Acquiring training data, the training data including real-time road images of a training highway of a road section to be analyzed collected by a drone, and a true value of whether to generate a highway traffic safety warning; Segmenting the training expressway real-time road image along the extension direction of the road section to be analyzed to obtain a sequence of training expressway real-time road image blocks; Passing the sequence of the training highway real-time road image blocks through the road state feature extractor using the spatial attention mechanism to obtain a sequence of training highway real-time road state feature matrices; Aggregating the sequence of the training highway real-time road state feature matrix along the channel dimension into a training highway real-time road state input tensor, and then passing it through the inter-section road state association feature extractor using the channel attention mechanism to obtain a training highway inter-section road state association feature map; Passing the road state association feature map between the training highway sections through a classifier to obtain a classification loss function value; and The road state feature extractor using the spatial attention mechanism, the road state association feature extractor between road sections using the channel attention mechanism, and the classifier are trained using the classification loss function value, wherein in each iteration of the training, the training highway section road state association feature vector obtained by expanding the training highway section road state association feature graph is subjected to iterative optimization of the weight matrix in the weight space; Wherein, in each round of iteration of the training, the road state association feature vectors between the training highway sections obtained after the road state association feature graph between the training highway sections is expanded are subjected to iterative optimization of the weight matrix in the weight space, including: in each round of iteration of the training, the road state association feature vectors between the training highway sections obtained after the road state association feature graph between the training highway sections is expanded are subjected to iterative optimization of the weight matrix in the weight space according to the following optimization formula; wherein, the optimization formula is: Among them, M1 and M2 are the weight matrices of the previous and current iterations, respectively, and V c is the road state association feature vector between the training highway sections to be classified, and Represent the global mean of the eigenvectors V1 and V2, and M b is the bias matrix, represents vector multiplication, represents matrix addition, ⊙ represents positional multiplication, (·) T represents the transpose operation, max(·) represents the maximum function, and M2' represents the optimized weight matrix.

2. The traffic safety early warning method for highways according to claim 1, characterized in that: The real-time highway road image is subjected to data preprocessing to obtain a sequence of real-time highway road image blocks, including: The real-time highway road image is segmented along the extension direction of the road section to be analyzed to obtain a sequence of real-time highway road image blocks.

3. The traffic safety early warning method for highways according to claim 2, characterized in that: The sequence of the highway real-time road image blocks is obtained by using a road state feature extractor using a spatial attention mechanism to obtain a sequence of the highway real-time road state feature matrix, including: Each layer of the road state feature extractor using the spatial attention mechanism performs the following operations on the input data during the forward pass of the layer: Perform convolution on the input data to generate a sequence of convolution feature maps; Performing pooling processing on the sequence of convolutional feature maps respectively to generate a sequence of pooled feature maps; Performing nonlinear activation on the sequence of pooled feature maps respectively to generate a sequence of activated feature maps; Respectively calculating the mean of each position of each activation feature map in the sequence of activation feature maps along the channel dimension to generate a sequence of spatial feature matrices; Respectively calculating the Softmax-like function value of each position of each spatial feature matrix in the sequence of the spatial feature matrix to obtain a sequence of spatial score matrices; and Respectively calculating the positional point multiplication of the sequence of the spatial feature matrix and the sequence of the spatial score matrix to obtain a plurality of feature matrices; Among them, the multiple feature matrices output by the last layer of the road state feature extractor using the spatial attention mechanism are a sequence of the real-time road state feature matrices of the highway.

4. A traffic safety warning system for highways, characterized in that: include: An image acquisition module, used to acquire a real-time highway road image of the road section to be analyzed collected by a drone; A data preprocessing module, used for performing data preprocessing on the real-time highway road image to obtain a sequence of real-time highway road image blocks; A local real-time road state feature extraction module, used to extract local real-time road state features of the sequence of highway real-time road image blocks to obtain a sequence of highway real-time road state feature matrices; A road state correlation feature extraction module between sections, used to extract the road state correlation features between sections between the sequences of the highway real-time road state feature matrix to obtain a highway road state correlation feature map between sections; as well as A safety warning module, used to determine whether to generate a highway traffic safety warning based on the road state association feature map between the highway sections; Wherein, the local real-time road state feature extraction module is used to: The sequence of the highway real-time road image blocks is extracted by a road state feature extractor using a spatial attention mechanism to obtain a sequence of the highway real-time road state feature matrices; The road state correlation feature extraction model between road sections includes: After aggregating the sequence of the highway real-time road state feature matrix along the channel dimension into a highway real-time road state input tensor, a road state correlation feature map between highway sections is obtained by using a road state correlation feature extractor between road sections using a channel attention mechanism; Wherein, the safety warning module includes: Passing the highway road state correlation feature map between sections through a classifier to obtain a classification result, wherein the classification result is used to indicate whether to generate a highway traffic safety warning; The method also includes: training a road state feature extractor using a spatial attention mechanism, and a road state association feature extractor and classifier between road sections using a channel attention mechanism; Among them, the road state feature extractor using the spatial attention mechanism, the road state association feature extractor and classifier between road sections using the channel attention mechanism are trained, including: Acquiring training data, the training data including real-time road images of a training highway of a road section to be analyzed collected by a drone, and a true value of whether to generate a highway traffic safety warning; Segmenting the training expressway real-time road image along the extension direction of the road section to be analyzed to obtain a sequence of training expressway real-time road image blocks; Passing the sequence of the training highway real-time road image blocks through the road state feature extractor using the spatial attention mechanism to obtain a sequence of training highway real-time road state feature matrices; Aggregating the sequence of the training highway real-time road state feature matrix along the channel dimension into a training highway real-time road state input tensor, and then passing it through the inter-section road state association feature extractor using the channel attention mechanism to obtain a training highway inter-section road state association feature map; Passing the road state association feature map between the training highway sections through a classifier to obtain a classification loss function value; and The road state feature extractor using the spatial attention mechanism, the road state association feature extractor between road sections using the channel attention mechanism, and the classifier are trained using the classification loss function value, wherein in each iteration of the training, the training highway section road state association feature vector obtained by expanding the training highway section road state association feature graph is subjected to iterative optimization of the weight matrix in the weight space; Wherein, in each round of iteration of the training, the road state association feature vectors between the training highway sections obtained after the road state association feature graph between the training highway sections is expanded are subjected to iterative optimization of the weight matrix in the weight space, including: in each round of iteration of the training, the road state association feature vectors between the training highway sections obtained after the road state association feature graph between the training highway sections is expanded are subjected to iterative optimization of the weight matrix in the weight space according to the following optimization formula; wherein, the optimization formula is: Among them, M1 and M2 are the weight matrices of the previous and current iterations, respectively, and V c is the road state association feature vector between the training highway sections to be classified, and Represent the global mean of the eigenvectors V1 and V2, and M b is the bias matrix, represents vector multiplication, represents matrix addition, ⊙ represents positional multiplication, (·) T represents the transpose operation, max(·) represents the maximum function, and M2' represents the optimized weight matrix.

5. The traffic safety warning system for highways according to claim 4, characterized in that: The data preprocessing module is used to: The real-time highway road image is segmented along the extension direction of the road section to be analyzed to obtain a sequence of real-time highway road image blocks.

Images