Automatic traffic accident monitoring and alarming system and method

Abstract

The invention relates to the field of monitoring and alarming, and particularly discloses an automatic monitoring and alarming system and method for a traffic accident, and the method comprises the steps: firstly obtaining highway real-time video stream data collected by a front-facing camera, a vehicle obstacle distance echo signal collected by an ultrasonic sensor, and a wireless signal of a nearby vehicle collected by a vehicle-mounted probe; and then feature extraction and correlation analysis are carried out on the three by using a deep learning technology, and finally a classification result is obtained through a classifier to judge whether a traffic accident alarm is sent out, so that traffic accidents such as rear-end collision and collision are reduced through timely early warning, and the overall road traffic safety is improved.

Description

Technical Field

[0001] The present application relates to the field of monitoring and alarming, and more specifically, to a system and method for automatically monitoring and alarming traffic accidents. Background Art

[0002] Traffic accidents refer to unexpected events caused by the behavior of vehicles, pedestrians or other traffic participants during road traffic activities, usually accompanied by property damage, casualties or threats to public safety. Traffic accidents include collisions, rollovers, rear-end collisions and other situations. In the current road traffic management, the frequent occurrence of traffic accidents poses a serious threat to public safety, especially on highways, where vehicles travel at high speeds and the consequences of accidents are often more serious.

[0003] Existing accident warning methods mostly rely on manually setting warning signs. However, this method has problems such as insufficient response time and unsatisfactory warning effect in practical applications. Specifically, after an accident occurs, the rear vehicle may fail to obtain accident information in time, resulting in secondary accidents such as rear-end collisions. In addition, if the existing warning signs are placed too close to the accident scene, it will be difficult for the rear vehicle to make effective avoidance when driving at high speeds, further increasing the risk of accidents.

[0004] Therefore, a traffic accident automatic monitoring alarm system and method are desired. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a traffic accident automatic monitoring alarm system and method, which first obtains the real-time video stream data of the highway collected by the front camera, the vehicle obstacle distance echo signal collected by the ultrasonic sensor and the wireless signal of the nearby vehicle collected by the vehicle-mounted probe, and then uses the deep learning technology to perform feature extraction and correlation analysis on the three, and finally obtains the classification result through the classifier to determine whether to issue a traffic accident alarm, thereby reducing the occurrence of traffic accidents such as rear-end collisions and collisions through timely warnings, and improving the overall road traffic safety.

[0006] According to one aspect of the present application, there is provided a traffic accident automatic monitoring and alarm system, which comprises: The highway traffic data acquisition module is used to acquire the real-time video stream data of the highway collected by the front camera, the vehicle obstacle distance echo signal collected by the ultrasonic sensor, and the wireless signal of the nearby vehicles collected by the vehicle-mounted probe; A highway traffic data extraction module, used to extract highway road condition feature vectors and highway obstacle multimodal correlation feature vectors from the highway real-time video stream data collected by the front camera, the vehicle obstacle distance echo signal collected by the ultrasonic sensor, and the wireless signal of the nearby vehicle collected by the vehicle-mounted probe; The traffic accident alarm judgment module is used to judge whether to issue a traffic accident alarm based on the highway road condition feature vector and the high-speed obstacle multi-modal association feature vector.

[0007] According to another aspect of the present application, a method for automatic monitoring and alarming of traffic accidents is provided, which comprises: Acquire real-time video stream data of highways collected by the front camera, echo signals of vehicle obstacle distance collected by ultrasonic sensors, and wireless signals of nearby vehicles collected by vehicle-mounted probes; Extracting highway road condition feature vectors and highway obstacle multimodal correlation feature vectors from the highway real-time video stream data collected by the front camera, the vehicle obstacle distance echo signal collected by the ultrasonic sensor, and the wireless signal of nearby vehicles collected by the vehicle-mounted probe; Based on the highway road condition feature vector and the high-speed obstacle multi-modal association feature vector, it is determined whether to issue a traffic accident alarm.

[0008] Compared with the prior art, the present application provides an automatic traffic accident monitoring and alarm system and method, which first obtains real-time video stream data of the highway collected by the front camera, the vehicle obstacle distance echo signal collected by the ultrasonic sensor, and the wireless signal of the nearby vehicle collected by the on-board probe, and then uses deep learning technology to perform feature extraction and correlation analysis on the three, and finally obtains the classification result through the classifier to determine whether to issue a traffic accident alarm, thereby reducing the occurrence of traffic accidents such as rear-end collisions and collisions through timely warnings, and improving overall road traffic safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0010] Figure 1 It is a block diagram of an automatic traffic accident monitoring and alarm system according to an embodiment of the present application.

[0011] Figure 2It is a block diagram of a highway traffic data extraction module in the automatic traffic accident monitoring and alarm system according to an embodiment of the present application.

[0012] Figure 3 Schematic diagram of a block diagram of a nearby vehicle wireless signal feature extraction unit in an automatic traffic accident monitoring and alarm system according to an embodiment of the present application.

[0013] Figure 4 It is a block diagram of a traffic accident alarm determination module in the automatic traffic accident monitoring and alarm system according to an embodiment of the present application.

[0014] Figure 5 Flow chart of a method for automatic monitoring and alarming of traffic accidents according to an embodiment of the present application. DETAILED DESCRIPTION

[0015] Various exemplary embodiments, features and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0016] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0017] In addition, in order to better illustrate the present application, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present application can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present application.

[0018] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0019] Figure 1 Schematic diagram of the automatic monitoring and alarm system for traffic accidents in the embodiment of the present application. Figure 1As shown, the automatic traffic accident monitoring and alarm system 100 according to the embodiment of the present application includes: a highway traffic data acquisition module 110, which is used to acquire real-time video stream data of the highway collected by the front camera, the vehicle obstacle distance echo signal collected by the ultrasonic sensor, and the wireless signal of the nearby vehicle collected by the on-board probe; a highway traffic data extraction module 120, which is used to extract the highway road condition feature vector and the highway obstacle multimodal association feature vector from the highway real-time video stream data collected by the front camera, the vehicle obstacle distance echo signal collected by the ultrasonic sensor, and the wireless signal of the nearby vehicle collected by the on-board probe; a traffic accident alarm judgment module 130, which is used to judge whether to issue a traffic accident alarm based on the highway road condition feature vector and the highway obstacle multimodal association feature vector.

[0020] In the above-mentioned traffic accident automatic monitoring and alarm system 100, the highway traffic data acquisition module 110 is used to obtain the real-time video stream data of the highway collected by the front camera, the vehicle obstacle distance echo signal collected by the ultrasonic sensor, and the wireless signal of the nearby vehicle collected by the vehicle-mounted probe. It should be understood that a traffic accident is an unexpected event caused by the behavior of vehicles, pedestrians or other traffic participants in road traffic activities, which is usually accompanied by property loss, casualties or threats to public safety. Such events cover a variety of situations such as collisions, rollovers, and rear-end collisions. In the current road traffic management, the frequent occurrence of traffic accidents poses a major threat to public safety, especially on highways, where the consequences of accidents are often more serious due to the high speed of vehicles. At present, accident warnings mainly rely on manually set warning signs, but this method has the problems of insufficient response time and poor warning effect in practical applications. Specifically, after the accident occurs, the rear vehicle may cause secondary accidents such as rear-end collisions due to failure to obtain accident information in time. In addition, if the warning sign is placed too close to the accident scene, it is difficult for vehicles traveling at high speeds to make effective avoidance, thereby further increasing the risk of accidents.

[0021] In the technical solution of the present application, the front camera can capture road conditions, traffic flow and potential dangers in real time. The video stream data can be processed and analyzed by images to identify the positions of traffic signs, pedestrians and other vehicles. The ultrasonic sensor accurately measures the distance to surrounding obstacles by emitting sound waves and receiving echo signals, helping to determine whether there is a potential risk of collision. The on-board probe collects wireless signals from nearby vehicles, and through vehicle-to-vehicle communication (V2V), it can share vehicle status, speed and location information in real time, further enhancing the coordination and safety of traffic flow. Through the comprehensive application of the front camera, ultrasonic sensor and on-board probe, the traffic data of the highway can be obtained in real time, which not only improves traffic safety, but also lays the foundation for the realization of the intelligent transportation system. Therefore, in the technical solution of the present application, by obtaining the real-time video stream data of the highway collected by the front camera, the echo signal of the vehicle obstacle distance collected by the ultrasonic sensor and the wireless signal of the nearby vehicles collected by the on-board probe, and combining deep learning technology, it is determined whether a traffic accident alarm needs to be issued, so as to timely warn to reduce the occurrence of rear-end and collision traffic accidents and improve the overall road traffic safety.

[0022] In the above-mentioned traffic accident automatic monitoring and alarm system 100, the highway traffic data extraction module 120 is used to extract highway road condition feature vectors and highway obstacle multimodal correlation feature vectors from the highway real-time video stream data collected by the front camera, the vehicle obstacle distance echo signal collected by the ultrasonic sensor, and the wireless signal of the nearby vehicle collected by the vehicle-mounted probe. It should be understood that by analyzing and fusing the data from the front camera, the ultrasonic sensor and the vehicle-mounted probe, the highway road condition characteristics and the multimodal correlation characteristics of the obstacles can be effectively extracted, providing basic support for the real-time monitoring, early warning and decision-making of the intelligent transportation system.

[0023] Figure 2 FIG. 1 is a block diagram of a highway traffic data extraction module in an automatic traffic accident monitoring and alarm system according to an embodiment of the present application. Figure 2As shown, in a specific embodiment of the present application, the highway traffic data extraction module 120 includes: a highway video stream data feature extraction unit 121, which is used to perform feature extraction on the real-time highway video stream data collected by the front camera to obtain the highway road condition feature vector; an obstacle distance echo signal feature extraction unit 122, which is used to perform feature extraction on the vehicle obstacle distance echo signal collected by the ultrasonic sensor to obtain a vehicle obstacle distance echo feature vector; a nearby vehicle wireless signal feature extraction unit 123, which is used to perform feature extraction on the wireless signal of the nearby vehicle collected by the vehicle-mounted probe to obtain a nearby vehicle wireless signal text global feature vector; a highway obstacle multimodal feature association unit 124, which is used to associate the vehicle obstacle distance echo feature vector with the nearby vehicle wireless signal text global feature vector to obtain the highway obstacle multimodal association feature vector.

[0024] It should be understood that the first step in feature extraction of the real-time video stream data of the highway collected by the front camera is to preprocess these image frames, including denoising, contrast enhancement and image resizing. Next, computer vision technology is applied to identify and extract key elements in the image, such as vehicles, pedestrians, traffic lights and lane lines. Through algorithms such as edge detection, region segmentation and feature point extraction, the system can obtain the shape, position and motion state of the target object. The obtained highway road condition feature vector can not only be used for real-time monitoring, but also can be used for predictive analysis through machine learning models, such as identifying potential traffic congestion and collision risks. Through training on historical data, the model can be continuously optimized to improve the accuracy, thereby providing effective support in complex traffic environments.

[0025] Furthermore, ultrasonic sensors can measure the distance to surrounding obstacles in real time by emitting sound waves and receiving reflected echo signals. These signals contain rich environmental information and are crucial to intelligent transportation systems. Specifically, in the signal processing stage, the original echo signal needs to be preprocessed, including removing noise and interference, to ensure the accuracy of the measurement. This usually involves filtering techniques such as Kalman filtering or wavelet transform to extract effective signals. Next, the system will analyze the time delay of the echo signal and calculate the actual distance to the obstacle. Through the time-distance conversion formula, the distance data can be obtained and converted into digital features. In addition, feature extraction is not limited to simple distance calculation, but other important information can be extracted from the echo signal, such as the size, shape and motion state of the obstacle. For example, using the signal amplitude and time information, the outline of the obstacle can be identified and whether it is a stationary or moving object. This information can further construct a multidimensional feature vector containing the distance, type and speed of the obstacle. The extracted vehicle obstacle distance echo feature vector provides real-time environmental perception for the intelligent driving system, enabling the system to respond quickly in complex road conditions and avoid potential collision risks.

[0026] Furthermore, the vehicle-mounted probe can collect information about the speed, position, driving direction and status of other vehicles by exchanging real-time data with surrounding vehicles through wireless signals. In the process of feature extraction, the wireless signal needs to be decoded and processed first. This usually involves the analysis of signal strength, the parsing of signal format and the filtering of information. By analyzing the wireless signals sent by different vehicles, the system can obtain dynamic features such as vehicle identification, driving speed, acceleration and position. Then, this information is integrated into a feature vector to describe the relative position and motion state between vehicles. In order to construct a global feature vector, the time series characteristics and spatial distribution characteristics of the signal need to be considered in the extraction process. For example, the overall characteristics of traffic flow can be obtained by calculating the distance and relative speed between adjacent vehicles. In addition, the use of machine learning algorithms, such as principal component analysis (PCA) or cluster analysis, can further extract representative features, simplify data dimensions and highlight key information. Finally, the extracted global feature vector of the wireless signal text of nearby vehicles provides a comprehensive traffic situation awareness capability for the intelligent traffic management system.

[0027] In particular, by integrating data from different sources, more accurate and comprehensive traffic information can be obtained, helping the system to better identify and respond to potential risks. Among them, the vehicle obstacle distance echo feature vector provides spatial information about the surrounding obstacles, including distance, type, and dynamic state, while the global feature vector of the nearby vehicle wireless signal text describes the dynamic characteristics of the surrounding vehicles, such as speed, position, and driving direction. Correlating these two feature vectors can achieve multi-dimensional analysis of the environment. In specific operations, the association process can be achieved through data fusion technology. First, the system needs to ensure that the timestamps of the two feature vectors are consistent to ensure the timeliness of the data. Then, the features of the two are matched using mathematical models or machine learning algorithms. For example, the distance feature and the dynamic feature can be combined through weighted averaging or fusion algorithms to form a new feature vector that can simultaneously reflect the location of the obstacle and the motion state of nearby vehicles.

[0028] In a specific embodiment of the present application, the highway video stream data feature extraction unit 121 is used to: extract key frames from the real-time highway video stream data collected by the front camera to obtain a plurality of highway road condition feature maps; and pass the plurality of highway road condition feature maps through a highway road condition feature extractor based on a spatial attention module to obtain the highway road condition feature vector.

[0029] It should be understood that the process of extracting key frames is usually based on similarity analysis of video frames. The system compares consecutive frames and identifies frames with significant changes as key frames. This can be achieved by calculating the difference between images, such as using the mean square error (MSE) or structural similarity (SSIM) indicators. When the difference between a frame and the previous or next frame exceeds a preset threshold, the frame is selected as a key frame. In addition, combined with image processing techniques such as edge detection and feature point matching, frames containing important information can be identified more accurately.

[0030] Furthermore, multiple highway traffic feature maps are passed through a highway traffic feature extractor based on a spatial attention module, which can effectively extract and enhance important traffic information, thereby generating a highway traffic feature vector. The spatial attention mechanism dynamically weights different regions in the feature map, allowing the model to focus on parts with higher importance, thereby improving the effect of feature extraction. In actual operation, multiple feature maps represent traffic information in different time periods, which may include multiple features such as vehicle density, driving speed, and traffic signal status. Through the spatial attention module, different regions of these feature maps will be analyzed, and the model will learn which parts are more important to the overall traffic conditions. For example, in the case of high-density traffic, the model may give higher weights to congested areas, while paying relatively less attention to open sections. Specifically, multiple feature maps are first stacked to form a multi-channel input; then, the spatial attention module generates a weighted coefficient map by calculating the attention weight of each region. This coefficient map will be used to weight the original feature map so that the features of important areas are enhanced in the output. Finally, through feature fusion and pooling operations, the extractor will generate a unified highway traffic feature vector that brings together all key traffic information.

[0031] Specifically, the multiple highway road condition feature maps are passed through a highway road condition feature extractor based on a spatial attention module to obtain the highway road condition feature vector, including: using the convolution coding part of the highway road condition feature extractor based on the spatial attention module to perform deep convolution coding on the multiple highway road condition feature maps to obtain an initial convolution feature map; inputting the initial convolution feature map into the spatial attention part of the highway road condition feature extractor based on the spatial attention module to obtain a spatial attention map; passing the spatial attention map through a Softmax activation function to obtain a spatial attention feature map; calculating the point-by-point multiplication of the spatial attention feature map and the initial convolution feature map to obtain the spatial focus feature map; and pooling the spatial focus feature map into the highway road condition feature vector. More specifically, the initial convolutional feature map is input into the spatial attention part of the highway road condition feature extractor based on the spatial attention module to obtain a spatial attention map, including: performing average pooling and maximum pooling on the initial convolutional feature map along the channel dimension to obtain an average feature matrix and a maximum feature matrix; cascading and channel-adjusting the average feature matrix and the maximum feature matrix to obtain a channel feature matrix; and using the convolution layer of the spatial attention feature map to convolutionally encode the channel feature matrix to obtain the spatial attention map.

[0032] In a specific embodiment of the present application, the obstacle distance echo signal feature extraction unit 122 is used to: obtain a vehicle obstacle distance echo feature map by using a vehicle obstacle distance echo signal convolution neural network with a channel attention mechanism for the vehicle obstacle distance echo signal collected by the ultrasonic sensor; and perform maximum pooling on the vehicle obstacle distance echo feature map to obtain the vehicle obstacle distance echo feature vector.

[0033] It should be understood that the echo signals collected by ultrasonic sensors contain rich distance information, which can reflect the location and characteristics of obstacles in the surrounding environment. In order to convert these signals into useful feature maps, convolutional neural networks are used to process and analyze these signals. Through convolution operations, convolutional neural networks can extract local features in echo signals, such as the reflection intensity and distance changes of obstacles. However, since different channels may contain features of different importance, simple convolution processing may not be able to fully utilize this information. Therefore, the channel attention mechanism is introduced to dynamically adjust the weights of each channel so that the network can focus on the features that contribute most to obstacle detection. In the specific operation, the network first aggregates the features of each channel to generate channel descriptors. Subsequently, these descriptors are processed through the fully connected layer, and the output weight coefficients are used to adjust the importance of each channel feature. Finally, these weighted features are fused to form a vehicle obstacle distance echo feature map, which highlights key obstacle information. In this way, the generated feature map not only retains rich environmental information, but also can effectively distinguish different types of obstacles, providing more accurate input for subsequent intelligent decision-making, and ensuring the safety and reliability of vehicles in complex environments.

[0034] Specifically, the vehicle obstacle distance echo signal collected by the ultrasonic sensor is passed through a vehicle obstacle distance echo signal convolution neural network using a channel attention mechanism to obtain a vehicle obstacle distance echo feature map, including: each layer of the vehicle obstacle distance echo signal convolution neural network using the channel attention mechanism performs the following steps on the input data in the forward transfer of the layer: convolution processing on the input data based on a convolution kernel to generate a convolution feature map; pooling processing on the convolution feature map to generate a pooling feature map; activation processing on the pooling feature map to generate an activation feature map; calculating the quotient of the eigenvalue mean of the feature matrix corresponding to each channel in the activation feature map and the sum of the eigenvalue means of the feature matrices corresponding to all channels as the weighting coefficient of the feature matrix corresponding to each channel; and weighting the feature matrix of each channel with the weighting coefficient of each channel in the activation feature map to generate a channel attention feature map; wherein the output of the last layer of the vehicle obstacle distance echo signal convolution neural network using the channel attention mechanism is the vehicle obstacle distance echo feature map.

[0035] Furthermore, the maximum pooling operation can effectively retain the most representative signal in the feature map, while removing redundant information and enhancing the model's attention to important obstacle features. In the specific implementation, maximum pooling slides on the feature map by defining a fixed-size pooling window (such as 2x2 or 3x3). For each window position, the maximum value in the area is calculated and recorded as a new feature value in the pooled feature vector. This process can effectively reduce the size of the feature map, such as reducing a high-resolution feature map to a smaller feature vector. In this way, maximum pooling not only reduces the amount of calculation, but also retains important spatial information, allowing the model to focus more on the distance features of key obstacles.

[0036] Figure 3 FIG. 1 is a block diagram of a nearby vehicle wireless signal feature extraction unit in a traffic accident automatic monitoring and alarm system according to an embodiment of the present application. Figure 3 As shown, in a specific embodiment of the present application, the nearby vehicle wireless signal feature extraction unit 123 includes: a nearby vehicle wireless signal text semantic encoding subunit 1231, which is used to pass the wireless signal of the nearby vehicle collected by the vehicle-mounted probe through a text semantic encoder to obtain multiple nearby vehicle wireless signal text feature vectors; a two-dimensional arrangement subunit 1232, which is used to arrange the multiple nearby vehicle wireless signal text feature vectors in two dimensions into a nearby vehicle wireless signal text feature matrix; a nearby vehicle wireless signal convolution encoding subunit 1233, which is used to pass the nearby vehicle wireless signal text feature matrix through a convolutional neural network as a nearby vehicle wireless signal text feature extractor to obtain the nearby vehicle wireless signal text global feature vector.

[0037] It should be understood that wireless signals usually contain information about the vehicle's location, speed, and status, but these signals themselves are raw data and are not easy to directly perform machine learning and analysis. Therefore, using a text semantic encoder can convert these signals into more meaningful feature vectors. The text semantic encoder can process and understand the patterns in the signal and map the wireless signal into a high-dimensional space, making it easier to process later. In the specific implementation process, the vehicle-mounted probe first converts the collected wireless signal into a text format, which may contain signal strength, frequency, and delay information. Subsequently, these text data will be input into the text semantic encoder. The encoder uses a deep learning model, such as a recurrent neural network (RNN) or a transformer, to process the text data, identify the semantic relationship and context information therein, and generate multiple text feature vectors. These vectors not only retain the key information of the original signal, but also strengthen the correlation between the signals, and can effectively characterize the status of nearby vehicles. The multiple text feature vectors of nearby vehicle wireless signals generated in the end can provide rich data support for intelligent transportation systems.

[0038] In a specific embodiment of the present application, the text semantic encoding subunit 1231 of the nearby vehicle wireless signal is used to: pass the wireless signal of the nearby vehicle collected by the vehicle-mounted probe through the nearby vehicle wireless signal text semantic encoder including an embedding layer and a bidirectional long short-term memory model to obtain the multiple nearby vehicle wireless signal text feature vectors. It should be understood that the wireless signal itself is a time series data, which contains real-time information about nearby vehicles, such as position, speed and communication status. However, the original form of these signals is not easy to be used directly for subsequent analysis. Therefore, using the embedding layer of the text semantic encoder, the original signal can be converted into a dense vector representation. The embedding layer maps each signal feature to a high-dimensional space by learning, captures the relationship between signals, and makes the information more compact and expressive. Then, after being processed by the embedding layer, the signal feature vector is input into the bidirectional long short-term memory model. Bi-LSTM can take into account the forward and backward information of the sequence data at the same time, which is particularly important for understanding the changes in context in the time series. By learning the temporal dependency of the signal, Bi-LSTM can effectively capture the dynamic changes in the vehicle state and the mutual influence with other vehicles. Finally, after Bi-LSTM processing, the model will output text feature vectors of wireless signals of multiple nearby vehicles, which can fully express the characteristics of each vehicle and its temporal relationship. This process not only improves the ability of feature representation, but also enhances the system's perception of the surrounding environment.

[0039] Furthermore, the wireless signal text feature vector of each nearby vehicle represents the characteristic information of the vehicle, such as location, speed, communication status, etc. These vectors are arranged in two dimensions, that is, each feature vector is used as a row or column of the matrix to form a feature matrix. The number of rows of this matrix corresponds to the number of nearby vehicles, and the number of columns corresponds to the dimension of each feature vector, forming an overall data structure containing information of multiple vehicles. This organization method not only improves the readability of the data, but also facilitates matrix operations such as linear transformation, feature normalization, and batch processing.

[0040] Furthermore, the feature matrix of the text of the wireless signals of nearby vehicles is processed by a convolutional neural network to obtain the global feature vector of the text of the wireless signals of nearby vehicles, mainly to extract the potential deep features and patterns in the matrix, thereby improving the understanding of the vehicle status and behavior. Specifically, the rows of the feature matrix represent different vehicles, while the columns represent the characteristics of each vehicle. Before being input into the convolutional neural network, the feature matrix is ​​first preprocessed to adapt to the requirements of the network structure. Then, the convolutional neural network processes the feature matrix through a series of convolutional layers and pooling layers. In the convolutional layer, the network convolves the matrix with different convolution kernels to extract local features, which reflect the relationship between vehicles and the complexity of their status. Next, the pooling layer reduces the dimension of the feature map to reduce the computational complexity while retaining important information. In this process, the convolutional neural network can learn various important spatial patterns, thereby enhancing the ability to express features. Finally, after multiple layers of processing, the network integrates all the extracted local features into a global feature vector, which comprehensively describes the overall characteristics and status of nearby vehicles, providing efficient and rich information for subsequent decision-making. Specifically, the nearby vehicle wireless signal text feature matrix is ​​passed through a convolutional neural network serving as a nearby vehicle wireless signal text feature extractor to obtain the nearby vehicle wireless signal text global feature vector, including: using each layer of the convolutional neural network serving as a nearby vehicle wireless signal text feature extractor to perform convolution processing, mean pooling processing based on the local feature matrix and nonlinear activation processing on the input data in the forward transfer of the layer to output the nearby vehicle wireless signal text global feature vector from the last layer of the convolutional neural network serving as a nearby vehicle wireless signal text feature extractor, wherein the input of the convolutional neural network serving as a nearby vehicle wireless signal text feature extractor is the nearby vehicle wireless signal text feature matrix.

[0041] In the above-mentioned traffic accident automatic monitoring and alarm system 100, the traffic accident alarm judgment module 130 is used to judge whether to issue a traffic accident alarm based on the highway road condition feature vector and the high-speed obstacle multimodal association feature vector. It should be understood that judging whether to issue a traffic accident alarm based on the highway road condition feature vector and the high-speed obstacle multimodal association feature vector, this data-driven method can not only monitor and predict traffic safety risks in real time, but also can promptly warn the driver or traffic management system when a potential accident occurs, so as to take necessary preventive measures to reduce the possibility of accidents.

[0042] Figure 4 FIG. 1 is a block diagram of a traffic accident alarm determination module in an automatic traffic accident monitoring and alarm system according to an embodiment of the present application. Figure 4As shown, in a specific embodiment of the present application, the traffic accident alarm judgment module 130 includes: a highway traffic feature fusion unit 131, which is used to perform topological correlation modulation on the highway road condition feature vector and the highway obstacle multimodal correlation feature vector based on the heterogeneous feature manifold to obtain a traffic accident alarm judgment classification feature vector; a traffic accident judgment classification unit 132, which is used to pass the traffic accident alarm judgment classification feature vector through a classifier to obtain a classification result, and the classification result is used to determine whether to issue a traffic accident alarm.

[0043] It should be understood that the highway road condition feature vector provides information on traffic flow, vehicle speed and road surface status, while the highway obstacle multimodal association feature vector contains detailed data on the type, location and movement of obstacles. The combination of the two can fully reflect the complexity of the traffic environment. By integrating these two feature vectors, the system can comprehensively consider traffic flow conditions and obstacle information, improve the sensitivity and accuracy of accident warning, and thus better ensure road safety.

[0044] In particular, in the technical solution of the present application, the highway road condition feature vector is the visual information extracted from the key frame extracted by the front camera, which mainly reflects the dynamic state of the road. These dynamic states have timeliness and spatial continuity, and are suitable for describing the overall traffic state of the highway. The high-speed obstacle multimodal association feature vector combines the text features of the distance echo signal from the ultrasonic sensor and the wireless signal of the nearby vehicle. The former mainly provides the physical location and distance information of the obstacle, and the latter reflects the existence of the surrounding vehicles and their possible behavior patterns. Such features are often related to the local information of time and space, and are more focused on detecting potential security threats. Since the highway road condition feature vector and the high-speed obstacle multimodal association feature vector have different feature spaces and manifold structures, they may be incompatible during the feature fusion process. The incompatibility of the feature manifold means that in the multidimensional space, they may be in different distribution states, which will result in the inability to find a common feature representation when performing feature fusion, so that the fused traffic accident alarm judgment classification feature vector lacks smoothness. The smoothness of the feature manifold refers to the connectivity and smoothness between points in the feature space. If the feature manifold is not smooth, it may cause uncertainty in the model when making traffic accident alarm judgments, thereby affecting the certainty of the feature expression, and ultimately causing the accuracy of the alarm to decrease. Therefore, in the technical solution of the present application, the highway road condition feature vector and the high-speed obstacle multimodal correlation feature vector are modulated by topological association based on heterogeneous feature manifolds to obtain the traffic accident alarm judgment classification feature vector.

[0045] Among them, the highway road condition feature vector and the highway obstacle multimodal association feature vector are subjected to topological association modulation based on heterogeneous feature manifold to obtain the traffic accident alarm judgment classification feature vector, including: constructing a road condition distance topological matrix between the highway road condition feature vector and the highway obstacle multimodal association feature vector; constructing a road condition feature value granularity association matrix between the highway road condition feature vector and the highway obstacle multimodal association feature vector; performing topological association modulation on the road condition feature value granularity association matrix based on the road condition distance topological matrix to obtain a road condition topological modulation association matrix; using the road condition topological modulation association matrix as a perspective co-projection space, projecting the highway road condition feature vector and the highway obstacle multimodal association feature vector to the perspective co-projection space to obtain a perspective modulated highway road condition feature vector and a perspective modulated high-speed obstacle multimodal association feature vector; fusing the perspective modulated highway road condition feature vector and the perspective modulated high-speed obstacle multimodal association feature vector to obtain the traffic accident alarm judgment classification feature vector.

[0046] The fusion step is specifically expressed as: ; is the road condition distance topological matrix between the highway road condition feature vector and the high-speed obstacle multimodal association feature vector, that is, ,and and are column vectors.

[0047] ;in, represents the highway traffic feature vector, represents the multi-modal association feature vector of the high-speed obstacle, represents the road distance topology matrix, represents the transpose of a vector, represents matrix multiplication, represents the granularity correlation matrix of road condition feature values, represents the traffic topology modulation correlation matrix, represents the view-modulated highway road condition feature vector, represents the multi-modal correlation feature vector of high-speed obstacles modulated by viewing angle, and represents the weighted hyperparameter, Represents the traffic accident alarm judgment classification feature vector.

[0048] In the technical solution of the present application, the highway road condition feature vector and the highway obstacle multimodal association feature vector have relatively significant feature heterogeneity, which results in that when performing feature fusion of the highway road condition feature vector and the highway obstacle multimodal association feature vector, the feature manifold and feature space mismatch caused by the heterogeneity will cause the traffic accident alarm judgment classification feature vector to fail to have feature manifold smoothness, thereby affecting the certainty of its feature expression.

[0049] Based on this, in the technical solution of the present application, the highway road condition feature vector and the highway obstacle multimodal correlation feature vector are modulated by topological correlation based on heterogeneous feature manifolds, which first constructs the road condition distance topological matrix between the highway road condition feature vector and the highway obstacle multimodal correlation feature vector. This step utilizes the idea of ​​graph theory, takes the eigenvalues ​​of each position in the highway road condition feature vector and the highway obstacle multimodal correlation feature vector as nodes, and uses the distance metric function to determine the low-dimensional embedding expression of the edges between nodes, so as to construct a feature expression that can quantify the topological correlation relationship between the highway road condition feature vector and the highway obstacle multimodal correlation feature vector. This step is crucial to understanding the spatial distribution of data points and their mutual connections, and also provides basic structural information for subsequent steps.

[0050] Next, a road condition feature value granularity association matrix between the highway road condition feature vector and the highway obstacle multimodal association feature vector is constructed. This step focuses on exploring the correlation between the internal attributes of the feature vectors. In a specific example, the product between the transposed vectors of the highway road condition feature vector and the highway obstacle multimodal association feature vector is calculated to obtain the road condition feature value granularity association matrix. Furthermore, the road condition feature value granularity association matrix is ​​topologically modulated based on the road condition distance topological matrix to obtain a road condition topological modulation association matrix. That is, the feature value granularity association information of the highway road condition feature vector and the highway obstacle multimodal association feature vector is driven to wander on the distance topological map to construct a low-dimensional modulated perspective co-projection space domain for mapping the highway road condition feature vector and the highway obstacle multimodal association feature vector to a continuous high-dimensional regression space attribute.

[0051] Next, the road condition topology modulation association matrix is ​​used as the perspective co-projection space, and the highway road condition feature vector and the high-speed obstacle multimodal association feature vector are projected into the perspective co-projection space to obtain the perspective modulated highway road condition feature vector and the perspective modulated high-speed obstacle multimodal association feature vector. That is, the key to this technical solution is to find an implicit third space with modulation capability, in which as much information of the original high-dimensional data as possible is retained, so that the converted new feature vector not only reduces the dimension, but also maintains the relative position relationship between the original features.

[0052] Finally, the perspective modulated highway road condition feature vector and the perspective modulated highway obstacle multimodal association feature vector are fused to obtain the traffic accident alarm judgment classification feature vector. Fusion can be achieved in a variety of forms, ranging from simple arithmetic average to complex ensemble learning algorithms, depending on the application scenario and the goals pursued. The final traffic accident alarm judgment classification feature vector combines the main features of the two sets of input data and provides a more comprehensive data representation, which is not only conducive to improving the performance of machine learning models, but also promotes the ability of cross-modal or multi-source data analysis, thereby significantly enhancing the understanding and utilization efficiency of complex data structures.

[0053] Furthermore, the traffic accident alarm judgment classification feature vector is processed by a classifier to obtain a classification result, the main purpose of which is to automatically determine whether there is a potential accident risk in the current traffic situation. Among them, the traffic accident alarm judgment classification feature vector integrates the highway road conditions and obstacle information, reflecting the complexity of the current environment. The traffic accident alarm judgment classification feature vector is input into the classifier, and the classifier analyzes the features through the trained model parameters. The model evaluates the importance of each feature in the traffic accident alarm judgment classification feature vector based on the rules learned during training, and outputs a classification result, such as "safe" or "dangerous". In this process, the classifier can identify the potential relationship between the features and make judgments based on historical data and real-time information. In this way, the automated classifier not only improves the response speed, but also reduces the error of human judgment, thereby enhancing the intelligent level of road safety management.

[0054] In summary, the embodiment of the present application first obtains the real-time video stream data of the highway collected by the front camera, the vehicle obstacle distance echo signal collected by the ultrasonic sensor, and the wireless signal of the nearby vehicle collected by the vehicle-mounted probe, and then uses deep learning technology to perform feature extraction and correlation analysis on the three. Finally, the classification result is obtained through the classifier to determine whether to issue a traffic accident alarm, thereby reducing the occurrence of traffic accidents such as rear-end collisions and collisions through timely warnings, and improving overall road traffic safety.

[0055] As described above, the automatic traffic accident monitoring and alarm system 100 according to the embodiment of the present application can be implemented in various terminal devices. In one example, the automatic traffic accident monitoring and alarm system 100 can be integrated into the terminal device as a software module and / or a hardware module. For example, the automatic traffic accident monitoring and alarm system 100 can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the automatic traffic accident monitoring and alarm system 100 can also be one of the many hardware modules of the terminal device.

[0056] Alternatively, in another example, the automatic traffic accident monitoring alarm system 100 and the terminal device may also be separate devices, and the automatic traffic accident monitoring alarm system 100 may be connected to the terminal device via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0057] Figure 5 FIG. 1 is a flow chart of a method for automatically monitoring and alarming a traffic accident according to an embodiment of the present application. Figure 5 As shown, according to the automatic monitoring and alarm method for traffic accidents in an embodiment of the present application, it includes: S110, acquiring real-time video stream data of the highway collected by the front camera, the vehicle obstacle distance echo signal collected by the ultrasonic sensor, and the wireless signal of the nearby vehicle collected by the on-board probe; S120, extracting the highway road condition feature vector and the highway obstacle multimodal association feature vector from the real-time video stream data of the highway collected by the front camera, the vehicle obstacle distance echo signal collected by the ultrasonic sensor, and the wireless signal of the nearby vehicle collected by the on-board probe; S130, judging whether to issue a traffic accident alarm based on the highway road condition feature vector and the highway obstacle multimodal association feature vector.

[0058] Here, those skilled in the art will appreciate that the specific operations of each step in the above-mentioned traffic accident automatic monitoring and alarm method have been described in detail above. Figures 1 to 4 The invention has been described in detail in the description of the automatic traffic accident monitoring alarm system, and therefore, its repeated description will be omitted.

[0059] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0060] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0061] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0062] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0063] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0064] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the system claim can also be implemented by one unit or device through software or hardware. The second and other words are used to indicate names, but not to indicate any particular order.

[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit of the technical solution of the present invention.

Claims

1. A traffic accident automatic monitoring and alarm system, characterized in that: include: The highway traffic data acquisition module is used to acquire the real-time video stream data of the highway collected by the front camera, the vehicle obstacle distance echo signal collected by the ultrasonic sensor, and the wireless signal of the nearby vehicles collected by the vehicle-mounted probe; A highway traffic data extraction module, used to extract highway road condition feature vectors and highway obstacle multimodal correlation feature vectors from the highway real-time video stream data collected by the front camera, the vehicle obstacle distance echo signal collected by the ultrasonic sensor, and the wireless signal of the nearby vehicle collected by the vehicle-mounted probe; The traffic accident alarm judgment module is used to judge whether to issue a traffic accident alarm based on the highway road condition feature vector and the high-speed obstacle multi-modal association feature vector.

2. The automatic traffic accident monitoring and alarm system according to claim 1 is characterized in that: The highway traffic data extraction module comprises: A highway video stream data feature extraction unit, used for extracting features from the real-time highway video stream data collected by the front camera to obtain the highway road condition feature vector; An obstacle distance echo signal feature extraction unit, used for performing feature extraction on the vehicle obstacle distance echo signal collected by the ultrasonic sensor to obtain a vehicle obstacle distance echo feature vector; A nearby vehicle wireless signal feature extraction unit, used for extracting features from the wireless signals of the nearby vehicles collected by the vehicle-mounted probe to obtain a global feature vector of the nearby vehicle wireless signal text; The high-speed obstacle multimodal feature association unit is used to associate the vehicle obstacle distance echo feature vector with the nearby vehicle wireless signal text global feature vector to obtain the high-speed obstacle multimodal association feature vector.

3. The automatic traffic accident monitoring and alarm system according to claim 2 is characterized in that: The highway video stream data feature extraction unit is used to: Extracting key frames from the real-time video stream data of the highway collected by the front camera to obtain a plurality of highway road condition feature maps; The plurality of highway traffic condition feature maps are passed through a highway traffic condition feature extractor based on a spatial attention module to obtain the highway traffic condition feature vector.

4. The automatic traffic accident monitoring and alarm system according to claim 3 is characterized in that: The obstacle distance echo signal feature extraction unit is used to: The vehicle obstacle distance echo signal collected by the ultrasonic sensor is passed through a vehicle obstacle distance echo signal convolutional neural network using a channel attention mechanism to obtain a vehicle obstacle distance echo feature map; Maximum value pooling is performed on the vehicle obstacle distance echo feature map to obtain the vehicle obstacle distance echo feature vector.

5. The automatic traffic accident monitoring and alarm system according to claim 4 is characterized in that: The nearby vehicle wireless signal feature extraction unit comprises: A text semantic encoding subunit for nearby vehicle wireless signals, used for passing the wireless signals of nearby vehicles collected by the vehicle-mounted probe through a text semantic encoder to obtain a plurality of text feature vectors of nearby vehicle wireless signals; A two-dimensional arrangement subunit, used for two-dimensionally arranging the plurality of nearby vehicle wireless signal text feature vectors into a nearby vehicle wireless signal text feature matrix; The nearby vehicle wireless signal convolutional coding subunit is used to obtain the nearby vehicle wireless signal text global feature vector by passing the nearby vehicle wireless signal text feature matrix through a convolutional neural network serving as a nearby vehicle wireless signal text feature extractor.

6. The automatic traffic accident monitoring and alarm system according to claim 5 is characterized in that: The nearby vehicle wireless signal text semantic encoding subunit is used to: pass the wireless signals of nearby vehicles collected by the vehicle-mounted probe through a nearby vehicle wireless signal text semantic encoder including an embedding layer and a bidirectional long short-term memory model to obtain the multiple nearby vehicle wireless signal text feature vectors.

7. The automatic traffic accident monitoring and alarm system according to claim 6 is characterized in that: The traffic accident alarm determination module comprises: A highway traffic feature fusion unit is used to perform topological correlation modulation based on heterogeneous feature manifolds on highway road condition feature vectors and multi-modal correlation feature vectors of highway obstacles to obtain a traffic accident alarm judgment classification feature vector; The traffic accident judgment classification unit is used to pass the traffic accident alarm judgment classification feature vector through a classifier to obtain a classification result, and the classification result is used to determine whether to issue a traffic accident alarm.

8. The automatic traffic accident monitoring and alarm system according to claim 7 is characterized in that: The highway traffic feature fusion unit comprises: Constructing a road condition distance topological matrix between the highway road condition feature vector and the high-speed obstacle multi-modal association feature vector; Constructing a road condition feature value granularity association matrix between the highway road condition feature vector and the high-speed obstacle multimodal association feature vector; Performing topological correlation modulation on the road condition characteristic value granularity correlation matrix based on the road condition distance topological matrix to obtain a road condition topological modulation correlation matrix; Taking the road condition topology modulation association matrix as the perspective co-projection space, projecting the highway road condition feature vector and the high-speed obstacle multimodal association feature vector into the perspective co-projection space to obtain the perspective-modulated highway road condition feature vector and the perspective-modulated high-speed obstacle multimodal association feature vector; The view-modulated highway road condition feature vector and the view-modulated highway obstacle multi-modal association feature vector are fused to obtain the traffic accident alarm judgment classification feature vector.

9. A method for automatic monitoring and alarming of traffic accidents, characterized in that: include: Acquire real-time video stream data of highways collected by the front camera, echo signals of vehicle obstacle distance collected by ultrasonic sensors, and wireless signals of nearby vehicles collected by vehicle-mounted probes; Extracting highway road condition feature vectors and highway obstacle multimodal correlation feature vectors from the highway real-time video stream data collected by the front camera, the vehicle obstacle distance echo signal collected by the ultrasonic sensor, and the wireless signal of nearby vehicles collected by the vehicle-mounted probe; Based on the highway road condition feature vector and the high-speed obstacle multi-modal association feature vector, it is determined whether to issue a traffic accident alarm.

10. The method for automatic monitoring and alarming of traffic accidents according to claim 9, characterized in that: Extracting a highway road condition feature vector and a highway obstacle multimodal correlation feature vector from the highway real-time video stream data collected by the front camera, the vehicle obstacle distance echo signal collected by the ultrasonic sensor, and the wireless signal of the nearby vehicle collected by the vehicle-mounted probe, including: Performing feature extraction on the real-time video stream data of the highway collected by the front camera to obtain the highway road condition feature vector; Performing feature extraction on the vehicle obstacle distance echo signal collected by the ultrasonic sensor to obtain a vehicle obstacle distance echo feature vector; Performing feature extraction on the wireless signals of the nearby vehicles collected by the vehicle-mounted probe to obtain a global feature vector of the wireless signal text of the nearby vehicles; The vehicle obstacle distance echo feature vector and the nearby vehicle wireless signal text global feature vector are associated to obtain the high-speed obstacle multi-modal association feature vector.

Images