Traffic accident automatic monitoring and alarm system and method

Abstract

The present application relates to the field of monitoring and alarm, and specifically discloses a traffic accident automatic monitoring and alarm system and method. The system first obtains real-time video stream data of a highway collected by a front camera, the vehicle obstacle distance echo signal collected by an ultrasonic sensor, and the wireless signal of nearby vehicles collected by an on-board probe. Then, the system uses deep learning technology to perform feature extraction and correlation analysis on the three. Finally, a classification result is obtained through a classifier to determine whether to issue a traffic accident alarm. Thus, through timely warning, the occurrence of traffic accidents such as rear-end collisions and collisions is reduced, and overall road traffic safety is improved.

Description

Technical Field

[0001] The present application relates to the field of monitoring and alarming, and more specifically, to a system and method for automatically monitoring and alarming traffic accidents. Background Art

[0002] A traffic accident is an unexpected incident during road traffic caused by the actions of vehicles, pedestrians, or other road users, typically resulting in property damage, casualties, or threats to public safety. Traffic accidents include collisions, rollovers, and rear-end collisions. The frequent occurrence of traffic accidents in current road traffic management poses a serious threat to public safety, especially on highways, where vehicles travel at high speeds and the consequences of accidents are often more serious.

[0003] Existing accident warning methods often rely on manually placed warning signs. However, this approach suffers from insufficient response time and suboptimal warning effectiveness in practical applications. Specifically, after an accident occurs, following vehicles may fail to receive timely information about the accident, leading to secondary accidents such as rear-end collisions. Furthermore, if existing warning signs are placed too close to the accident scene, it will be difficult for following vehicles to effectively evade the vehicle at high speeds, further increasing the risk of accidents.

[0004] Therefore, a traffic accident automatic monitoring and alarm system and method are desired. Summary of the Invention

[0005] The present application is proposed to address the above-mentioned technical problems. The embodiments of the present application provide an automatic traffic accident monitoring and alarm system and method. The system first acquires real-time highway video stream data collected by a front-facing camera, vehicle obstacle distance echo signals collected by an ultrasonic sensor, and wireless signals of nearby vehicles collected by an on-board probe. Deep learning technology is then used to perform feature extraction and correlation analysis on the three. Finally, a classifier is used to obtain a classification result to determine whether to issue a traffic accident alarm. This system, through timely warnings, reduces the occurrence of traffic accidents such as rear-end collisions and collisions, and improves overall road traffic safety.

[0006] According to one aspect of the present application, there is provided a traffic accident automatic monitoring and alarm system, comprising:

[0007] Highway traffic data acquisition module, used to obtain real-time highway video stream data collected by the front camera, vehicle obstacle distance echo signals collected by the ultrasonic sensor, and wireless signals of nearby vehicles collected by the on-board probe;

[0008] A highway traffic data extraction module is used to extract highway road condition feature vectors and highway obstacle multimodal correlation feature vectors from the real-time highway video stream data collected by the front camera, the vehicle obstacle distance echo signal collected by the ultrasonic sensor, and the wireless signal of nearby vehicles collected by the on-board probe;

[0009] The traffic accident alarm judgment module is used to judge whether to issue a traffic accident alarm based on the highway road condition feature vector and the high-speed obstacle multimodal correlation feature vector.

[0010] According to another aspect of the present application, a method for automatically monitoring and alarming a traffic accident is provided, comprising:

[0011] Acquire real-time highway video stream data collected by the front camera, vehicle obstacle distance echo signals collected by the ultrasonic sensor, and wireless signals of nearby vehicles collected by the on-board probe;

[0012] Extracting a highway road condition feature vector and a highway obstacle multimodal correlation feature vector from the highway real-time video stream data collected by the front camera, the vehicle obstacle distance echo signal collected by the ultrasonic sensor, and the wireless signal of the nearby vehicle collected by the on-board probe;

[0013] Based on the highway road condition feature vector and the high-speed obstacle multimodal association feature vector, it is determined whether to issue a traffic accident alarm.

[0014] Compared with the existing technology, the present application provides an automatic traffic accident monitoring and alarm system and method, which first obtains real-time video stream data of the highway collected by the front camera, the vehicle obstacle distance echo signal collected by the ultrasonic sensor, and the wireless signal of the nearby vehicle collected by the on-board probe, and then uses deep learning technology to perform feature extraction and correlation analysis on the three. Finally, the classification result is obtained through the classifier to determine whether to issue a traffic accident alarm, thereby reducing the occurrence of traffic accidents such as rear-end collisions and collisions through timely warnings, and improving overall road traffic safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above and other purposes, features, and advantages of the present application will become more apparent through a more detailed description of the embodiments of the present application in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0016] Figure 1 Schematic diagram of a block diagram of an automatic traffic accident monitoring and alarm system according to an embodiment of the present application.

[0017] Figure 2 Schematic diagram of a block diagram of a highway traffic data extraction module in an automatic traffic accident monitoring and alarm system according to an embodiment of the present application.

[0018] Figure 3 Schematic diagram of a block diagram of a nearby vehicle wireless signal feature extraction unit in an automatic traffic accident monitoring and alarm system according to an embodiment of the present application.

[0019] Figure 4 Schematic diagram of a block diagram of a traffic accident alarm determination module in an automatic traffic accident monitoring and alarm system according to an embodiment of the present application.

[0020] Figure 5 Flowchart of a method for automatic monitoring and alarming of traffic accidents according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] Various exemplary embodiments, features, and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

[0022] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0023] In addition, numerous specific details are provided in the following detailed description to better illustrate the present application. Those skilled in the art will appreciate that the present application can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art are not described in detail in order to highlight the main purpose of the present application.

[0024] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. Throughout the description of this application, "plurality" means two or more, unless otherwise specifically defined.

[0025] Figure 1 This is a block diagram of the automatic traffic accident monitoring and alarm system according to the embodiment of the present application. Figure 1As shown, the automatic traffic accident monitoring and alarm system 100 according to an embodiment of the present application includes: a highway traffic data acquisition module 110, which is used to acquire real-time highway video stream data collected by a front camera, vehicle obstacle distance echo signals collected by an ultrasonic sensor, and wireless signals of nearby vehicles collected by an on-board probe; a highway traffic data extraction module 120, which is used to extract highway road condition feature vectors and highway obstacle multimodal correlation feature vectors from the highway real-time video stream data collected by the front camera, the vehicle obstacle distance echo signals collected by the ultrasonic sensor, and the wireless signals of nearby vehicles collected by the on-board probe; and a traffic accident alarm judgment module 130, which is used to judge whether to issue a traffic accident alarm based on the highway road condition feature vectors and the highway obstacle multimodal correlation feature vectors.

[0026] In the aforementioned automatic traffic accident monitoring and alarm system 100, the highway traffic data acquisition module 110 is used to acquire real-time highway video stream data captured by a front-facing camera, vehicle obstacle distance echo signals collected by ultrasonic sensors, and wireless signals from nearby vehicles collected by on-board probes. It should be understood that a traffic accident is an unexpected event in road traffic caused by the behavior of vehicles, pedestrians, or other traffic participants, typically resulting in property damage, casualties, or threats to public safety. Such incidents include collisions, rollovers, and rear-end collisions. In current road traffic management, the frequent occurrence of traffic accidents poses a significant threat to public safety, especially on highways, where the consequences are often more severe due to the high speeds of vehicles. Currently, accident warnings primarily rely on manually placed warning signs, but this approach suffers from insufficient response time and poor warning effectiveness in practical applications. Specifically, after an accident occurs, following vehicles may fail to receive accident information in a timely manner, leading to secondary accidents such as rear-end collisions. Furthermore, if warning signs are placed too close to the accident scene, it will be difficult for high-speed vehicles to effectively avoid the vehicle, further increasing the risk of accidents.

[0027] In the technical solution of this application, a front-facing camera can capture road conditions, traffic flow, and potential hazards in real time. Through image processing and analysis, the video stream data can identify the locations of traffic signs, pedestrians, and other vehicles. Ultrasonic sensors transmit sound waves and receive echo signals to accurately measure the distance to surrounding obstacles, helping to determine whether there is a potential collision risk. Onboard probes collect wireless signals from nearby vehicles. Through vehicle-to-vehicle (V2V) communication, they can share vehicle status, speed, and location information in real time, further enhancing traffic coordination and safety. The integrated application of front-facing cameras, ultrasonic sensors, and onboard probes enables real-time acquisition of highway traffic data, which not only improves traffic safety but also lays the foundation for the implementation of intelligent transportation systems. Therefore, in the technical solution of this application, by acquiring real-time highway video stream data collected by the front-facing camera, echo signals of vehicle obstacle distances collected by the ultrasonic sensor, and wireless signals from nearby vehicles collected by the onboard probe, combined with deep learning technology, it is possible to determine whether a traffic accident alert is necessary. This provides timely warnings, reduces the occurrence of rear-end collisions and collisions, and improves overall road traffic safety.

[0028] In the aforementioned automatic traffic accident monitoring and alarm system 100, the highway traffic data extraction module 120 is configured to extract highway road condition feature vectors and highway obstacle multimodal correlation feature vectors from the real-time highway video stream data captured by the front-facing camera, the vehicle obstacle range echo signals collected by the ultrasonic sensor, and the wireless signals from nearby vehicles collected by the on-board probe. It should be understood that by analyzing and fusing data from the front-facing camera, ultrasonic sensor, and on-board probe, highway road condition features and obstacle multimodal correlation features can be effectively extracted, providing fundamental support for real-time monitoring, early warning, and decision-making in intelligent transportation systems.

[0029] Figure 2 FIG. 1 is a block diagram of a highway traffic data extraction module in an automatic traffic accident monitoring and alarm system according to an embodiment of the present application. Figure 2As shown, in a specific embodiment of the present application, the highway traffic data extraction module 120 includes: a highway video stream data feature extraction unit 121, which is used to extract features from the real-time highway video stream data collected by the front camera to obtain the highway road condition feature vector; an obstacle distance echo signal feature extraction unit 122, which is used to extract features from the vehicle obstacle distance echo signal collected by the ultrasonic sensor to obtain a vehicle obstacle distance echo feature vector; a nearby vehicle wireless signal feature extraction unit 123, which is used to extract features from the wireless signals of nearby vehicles collected by the on-board probe to obtain a global feature vector of the nearby vehicle wireless signal text; and a highway obstacle multimodal feature association unit 124, which is used to associate the vehicle obstacle distance echo feature vector with the global feature vector of the nearby vehicle wireless signal text to obtain the highway obstacle multimodal association feature vector.

[0030] It should be understood that the first step in extracting features from the real-time highway video stream data captured by the front camera is to preprocess these image frames, including denoising, contrast enhancement, and image resizing. Next, computer vision technology is applied to identify and extract key elements in the image, such as vehicles, pedestrians, traffic lights, and lane lines. Through algorithms such as edge detection, region segmentation, and feature point extraction, the system can obtain the shape, position, and motion state of the target object. The resulting highway road condition feature vector can not only be used for real-time monitoring, but can also be used for predictive analysis through machine learning models, such as identifying potential traffic congestion and collision risks. By training on historical data, the model can be continuously optimized to improve accuracy, thereby providing effective support in complex traffic environments.

[0031] Furthermore, ultrasonic sensors can measure the distance to surrounding obstacles in real time by emitting sound waves and receiving reflected echo signals. These signals contain rich environmental information and are crucial for intelligent transportation systems. Specifically, during the signal processing stage, the raw echo signals require preprocessing, including removing noise and interference, to ensure measurement accuracy. This typically involves filtering techniques such as Kalman filtering or wavelet transforms to extract valid signals. Next, the system analyzes the time delay of the echo signals to calculate the actual distance to the obstacle. Using the time-to-distance conversion formula, distance data is obtained and converted into digital features. Furthermore, feature extraction is not limited to simple distance calculation; other important information can be extracted from the echo signals, such as the size, shape, and motion of the obstacle. For example, using signal amplitude and time information, the outline of the obstacle can be identified and whether it is stationary or moving. This information can be further used to construct a multidimensional feature vector containing the obstacle's distance, type, and speed. The extracted vehicle-obstacle distance echo feature vector provides the intelligent driving system with real-time environmental awareness, enabling the system to react quickly in complex road conditions and avoid potential collisions.

[0032] Furthermore, on-board probes exchange real-time data with surrounding vehicles via wireless signals, collecting information about other vehicles' speed, location, direction, and status. The feature extraction process first requires decoding and processing the wireless signals. This typically involves analyzing signal strength, parsing the signal format, and filtering the information. By analyzing the wireless signals transmitted by different vehicles, the system can obtain dynamic features such as vehicle identification, speed, acceleration, and position. This information is then integrated into a feature vector that describes the relative positions and motion states of the vehicles. To construct a global feature vector, the extraction process also considers the signal's time series and spatial distribution characteristics. For example, the distance and relative speed between adjacent vehicles can be calculated to capture the overall characteristics of traffic flow. Furthermore, machine learning algorithms such as principal component analysis (PCA) or cluster analysis can further extract representative features, simplify data dimensionality, and highlight key information. Ultimately, the extracted global feature vector of nearby vehicle wireless signal text provides comprehensive traffic situational awareness for intelligent traffic management systems.

[0033] In particular, integrating data from different sources can yield more accurate and comprehensive traffic information, helping the system better identify and address potential risks. The vehicle obstacle range echo feature vector provides spatial information about surrounding obstacles, including distance, type, and dynamic state, while the global feature vector of the nearby vehicle wireless signal text describes dynamic characteristics such as the speed, position, and direction of travel of surrounding vehicles. Correlating these two feature vectors enables multi-dimensional analysis of the environment. In practice, this correlation process can be achieved through data fusion technology. First, the system must ensure that the timestamps of the two feature vectors are consistent to ensure data timeliness. Then, using mathematical models or machine learning algorithms, the two features are matched. For example, distance features and dynamic features can be combined through weighted averaging or fusion algorithms to form a new feature vector that simultaneously reflects the location of obstacles and the motion state of nearby vehicles.

[0034] In a specific embodiment of the present application, the highway video stream data feature extraction unit 121 is used to: extract key frames from the real-time highway video stream data collected by the front camera to obtain multiple highway road condition feature maps; and pass the multiple highway road condition feature maps through a highway road condition feature extractor based on a spatial attention module to obtain the highway road condition feature vector.

[0035] As you can understand, the process of extracting keyframes is typically based on similarity analysis of video frames. The system compares consecutive frames and identifies frames with significant changes as keyframes. This is achieved by calculating the differences between images, such as using the mean squared error (MSE) or structural similarity (SSIM) metrics. When a frame's difference from the previous or next frame exceeds a preset threshold, it is selected as a keyframe. Furthermore, incorporating image processing techniques such as edge detection and feature point matching can more accurately identify frames containing important information.

[0036] Furthermore, passing multiple highway traffic feature maps through a highway traffic feature extractor based on a spatial attention module can effectively extract and enhance important traffic information, thereby generating a highway traffic feature vector. The spatial attention mechanism dynamically weights different regions within the feature map, enabling the model to focus on more important areas, thereby improving feature extraction. In practice, multiple feature maps represent traffic conditions at different time periods, which may include various features such as vehicle density, driving speed, and traffic signal status. The spatial attention module analyzes different regions of these feature maps, and the model learns which parts are more important to the overall traffic situation. For example, in high-density traffic conditions, the model may give higher weight to congested areas and less attention to open sections. Specifically, multiple feature maps are first stacked to form a multi-channel input. Next, the spatial attention module calculates the attention weights for each region to generate a weighted coefficient map. This coefficient map is used to weight the original feature map, enhancing the features of important regions in the output. Finally, through feature fusion and pooling operations, the extractor will generate a unified highway traffic feature vector that brings together all key traffic information.

[0037] Specifically, the multiple highway road condition feature maps are passed through a highway road condition feature extractor based on a spatial attention module to obtain the highway road condition feature vector, including: using the convolution coding part of the highway road condition feature extractor based on the spatial attention module to perform deep convolution coding on the multiple highway road condition feature maps to obtain an initial convolution feature map; inputting the initial convolution feature map into the spatial attention part of the highway road condition feature extractor based on the spatial attention module to obtain a spatial attention map; passing the spatial attention map through a Softmax activation function to obtain a spatial attention feature map; calculating the position point multiplication of the spatial attention feature map and the initial convolution feature map to obtain the spatial focus feature map; and pooling the spatial focus feature map into the highway road condition feature vector. More specifically, the initial convolution feature map is input into the spatial attention part of the highway road condition feature extractor based on the spatial attention module to obtain a spatial attention map, including: performing average pooling and maximum pooling along the channel dimension on the initial convolution feature map to obtain an average feature matrix and a maximum feature matrix; cascading and channel-adjusting the average feature matrix and the maximum feature matrix to obtain a channel feature matrix; and using the convolution layer of the spatial attention feature map to perform convolution encoding on the channel feature matrix to obtain a spatial attention map.

[0038] In a specific embodiment of the present application, the obstacle distance echo signal feature extraction unit 122 is used to: obtain a vehicle obstacle distance echo feature map by using a vehicle obstacle distance echo signal convolution neural network with a channel attention mechanism on the vehicle obstacle distance echo signal collected by the ultrasonic sensor; and perform maximum pooling on the vehicle obstacle distance echo feature map to obtain the vehicle obstacle distance echo feature vector.

[0039] As you can understand, the echo signals collected by ultrasonic sensors contain rich distance information, reflecting the location and characteristics of obstacles in the surrounding environment. To convert these signals into useful feature maps, convolutional neural networks are used to process and analyze them. Through convolution operations, convolutional neural networks can extract local features from the echo signals, such as the reflection intensity and distance variation of obstacles. However, since different channels may contain features of varying importance, simple convolution processing may not fully utilize this information. Therefore, a channel attention mechanism is introduced to dynamically adjust the weights of each channel, allowing the network to focus on the features that contribute most to obstacle detection. In practice, the network first aggregates the features of each channel to generate channel descriptors. These descriptors are then processed through a fully connected layer, and the output weight coefficients are used to adjust the importance of each channel feature. Finally, these weighted features are fused to form a feature map of the vehicle's obstacle distance echo, which further highlights key obstacle information. In this way, the generated feature map not only retains rich environmental information but also effectively distinguishes different types of obstacles, providing more accurate input for subsequent intelligent decision-making and ensuring the safety and reliability of the vehicle in complex environments.

[0040] Specifically, the vehicle obstacle distance echo signal collected by the ultrasonic sensor is passed through a vehicle obstacle distance echo signal convolutional neural network using a channel attention mechanism to obtain a vehicle obstacle distance echo feature map, including: each layer of the vehicle obstacle distance echo signal convolutional neural network using the channel attention mechanism performs the following steps on the input data in the forward pass of the layer: convolution processing on the input data based on the convolution kernel to generate a convolution feature map; pooling processing on the convolution feature map to generate a pooling feature map; activation processing on the pooling feature map to generate an activation feature map; calculating the quotient of the eigenvalue mean of the feature matrix corresponding to each channel in the activation feature map and the sum of the eigenvalue mean of the feature matrix corresponding to all channels as the weighting coefficient of the feature matrix corresponding to each channel; and weighting the feature matrix of each channel with the weighting coefficient of each channel in the activation feature map to generate a channel attention feature map; wherein the output of the last layer of the vehicle obstacle distance echo signal convolutional neural network using the channel attention mechanism is the vehicle obstacle distance echo feature map.

[0041] Furthermore, the maximum pooling operation can effectively retain the most representative signals in the feature map while removing redundant information, enhancing the model's focus on important obstacle features. In specific implementations, maximum pooling slides across the feature map by defining a fixed-size pooling window (e.g., 2x2 or 3x3). For each window position, the maximum value within the area is calculated and recorded as the new feature value in the pooled feature vector. This process can effectively reduce the size of the feature map, for example, by shrinking a high-resolution feature map into a smaller feature vector. In this way, maximum pooling not only reduces the amount of computation but also retains important spatial information, allowing the model to focus more on the distance features of key obstacles.

[0042] Figure 3 FIG. 1 is a block diagram of a nearby vehicle wireless signal feature extraction unit in an automatic traffic accident monitoring and alarm system according to an embodiment of the present application. Figure 3 As shown, in a specific embodiment of the present application, the nearby vehicle wireless signal feature extraction unit 123 includes: a nearby vehicle wireless signal text semantic encoding subunit 1231, which is used to pass the wireless signal of the nearby vehicle collected by the vehicle-mounted probe through a text semantic encoder to obtain a plurality of nearby vehicle wireless signal text feature vectors; a two-dimensional arrangement subunit 1232, which is used to two-dimensionally arrange the plurality of nearby vehicle wireless signal text feature vectors into a nearby vehicle wireless signal text feature matrix; a nearby vehicle wireless signal convolution encoding subunit 1233, which is used to pass the nearby vehicle wireless signal text feature matrix through a convolutional neural network as a nearby vehicle wireless signal text feature extractor to obtain the nearby vehicle wireless signal text global feature vector.

[0043] It's understandable that wireless signals typically contain information about vehicle location, speed, and status. However, these signals are raw data and not easily amenable to direct machine learning and analysis. Therefore, a text semantic encoder can be used to convert these signals into more meaningful feature vectors. This encoder processes and understands patterns within the signals, mapping them into a high-dimensional space for easier subsequent processing. In practice, on-board probes first convert collected wireless signals into text format, which may include information about signal strength, frequency, and latency. This text data is then input into the text semantic encoder. The encoder utilizes deep learning models, such as recurrent neural networks (RNNs) or transformers, to process the text data, identifying semantic relationships and contextual information, thereby generating multiple text feature vectors. These vectors not only retain key information from the original signals but also enhance correlations between signals, effectively characterizing the status of nearby vehicles. The resulting multiple text feature vectors of nearby vehicle wireless signals can provide rich data support for intelligent transportation systems.

[0044] In a specific embodiment of the present application, the nearby vehicle wireless signal text semantic encoding subunit 1231 is configured to pass the wireless signals of nearby vehicles collected by the vehicle-mounted probe through a nearby vehicle wireless signal text semantic encoder comprising an embedding layer and a bidirectional long short-term memory model to obtain the multiple nearby vehicle wireless signal text feature vectors. It should be understood that wireless signals themselves are time series data that contain real-time information about nearby vehicles, such as location, speed, and communication status. However, the raw form of these signals is not easily usable for subsequent analysis. Therefore, the embedding layer of the text semantic encoder can be used to convert the raw signals into a dense vector representation. The embedding layer learns to map each signal feature into a high-dimensional space, capturing the relationships between the signals and making the information more compact and expressive. After processing by the embedding layer, the signal feature vectors are then input into the bidirectional long short-term memory model. The Bi-LSTM can simultaneously consider both forward and backward information in the sequence data, which is particularly important for understanding contextual changes in time series. By learning the temporal dependencies of the signals, the Bi-LSTM can effectively capture the dynamic changes in the vehicle's state and its interactions with other vehicles. Finally, after Bi-LSTM processing, the model outputs text feature vectors of the wireless signals of multiple nearby vehicles. These vectors fully represent the characteristics of each vehicle and their temporal relationships. This process not only improves the ability to represent features, but also enhances the system's perception of the surrounding environment.

[0045] Furthermore, the textual feature vectors of the wireless signals from each nearby vehicle represent characteristic information about that vehicle, such as its location, speed, and communication status. These vectors are arranged two-dimensionally, with each feature vector serving as a row or column of a matrix, to form a feature matrix. The number of rows in this matrix corresponds to the number of nearby vehicles, while the number of columns corresponds to the dimensions of each feature vector, forming a single data structure containing information about multiple vehicles. This organization not only improves data readability but also facilitates matrix operations such as linear transformations, feature normalization, and batch processing.

[0046] Furthermore, the feature matrix of nearby vehicle wireless signal text is processed through a convolutional neural network to obtain a global feature vector of nearby vehicle wireless signal text. This is primarily intended to extract underlying deep features and patterns within the matrix, thereby improving understanding of vehicle status and behavior. Specifically, the rows of the feature matrix represent different vehicles, while the columns represent the characteristics of each vehicle. Before being input into the convolutional neural network, the feature matrix is ​​first preprocessed to adapt to the network's structural requirements. The convolutional neural network then processes the feature matrix through a series of convolutional and pooling layers. In the convolutional layers, the network convolves the matrix with different convolution kernels to extract local features that reflect the complexity of the relationships between vehicles and their status. Next, the pooling layer reduces the dimensionality of the feature map to reduce computational complexity while retaining important information. In this process, the convolutional neural network learns various important spatial patterns, thereby enhancing its ability to express features. Finally, after multiple layers of processing, the network integrates all extracted local features into a global feature vector that comprehensively describes the overall characteristics and status of nearby vehicles, providing efficient and rich information for subsequent decision-making. Specifically, the nearby vehicle wireless signal text feature matrix is ​​passed through a convolutional neural network serving as a nearby vehicle wireless signal text feature extractor to obtain the nearby vehicle wireless signal text global feature vector, including: using each layer of the convolutional neural network serving as the nearby vehicle wireless signal text feature extractor to perform convolution processing, mean pooling processing based on the local feature matrix, and nonlinear activation processing on the input data in the forward pass of the layer to output the nearby vehicle wireless signal text global feature vector from the last layer of the convolutional neural network serving as the nearby vehicle wireless signal text feature extractor, wherein the input of the convolutional neural network serving as the nearby vehicle wireless signal text feature extractor is the nearby vehicle wireless signal text feature matrix.

[0047] In the aforementioned automatic traffic accident monitoring and alarm system 100, the traffic accident alarm determination module 130 is configured to determine whether to issue a traffic accident alarm based on the highway road condition feature vector and the highway obstacle multimodal correlation feature vector. It should be understood that this data-driven approach, which determines whether to issue a traffic accident alarm based on the highway road condition feature vector and the highway obstacle multimodal correlation feature vector, not only enables real-time monitoring and prediction of traffic safety risks but also promptly alerts drivers or traffic management systems when potential accidents occur, allowing them to take necessary preventative measures and reduce the likelihood of accidents.

[0048] Figure 4 FIG. 1 is a block diagram of a traffic accident alarm determination module in an automatic traffic accident monitoring and alarm system according to an embodiment of the present application. Figure 4As shown, in a specific embodiment of the present application, the traffic accident alarm judgment module 130 includes: a highway traffic feature fusion unit 131, which is used to perform topological correlation modulation on the highway road condition feature vector and the highway obstacle multimodal correlation feature vector based on the heterogeneous feature manifold to obtain a traffic accident alarm judgment classification feature vector; a traffic accident judgment classification unit 132, which is used to pass the traffic accident alarm judgment classification feature vector through a classifier to obtain a classification result, and the classification result is used to determine whether to issue a traffic accident alarm.

[0049] As you can see, the highway road condition feature vector provides information on traffic flow, vehicle speed, and road surface conditions, while the highway obstacle multimodal association feature vector contains detailed data on obstacle type, location, and movement. The combination of the two fully reflects the complexity of the traffic environment. By integrating these two feature vectors, the system can comprehensively consider traffic flow conditions and obstacle information, improving the sensitivity and accuracy of accident warnings and thus better ensuring road safety.

[0050] In particular, in the technical solution of this application, the highway road condition feature vector is visual information extracted from keyframes captured by the front-facing camera, primarily reflecting the dynamic state of the road. These dynamic states are time-sensitive and spatially continuous, making them suitable for describing the overall traffic state of the highway. The highway obstacle multimodal correlation feature vector combines the range echo signal from the ultrasonic sensor and the textual features of the wireless signals from nearby vehicles. The former primarily provides information about the physical location and distance of the obstacle, while the latter reflects the presence of surrounding vehicles and their possible behavior patterns. Such features are often related to local information in time and space and are more focused on detecting potential safety threats. Because the highway road condition feature vector and the highway obstacle multimodal correlation feature vector are constructed with different feature spaces and manifold structures, they may become incompatible during the feature fusion process. Feature manifold incompatibility means that they may be in different distribution states in multidimensional space. This can make it impossible to find a common feature representation during feature fusion, resulting in a lack of smoothness in the fused traffic accident alarm classification feature vector. Feature manifold smoothness refers to the connectivity and smoothness between points in the feature space. If the feature manifold is not smooth, it may lead to uncertainty in the model's traffic accident warning judgment, thereby affecting the certainty of the feature expression and ultimately reducing the accuracy of the warning. Therefore, in the technical solution of this application, the highway road condition feature vector and the highway obstacle multimodal correlation feature vector are modulated by topological correlation based on heterogeneous feature manifolds to obtain the traffic accident warning judgment classification feature vector.

[0051] Among them, topological correlation modulation based on heterogeneous feature manifolds is performed on the highway road condition feature vector and the highway obstacle multimodal correlation feature vector to obtain the traffic accident alarm judgment classification feature vector, including: constructing a road condition distance topological matrix between the highway road condition feature vector and the highway obstacle multimodal correlation feature vector; constructing a road condition feature value granularity correlation matrix between the highway road condition feature vector and the highway obstacle multimodal correlation feature vector; performing topological correlation modulation on the road condition feature value granularity correlation matrix based on the road condition distance topological matrix to obtain a road condition topological modulation correlation matrix; using the road condition topological modulation correlation matrix as a perspective co-projection space, projecting the highway road condition feature vector and the highway obstacle multimodal correlation feature vector into the perspective co-projection space to obtain a perspective modulated highway road condition feature vector and a perspective modulated high-speed obstacle multimodal correlation feature vector; and fusing the perspective modulated highway road condition feature vector and the perspective modulated high-speed obstacle multimodal correlation feature vector to obtain the traffic accident alarm judgment classification feature vector.

[0052] The fusion step is specifically expressed as: ; is the road condition distance topology matrix between the highway road condition feature vector and the high-speed obstacle multimodal association feature vector, that is, ,and and are column vectors.

[0053] ; in, represents the highway traffic feature vector, represents the multimodal association feature vector of the high-speed obstacle, represents the road distance topology matrix, represents the transpose of a vector, represents matrix multiplication, represents the granularity correlation matrix of road condition eigenvalues, represents the traffic topology modulation correlation matrix, represents the view-modulated highway road condition feature vector, represents the multimodal correlation feature vector of the high-speed obstacle modulated by the viewing angle, and represents the weighted hyperparameter, Represents the traffic accident alarm judgment classification feature vector.

[0054] In the technical solution of the present application, the highway road condition feature vector and the highway obstacle multimodal association feature vector have relatively significant feature heterogeneity, which results in the feature manifold and feature space mismatch caused by the heterogeneity when performing feature fusion of the highway road condition feature vector and the highway obstacle multimodal association feature vector, resulting in the traffic accident alarm judgment classification feature vector being unable to have feature manifold smoothness, thereby affecting the certainty of its feature expression.

[0055] Based on this, in the technical solution of the present application, the highway road condition feature vector and the highway obstacle multimodal correlation feature vector are modulated by topological correlation based on heterogeneous feature manifolds, which first constructs the road condition distance topological matrix between the highway road condition feature vector and the highway obstacle multimodal correlation feature vector. This step utilizes the idea of ​​graph theory, takes the eigenvalues ​​of each position in the highway road condition feature vector and the highway obstacle multimodal correlation feature vector as nodes, and uses the distance metric function to determine the low-dimensional embedding expression of the edges between nodes, so as to construct a feature expression that can quantify the topological correlation relationship between the highway road condition feature vector and the highway obstacle multimodal correlation feature vector. This step is crucial for understanding the spatial distribution of data points and their mutual connections, and also provides basic structural information for subsequent steps.

[0056] Next, a road condition eigenvalue granularity correlation matrix is ​​constructed between the highway road condition feature vector and the highway obstacle multimodal correlation feature vector. This step focuses on exploring the correlation between the internal attributes of the feature vectors. In a specific example, the product between the transposed vectors of the highway road condition feature vector and the highway obstacle multimodal correlation feature vector is calculated to obtain the road condition eigenvalue granularity correlation matrix. Furthermore, the road condition eigenvalue granularity correlation matrix is ​​topologically correlated modulated based on the road condition distance topology matrix to obtain a road condition topology modulation correlation matrix. That is, the eigenvalue granularity correlation information of the highway road condition feature vector and the highway obstacle multimodal correlation feature vector is driven to wander on the distance topology map to construct a low-dimensional modulation perspective co-projection space domain for mapping the highway road condition feature vector and the highway obstacle multimodal correlation feature vector to a continuous high-dimensional regression space property.

[0057] Next, using the road condition topology modulation correlation matrix as the perspective co-projection space, the highway road condition feature vector and the highway obstacle multimodal correlation feature vector are projected into the perspective co-projection space to obtain the perspective-modulated highway road condition feature vector and the perspective-modulated highway obstacle multimodal correlation feature vector. In other words, the key to this technical solution lies in finding an implicit third space with modulation capabilities that retains as much information as possible from the original high-dimensional data. This conversion results in a new feature vector that not only reduces dimensionality but also maintains the relative positional relationships between the original features.

[0058] Finally, the perspective-modulated highway road condition feature vector and the perspective-modulated highway obstacle multimodal correlation feature vector are fused to obtain the traffic accident warning judgment classification feature vector. Fusion can be implemented in various forms, ranging from simple arithmetic averaging to complex ensemble learning algorithms, depending on the application scenario and the desired goal. The resulting traffic accident warning judgment classification feature vector integrates the key features of the two sets of input data, providing a more comprehensive data representation. This not only improves the performance of machine learning models but also facilitates cross-modal or multi-source data analysis, significantly enhancing the understanding and utilization efficiency of complex data structures.

[0059] The traffic accident alert judgment classification feature vector is then processed by a classifier to produce a classification result. This is primarily intended to automatically determine whether the current traffic situation presents a potential accident risk. The traffic accident alert judgment classification feature vector integrates information about highway road conditions and obstacles, reflecting the complexity of the current environment. The traffic accident alert judgment classification feature vector is input into the classifier, which analyzes the features using trained model parameters. Based on the patterns learned during training, the model assesses the importance of each feature in the traffic accident alert judgment classification feature vector and outputs a classification result, such as "safe" or "dangerous." During this process, the classifier identifies potential relationships between features and makes judgments based on historical data and real-time information. In this way, the automated classifier not only improves response speed but also reduces errors in human judgment, thereby enhancing the intelligent level of road safety management.

[0060] In summary, the embodiment of the present application first obtains real-time video stream data of the highway collected by the front camera, the vehicle obstacle distance echo signal collected by the ultrasonic sensor, and the wireless signal of the nearby vehicle collected by the on-board probe, and then uses deep learning technology to perform feature extraction and correlation analysis on the three. Finally, the classification result is obtained through the classifier to determine whether to issue a traffic accident alarm, thereby reducing the occurrence of traffic accidents such as rear-end collisions and collisions through timely warnings, and improving overall road traffic safety.

[0061] As described above, the automatic traffic accident monitoring and alarm system 100 according to the embodiment of the present application can be implemented in various terminal devices. In one example, the automatic traffic accident monitoring and alarm system 100 can be integrated into the terminal device as a software module and / or hardware module. For example, the automatic traffic accident monitoring and alarm system 100 can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the automatic traffic accident monitoring and alarm system 100 can also be one of the many hardware modules of the terminal device.

[0062] Alternatively, in another example, the automatic traffic accident monitoring alarm system 100 and the terminal device may also be separate devices, and the automatic traffic accident monitoring alarm system 100 may be connected to the terminal device via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0063] Figure 5 FIG. 1 is a flow chart of a method for automatically monitoring and alarming traffic accidents according to an embodiment of the present application. Figure 5 As shown, the automatic monitoring and alarm method for traffic accidents according to an embodiment of the present application includes: S110, obtaining real-time video stream data of the highway collected by the front camera, the vehicle obstacle distance echo signal collected by the ultrasonic sensor, and the wireless signal of the nearby vehicle collected by the on-board probe; S120, extracting the highway road condition feature vector and the highway obstacle multimodal association feature vector from the real-time video stream data of the highway collected by the front camera, the vehicle obstacle distance echo signal collected by the ultrasonic sensor, and the wireless signal of the nearby vehicle collected by the on-board probe; S130, judging whether to issue a traffic accident alarm based on the highway road condition feature vector and the highway obstacle multimodal association feature vector.

[0064] Here, those skilled in the art will appreciate that the specific operations of each step in the above-mentioned traffic accident automatic monitoring and alarm method have been described in detail above. Figures 1 to 4 The description of the automatic traffic accident monitoring alarm system has been described in detail, and therefore, its repeated description will be omitted.

[0065] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the module division is merely a logical function division, and other division methods may be used in actual implementation.

[0066] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0067] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional modules.

[0068] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0069] Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0070] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. Second-order terms are used to indicate names and do not imply any particular order.

[0071] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit of the technical solutions of the present invention.

Claims

1. A traffic accident automatic monitoring and alarm system, characterized in that: include: Highway traffic data acquisition module, used to obtain real-time highway video stream data collected by the front camera, vehicle obstacle distance echo signals collected by the ultrasonic sensor, and wireless signals of nearby vehicles collected by the on-board probe; A highway traffic data extraction module is used to extract highway road condition feature vectors and highway obstacle multimodal correlation feature vectors from the real-time highway video stream data collected by the front camera, the vehicle obstacle distance echo signal collected by the ultrasonic sensor, and the wireless signal of nearby vehicles collected by the on-board probe; A traffic accident alarm judgment module for judging whether to issue a traffic accident alarm based on the highway road condition feature vector and the high-speed obstacle multimodal association feature vector; The traffic accident alarm judgment module includes: a highway traffic feature fusion unit, configured to perform topological correlation modulation based on heterogeneous feature manifolds on the highway road condition feature vector and the highway obstacle multimodal correlation feature vector to obtain a traffic accident alarm judgment classification feature vector; a traffic accident judgment classification unit, configured to pass the traffic accident alarm judgment classification feature vector through a classifier to obtain a classification result, and the classification result is used to determine whether to issue a traffic accident alarm; Among them, the highway traffic feature fusion unit includes: constructing a road condition distance topological matrix between the highway road condition feature vector and the highway obstacle multimodal association feature vector; constructing a road condition feature value granularity association matrix between the highway road condition feature vector and the highway obstacle multimodal association feature vector; performing topological association modulation on the road condition feature value granularity association matrix based on the road condition distance topological matrix to obtain a road condition topological modulation association matrix; using the road condition topological modulation association matrix as a perspective co-projection space, projecting the highway road condition feature vector and the highway obstacle multimodal association feature vector into the perspective co-projection space to obtain a perspective-modulated highway road condition feature vector and a perspective-modulated high-speed obstacle multimodal association feature vector; and fusing the perspective-modulated highway road condition feature vector and the perspective-modulated high-speed obstacle multimodal association feature vector to obtain the traffic accident alarm judgment classification feature vector.

2. The automatic traffic accident monitoring and alarm system according to claim 1, characterized in that: The highway traffic data extraction module includes: a highway video stream data feature extraction unit, configured to extract features from the real-time highway video stream data collected by the front camera to obtain the highway road condition feature vector; an obstacle distance echo signal feature extraction unit, configured to extract features from the vehicle obstacle distance echo signal collected by the ultrasonic sensor to obtain a vehicle obstacle distance echo feature vector; A nearby vehicle wireless signal feature extraction unit is used to extract features from the wireless signals of nearby vehicles collected by the vehicle-mounted probe to obtain a global feature vector of the nearby vehicle wireless signal text; The high-speed obstacle multimodal feature association unit is used to associate the vehicle obstacle distance echo feature vector with the global feature vector of the nearby vehicle wireless signal text to obtain the high-speed obstacle multimodal association feature vector.

3. The automatic traffic accident monitoring and alarm system according to claim 2, characterized in that: The highway video stream data feature extraction unit is used to: Extracting key frames from the real-time video stream data of the highway collected by the front camera to obtain a plurality of highway road condition feature maps; The plurality of highway traffic condition feature maps are passed through a highway traffic condition feature extractor based on a spatial attention module to obtain the highway traffic condition feature vector.

4. The automatic traffic accident monitoring and alarm system according to claim 3, characterized in that: The obstacle distance echo signal feature extraction unit is used to: The vehicle obstacle distance echo signal collected by the ultrasonic sensor is subjected to a vehicle obstacle distance echo signal convolutional neural network using a channel attention mechanism to obtain a vehicle obstacle distance echo feature map; Maximum pooling is performed on the vehicle obstacle distance echo feature map to obtain the vehicle obstacle distance echo feature vector.

5. The automatic traffic accident monitoring and alarm system according to claim 4, characterized in that: The nearby vehicle wireless signal feature extraction unit includes: A text semantic encoding subunit for nearby vehicle wireless signals, configured to pass the wireless signals of nearby vehicles collected by the vehicle-mounted probe through a text semantic encoder to obtain a plurality of text feature vectors of nearby vehicle wireless signals; a two-dimensional arrangement subunit, configured to two-dimensionally arrange the plurality of nearby vehicle wireless signal text feature vectors into a nearby vehicle wireless signal text feature matrix; The nearby vehicle wireless signal convolution encoding subunit is used to pass the nearby vehicle wireless signal text feature matrix through a convolutional neural network serving as a nearby vehicle wireless signal text feature extractor to obtain the nearby vehicle wireless signal text global feature vector.

6. The automatic traffic accident monitoring and alarm system according to claim 5, characterized in that: The nearby vehicle wireless signal text semantic encoding subunit is used to: pass the wireless signals of nearby vehicles collected by the vehicle-mounted probe through a nearby vehicle wireless signal text semantic encoder including an embedding layer and a bidirectional long short-term memory model to obtain the multiple nearby vehicle wireless signal text feature vectors.

7. A method for automatic monitoring and alarming of traffic accidents, characterized in that: include: Acquire real-time highway video stream data collected by the front camera, vehicle obstacle distance echo signals collected by the ultrasonic sensor, and wireless signals of nearby vehicles collected by the on-board probe; Extracting a highway road condition feature vector and a highway obstacle multimodal correlation feature vector from the highway real-time video stream data collected by the front camera, the vehicle obstacle distance echo signal collected by the ultrasonic sensor, and the wireless signal of the nearby vehicle collected by the on-board probe; Determining whether to issue a traffic accident alert based on the highway road condition feature vector and the high-speed obstacle multimodal association feature vector; The method comprises: performing topological correlation modulation on the highway road condition feature vector and the highway obstacle multimodal correlation feature vector based on a heterogeneous feature manifold to obtain a traffic accident alarm judgment classification feature vector; passing the traffic accident alarm judgment classification feature vector through a classifier to obtain a classification result, and using the classification result to determine whether to issue a traffic accident alarm; Among them, topological correlation modulation based on heterogeneous feature manifolds is performed on the highway road condition feature vector and the highway obstacle multimodal correlation feature vector to obtain the traffic accident alarm judgment classification feature vector, including: constructing a road condition distance topological matrix between the highway road condition feature vector and the highway obstacle multimodal correlation feature vector; constructing a road condition feature value granularity correlation matrix between the highway road condition feature vector and the highway obstacle multimodal correlation feature vector; performing topological correlation modulation on the road condition feature value granularity correlation matrix based on the road condition distance topological matrix to obtain a road condition topological modulation correlation matrix; using the road condition topological modulation correlation matrix as a perspective co-projection space, projecting the highway road condition feature vector and the highway obstacle multimodal correlation feature vector into the perspective co-projection space to obtain a perspective modulated highway road condition feature vector and a perspective modulated high-speed obstacle multimodal correlation feature vector; and fusing the perspective modulated highway road condition feature vector and the perspective modulated high-speed obstacle multimodal correlation feature vector to obtain the traffic accident alarm judgment classification feature vector.

8. The method for automatic monitoring and alarming of traffic accidents according to claim 7, characterized in that: Extracting a highway road condition feature vector and a highway obstacle multimodal correlation feature vector from the highway real-time video stream data collected by the front camera, the vehicle obstacle distance echo signal collected by the ultrasonic sensor, and the wireless signal of the nearby vehicle collected by the vehicle-mounted probe, including: Performing feature extraction on the real-time video stream data of the highway collected by the front camera to obtain the highway road condition feature vector; Performing feature extraction on the vehicle obstacle distance echo signal collected by the ultrasonic sensor to obtain a vehicle obstacle distance echo feature vector; Performing feature extraction on the wireless signals of the nearby vehicles collected by the vehicle-mounted probe to obtain a global feature vector of the wireless signal text of the nearby vehicles; The vehicle obstacle distance echo feature vector and the global feature vector of the nearby vehicle wireless signal text are associated to obtain the high-speed obstacle multimodal association feature vector.

Images