5G + V2X module and risk early warning terminal for smart traffic and application method of 5G + V2X module and risk early warning terminal

Abstract

The invention relates to the technical field of intelligent traffic, and discloses an intelligent traffic-oriented 5G + V2X module and risk early warning terminal and an application method thereof, and the intelligent traffic-oriented 5G + V2X module and risk early warning terminal comprises an equipment end and a cloud platform. The equipment end is mounted on a vehicle and comprises a real-time data acquisition module, a front-end anomaly detection model and a risk alarm; the real-time data acquisition module is responsible for acquiring data related to a vehicle and real-time data of the surrounding environment of the vehicle; the front-end anomaly detection model is deployed at an equipment end; the risk alarm triggers an alarm device when the front-end anomaly detection model detects a potential risk or an abnormal condition; the cloud platform comprises a data extraction module, a data center module and a real-time calculation module; wherein the data extraction is responsible for receiving, extracting and storing data acquired by a vehicle sensor and a camera from an equipment end; the data center stores and manages a large amount of vehicle data; accurate collection of the vehicle body motion state is improved, the position information of the vehicle is more accurate, and the tracking capacity of the system on the vehicle position and the driving track is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent transportation, and particularly to a 5G+V2X module and a risk warning terminal for intelligent transportation and their application methods. Background Art

[0002] Intelligent transportation systems have become increasingly important in modern cities to improve traffic safety, optimize traffic flow, reduce traffic congestion, and enhance the driving experience. With the development of 5G technology, the ability of vehicle-to-infrastructure (V2X) communication has been significantly enhanced, providing opportunities for building a more intelligent and efficient traffic management system.

[0003] Existing traffic monitoring systems mainly rely on traditional camera monitoring and sensor technologies. Although these technologies can monitor traffic conditions to a certain extent, there are still some limitations. For example, these systems usually can only provide limited real-time data and analysis, and have relatively limited ability to accurately predict and warn of upcoming traffic risks.

[0004] In addition, the driving behavior monitoring technologies in existing systems usually rely on single-modal data, such as video images or vehicle sensor data, and it is difficult to comprehensively and accurately evaluate the driver's behavior. In case of emergencies, the ability of existing systems to respond to real-time rescue also needs to be improved.

[0005] Disadvantages of the prior art: Limited real-time performance and accuracy: Traditional traffic monitoring systems have limitations in real-time performance and accuracy, and cannot quickly identify and respond to potential traffic risks.

[0006] Single-modal monitoring: Existing systems usually rely on single-modal data monitoring, such as camera images or sensor data, and it is difficult to comprehensively evaluate driving behavior.

[0007] Lack of comprehensive risk prediction and rescue response: The prior art often cannot provide a comprehensive risk prediction and real-time rescue response mechanism, especially in case of emergencies. Summary of the Invention

[0008] The present invention provides a 5G+V2X module and a risk warning terminal for intelligent transportation, as well as a method, to solve the problems mentioned in the above background art.

[0009] The present invention provides the following technical solutions: A 5G+V2X module and a risk warning terminal for intelligent transportation, including a device end and a cloud platform; The device end is installed on a vehicle and includes a real-time data acquisition module, a front-end anomaly detection model, and a risk alarm; Among them, the real-time data acquisition module is responsible for collecting vehicle-related data and real-time data of the vehicle's surrounding environment; The front-end anomaly detection model is deployed on the device side to receive the data collected by the real-time data acquisition module and perform front-end analysis; When the front-end anomaly detection model detects potential risks or abnormal situations, the risk alarm triggers the alarm device to remind the driver or relevant personnel to take necessary measures; The cloud platform includes data extraction, a data center, and a real-time computing module; Among them, data extraction is responsible for receiving, extracting, and storing the data collected by vehicle sensors and cameras from the device side; The data center stores and manages a large amount of vehicle data and provides data query and analysis functions; The real-time computing module processes the real-time data transmitted from the device side and performs risk prediction and analysis; The device side is signal-connected to the cloud platform through a communication module. The communication module includes a 5G+V2X integrated communication network and a 5G+Beidou positioning network, which are used to realize network communication and enhanced positioning, transmit the data stream collected by the real-time data acquisition module to the cloud platform, and the communication module is also used to receive cloud platform instructions in real time.

[0010] Preferably, the data center fuses the data collected by the real-time data acquisition module to achieve position calculation and real-time trajectory recording of the vehicle when it is indoors or without Beidou positioning signals; Specifically, it includes an inertial navigation system, a visual positioning system, a hybrid positioning system, and a wireless signal positioning system.

[0011] Preferably, the cloud platform is built-in with a 5G+V2X intelligent network connection terminal module. The 5G+V2X intelligent network connection terminal module includes an abnormal driving behavior recognition module and a driving risk warning module, and at the same time provides decision-making support for the control program of the real-time computing module; Among them, the abnormal driving behavior recognition module is used to analyze the characteristics of various abnormal driving behaviors of the vehicle by combining the data of the vehicle in the real traffic environment and dangerous scenarios under different driving behavior conditions, and uses machine learning algorithms to model various abnormal driving behaviors to achieve fine-grained perception and recognition of abnormal driving behaviors of the vehicle; The driving risk warning module provides timely driving risk warnings for the abnormal driving behaviors identified by the abnormal driving behavior recognition module.

[0012] Preferably, the cloud platform is used to be responsible for resource management, data storage, real-time traffic knowledge base extraction, multi-task warning model training, terminal model distribution, terminal model update, and key traffic event reporting / warning. The cloud platform is also used for: The multi-modal collision detection module uses existing vehicle collision samples to train multi-modal collision detection based on sensor data related to the vehicle's motion state and the video stream of the driving scenario, detects collision accidents in real-time driving scenarios, calculates the severity and the scope of influence of the accident, and sends real-time collision accident alarms to the rescue center; The deep learning accident recognition model module uses the historical trajectory data uploaded by a large number of users and adopts the driving behavior knowledge distillation algorithm to train the driving risk assessment model for each user, evaluates the user's driving behavior in real-time, and in the case of a high risk coefficient, sends driving risk warning information to the vehicle terminal to trigger the horn of the cellular alarm. At the same time, deep learning is carried out according to the user's spatio-temporal habits to train the user's spatio-temporal graph neural network to predict the user's future travel trajectory; The risk warning module, after receiving the key video stream, uses a deeper and unpruned deep network for further identification, and distributes the identified warning information to the user's spatio-temporal graph neural network to achieve warning of the target user.

[0013] Preferably, the multi-modal collision detection module includes: The collision sample data storage module is used to store existing vehicle collision sample data; The training model module is used to train the multi-modal collision detection algorithm based on vision and sensor data; The accident information sending module is used to send collision accident alarm information to the rescue center in real-time; Among them, the multi-modal collision detection algorithm includes: fusing visual and sensor data, using the characteristics of sensor data for severity classification training, then using a deep network for collision vision model training, fusing the results of the severity classification training and the collision vision training models to make a collision severity classification output, obtaining data packets in the time interval T1 before and the time interval T2 after the event time point from a large number of labeled collision positive and negative samples, and extracting motion data and position data from the data packets; Segment the motion data, perform data processing on the data segments in units of segments, extract the data characteristics of various motion parameter data segments, and then perform training on the multi-classification task of collision severity, and output the sensor data classification model; At the same time, obtain the video stream in the same time period, train the visual classification model, and calculate and process the combined collision severity level by combining the severity classification training and the collision vision training models.

[0014] Preferably, the specific method of the knowledge distillation algorithm includes: S1. Data collection: The historical trajectory data uploaded by a large number of users is used as training data, and this data includes the user's driving route, speed, and parking information; S2, teacher model training: Using historical trajectory data, a deep learning model is trained to learn the driving behavior data extracted from S1. The model can capture various driving behavior patterns, including normal driving, sudden braking, and speeding. S3, knowledge distillation: transfer the knowledge of the teacher model to the student model, including the output probability distribution of the teacher model as the target of the student model, or using the intermediate layer representation of the teacher model as the input of the student model; S4. Student model training: Use historical trajectory data and data transmitted by the teacher model to train the student model. The student model is lighter than the teacher model and is more suitable for running in an environment with limited mobile terminal resources; S5. Real-time evaluation: The student model is used to evaluate the user's driving behavior in real time. In high-risk situations, the early warning mechanism is triggered based on the model's prediction results to provide real-time warnings of driving risks.

[0015] Preferably, the data fusion adopts a multi-source heterogeneous data fusion method for accurate acquisition of vehicle body motion state, and calculation of vehicle speed, steering, and six-axis data stability motion state data over a period of time, specifically including: Multi-source sensor time synchronization and joint calibration module, which improves the accuracy of sensor data by performing time synchronization and joint calibration on Wi-Fi, GNSS data, base station data, Beidou positioning, and six-axis gyroscope multi-sensor data; A spatial registration module based on time synchronization and joint calibration enables real-time fusion of multi-source information, overcoming the limitations of single-sensor environmental perception and providing effective environmental perception information for vehicles in actual traffic scenarios. A target detection algorithm based on multi-source heterogeneous information fusion is proposed using the fused data to improve the detection accuracy of dangerous scenes.

[0016] Preferably, the standardized detection method for dangerous scenes is: We used data from various types of traffic accidents in the National Accident In-depth Investigation System to extract accident scenario elements. We analyzed and sorted the variables contained in the cases to obtain relevant variables. We used the Spearman correlation analysis method to extract the variables and variable values that were closely related to the accidents. This was used to construct a scenario element table. From this scenario element table, we freely arranged and combined various accident scenarios to analyze the most dangerous ones. A correlation quantification module that considers the logical correlation between variables and the importance of variables in the objective description of the scene; A variable analysis module that considers high-measure variable data and relatively low-measure nominal-scale data in traffic accident scenarios; A clustering algorithm correction module that corrects traditional clustering algorithms and comprehensively analyzes traffic accident theories to quantify the correlation between scenario variables and analyze variable weights.

[0017] Preferably, the machine learning algorithm is used to model various abnormal driving behaviors, specifically including: A feature extraction module that converts the acceleration and direction angle sensing data of driving behavior segments into feature data; A DNN model training module that uses a deep neural network algorithm to classify and train the feature data of driving behavior segments; An algorithm model that real-time detects and identifies abnormal driving behaviors during driving, takes the features of newly emerging driving behaviors as input, and takes what kind of driving behavior it belongs to as output, an abnormal driving behavior recognition module.

[0018] This application also discloses an application method of a 5G+V2X module and a risk warning terminal for intelligent transportation: Apply this system to urban traffic monitoring, collect vehicle and surrounding environment data through the device end, and transmit it to the cloud platform in real time; The cloud platform performs real-time calculations, executes risk prediction and analysis to identify potential traffic risks and abnormal situations; Use the abnormal driving behavior recognition module and driving risk warning module on the device end to monitor driving behavior in real time; When an abnormal driving behavior is detected, remind the driver to take necessary measures through a risk alarm, and at the same time transmit the data to the cloud platform for further analysis and recording; Use the multi-modal collision detection module on the cloud platform to monitor the surrounding environment of the vehicle in real time and detect collision accidents; When a collision accident is detected, send a real-time collision accident alarm to the rescue center, including information on the severity and scope of the accident; Use the deep learning accident recognition model and risk warning module on the cloud platform to evaluate the driving behavior of users in real time; In the case of a high risk coefficient, send a driving risk warning message to the vehicle terminal, and at the same time trigger a cellular alarm to remind the driver to take safety measures.

[0019] The present invention has the following beneficial effects: By fusing multi-source heterogeneous data on the cloud platform, including inertial navigation systems, visual positioning systems, hybrid positioning systems, and wireless signal positioning systems, the position calculation and real-time trajectory recording of the vehicle when there is no Beidou positioning signal indoors are realized. This data fusion method improves the accurate acquisition of the vehicle body's motion state, makes the vehicle's position information more accurate, and thus improves the tracking and recording ability of the entire system for the vehicle's position and driving trajectory.

[0020] There is a 5G+V2X intelligent network connection terminal module built into the cloud platform. This module includes an abnormal driving behavior recognition module and a driving risk warning module. Through machine learning algorithms, the abnormal driving behavior recognition module can combine data in real traffic environments and dangerous scenarios to perform fine-grained modeling of various abnormal driving behaviors of the vehicle, achieving the perception and recognition of driving behaviors. The driving risk warning module provides timely driving risk warnings and provides decision-making support for the control program of the real-time calculation module through the intelligent network connection terminal module. The introduction of this intelligent network connection terminal effectively improves the evaluation accuracy of driving behaviors, more accurately identifies driving risks, and improves the practicality and safety of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic diagram of the system flow of the present invention.

[0022] Figure 2 It is a schematic diagram of the knowledge distillation algorithm system in an embodiment of the present invention; Figure 3 It is a schematic diagram of the implementation method in an embodiment of the present invention; Figure 4 It is a schematic diagram of the system in an embodiment of the present invention; Figure 5 It is a schematic diagram of the multi-modal collision detection system in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0024] Embodiment

[0025] Please refer to Figures 1-5 This application discloses a 5G+V2X module and a risk warning terminal for intelligent transportation, including a device end and a cloud platform; The device end is installed on the vehicle and includes a real-time data acquisition module, a front-end anomaly detection model, and a risk alarm; Among them, the real-time data acquisition module is responsible for collecting data related to the vehicle and real-time data of the vehicle's surrounding environment; The front-end anomaly detection model is deployed on the device side to receive the data collected by the real-time data acquisition module and perform front-end analysis. Introducing the front-end anomaly detection model enables comprehensive analysis of the real-time collected data on the device side, including multiple aspects such as vehicle driving status and driver behavior. It helps to comprehensively evaluate driving behavior, identify potential risks and abnormal situations, and improve the early warning and monitoring level of traffic safety.

[0026] When the front-end anomaly detection model detects potential risks or abnormal situations, the risk alarm triggers the alarm device to remind the driver or relevant personnel to take necessary measures. The cloud platform includes a data extraction module, a data center, and a real-time computing module. Among them, the data extraction is responsible for receiving, extracting, and storing the data collected by vehicle sensors and cameras from the device side. The data center stores and manages a large amount of vehicle data and provides data query and analysis functions. The real-time computing module processes the real-time data transmitted from the device side and performs risk prediction and analysis. Through the real-time computing module of the cloud platform, it is possible to quickly and accurately perform risk prediction and analysis on the real-time data transmitted from the device side. Once the front-end anomaly detection model detects potential risks or abnormal situations, the risk alarm immediately triggers the alarm device to remind the driver or relevant personnel to take necessary measures, enhancing the ability to respond promptly to abnormal driving behaviors.

[0027] The device side is signal-connected to the cloud platform through a communication module. The communication module includes a 5G+V2X integrated communication network and a 5G+Beidou positioning network, which are used to achieve network communication and enhanced positioning, transmit the data stream collected by the real-time data acquisition module to the cloud platform, and the communication module is also used to receive cloud platform instructions in real time.

[0028] It solves the problem of insufficient real-time performance and accuracy in traditional traffic monitoring systems. This technology enables the device side to collect data on vehicles and the surrounding environment in real time and transmit the data to the cloud platform through a high-speed 5G communication network to achieve real-time calculation and analysis, thereby improving the ability to promptly perceive traffic risks and abnormal situations.

[0029] The data center fuses the data collected by the real-time data acquisition module to achieve position calculation and real-time trajectory recording of the vehicle when it is indoors or without Beidou positioning signals. Indoor or Beidou positioning signal-free positioning methods include: an inertial navigation system, a visual positioning system, a hybrid positioning system, and a wireless signal positioning system.

[0030] Specifically, for the inertial navigation system: A high-precision inertial measurement unit (IMU) is introduced. By measuring the acceleration and angular velocity of the vehicle, the position and attitude changes of the vehicle are deduced, thereby realizing position calculation when there is no Beidou positioning signal indoors.

[0031] For the vision positioning system: The on-vehicle camera is used to collect image information of the road and the surrounding environment. Through computer vision algorithms, the images are processed and analyzed to realize the estimation of the vehicle position and real-time trajectory recording.

[0032] For the hybrid positioning system: Multiple positioning technologies are combined, such as the inertial navigation system, vision positioning technology, and positioning methods based on map matching. By fusing multiple data sources, the accuracy and reliability of positioning are improved.

[0033] For wireless signal positioning: Wi-Fi, Bluetooth and other wireless signals are used for positioning. By associating the signal strength with the position, the position of the vehicle is deduced to supplement the deficiency of the Beidou positioning signal.

[0034] The cloud platform is built-in with a 5G+V2X intelligent network connection terminal module. The 5G+V2X intelligent network connection terminal module includes an abnormal driving behavior recognition module and a driving risk warning module, and at the same time provides decision support for the control program of the real-time calculation module; Among them, the abnormal driving behavior recognition module is used to combine the data of the vehicle in the real traffic environment and dangerous scenarios under different driving behavior conditions, analyze the characteristics of various abnormal driving behaviors of the vehicle, and use machine learning algorithms to model various abnormal driving behaviors to realize fine-grained perception and recognition of vehicle abnormal driving behaviors; In addition to the vehicle-related data and the real-time data of the vehicle surrounding environment, physiological index data such as heart rate monitoring and facial expression analysis are also set to assist in identifying complex abnormal driving behaviors such as fatigue driving and distracted attention.

[0035] The driving risk warning module provides timely driving risk warnings for the abnormal driving behaviors identified by the abnormal driving behavior recognition module.

[0036] Specifically, geographical information system (GIS) and meteorological information data are combined. The GIS data includes: map information, road network, intersection topology structure, etc.; The meteorological information includes temperature, humidity, wind speed, etc., which are factors affecting the driving environment.

[0037] A reasonable data update frequency is set and adjusted according to the real-time nature and change situation of the data source.

[0038] For data with large dynamic changes (such as meteorological information), a shorter update period can be adopted.

[0039] Establish a data quality monitoring mechanism to detect and handle data source anomalies in a timely manner, ensuring the accuracy and reliability of data.

[0040] Set data quality thresholds. When the data quality is below the threshold, trigger an alarm and take corresponding corrective measures.

[0041] Specifically, based on historical temperature data, conduct statistical analysis and set the threshold for abnormal temperature. For example, trigger an alarm when the temperature exceeds 40 degrees Celsius or is below -10 degrees Celsius.

[0042] Setting the threshold for fog visibility: Combined with the data from fog visibility sensors, set the threshold for abnormal fog visibility. For example, trigger an alarm when the visibility is below 100 meters.

[0043] Setting the threshold for geographical location: Utilize the satellite positioning system to obtain the real-time geographical location data of the vehicle, and set the threshold for abnormal location. For example, trigger an alarm when the vehicle moves more than 10 kilometers within a short period of time.

[0044] Setting the comprehensive threshold: Jointly consider factors such as temperature, fog visibility, and geographical location, and formulate a comprehensive data quality threshold strategy to ensure the adaptability of the system under different circumstances.

[0045] The cloud platform is used for resource management (automatically allocating cloud computing resources to ensure the efficient operation of each module), data storage (providing reliable data storage, including historical trajectory data, vehicle collision samples, and real-time traffic scene videos), extracting real-time traffic knowledge bases (extracting traffic knowledge from multiple sources to update the knowledge base of the cloud platform for better understanding of the traffic environment), training multi-task warning models (training warning models for various traffic events, including collisions, illegal driving, etc.), distributing terminal models (managing the models on terminal devices to ensure their synchronization with the latest warning models), updating terminal models, and reporting / warning key traffic events. The cloud platform is also used for: The multi-modal collision detection module uses existing vehicle collision samples to train multi-modal collision detection based on sensor data related to the vehicle body's motion state and the video stream of the driving scene, detects collision accidents in the real-time driving scene, calculates the severity and impact range of the accident, and sends real-time collision accident alarms to the rescue center. Specifically, for collision sample collection: collect a large number of vehicle collision samples, including sensor data (such as acceleration, angular velocity, etc.) and the video stream of the driving scene.

[0046] Algorithm design: Sensor data processing: Preprocess the sensor data, including filtering, normalization, etc., to ensure the quality and consistency of the data.

[0047] Video stream processing: Use computer vision technology to extract key information in the video, such as vehicle position, speed, etc.

[0048] Model training: Multi-modal training: Use neural networks to fuse sensor data and video data to establish a multi-modal collision detection model.

[0049] Severity classification: Add a classifier to the model to classify the severity of the detected collisions.

[0050] Real-time application scenarios: Real-time detection: In a traffic monitoring system, monitor the driving scenario in real time. After detecting a collision accident, calculate the severity and the scope of the accident impact.

[0051] Alarm and rescue: Send real-time collision accident alarms to the rescue center, providing accurate accident information for quick rescue measures.

[0052] Deep learning accident recognition model module. Using historical trajectory data uploaded by a large number of users, adopting the driving behavior knowledge distillation algorithm, train the driving risk assessment model for each user, evaluate the user's driving behavior in real time. In case of a high risk coefficient, send a driving risk warning message to the vehicle terminal, trigger the cellular alarm to sound. At the same time, perform deep learning according to the user's spatio-temporal habits, train the user's spatio-temporal graph neural network, and predict the user's future travel trajectory; Specifically, data preparation: Historical trajectory data collection: Collect historical trajectory data uploaded by a large number of users, including vehicle driving paths, speeds, driving behaviors, etc.

[0053] Algorithm design: Driving behavior knowledge distillation: Use the knowledge distillation algorithm to transform the large-scale historical trajectory data into a risk assessment model of driving behavior.

[0054] Spatio-temporal graph neural network design: Construct a user spatio-temporal graph neural network to predict the user's future travel trajectory in combination with the user's spatio-temporal habits.

[0055] Model training: User risk assessment model: Use the large-scale historical trajectory data to train the driving risk assessment model for each user.

[0056] Spatio-temporal graph neural network training: Train the spatio-temporal graph neural network according to the user's spatio-temporal habits.

[0057] Real-time application scenarios: Real-time evaluation: Evaluate the user's driving behavior in real time. When a high risk coefficient is detected, send a driving risk warning message to the vehicle terminal.

[0058] Alarm and Siren: Trigger the vehicle's cellular alarm to sound the siren and remind the driver to take measures to reduce risks.

[0059] Risk warning module, after receiving key data, uses a deeper and unpruned deep network for further identification, and distributes the identified warning information to the user spatio-temporal graph neural network to achieve warning of the target user.

[0060] Among them, the multi-modal collision detection module includes: Collision sample data storage module, used to store existing vehicle collision sample data; Training model module, used to train a multi-modal collision detection algorithm based on vision and sensor data; Accident information sending module, used to send collision accident alarm information to the rescue center in real time; Among them, the multi-modal collision detection algorithm includes: fusing vision and sensor data, using sensor data features for severity classification training, then using a deep network for collision vision model training, fusing the results of the two types of models of severity classification training and collision vision training to make a collision severity classification output, obtaining data packets in the time interval of T1 before and T2 after the event time point from a large number of labeled collision positive and negative samples, and extracting motion data and position data from the data packets; Segment the motion data, perform data processing on the data segments in units of segments, extract the data features of various motion parameter data segments, and then perform training on the multi-classification task of collision severity, and output a sensor data classification model; At the same time, obtain the video stream in the same time period, train the vision classification model, and calculate and process the combined collision severity level by combining the severity classification training and the collision vision training model.

[0061] The multi-modal collision detection algorithm can be specifically divided into: data preprocessing stage, model training stage, real-time application stage, parameter tuning and performance optimization stage; Among them, the data preprocessing stage includes: Sensor data feature extraction: Sensor type: Select appropriate sensors, such as accelerometers, gyroscopes, etc., and collect data related to the vehicle's motion state.

[0062] Feature extraction: Preprocess the sensor data and extract key motion features, such as acceleration peaks, direction changes, etc.

[0063] Vision data preprocessing: Vision data acquisition: Use on-vehicle cameras or other vision sensors to collect images or video streams of the driving scene.

[0064] Image processing: Preprocess the image, such as cropping, scaling, denoising, etc., to obtain clear visual information.

[0065] The model training stage includes: Severity classification model: Model type: Use deep learning models, such as convolutional neural networks (CNNs).

[0066] Training data: Utilize a large number of labeled collision samples to train the severity classification model based on the characteristics of sensor data.

[0067] Model parameters: Include learning rate, batch size, number of hidden layer nodes, etc., and are tuned through cross-validation.

[0068] Collision vision model: Model type: A CNN-based vision model that can use pre-trained models such as ResNet.

[0069] Training data: Use labeled visual data and train through a deep network.

[0070] Model parameters: Include learning rate, convolutional kernel size, pooling layer configuration, etc., and need to be adjusted according to the dataset.

[0071] The collision vision model specifically is: Data input: The input of the collision vision model is a video stream from an in-vehicle camera or other visual sensors. These video streams capture the driving scenarios around the vehicle.

[0072] Data preprocessing: After data input, perform a series of preprocessing steps, including cropping, scaling, and denoising of the image, etc., to ensure the quality and consistency of the input data.

[0073] Network architecture: Adopt a deep learning network, usually a convolutional neural network (CNN), to extract features from the image. You can choose to use a pre-trained model, such as ResNet, to obtain better feature representation on a larger image dataset.

[0074] Convolutional layer: Capture local features in the image through a series of convolutional layers. Helps the model learn visual structures and patterns and identify collision-related features.

[0075] Pooling layer: Use a pooling layer after the convolutional layer to reduce the dimension of the feature map and retain the most important information. Helps improve the computational efficiency of the model.

[0076] Fully connected layer: The feature maps output by the convolutional layer are mapped to the final output layer through the fully connected layer. This layer is usually a binary classification layer for determining whether a collision accident has occurred.

[0077] Training process: Supervised learning is carried out using labeled collision and non - collision samples. An appropriate loss function, such as cross - entropy loss, is used to optimize the network parameters. During the training process, optimization algorithms such as gradient descent are used to update the model parameters.

[0078] Output: The final output is a binary classification indicating whether a collision accident has occurred and the possible severity level. This output can be used to trigger a real - time alarm system, notify the rescue center or take other emergency measures.

[0079] Motion data classification model: Data segmentation: The sensor data is divided according to T1 before and T2 after the event time point.

[0080] Feature extraction: Feature extraction is performed on each data segment, which may include motion parameters such as peak value, mean value, variance, etc.

[0081] Classification model: A deep learning model, such as a recurrent neural network (RNN), is used to train for multi - classification tasks.

[0082] Visual classification model: Data acquisition: The video stream within the same time period is used for training.

[0083] Model design: A structure similar to the collision vision model is used to train for visual information classification.

[0084] Model parameters: Parameters such as the learning rate and convolutional kernel size are adjusted to adapt to different visual data.

[0085] Comprehensive model output: Model fusion: The results of the severity classification model, collision vision model, motion data classification model, and visual classification model are fused.

[0086] Level calculation: Combining the outputs of different models, the final collision severity level is calculated.

[0087] The real - time application stage includes Real - time data processing: Data stream reception: Receive real - time sensor data and video stream.

[0088] Data packet extraction: Obtain the data packets within the time interval of T1 before and T2 after the event time point when the event occurs.

[0089] Feature extraction and classification: Sensor data processing: Extract motion data and location data, and perform real-time classification using a motion data classification model.

[0090] Visual data processing: Process the real-time video stream and perform real-time classification using a visual classification model.

[0091] Model comprehensive output: Fusion model output: Fuse the results of the sensor data classification model and the visual classification model, and calculate the comprehensive collision severity level.

[0092] Alarm and notification:[[ID=I5]] Alarm trigger: When the comprehensive collision severity level reaches a predetermined threshold, trigger the alarm mechanism.

[0093] Real-time notification: Send a collision accident alarm to the rescue center in real time, and send a driving risk warning message to the vehicle terminal.

[0094] The parameter tuning and performance optimization stage includes: Parameter tuning: Cross-validation: Use cross-validation techniques to adjust the model parameters to improve the generalization ability of the model.

[0095] Hyperparameter search: Adopt methods such as grid search or random search to find the optimal combination of model hyperparameters.

[0096] Performance optimization: Hardware optimization: Use GPUs for model training to accelerate the calculation process of deep learning algorithms.

[0097] Parallel computing: In the real-time application stage, improve the real-time performance of the algorithm through parallel computing.

[0098] The specific method of the knowledge distillation algorithm includes: S1. Data collection: Historical trajectory data uploaded by a large number of users is used as training data, which includes users' driving routes, speeds, and parking information; S2. Teacher model training: Use historical trajectory data to train a deep learning model to learn the driving behavior data extracted from S1. This model can capture various driving behavior patterns, including normal driving, hard braking, and speeding; S3. Knowledge distillation: Transfer the knowledge of the teacher model to the student model, including using the output probability distribution of the teacher model as the target of the student model, or using the intermediate layer representation of the teacher model as the input of the student model; S4. Student model training: Use historical trajectory data and data transmitted by the teacher model to train the student model. The student model is lighter than the teacher model and is more suitable for running in an environment with limited mobile terminal resources; S5. Real-time evaluation: The student model is used to evaluate the user's driving behavior in real time. In high-risk situations, the early warning mechanism is triggered based on the model's prediction results to provide real-time warnings of driving risks.

[0099] The data fusion adopts a multi-source heterogeneous data fusion method to accurately collect the vehicle body motion state and calculate the vehicle's speed, steering, and six-axis data stability motion state data over a period of time, specifically including: Multi-source sensor time synchronization and joint calibration module, which improves the accuracy of sensor data by performing time synchronization and joint calibration on Wi-Fi, GNSS data, base station data, Beidou positioning, and six-axis gyroscope multi-sensor data; A spatial registration module based on time synchronization and joint calibration enables real-time fusion of multi-source information, overcoming the limitations of single-sensor environmental perception and providing effective environmental perception information for vehicles in actual traffic scenarios. A target detection algorithm based on multi-source heterogeneous information fusion is proposed using the fused data to improve the detection accuracy of dangerous scenes.

[0100] The standardized detection method for the dangerous scene is as follows: We used data from various types of traffic accidents in the National Accident In-depth Investigation System to extract accident scenario elements. We analyzed and sorted the variables contained in the cases to obtain relevant variables. We used the Spearman correlation analysis method to extract the variables and variable values that were closely related to the accidents. This was used to construct a scenario element table. From this scenario element table, we freely arranged and combined various accident scenarios to analyze the most dangerous ones. A correlation quantification module that considers the logical correlation between variables and the importance of variables in the objective description of the scene; A variable analysis module that considers high-measure variable data and relatively low-measure nominal-scale data in traffic accident scenarios; By modifying the traditional clustering algorithm and integrating traffic accident analysis theory, a clustering algorithm modification module is developed to quantify the correlation between scene variables and analyze variable weights.

[0101] Using machine learning algorithms to model various abnormal driving behaviors includes: A feature extraction module that converts acceleration and angular sensing data of driving behavior segments into feature data; A DNN model training module that uses a deep neural network algorithm to classify and train the feature data of driving behavior segments; An algorithm model for real-time detection and recognition of abnormal driving behaviors that occur during driving, which takes the characteristics of newly emerging driving behaviors as input and the classification of the driving behaviors as output, is an abnormal driving behavior recognition module.

[0102] An application method of a 5G+V2X module and a risk warning terminal for intelligent transportation: Apply this system to urban traffic monitoring. Collect vehicle and surrounding environment data through the device side and transmit it to the cloud platform in real time; The cloud platform performs real-time calculations, executes risk prediction and analysis to identify potential traffic risks and abnormal situations; Utilize the abnormal driving behavior recognition module and the driving risk warning module on the device side to monitor driving behaviors in real time; When an abnormal driving behavior is detected, remind the driver to take necessary measures through a risk alarm, and at the same time transmit the data to the cloud platform for further analysis and recording; Utilize the multi-modal collision detection module on the cloud platform to monitor the surrounding environment of the vehicle in real time and detect collision accidents; When a collision accident is detected, send a real-time collision accident alarm to the rescue center, including information on the severity and scope of the accident; Utilize the deep learning accident recognition model and the risk warning module on the cloud platform to conduct real-time evaluation of the user's driving behavior; In the case of a high risk coefficient, send a driving risk warning message to the vehicle terminal and trigger a cellular alarm to remind the driver to take safety measures.

[0103] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0104] The above is only the preferred embodiment of the present invention. It should be pointed out that for those of ordinary skill in the art, without departing from the technical principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A 5G+V2X module and a risk warning terminal for intelligent transportation, characterized in that, It includes a device side and a cloud platform; The device side is installed on the vehicle and includes a real-time data acquisition module, a front-end anomaly detection model, and a risk alarm; Among them, the real-time data acquisition module is responsible for collecting data related to the vehicle and real-time data of the vehicle's surrounding environment; The front-end anomaly detection model is deployed on the device side and is used to receive the data collected by the real-time data acquisition module and perform front-end analysis; When the front-end anomaly detection model detects potential risks or abnormal situations, the risk alarm triggers an alarm device to remind the driver or relevant personnel to take necessary measures; The cloud platform includes a data extraction module, a data center, and a real-time computing module; Among them, the data extraction is responsible for receiving, extracting, and storing the data collected by vehicle sensors and cameras from the device side; The data center stores and manages a large amount of vehicle data and provides data query and analysis functions; The real-time computing module processes the real-time data transmitted from the device side and performs risk prediction and analysis; The device side and the cloud platform are signal-connected through a communication module. The communication module includes a 5G+V2X integrated communication network and a 5G+Beidou positioning network, which are used to realize network communication and enhanced positioning, transmit the data stream collected by the real-time data acquisition module to the cloud platform, and the communication module is also used to receive cloud platform instructions in real time.

2. The 5G+V2X module and risk warning terminal for intelligent transportation according to claim 1, wherein, The data center fuses the data collected by the real-time data acquisition module to realize position calculation and real-time trajectory recording of the vehicle when it is indoors or without Beidou positioning signal; Specifically, it includes an inertial navigation system, a visual positioning system, a hybrid positioning system, and a wireless signal positioning system.

3. The 5G+V2X module and risk warning terminal for intelligent transportation according to claim 2, wherein, The cloud platform is built-in with a 5G+V2X intelligent network connection terminal module. The 5G+V2X intelligent network connection terminal module includes an abnormal driving behavior recognition module and a driving risk warning module; Among them, the abnormal driving behavior recognition module is used to analyze the characteristics of various abnormal driving behaviors of the vehicle by combining the data of the vehicle in real traffic environments and dangerous scenarios under different driving behavior conditions, and uses machine learning algorithms to model various abnormal driving behaviors to realize fine-grained perception and recognition of abnormal driving behaviors of the vehicle; The driving risk warning module provides timely driving risk warnings for the abnormal driving behaviors recognized by the abnormal driving behavior recognition module.

4. The 5G+V2X module and risk warning terminal for intelligent transportation according to claim 3, wherein The cloud platform is used to be responsible for resource management, data storage, extraction of real-time traffic knowledge bases, training of multi-task warning models, distribution of terminal models, update of terminal models, and reporting / warning of key traffic events. The cloud platform is also used for: The multi-modal collision detection module uses existing vehicle collision samples to train multi-modal collision detection based on sensor data related to the vehicle's body motion state and video streams of driving scenarios, detects collision accidents in real driving scenarios, calculates the severity and impact range of the accidents, and sends collision accident alarms to the rescue center in real time; Deep learning accident recognition model module, which uses historical trajectory data uploaded by a large number of users, adopts a driving behavior knowledge distillation algorithm to train the driving risk assessment model for each user, evaluates the user's driving behavior in real time, and issues a driving risk warning message to the vehicle terminal in case of a high risk coefficient, triggering the horn of the cellular alarm. At the same time, it conducts deep learning based on the user's spatio-temporal habits to train the user's spatio-temporal graph neural network and predict the user's future travel trajectory; Risk warning module, after receiving key data, uses a deeper and unpruned deep network for further identification, and distributes the identified warning information to the user's spatio-temporal graph neural network to achieve warning of the target user.

5. The 5G+V2X module and risk warning terminal for intelligent transportation according to claim 4, wherein The multi-modal collision detection module includes: Collision sample data storage module, which is used to store existing vehicle collision sample data; Training model module, which is used to train a multi-modal collision detection algorithm based on visual and sensor data; Accident information sending module, which is used to send collision accident alarm information to the rescue center in real time; Among them, the multi-modal collision detection algorithm includes: fusing visual and sensor data, using the characteristics of sensor data for severity classification training, then using a deep network for collision visual model training, fusing the results of the two types of models of severity classification training and collision visual training to make a collision severity classification output, obtaining data packets with a time interval of T1 before and a time interval of T2 after the event time point from a large number of labeled collision positive and negative samples, and extracting motion data and position data from the data packets; Segment the motion data, process the data segments in units of segments, extract the data characteristics of various motion parameter data segments, and then conduct training for the multi-classification task of collision severity, and output a sensor data classification model; At the same time, obtain the video stream in the same time period, train the visual classification model, and calculate and process the combined collision severity level by combining the severity classification training and the collision visual training model.

6. The 5G+V2X module and risk warning terminal for intelligent transportation according to claim 4, characterized in that, The specific method of the knowledge distillation algorithm includes: S1. Data collection: Historical trajectory data uploaded by a large number of users is used as training data, which contains the user's driving route, speed, and parking information; S2. Teacher model training: Use historical trajectory data to train a deep learning model to learn the driving behavior data extracted from S1. This model can capture various driving behavior patterns, including normal driving, sudden braking, and speeding; S3. Knowledge distillation: Transfer the knowledge of the teacher model to the student model, including using the output probability distribution of the teacher model as the target of the student model, or using the intermediate layer representation of the teacher model as the input of the student model; S4. Student model training: Use historical trajectory data and the data transferred by the teacher model to train the student model. The student model is lighter than the teacher model and is more suitable for running in an environment with limited resources on the mobile terminal; S5. Real-time evaluation: The student model is used to evaluate the user's driving behavior in real time. In case of a high risk coefficient, according to the prediction result of the model, trigger the warning mechanism to provide a real-time warning of driving risk.

7. The 5G+V2X module and risk warning terminal for intelligent transportation according to claim 2, characterized in that The data fusion adopts a multi-source heterogeneous data fusion method to accurately collect the vehicle body motion state and calculate the vehicle's speed, steering, and six-axis data stability motion state data over a period of time, specifically including: Multi-source sensor time synchronization and joint calibration module, which improves the accuracy of sensor data by performing time synchronization and joint calibration on Wi-Fi, GNSS data, base station data, Beidou positioning, and six-axis gyroscope multi-sensor data; A spatial registration module based on time synchronization and joint calibration enables real-time fusion of multi-source information, overcoming the limitations of single-sensor environmental perception and providing effective environmental perception information for vehicles in actual traffic scenarios. A target detection algorithm based on multi-source heterogeneous information fusion is proposed using the fused data to improve the detection accuracy of dangerous scenes.

8. The 5G+V2X module and risk warning terminal for intelligent transportation according to claim 7, wherein, The standardized detection method for the dangerous scene is as follows: We used data from various types of traffic accidents in the National Accident In-depth Investigation System to extract accident scenario elements. We analyzed and sorted the variables contained in the cases to obtain relevant variables. We used the Spearman correlation analysis method to extract the variables and variable values that were closely related to the accidents. This was used to construct a scenario element table. From this scenario element table, we freely arranged and combined various accident scenarios to analyze the most dangerous ones. A correlation quantification module that considers the logical correlation between variables and the importance of variables in the objective description of the scene; A variable analysis module that considers high-measurement variable data and relatively low-measurement nominal-scale data in traffic accident scenarios; By modifying the traditional clustering algorithm and integrating traffic accident analysis theory, a clustering algorithm modification module is developed to quantify the correlation between scene variables and analyze variable weights.

9. The 5G+V2X module and risk warning terminal for intelligent transportation according to claim 3, characterized in that, Using machine learning algorithms to model various abnormal driving behaviors includes: A feature extraction module that converts acceleration and angular sensing data of driving behavior segments into feature data; A DNN model training module that uses a deep neural network algorithm to classify and train the feature data of driving behavior segments; An algorithm model for real-time detection and identification of abnormal driving behaviors that occur during driving. The abnormal driving behavior recognition module takes the characteristics of the newly emerged driving behavior as input and outputs the type of driving behavior it belongs to.

10. An application method of a 5G+V2X module and a risk warning terminal for smart transportation, implemented based on the 5G+V2X module and risk warning terminal for smart transportation according to any one of claims 1 to 9, characterized in that: The system is applied to urban traffic monitoring, collecting vehicle and surrounding environment data through the device end and transmitting it to the cloud platform in real time; The cloud platform performs real-time calculations, risk predictions, and analysis to identify potential traffic risks and anomalies; Utilize the abnormal driving behavior recognition module and driving risk warning module on the device side to monitor driving behavior in real time; When abnormal driving behavior is detected, the driver is reminded to take necessary measures through the risk alarm, and the data is transmitted to the cloud platform for further analysis and recording; Utilize the multimodal collision detection module of the cloud platform to monitor the vehicle's surrounding environment in real time and detect collision accidents; When a collision accident is detected, send a real-time collision accident alarm to the rescue center, including information on the severity and impact scope of the accident; Utilize the deep learning accident recognition model and risk warning module of the cloud platform to conduct real-time evaluation of the user's driving behavior; In the case of a high risk coefficient, send a driving risk warning message to the vehicle terminal and trigger the cellular alarm at the same time to remind the driver to take safety measures.

Images