Vehicle safety early warning method and system based on Internet of Vehicles

Abstract

The invention relates to the field of intelligent driving, and provides a vehicle safety early warning method and system based on the Internet of Vehicles, and the method comprises the steps: carrying out the real-time analysis of the speed data, acceleration data and direction data of a current driving vehicle, and obtaining the driving state of the vehicle; meanwhile, the driving environment around the vehicle is known by analyzing radar reflection signals, and the driving state of the vehicle and the driving environment around the vehicle are comprehensively considered to further judge whether the current driving vehicle has the collision risk or not. In this way, the collision risk can be evaluated more accurately, and then a powerful guarantee is provided for vehicle driving safety.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving, and more specifically, to a vehicle safety warning method and system based on the Internet of Vehicles. Background Art

[0002] In today's rapidly developing digital age, Internet of Vehicles technology has become an important research direction in the field of intelligent transportation. By connecting vehicles to the Internet, Internet of Vehicles can achieve real-time monitoring and data analysis of vehicle information, thus providing strong support for vehicle safety warnings.

[0003] In the past, when analyzing vehicle driving-related data, traditional vehicle safety warnings often simply considered the state value at a certain moment, while ignoring the state change trend of the vehicle over a continuous period of time. This made it impossible to accurately grasp the actual driving conditions of the vehicle, which in turn led to the deficiency of only looking at the instantaneous value and ignoring the dynamic changes in the warning judgment.

[0004] Therefore, an optimized vehicle safety warning solution is needed. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present application provides a vehicle safety warning method and system based on the Internet of Vehicles.

[0006] A vehicle safety early warning method based on Internet of Vehicles, comprising:

[0007] Acquire vehicle speed data, vehicle acceleration data, direction data of the currently traveling vehicle at multiple predetermined time points within a predetermined time period, and radar reflection signals received within the predetermined time period;

[0008] Performing time series analysis on the vehicle speed data, vehicle acceleration data, and direction data at the plurality of predetermined time points to obtain a time series characteristic matrix of the vehicle's own driving state;

[0009] Performing signal analysis on the radar reflection signals received in the predetermined time period to obtain a driving environment feature matrix around the vehicle;

[0010] Fusion of the vehicle surrounding driving environment feature matrix and the vehicle's own driving state time series feature matrix to obtain a vehicle collision warning related feature fusion matrix;

[0011] Based on the vehicle collision warning related feature fusion matrix, it is determined whether there is a collision risk.

[0012] A vehicle safety warning system based on Internet of Vehicles, comprising:

[0013] The vehicle safe driving related data acquisition module is used to obtain the vehicle speed data, vehicle acceleration data, direction data of the currently driving vehicle at multiple predetermined time points within a predetermined time period, and the radar reflection signal received during the predetermined time period;

[0014] A vehicle driving state time series feature extraction module is used to perform time series analysis on the vehicle speed data, vehicle acceleration data, and direction data at the plurality of predetermined time points to obtain a vehicle driving state time series feature matrix;

[0015] A vehicle surrounding driving environment feature extraction module, used for performing signal analysis on the radar reflection signals received in the predetermined time period to obtain a vehicle surrounding driving environment feature matrix;

[0016] A vehicle safety driving related feature fusion module is used to fuse the vehicle surrounding driving environment feature matrix and the vehicle's own driving state time series feature matrix to obtain a vehicle collision warning related feature fusion matrix;

[0017] The collision risk prediction module is used to determine whether there is a collision risk based on the vehicle collision warning related feature fusion matrix.

[0018] This application has significant technical effects due to the adoption of the above technical solutions:

[0019] The vehicle safety warning method and system based on the Internet of Vehicles provided by this application can understand the vehicle's own driving state by real-time analysis of the vehicle's speed data, vehicle acceleration data, and direction data, and can understand the vehicle's surrounding driving environment by analyzing the radar reflection signal, and can further determine whether the vehicle currently in motion has a collision risk by comprehensively considering the vehicle's own driving state and the vehicle's surrounding driving environment. In this way, the collision risk can be assessed more accurately, thereby providing a strong guarantee for vehicle driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0021] Figure 1 The present invention is a flowchart of a vehicle safety warning method based on the Internet of Vehicles according to an embodiment of the present application.

[0022] Figure 2 Schematic diagram of data flow of a vehicle safety warning method based on Internet of Vehicles according to an embodiment of the present application.

[0023] Figure 3 This is a flowchart of step S120 in the vehicle safety warning method based on the Internet of Vehicles according to an embodiment of the present application.

[0024] Figure 4 The system block diagram of the vehicle safety warning system based on the Internet of Vehicles according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0026] In today's digital era, where technology is changing with each passing day and developing rapidly, Internet of Vehicles technology has emerged as an important research direction that has attracted much attention in the field of intelligent transportation. With the popularization of the Internet and the deep penetration of information technology, Internet of Vehicles has built an intelligent network for information interaction by closely connecting each vehicle with the huge Internet system with its unique technical advantages.

[0027] In this network, various on-board sensors, smart terminals and other devices can continuously collect various vehicle information, covering many key factors such as vehicle speed, tire pressure, brake system status, engine operating conditions, etc. The Internet of Vehicles system can monitor these massive amounts of vehicle information in real time and conduct in-depth and detailed data analysis with advanced algorithms and powerful data processing capabilities.

[0028] In contrast, traditional vehicle safety warning methods often have obvious limitations when analyzing vehicle driving related data. Usually, they simply consider the state value at a certain moment, such as only paying attention to whether the speed of the vehicle at a certain moment exceeds the specified range, but ignoring the state change trend of the vehicle over a continuous period of time. This approach makes it impossible to accurately grasp the actual driving conditions of the vehicle, which in turn leads to many deficiencies in warning judgment by only looking at instantaneous values ​​and ignoring dynamic changes. Therefore, a vehicle safety warning solution based on the Internet of Vehicles is expected.

[0029] In recent years, deep learning and neural networks have been widely used in computer vision, natural language processing, text signal processing and other fields. In addition, deep learning and neural networks have also demonstrated a level close to or even beyond that of humans in image classification, object detection, semantic segmentation, text translation and other fields. The development of deep learning and neural networks provides new solutions and solutions for vehicle safety warning based on Internet of Vehicles.

[0030] Figure 1The present invention is a flowchart of a vehicle safety warning method based on the Internet of Vehicles according to an embodiment of the present application.

[0031] Figure 2 FIG. 1 is a data flow diagram of a vehicle safety warning method based on the Internet of Vehicles according to an embodiment of the present application. Figure 1 and Figure 2 As shown, according to the embodiment of the present application, the vehicle safety warning method based on the Internet of Vehicles includes: S110, obtaining the vehicle speed data, vehicle acceleration data, direction data of the currently traveling vehicle at multiple predetermined time points within a predetermined time period, and the radar reflection signal received in the predetermined time period; S120, performing time series analysis on the vehicle speed data, vehicle acceleration data, and direction data at the multiple predetermined time points to obtain the vehicle's own driving state time series feature matrix; S130, performing signal analysis on the radar reflection signal received in the predetermined time period to obtain the vehicle's surrounding driving environment feature matrix; S140, fusing the vehicle's surrounding driving environment feature matrix and the vehicle's own driving state time series feature matrix to obtain a vehicle collision warning related feature fusion matrix; S150, judging whether there is a collision risk based on the vehicle collision warning related feature fusion matrix.

[0032] In step S110, the speed data, vehicle acceleration data, direction data and radar reflection signals received in the predetermined time period of the current driving vehicle at multiple predetermined time points within the predetermined time period are obtained. It should be understood that the speed data reflects the driving speed of the vehicle at each predetermined time point. It is not just a simple value, but also contains many important information. For example, by comparing the speeds at different time points, it can be known whether the vehicle is in a uniform driving state, an accelerated driving state or a decelerated driving state. If the speed of the vehicle gradually increases at adjacent time points, it means that the vehicle is accelerating; conversely, if the speed gradually decreases, it means that the vehicle is decelerating. Moreover, the absolute value of the speed is also related to the risk of collision. When encountering an emergency at high speed, the braking distance will become longer, which is more likely to cause a collision accident than driving at low speed. The vehicle acceleration data reflects the speed of the vehicle. Positive acceleration means that the vehicle is accelerating, and negative acceleration means that the vehicle is decelerating. Its numerical value intuitively reflects the severity of the speed change. For example, a large positive acceleration means that the vehicle speed increases rapidly in a short period of time, which may make it more difficult for the vehicle to respond to the emergency ahead in time and increase the risk of collision. Direction data records the direction of the vehicle at each predetermined time point, including but not limited to the vehicle's heading angle, steering angle, etc. Direction data can reflect the vehicle's driving trajectory and path, and is an important basis for determining whether the vehicle has dangerous behaviors such as driving on a curve, changing lanes or turning. The radar reflection signal reflects the key information of the vehicle's surrounding environment. Specifically, it can reflect whether there are other objects within a certain range around the vehicle. These objects may be other driving vehicles, fixed obstacles on the roadside (such as street lamp poles, guardrails, etc.) or temporary obstacles (such as broken-down vehicles, dropped goods, etc.). From the perspective of distance, the radar reflection signal can determine the relative distance between these objects and the vehicle. The closer the distance, the higher the collision risk. For example, when it is detected that the distance between the vehicle in front and the vehicle is constantly shortening and is lower than the safety distance threshold, it means that the collision risk is increasing. From the perspective of orientation, it can also be clear where the object is relative to the vehicle, whether it is in front, on the side or behind, etc., which helps to determine in which direction the vehicle faces potential collision threats. For example, when a vehicle suddenly approaches from the side and the distance is close, it is easy to collide with it during operations such as changing lanes. In general, the data obtained reflect the vehicle's driving status and surrounding environment from different aspects. They are combined and verified with each other, and jointly affect the judgment of collision risk.

[0033] Obtaining the speed data, vehicle acceleration data, direction data of the current vehicle at multiple predetermined time points within a predetermined time period, as well as the radar reflection signal received during the predetermined time period, is the basis for building an accurate vehicle safety warning system. The following is a detailed description of its implementation process:

[0034] The acquisition of vehicle speed data mainly depends on the vehicle speed sensor equipped with the vehicle itself. Common vehicle speed sensors have various principles, among which electromagnetic induction vehicle speed sensors are more commonly used. At the wheel hub of the vehicle, a gear ring is installed that rotates synchronously with the wheel. When the wheel rotates, the gear ring rotates accordingly, causing the magnetic field in the sensor to change periodically. According to the law of electromagnetic induction, this magnetic field change will generate an induced electromotive force in the induction coil of the sensor, and its frequency is proportional to the rotation speed of the wheel, and then proportional to the vehicle speed. The vehicle's electronic control unit (ECU) can obtain vehicle speed data in real time by accurately measuring and calculating the frequency of the induced electromotive force. Within a predetermined time period, the vehicle speed sensor continues to work at an extremely high frequency, and records the vehicle speed value at a pre-set time interval (i.e., a predetermined time point), thereby forming a vehicle speed data sequence. In order to further improve the accuracy and reliability of vehicle speed measurement, some vehicles will also use the method of integrating wheel speed sensors with global positioning system (GPS) data. The wheel speed sensor accurately measures the rotation speed of the wheel and provides information on the vehicle's movement speed in a local range, while the GPS obtains the vehicle's absolute position and movement trajectory from a macro perspective. Through complex fusion algorithms, the data from both are comprehensively analyzed, which can effectively compensate for errors caused by factors such as tire wear, road conditions, and GPS signal interference, ensuring that vehicle speed data can maintain a high level of accuracy under various driving conditions.

[0035] The acquisition of vehicle acceleration data relies on acceleration sensors. Modern vehicles widely use micro-electromechanical system (MEMS) acceleration sensors, which are designed based on micromechanical structures. During the acceleration or deceleration of the vehicle, the mass block inside the acceleration sensor will be displaced due to the action of inertial force. This displacement is converted into an electrical signal output through physical mechanisms such as capacitance change or piezoelectric effect, and the magnitude of the electrical signal is proportional to the acceleration. The acceleration sensor can sense the acceleration changes of the vehicle in the three coordinate axes, including longitudinal acceleration (forward or backward direction), lateral acceleration (left and right steering direction) and vertical acceleration (vehicle bump or undulation direction). Within a predetermined time period, the acceleration sensor collects data at an extremely high sampling frequency, accurately records the acceleration value at each predetermined time point, and provides rich information for analyzing the dynamic behavior of the vehicle. These data are of key significance for judging the smoothness and safety of vehicle operations such as starting, braking, and turning, and also help to identify abnormal driving behaviors, such as sudden acceleration and sudden braking.

[0036] The acquisition of directional data involves the coordinated work of gyroscopes and magnetometers. A gyroscope is a sensor that can measure the angular velocity of an object. In a vehicle, it is mainly used to sense the rotational motion of the vehicle around various axes, thereby determining the change in the vehicle's attitude. By integrating the output data of the gyroscope, the vehicle's pitch angle (the tilt angle of the vehicle in the front and rear directions), roll angle (the tilt angle of the vehicle in the left and right directions), and heading angle (the angle of the vehicle's driving direction) can be obtained in real time. However, the gyroscope has a certain drift error, and long-term use alone will lead to cumulative deviations in the directional data. Therefore, the magnetometer is introduced as an auxiliary sensor. The magnetometer can sense the direction of the earth's magnetic field and provide an absolute directional reference for the vehicle. By fusing the data of the gyroscope and the magnetometer and using the magnetometer to correct the drift of the gyroscope, the real-time directional information of the vehicle during driving can be accurately obtained, including the vehicle's driving direction, steering angle, and the tilt state of the vehicle body. At each predetermined time point, the vehicle's electronic system will record these directional data to form a complete directional data trajectory, which provides an important basis for subsequent analysis of the vehicle's driving path and handling stability.

[0037] The acquisition of radar reflection signals mainly relies on the vehicle-mounted radar system. Currently, there are two common types: millimeter wave radar and laser radar. Millimeter wave radar uses electromagnetic waves in the millimeter wave frequency band to work. Millimeter waves have the characteristics of short wavelength, high frequency, and large bandwidth, which makes them have unique advantages in target detection and ranging. Millimeter wave radar emits millimeter wave signals. When these signals encounter objects around the vehicle (such as other vehicles, pedestrians, obstacles, etc.), they will be reflected. The radar receives the reflected signal and analyzes the parameters such as time delay, frequency change and phase difference of the signal through complex signal processing algorithms. According to the time delay of signal propagation, the distance between the object and the vehicle can be calculated; through frequency change (Doppler effect), the speed of the object relative to the vehicle can be obtained; using phase difference information, the azimuth of the object relative to the vehicle can be determined. In this way, millimeter wave radar can obtain information such as the position, speed and orientation of objects around the vehicle in real time, providing key environmental perception data for vehicle safety warning. Laser radar uses laser beams for detection. It emits laser pulses and measures the time from the emission to the reflection of the laser pulse. By accurately measuring time and combining it with the speed of laser propagation, the distance between the object and the vehicle can be calculated. LiDAR can quickly scan the vehicle's surroundings with extremely high resolution, creating a detailed three-dimensional point cloud map, which can not only accurately obtain the location information of the object, but also provide richer information such as the shape and outline of the object, which helps to more accurately identify and classify surrounding objects, especially in complex traffic scenes such as intersections and parking lots, so as to better cope with various potential dangerous situations. During the predetermined time period, the on-board radar system continuously transmits and receives signals, constantly updates the information of the surrounding environment, and provides real-time and accurate radar reflection signal data for the vehicle safety warning system.

[0038] These sensors and systems work closely together during vehicle operation to build a perception network for the vehicle. The data they collect in real time is efficiently transmitted to a dedicated safety warning system through the vehicle's internal communication network, such as the controller area network (CAN) bus. These massive amounts of data are processed, integrated, and analyzed in the system. At the same time, with the help of Internet of Vehicles technology, vehicles can transmit these data to the cloud to achieve data sharing and further analysis. In the cloud, more powerful computing resources can aggregate and deeply mine the data of a large number of vehicles, continuously optimize the vehicle safety warning model, and improve the accuracy and timeliness of the warning. In this way, the system can perceive the status of the vehicle and changes in the surrounding environment in real time, so as to predict potential safety risks in advance and provide accurate warning information to the driver, thereby effectively reducing the occurrence of traffic accidents.

[0039] In step S120, the vehicle speed data, vehicle acceleration data, and direction data at the plurality of predetermined time points are subjected to time series analysis to obtain a time series characteristic matrix of the vehicle's own driving state. Specifically, Figure 3 FIG. 1 is a flow chart of step S120 in the vehicle safety warning method based on the Internet of Vehicles according to an embodiment of the present application. Figure 3 As shown, the step S120 includes: S121, arranging the vehicle speed data, vehicle acceleration data, and direction data of the multiple predetermined time points according to the time dimension to obtain the vehicle's own driving state timing input matrix; S122, passing the vehicle's own driving state timing input matrix through a vehicle driving state timing feature capturer to obtain the vehicle's own driving state timing feature matrix.

[0040] In step S121, the vehicle speed data, vehicle acceleration data, and direction data at the plurality of predetermined time points are arranged according to the time dimension to obtain the vehicle's own driving state time series input matrix. It should be understood that during the actual driving process, the state of the vehicle is in a continuous dynamic change. Relying only on the data at a single time point can only reflect the static state of the vehicle at that moment, and cannot show the speed change trend, acceleration change, and direction adjustment and other continuous changes during its driving process. By obtaining data at multiple time points and arranging them according to the time dimension, the entire dynamic change process of the vehicle transitioning from one state to another can be fully captured, thereby more comprehensively and truly reflecting the actual driving conditions of the vehicle. And based on the historical state changes presented by the data arranged according to the time dimension, data analysis and prediction algorithms can be used to accurately predict the subsequent driving state of the vehicle to further improve the effectiveness of vehicle safety warnings.

[0041] In step S122, the vehicle's own driving state time series input matrix is ​​passed through the vehicle driving state time series feature capturer to obtain the vehicle's own driving state time series feature matrix. It should be understood that the vehicle's own driving state time series input matrix contains the vehicle speed, acceleration, and direction data at multiple time points, and the data volume is relatively large and may contain certain redundant information. For example, in a continuous time series, the vehicle state changes at some adjacent time points may be very subtle. Directly based on such original and complex data for subsequent collision risk judgment will increase the calculation burden and analysis difficulty. In addition, although the original input matrix records the changes in the vehicle state over time, some deep-level, representative features and change laws may be hidden in a large amount of data. In order to screen out the key information that best reflects the essential characteristics of the vehicle's driving state and deeply explore the key information and change laws hidden behind the data, it is necessary to perform feature extraction processing on the vehicle's own driving state time series input matrix. Specifically, the vehicle's own driving state time series input matrix is ​​input into the vehicle driving state time series feature capturer. Here, the vehicle driving state time series feature capturer is a convolutional neural network model using a two-dimensional convolution kernel. When the convolutional neural network model using a two-dimensional convolution kernel processes the time series input matrix of the vehicle's own driving state, the convolution kernel can slide on the two-dimensional plane of the matrix (the time dimension and the dimensions of different state parameters, such as the dimensions of vehicle speed, acceleration, and direction), and automatically capture the correlation and characteristic patterns of data in the local area through convolution operations. For example, it can discover the coordinated change characteristics between the vehicle's speed and acceleration in a short continuous period of time, or the corresponding relationship between the direction adjustment and the speed change in the local time window. This is conducive to accurately extracting the characteristics that reflect the vehicle's driving state, laying the foundation for subsequent accurate judgment of collision risks.

[0042] In step S130, the radar reflection signal received in the predetermined time period is subjected to signal analysis to obtain a feature matrix of the driving environment around the vehicle. Specifically, in the embodiment of the present application, the step S130 includes: passing the radar reflection signal received in the predetermined time period through a radar signal capturer to obtain a feature matrix of the driving environment around the vehicle. It should be understood that the radar reflection signal contains a large amount of raw data, which reflects various aspects of the vehicle's surrounding environment at different times, different directions and different distances. For example, the radar will receive information such as the intensity and phase of countless reflected waves during continuous monitoring. Directly judging the driving environment conditions and collision risks around the vehicle based on these complex and unprocessed data requires a large amount of calculation and there will be many interference factors. In order to screen out key information that is substantially helpful for analyzing the driving environment, eliminate redundant and meaningless data, simplify subsequent processing procedures, and improve analysis efficiency, it is necessary to perform feature extraction processing on the radar reflection signal. The extracted features include but are not limited to object position features and motion state features. Specifically, the radar reflection signal received in the predetermined time period is input into the radar signal capturer. The radar signal capturer described here is a deep separable convolutional neural network model. The deep separable convolutional neural network model can focus on the feature mining of the data in each channel during the deep convolution stage, fully capturing the detailed information of the characteristics of each dimension of the radar reflection signal, such as more accurately extracting the changes in the intensity of the reflection signal at different directions and distances. Then, the independent features of these different channels are reasonably integrated through point-by-point convolution to form a more comprehensive and representative feature matrix of the driving environment around the vehicle.

[0043] In step S140, the vehicle surrounding driving environment feature matrix and the vehicle's own driving state time series feature matrix are fused to obtain a vehicle collision warning related feature fusion matrix. It should be understood that relying solely on the vehicle's own driving state time series feature matrix to judge the collision risk can only understand the vehicle's own movement trend and stability, but ignore the potential threats brought by the external environment, which will lead to an incomplete assessment of the risk. For example, if it is judged that it is in a safe state only based on the vehicle's own acceleration and speed changes, but does not take into account the road ahead that is under construction and roadblocks are set up (surrounding environmental factors), it is likely to cause a collision accident. Similarly, judging only by the vehicle's surrounding driving environment feature matrix, only what objects are around and their positions, motion states, etc., but it is impossible to combine the vehicle itself to make timely response actions (such as deceleration, steering, etc.) to comprehensively assess the risk. Only by fusing these two matrices can we comprehensively consider various factors that may cause collisions from the matching degree between the vehicle's own response capabilities and external environmental threats, so as to accurately judge whether there is a collision risk and avoid misjudgment or missed judgment due to one-sided information.

[0044] In particular, considering that the data such as vehicle speed, acceleration, and direction are relatively small in magnitude and fixed in range, the time series feature matrix of the vehicle's own driving state obtained after time series arrangement and feature capture has the value size and distribution range of its elements related to the original magnitude and processing logic of such data, and is generally in a relatively specific scale range; while the radar reflection signal is affected by many factors, such as the radar's transmission power, detection distance, and reflection characteristics of the target object. The feature matrix of the vehicle's surrounding driving environment obtained after it is processed by the radar signal capturer is quite different from the feature matrix obtained based on the vehicle's own state data in terms of numerical magnitude and range, resulting in different scale ranges. When fusing the feature matrix of the vehicle's surrounding driving environment and the time series feature matrix of the vehicle's own driving state, many fusion algorithms (such as simple weighted fusion, splicing followed by a fully connected layer, etc.) often comprehensively consider the contribution of each feature based on some default weight settings or calculation logic. If the scale ranges of the two matrices are very different, for example, the values ​​of one matrix element are generally between 0-1, and the values ​​of the other matrix element are generally between 100-1000, then in the fusion process, the features of the matrix with the larger value may dominate the numerical calculation, resulting in incorrect importance assessment of features that should be equally important or have a specific importance ratio according to actual conditions. In order to solve this problem, the intrinsic adaptation method of features based on vector norm measurement can effectively solve the alignment problem between features of different scales. This method analyzes and adjusts the feature scale in the feature matrix, and then extracts the essential features of the adjusted features, ensuring that various key details are retained during the fusion process. In this way, the actual conditions of the vehicle driving process can be more accurately reflected, and more reliable data support can be provided for subsequent collision judgment.

[0045] Specifically, in an embodiment of the present application, the vehicle surrounding driving environment feature matrix and the vehicle's own driving state time series feature matrix are fused to obtain a vehicle collision warning related feature fusion matrix, including: expanding the vehicle surrounding driving environment feature matrix and the vehicle's own driving state time series feature matrix to obtain a vehicle surrounding driving environment feature vector and a vehicle's own driving state time series feature vector; performing feature intrinsic adaptation based on vector norm measurement on the vehicle surrounding driving environment feature vector and the vehicle's own driving state time series feature vector to obtain a vehicle collision warning related feature fusion vector; and performing dimensional reconstruction on the vehicle collision warning related feature fusion vector to obtain the vehicle collision warning related feature fusion matrix.

[0046] More specifically, in an embodiment of the present application, the feature intrinsic adaptation based on vector norm measurement is performed on the feature vector of the vehicle's surrounding driving environment and the vehicle's own driving state time series feature vector to obtain a vehicle collision warning related feature fusion vector, including: performing normalization modulation based on vector norming on the feature vector of the vehicle's surrounding driving environment and the vehicle's own driving state time series feature vector to obtain a pre-aligned vehicle's surrounding driving environment feature vector and a pre-aligned vehicle's own driving state time series feature vector; performing intrinsic feature extraction on the pre-aligned vehicle's surrounding driving environment feature vector and the pre-aligned vehicle's own driving state time series feature vector to obtain a vehicle's surrounding driving environment principal component feature vector and a vehicle's own driving state time series principal component feature vector; performing intrinsic fine-grained alignment and fusion on the vehicle's surrounding driving environment principal component feature vector and the vehicle's own driving state time series principal component feature vector to obtain the vehicle collision warning related feature fusion vector.

[0047] In the embodiment of the present application, another implementation expression of performing feature intrinsic adaptation based on vector norm measurement on the feature vector of the vehicle's surrounding driving environment and the vehicle's own driving state time series feature vector to obtain a vehicle collision warning related feature fusion vector can be: processing the vehicle's surrounding driving environment feature vector and the vehicle's own driving state time series feature vector using the following formula to obtain a vehicle collision warning related feature fusion vector; wherein the formula is:

[0048]

[0049]

[0050] Among them, V1 represents the feature vector of the driving environment around the vehicle, V2 represents the time series feature vector of the vehicle's own driving state, and v 1i represents the eigenvalue of the ith position of the feature vector of the driving environment around the vehicle, v 2i represents the eigenvalue of the i-th position of the time series eigenvector of the vehicle's own driving state, exp represents the natural exponential function, represents the square of the second norm of the vector, V1' represents the feature vector of the driving environment around the pre-aligned vehicle, V2' represents the time series feature vector of the driving state of the pre-aligned vehicle itself, PCA(·) represents the intrinsic feature extraction, U1 represents the sequence of the first intrinsic decomposition vector, Λ1 represents the first diagonal matrix, λ 11 and λ 1m denote the eigenvalues ​​of the first and mth positions of the first diagonal matrix, respectively. T denotes the transpose of the vector. concat{·;·} denotes vector concatenation. v 11 、v 12 、v 1mRespectively represent the first, second and mth eigenvectors of the sequence of principal component eigenvectors of the driving environment around the vehicle, V k represents the principal component eigenvector of the driving environment around the vehicle, U2 represents the sequence of the second eigendecomposition vector, Λ2 represents the second diagonal matrix, λ 21 and λ 2m denote the eigenvalues ​​of the first and mth positions of the second diagonal matrix, respectively, v 21 、v 22 、v 2m Respectively represent the first, second and mth eigenvectors of the sequence of principal component eigenvectors of the vehicle's own driving state time series, V s Represents the principal component eigenvector of the vehicle's own driving state time series, represents positional subtraction, ⊙ represents positional multiplication, Indicates positional addition, Conv 1×1 represents point convolution, f vp (V k ,V s ) represents the vehicle collision warning related feature fusion vector.

[0051] That is, in the technical solution of the present application, the feature vector of the vehicle's surrounding driving environment and the vehicle's own driving state time series feature vector are subjected to feature intrinsic adaptation based on vector norm measurement to obtain a vehicle collision warning related feature fusion vector, which first performs normalization modulation based on vector norming on the vehicle's surrounding driving environment feature vector and the vehicle's own driving state time series feature vector to obtain a pre-aligned vehicle surrounding driving environment feature vector and a pre-aligned vehicle's own driving state time series feature vector. This process is not just for simply adjusting the scale, but an important technology in feature engineering, which aims to solve the problem of inconsistent scales between features in the data set. By calculating the modulus of the feature vector and dividing each vector by its own modulus, it can be ensured that all feature vectors have the same length, thereby eliminating the impact of scale differences between different feature vectors. Normalization processing not only helps numerical stability in subsequent processing, but also makes the comparison between feature vectors more fair and reasonable, laying the foundation for subsequent feature alignment. In machine learning models, especially algorithms such as neural networks, the scale of the input features directly affects the speed and direction of weight update, so normalization can be regarded as a regularization method, which helps to prevent overfitting and also helps to avoid the problem of gradient disappearance or explosion.

[0052] Next, the characteristic features of the driving environment around the pre-aligned vehicle and the time series characteristic vector of the pre-aligned vehicle's own driving state are extracted to obtain the main component characteristic vector of the driving environment around the vehicle and the main component characteristic vector of the vehicle's own driving state. That is, after the normalization modulation is completed, the characteristic features of the driving environment around the pre-aligned vehicle and the time series characteristic vector of the vehicle's own driving state are extracted. As an unsupervised linear dimensionality reduction method, intrinsic feature extraction is widely used in data compression and feature extraction. This process determines the principal components by finding the direction of the maximum variance of the data. These principal components are new coordinate axes in the original feature space, which can capture the structure of the data with minimal information loss. It is worth mentioning that by selecting the eigenvectors corresponding to the first few maximum eigenvalues, the dimension of the feature space can be effectively reduced while maintaining the key information of the data. This dimensionality reduction can not only accelerate the model training process, but also help the model to generalize better, reduce the impact of noise, and thus improve the performance of the model.

[0053] Finally, the principal component feature vector of the driving environment around the vehicle and the principal component feature vector of the vehicle's own driving state time series are intrinsically aligned and fused to obtain the vehicle collision warning related feature fusion vector. The goal of this step is to find the best matching point or area between the principal component feature vectors of the two target analysis original data to achieve alignment between the two. Fine-grained alignment goes beyond the traditional rough fusion methods such as simple weighted average or splicing, and needs to consider the time series characteristics or spatial distribution characteristics of the feature vector. The fusion after alignment is not a simple numerical operation, but requires the design of a specific mechanism to determine how to combine the information of the two feature vectors. This content-based intelligent fusion method can more effectively utilize multi-source information and improve the representation ability and decision-making quality of the model.

[0054] In step S150, based on the vehicle collision warning related feature fusion matrix, it is determined whether there is a collision risk. Specifically, in the embodiment of the present application, the step S150 includes: passing the vehicle collision warning related feature fusion matrix through a collision warning device based on a classifier to obtain a collision classification result, and the collision classification result is used to indicate whether there is a collision risk. It should be understood that although the vehicle collision warning related feature fusion matrix integrates detailed feature information of various aspects such as the vehicle's own driving state and the surrounding driving environment, this information is still a relatively complex and original data presentation form. In order to convert complex feature information into intuitive and easy-to-understand classification results, so that the driver or the vehicle safety system can quickly understand and take corresponding measures, in the technical solution of the present application, it is necessary to input the vehicle collision warning related feature fusion matrix into a collision warning device based on a classifier. As a machine learning model, the classifier can analyze and judge according to the input data and map it to different categories, specifically referring to the two categories of collision risk and no collision risk. The most direct role of the collision classification result is to trigger the prompt function of the vehicle safety warning system. If the classification result shows that there is a collision risk, the system can immediately warn the driver through multiple methods such as sound (such as a rapid alarm), vision (such as warning lights on the dashboard, images of the collision risk area, etc.) or touch (such as seat vibration, etc.), reminding the driver to take corresponding safety operations such as braking and avoidance in time to avoid collision accidents and ensure driving safety.

[0055] In summary, the vehicle safety warning method based on the vehicle network in the embodiment of the present application is explained, which understands the vehicle's own driving state by real-time analysis of the vehicle's speed data, vehicle acceleration data, and direction data, and understands the vehicle's surrounding driving environment by analyzing the radar reflection signal, and comprehensively considers the vehicle's own driving state and the vehicle's surrounding driving environment to further determine whether the current driving vehicle has a collision risk. In this way, the collision risk can be assessed more accurately, thereby providing a strong guarantee for vehicle driving safety.

[0056] Figure 4 FIG. 1 is a system block diagram of a vehicle safety warning system based on the Internet of Vehicles according to an embodiment of the present application. Figure 4As shown, according to the embodiment of the present application, the vehicle safety warning system 100 based on the Internet of Vehicles includes: a vehicle safety driving related data acquisition module 110, which is used to obtain the vehicle speed data, vehicle acceleration data, direction data of the current driving vehicle at multiple predetermined time points within a predetermined time period, and the radar reflection signal received in the predetermined time period; a vehicle's own driving state time series feature extraction module 120, which is used to perform time series analysis on the vehicle speed data, vehicle acceleration data, and direction data at the multiple predetermined time points to obtain the vehicle's own driving state time series feature matrix; a vehicle surrounding driving environment feature extraction module 130, which is used to perform signal analysis on the radar reflection signal received in the predetermined time period to obtain the vehicle surrounding driving environment feature matrix; a vehicle safety driving related feature fusion module 140, which is used to fuse the vehicle surrounding driving environment feature matrix and the vehicle's own driving state time series feature matrix to obtain a vehicle collision warning related feature fusion matrix; a collision risk prediction module 150, which is used to determine whether there is a collision risk based on the vehicle collision warning related feature fusion matrix.

[0057] Here, those skilled in the art can understand that the specific functions and operations of the various units and modules in the above-mentioned vehicle safety warning system 100 based on the Internet of Vehicles have been referred to above. Figures 1 to 3 The invention has been introduced in detail in the description of the vehicle safety warning method based on the Internet of Vehicles, and therefore, its repeated description will be omitted.

[0058] In summary, the vehicle safety warning system 100 based on the vehicle network according to the embodiment of the present application is explained, which understands the vehicle's own driving state by performing real-time analysis on the vehicle speed data, vehicle acceleration data and direction data of the current driving vehicle, and understands the driving environment around the vehicle by analyzing the radar reflection signal, and comprehensively considers the vehicle's own driving state and the driving environment around the vehicle to further determine whether the current driving vehicle has a collision risk. In this way, the collision risk can be assessed more accurately, thereby providing a strong guarantee for vehicle driving safety.

[0059] As described above, the vehicle safety warning system 100 based on the Internet of Vehicles according to the embodiment of the present application can be implemented in various wireless terminals, such as a server for vehicle safety warning based on the Internet of Vehicles. In one example, the vehicle safety warning system 100 based on the Internet of Vehicles according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the vehicle safety warning system 100 based on the Internet of Vehicles can be a software module in the operating system of the wireless terminal, or can be an application developed for the wireless terminal; of course, the vehicle safety warning system 100 based on the Internet of Vehicles can also be one of the many hardware modules of the wireless terminal.

[0060] Alternatively, in another example, the vehicle safety warning system 100 based on the Internet of Vehicles and the wireless terminal may also be separate devices, and the vehicle safety warning system 100 based on the Internet of Vehicles may be connected to the wireless terminal via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

Claims

1. A vehicle safety warning method based on Internet of Vehicles, characterized in that: include: Acquire vehicle speed data, vehicle acceleration data, direction data of the currently traveling vehicle at multiple predetermined time points within a predetermined time period, and radar reflection signals received within the predetermined time period; Performing time series analysis on the vehicle speed data, vehicle acceleration data, and direction data at the plurality of predetermined time points to obtain a time series characteristic matrix of the vehicle's own driving state; Performing signal analysis on the radar reflection signals received in the predetermined time period to obtain a driving environment feature matrix around the vehicle; Fusion of the vehicle surrounding driving environment feature matrix and the vehicle's own driving state time series feature matrix to obtain a vehicle collision warning related feature fusion matrix; Based on the vehicle collision warning related feature fusion matrix, it is determined whether there is a collision risk.

2. The vehicle safety warning method based on the Internet of Vehicles according to claim 1 is characterized in that: The vehicle speed data, vehicle acceleration data, and direction data at the plurality of predetermined time points are subjected to time series analysis to obtain a time series characteristic matrix of the vehicle's own driving state, including: Arrange the vehicle speed data, vehicle acceleration data, and direction data at the plurality of predetermined time points according to the time dimension to obtain a time series input matrix of the vehicle's own driving state; The vehicle's own driving state time series input matrix is ​​passed through a vehicle's own driving state time series feature capturer to obtain the vehicle's own driving state time series feature matrix.

3. The vehicle safety warning method based on the Internet of Vehicles according to claim 2 is characterized in that: The radar reflection signals received in the predetermined time period are analyzed to obtain a characteristic matrix of the driving environment around the vehicle, including: the radar reflection signals received in the predetermined time period are passed through a radar signal capturer to obtain the characteristic matrix of the driving environment around the vehicle.

4. The vehicle safety warning method based on the Internet of Vehicles according to claim 3 is characterized in that: The vehicle driving state temporal feature capturer is a convolutional neural network model using a two-dimensional convolution kernel, and the radar signal capturer is a depth-separable convolutional neural network model.

5. The vehicle safety warning method based on the Internet of Vehicles according to claim 4 is characterized in that: The vehicle collision warning related feature fusion matrix is ​​obtained by fusing the vehicle surrounding driving environment feature matrix and the vehicle's own driving state time series feature matrix, including: Expanding the vehicle surrounding driving environment characteristic matrix and the vehicle's own driving state time series characteristic matrix to obtain the vehicle surrounding driving environment characteristic vector and the vehicle's own driving state time series characteristic vector; Performing feature intrinsic adaptation based on vector norm measurement on the feature vector of the driving environment around the vehicle and the time series feature vector of the driving state of the vehicle itself to obtain a vehicle collision warning related feature fusion vector; The vehicle collision warning related feature fusion vector is dimensionally reconstructed to obtain the vehicle collision warning related feature fusion matrix.

6. The vehicle safety warning method based on the Internet of Vehicles according to claim 5 is characterized in that: The feature intrinsic adaptation based on vector norm measurement is performed on the feature vector of the driving environment around the vehicle and the time series feature vector of the driving state of the vehicle itself to obtain a vehicle collision warning related feature fusion vector, including: Performing normalization modulation based on vector norming on the driving environment feature vector around the vehicle and the driving state time series feature vector of the vehicle itself to obtain the driving environment feature vector around the pre-aligned vehicle and the driving state time series feature vector of the pre-aligned vehicle itself; Performing intrinsic feature extraction on the feature vector of the driving environment around the pre-aligned vehicle and the time series feature vector of the driving state of the pre-aligned vehicle itself to obtain a principal component feature vector of the driving environment around the vehicle and a principal component feature vector of the time series of the driving state of the vehicle itself; The principal component feature vector of the driving environment around the vehicle and the principal component feature vector of the driving state time series of the vehicle itself are aligned and fused in an intrinsic fine-grained manner to obtain the vehicle collision warning related feature fusion vector.

7. The vehicle safety warning method based on the Internet of Vehicles according to claim 6 is characterized in that: Based on the vehicle collision warning related feature fusion matrix, judging whether there is a collision risk includes: passing the vehicle collision warning related feature fusion matrix through a classifier-based collision warning device to obtain a collision classification result, and the collision classification result is used to indicate whether there is a collision risk.

8. A vehicle safety warning system based on Internet of Vehicles, characterized in that: include: The vehicle safe driving related data acquisition module is used to obtain the vehicle speed data, vehicle acceleration data, direction data of the currently driving vehicle at multiple predetermined time points within a predetermined time period, and the radar reflection signal received during the predetermined time period; A vehicle driving state time series feature extraction module is used to perform time series analysis on the vehicle speed data, vehicle acceleration data, and direction data at the plurality of predetermined time points to obtain a vehicle driving state time series feature matrix; A vehicle surrounding driving environment feature extraction module, used for performing signal analysis on the radar reflection signals received in the predetermined time period to obtain a vehicle surrounding driving environment feature matrix; A vehicle safety driving related feature fusion module is used to fuse the vehicle surrounding driving environment feature matrix and the vehicle's own driving state time series feature matrix to obtain a vehicle collision warning related feature fusion matrix; The collision risk prediction module is used to determine whether there is a collision risk based on the vehicle collision warning related feature fusion matrix.

9. The vehicle safety warning system based on the Internet of Vehicles according to claim 8 is characterized in that: The vehicle's own driving state time series feature extraction module includes: The vehicle's own driving state data sorting unit is used to arrange the vehicle speed data, vehicle acceleration data, and direction data at the plurality of predetermined time points according to the time dimension to obtain the vehicle's own driving state time series input matrix; The vehicle's own driving state data encoding unit is used to pass the vehicle's own driving state time series input matrix through the vehicle's own driving state time series feature capturer to obtain the vehicle's own driving state time series feature matrix.

10. The vehicle safety warning system based on the Internet of Vehicles according to claim 9, characterized in that: The collision risk prediction module is used to: pass the vehicle collision warning related feature fusion matrix through a classifier-based collision warning device to obtain a collision classification result, and the collision classification result is used to indicate whether there is a collision risk.

Images