Vehicle intelligent safety early warning method and device, medium and equipment

Abstract

The invention discloses a vehicle intelligent safety early warning method and device, a medium and equipment, and the method comprises the steps: collecting environment perception information around a vehicle, the environment perception information comprises vehicle peripheral image information, vehicle front obstacle distance information, vehicle geographic position information, and vehicle acceleration and angular velocity information; the environment sensing information is preprocessed; fusing the preprocessed environment perception information to obtain fused environment perception information; constructing a vehicle intelligent safety early warning model, and training the model; and inputting the fused environmental perception information into the trained vehicle intelligent early warning model so as to identify the current safety risk of the vehicle, and executing corresponding early warning strategies based on different safety risks. According to the invention, comprehensive perception and intelligent safety early warning of the surrounding environment of the vehicle can be realized.

Description

Technical Field

[0001] The present application belongs to the field of vehicle safety technology, and specifically relates to a vehicle intelligent safety warning method, device, medium and equipment. Background Art

[0002] With the continuous increase in the number of vehicles, traffic safety issues are becoming increasingly prominent. Traditional vehicle safety systems mainly rely on single sensors (such as radars and cameras) for environmental perception, which have problems such as low detection accuracy and high false alarm rate. In addition, existing safety systems often lack comprehensive perception and intelligent decision-making capabilities for complex traffic environments. Therefore, it is of great significance to design an intelligent safety warning system that can integrate multiple sensor data, improve detection accuracy and reduce false alarm rate. Summary of the invention

[0003] The main purpose of this application is to provide a vehicle intelligent safety warning method, device, medium and equipment, which can improve the vehicle safety detection accuracy and reduce the false alarm rate, thereby improving the safety of the vehicle.

[0004] To achieve the above objectives, this application provides the following technical solutions:

[0005] A vehicle intelligent safety warning method, the method comprising: collecting environmental perception information around the vehicle, the environmental perception information including vehicle surrounding image information, vehicle front obstacle distance information, vehicle geographic location information, and vehicle acceleration and angular velocity information; preprocessing the environmental perception information; fusing the preprocessed environmental perception information to obtain fused environmental perception information; constructing a vehicle intelligent safety warning model and training the model; inputting the fused environmental perception information into the trained vehicle intelligent safety warning model to identify the current safety risk of the vehicle and execute corresponding warning strategies based on different safety risks.

[0006] Optionally, the environmental perception information is preprocessed, including: cleaning the environmental perception information; performing data correction on the cleaned environmental perception information; performing format conversion on the corrected environmental perception information; performing data normalization on the environmental perception information after format conversion; and performing data enhancement on the environmental perception information after data normalization.

[0007] Optionally, the preprocessed environmental perception information is fused, including: representing the environmental perception information as a multimodal feature vector; dynamically adjusting the attention weight of the multimodal feature vector through an attention mechanism; and performing weighted averaging of the multimodal feature vector according to the calculated attention weight to obtain a fused feature vector.

[0008] Optionally, the vehicle intelligent safety warning model includes the following connected in sequence: an input layer for inputting the fused environmental perception information; a feature extraction layer for extracting features from the fused environmental perception information; a feature fusion layer for fusing the extracted features; and an output layer for outputting predicted probabilities of different safety risks based on the fused features.

[0009] Optionally, the feature extraction layer includes: a first feature extraction module, used to extract image features in the fused environmental perception information; a second feature extraction module, used to extract time series features in the fused environmental perception information; and a third feature extraction module, used to extract geographic location features and acceleration and angular velocity features in the fused environmental perception information.

[0010] Optionally, the vehicle intelligent safety warning model is trained through the following steps: collecting historical environmental perception information to obtain a data set, and labeling each data sample in the data set with a corresponding safety risk label; dividing the labeled data set into a training set and a validation set in proportion; setting training parameters, and training the vehicle intelligent safety warning model through the training set. During the training process, when the loss function converges, the model training is completed; verifying the trained model through the validation set, and using accuracy, precision, recall rate and F1 score as indicators to evaluate the model performance. When the accuracy, precision, recall rate and F1 score are all greater than or equal to the threshold, the model verification passes; otherwise, adjusting the training parameters or expanding the training samples to retrain the model until the model verification passes.

[0011] In addition, the present application also provides a vehicle intelligent safety warning device, which includes: an acquisition module, which is used to collect environmental perception information around the vehicle, and the environmental perception information includes vehicle surrounding image information, obstacle distance information in front of the vehicle, vehicle geographic location information, and vehicle acceleration and angular velocity information; a preprocessing module, which is used to preprocess the environmental perception information; a fusion module, which is used to fuse the preprocessed environmental perception information to obtain fused environmental perception information; a model construction and training module, which is used to construct a vehicle intelligent safety warning model and train the model; an identification and execution module, which is used to input the fused environmental perception information into the trained vehicle intelligent safety warning model to identify the current safety risks of the vehicle and execute corresponding warning strategies based on different safety risks.

[0012] Optionally, the preprocessing module includes: a cleaning submodule for cleaning the environmental perception information; a correction submodule for performing data correction on the cleaned environmental perception information; a conversion submodule for performing format conversion on the corrected environmental perception information; a normalization submodule for performing data normalization on the environmental perception information after format conversion; and an enhancement submodule for performing data enhancement on the environmental perception information after data normalization.

[0013] In addition, the present application also provides a storage medium, which includes instructions, and when the instructions are executed on a computer, the computer executes the method as described in any of the preceding items.

[0014] In addition, the present application also provides an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the methods described above when executing the program.

[0015] Compared with the prior art, this application can bring the following technical effects:

[0016] This application integrates multiple sensor data (including image information, obstacle distance information, geographic location information, acceleration, angular velocity information), and uses multimodal feature extraction, attention mechanism, and deep learning model to achieve comprehensive perception of the vehicle's surrounding environment and intelligent safety warning. This application not only improves the accuracy and real-time performance of safety risk identification, but also significantly enhances the robustness and generalization ability of the system through dynamic feature fusion and adaptive warning strategies, effectively reducing the false alarm rate and missed alarm rate, thereby greatly improving the safety and reliability of vehicle driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of a vehicle intelligent safety warning method provided by an embodiment of the present application;

[0018] Figure 2 is a structural diagram of a vehicle intelligent safety warning model provided by another embodiment of the present application;

[0019] Figure 3 It is a structural schematic diagram of a vehicle intelligent safety warning device provided by an embodiment of the present application;

[0020] Figure 4 is a schematic diagram of the structure of a storage medium provided by another embodiment of the present application;

[0021] Figure 5 It is a schematic diagram of the structure of an electronic device provided in another embodiment of the present application. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0024] In the present invention, unless otherwise clearly specified and limited, the terms "connection", "fixation", etc. should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0025] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the meaning of "and / or" appearing in the full text includes three parallel schemes. Taking "A and / or B" as an example, it includes scheme A, or scheme B, or a scheme that satisfies both A and B. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0026] Figure 1 is a structural block diagram of a power output circuit according to an embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps:

[0027] S100: Collect environmental perception information around the vehicle, that is, collect image information such as lane lines, pedestrians and surrounding vehicles through cameras, monitor the distance information of obstacles in front of the vehicle through millimeter wave radar, obtain the vehicle's geographic location information through the GPS positioning system, and measure the vehicle's acceleration and angular velocity information through IMU (inertial measurement module); S200: Preprocess the environmental perception information; S300: Fuse the preprocessed environmental perception information to obtain fused environmental perception information; S400: Build a vehicle intelligent safety warning model and train the model; S500: Input the fused environmental perception information into the trained vehicle intelligent safety warning model to identify the current safety risks of the vehicle and execute corresponding warning strategies based on different safety risks.

[0028] This application achieves comprehensive perception of the vehicle's surrounding environment and intelligent safety warning by integrating multiple sensor data (including image information, obstacle distance information, geographic location information, acceleration, angular velocity information) and designing a vehicle intelligent safety warning model.

[0029] In another exemplary embodiment, in step S200, preprocessing the environment perception information includes the following steps:

[0030] S201: Cleaning the environmental perception information;

[0031] In this step, missing values ​​and outliers may appear in the environmental perception information. Therefore, it is necessary to detect and remove or repair these missing values ​​and outliers (such as data points beyond a reasonable range) to avoid these erroneous data affecting the accuracy of subsequent analysis.

[0032] S202: performing data correction on the cleaned environmental perception information;

[0033] In this step, since each sensor has its own error characteristics, these characteristics may lead to systematic deviations in the data. Through correction, these deviations can be eliminated, thereby improving the accuracy of the environmental perception information. In addition, the data of different sensors may be inconsistent. Through correction, the consistency of the data of each sensor can be ensured.

[0034] S203: Converting the format of the corrected environment perception information;

[0035] In this step, different sensors may output data in different formats, such as image data, distance data, etc. Format conversion can ensure the consistency of the format of all data, which is convenient for subsequent data fusion and processing.

[0036] S204: performing data normalization on the environment perception information after format conversion;

[0037] In this step, the data ranges of different features in the environmental perception information often vary greatly. Normalization can make all features on the same scale, thereby preventing some features from having too great an impact on model training due to their large numerical range.

[0038] S205: Performing data enhancement on the environmental perception information after data normalization.

[0039] In this step, data enhancement can generate more training samples to improve the generalization ability of the model; it can also simulate various changes in the real environment, making the model more robust when facing new data.

[0040] In another exemplary embodiment, in step S300, fusing the pre-processed environment perception information includes the following steps:

[0041] S301: Representing each environmental perception information as a multimodal feature vector;

[0042] In this step, a convolutional neural network (CNN) may be used to extract image features from image information; a recurrent neural network (RNN) or a long short-term memory network (LSTM) may be used to extract time series features from obstacle distance information; the geographic location information is encoded into a GPS feature vector (direct encoding or geocoding may be used, for example, the geocoding method includes: converting the vehicle's latitude and longitude information into a geocoding string, and then using a pre-trained word embedding model, such as Word2Vec, to convert the geocoding string into a feature vector); and acceleration and angular velocity features are extracted from the acceleration and angular velocity information (for example, first obtaining three-axis acceleration (x, y, z) and three-axis angular velocity (roll, pitch, yaw) from an IMU sensor, and then calculating the mean, variance, maximum, minimum, peak and other statistical features of the acceleration and angular velocity, and finally using Fourier transform (FFT) to convert the time domain signal into a frequency domain signal, and extracting frequency features, such as the main frequency and spectrum energy).

[0043] S302: Dynamically adjust the attention weight of the multimodal feature vector through an attention mechanism;

[0044] In this step, first, the multimodal feature vector needs to be mapped to a shared embedding space, secondly, the attention weight of each modality is calculated using the attention network, and finally, the feature vectors of different modalities are weighted averaged according to the calculated attention weights to obtain the final fused feature vector.

[0045] For example, assume that there are four modal data, including image features F img 、GPS feature vector Fgps , time series features F radar , acceleration and angular velocity characteristics F imu , the dimension of each feature vector is d img , d gps , d radar , d imu , map these feature vectors to a shared embedding space. Assuming the dimension of the embedding space is d, we have:

[0046] Image feature embedding:

[0047] E img =W img F img +b img

[0048] Among them, W img represents the weight matrix of image feature embedding, b img Represents the bias vector for image feature embedding.

[0049] GPS feature vector embedding:

[0050] E gps =W gps F gps +b gps

[0051] Among them, W gps represents the weight matrix of GPS feature vector embedding, b gps Represents the bias vector for GPS feature vector embedding.

[0052] Time Series Feature Embedding:

[0053] E radar =W radar F radar +b radar

[0054] Among them, W radar represents the weight matrix of time series feature embedding, b radar Represents the bias vector for time series feature embedding.

[0055] Acceleration and angular velocity feature embedding:

[0056] E imu =W imu F imu +b imu

[0057] Among them, W imu represents the weight matrix of acceleration and angular velocity feature embedding, b imuBias vector representing the acceleration and angular velocity feature embedding.

[0058] The attention weights of each modality are calculated using the attention network shown below, where the attention network is represented as:

[0059] h i =σ(W1E i +b1)i

[0060] α i =softmax(W2h i +b2)

[0061] Among them, E i represents the embedding vector of the i-th modality, σ represents the activation function (such as ReLU), W1 and W2 represent weight matrices, b1 and b2 represent bias vectors, and α i represents the attention weight of the i-th modality.

[0062] S303: Perform weighted averaging of feature vectors of different modalities according to the calculated attention weights to obtain the final fusion feature vector F fused , specifically expressed as:

[0063]

[0064] Among them, α i represents the attention weight of the i-th modality, E i Represents the embedding vector of the i-th modality.

[0065] In another exemplary embodiment, in step S400, Figure 2 As shown, the vehicle intelligent safety warning model includes the following connected in sequence:

[0066] Input layer, used to input fused environmental perception information;

[0067] A feature extraction layer, used for extracting features from the fused environmental perception information;

[0068] Feature fusion layer, used to fuse the extracted features;

[0069] The output layer is used to output the predicted probabilities of different security risks based on the fused features.

[0070] In this embodiment, the feature extraction layer includes a first feature extraction module for extracting image features from the fused environmental perception information; a second feature extraction module for extracting time series features from the fused environmental perception information; and a third feature extraction module for extracting geographic location features as well as acceleration and angular velocity features from the fused environmental perception information.

[0071] The first feature extraction module includes a sequentially connected multi-scale convolution layer (for example, a sequentially connected 3×3 convolution layer, a 5×5 convolution layer, and a 7×7 convolution layer), an adaptive attention module, and an adaptive pooling layer;

[0072] The second feature extraction module includes a bidirectional LSTM layer, a gated recurrent unit (GRU), an adaptive attention mechanism, and a fully connected layer;

[0073] The third feature extraction module includes an embedding layer (embedding geographic location information (such as longitude and latitude) and motion features (such as acceleration and angular velocity) into high-dimensional vectors respectively), a multi-head self-attention module (processing the embedded features to capture the correlation between different features), a dynamic feature fusion module (dynamically generating feature fusion weights through a small fully connected network, and then weighted fusion of different features, wherein the small fully connected network may include an input layer, a first hidden layer, a second hidden layer and an output layer, wherein the input layer is 128-dimensional, the first hidden layer includes 64 neurons, the second hidden layer includes 32 neurons, and the output layer includes 3 neurons, corresponding to the features of three types of information: geographic location information, acceleration and angular velocity) and a fully connected layer.

[0074] The feature fusion layer includes:

[0075] The multimodal feature input layer is used to receive the image features extracted by the first feature extraction module, the time series features extracted by the second feature extraction module, and the position and motion features extracted by the third feature extraction module.

[0076] The multimodal feature alignment layer is used to convert features of different modalities to the same dimension, ensuring that features of different modalities can be fused at the same scale.

[0077] The normalization layer is used to normalize the aligned features to ensure that the features are at the same scale.

[0078] The weighted fusion layer is used to perform weighted fusion of features of different modalities according to the normalized weights to generate preliminary fusion features.

[0079] The feature enhancement layer further processes the preliminary fused features through a convolutional layer or a fully connected layer to enhance the expressiveness of the features.

[0080] The feature interaction layer is used to enhance the interaction between different features. The feature interaction layer includes a multi-head self-attention mechanism connected in sequence (using residual connection to add the output of the self-attention mechanism to the input feature), a normalization layer, and a feedforward neural network (including two fully connected layers, the first fully connected layer is used to expand the feature dimension, and the second fully connected layer is used to restore the feature dimension to the original dimension. The two fully connected layers also use residual connection, that is, the input of the first fully connected layer and the output of the second fully connected layer are added, and a normalization layer is set after each fully connected layer).

[0081] In the feature extraction layer, the first feature extraction module is responsible for extracting image features from the fused environmental perception information. Specifically, the module processes image data through a multi-scale convolution layer, an adaptive attention module, and an adaptive pooling layer to extract useful visual features. Among them, the multi-scale convolution layer captures features of different scales through convolution kernels of different sizes (such as 3×3, 5×5, 7×7), thereby enhancing the model's ability to recognize objects of different sizes in the image, enabling the model to better recognize and understand various objects in complex scenes, such as lane lines, pedestrians, other vehicles, etc. The adaptive attention module dynamically adjusts the feature weights of different areas through the attention mechanism, allowing the model to focus on more important parts of the image, thereby improving the accuracy and efficiency of feature extraction and reducing the interference of noise and irrelevant information.

[0082] The second feature extraction module is responsible for extracting time series features from the fused environmental perception information. Specifically, the bidirectional LSTM layer can improve the model's ability to understand time series data by capturing long-term dependencies in time series data and considering past and future information at the same time, and can better capture dynamic changes and trends, such as vehicle speed changes, pedestrian behavior, etc. The adaptive attention mechanism dynamically adjusts the feature weights of different time steps, allowing the model to focus on the more important parts of the time series, thereby further improving the accuracy and efficiency of feature extraction. The fully connected layer further processes and integrates the extracted time features to generate a high-level feature representation, which has a higher degree of abstraction and expression ability, and is helpful for subsequent feature fusion and classification tasks.

[0083] The third feature extraction module is responsible for extracting geographic location features and acceleration and angular velocity features from the fused environmental perception information. Specifically, the embedding layer can convert different types of features into a unified vector representation, which is convenient for subsequent feature processing and fusion, and improves the operability and consistency of the features. The multi-head self-attention mechanism can capture the correlation between different features, so that the model can understand the interaction between features, thereby enhancing the model's ability to understand the complex relationship between features, improving the accuracy and robustness of feature extraction, and helping to better capture dynamic changes in the environment. The dynamic feature fusion module dynamically generates feature fusion weights through a small fully connected network, and then performs weighted fusion on different features, which can achieve adaptive adjustment of the importance of different features, improve the flexibility and effectiveness of feature fusion, and help generate more representative comprehensive features.

[0084] In the feature fusion layer, the multimodal feature input layer can ensure the integrity of features of different modalities, thereby improving the model's ability to process multi-source information and enhancing the model's robustness and generalization ability. The multimodal feature alignment layer can ensure that features of different modalities can be fused at the same scale by converting features of different modalities to the same dimension, thereby improving the accuracy and efficiency of feature fusion and reducing the risk of information loss. The weighted fusion layer generates preliminary fusion features by weighted fusion of features of different modalities, thereby achieving adaptive adjustment of the importance of features of different modalities, enhancing the model's attention to key features, thereby improving the accuracy and robustness of feature fusion. The feature enhancement layer further processes the preliminary fusion features to enhance the expressiveness of the features, thereby improving the abstraction and expressiveness of the features, so that the model can better understand and process information in complex scenes. The feature interaction layer captures the correlation between different features through a multi-head self-attention mechanism, enhancing the model's ability to understand the complex relationship between features. The use of a feedforward neural network can further enhance the expressiveness of features, so that the model can better capture the high-level information of features.

[0085] In summary, the feature fusion layer can effectively fuse and process features of different modalities through multiple modules such as multimodal feature input layer, multimodal feature alignment layer, normalization layer, weighted fusion layer, feature enhancement layer and feature interaction layer. The design of these modules not only improves the accuracy and efficiency of feature fusion, but also enhances the robustness and generalization ability of the model, providing high-quality feature representation for subsequent security risk identification.

[0086] In another exemplary embodiment, in step S400, the vehicle intelligent safety warning model is trained by the following steps:

[0087] S401: Collect historical environmental perception information to obtain a data set, and label each data sample in the data set with a corresponding safety risk label, where the label includes, for example, normal driving, potential collision risk, pedestrian approaching, etc.;

[0088] S402: Divide the labeled data set into a training set and a validation set in a ratio of 7:3, for example;

[0089] S403: Setting training parameters, for example, setting the learning rate to 0.001, the batch size to 64, and the number of iterations to 100, training the vehicle intelligent safety warning model through the training set, and during the training process, when the loss function converges, the model training is completed;

[0090] In this step, the loss function is expressed as:

[0091]

[0092] Among them, α i represents the category weight, y i represents the true label of the i-th sample, represents the predicted probability of the i-th sample, N represents the number of samples, and γ represents the focusing parameter.

[0093] It should be noted that in the vehicle intelligent warning scenario, there is usually a problem of class imbalance. For example, normal driving situations are far more common than potential safety risks. In this case, the traditional binary cross entropy loss may cause the model to over-focus on the majority class (normal driving) and ignore the minority class (potential safety risks). The loss function shown in this application introduces an adjustment factor To reduce the weight of samples that are easy to classify, the vehicle intelligent safety warning model can pay more attention to samples that are difficult to classify, which helps the model better learn the characteristics of minority classes during training, thereby improving the ability to detect potential safety risks.

[0094] S404: Verify the trained model through the validation set, and use accuracy, precision, recall and F1 score as indicators to evaluate the model performance. When the accuracy, precision, recall and F1 score are all greater than or equal to 0.9, the model verification is passed; otherwise, adjust the training parameters (for example, adjust the batch size to 32) or expand the training samples (for example, adjust the division ratio of the training set and the validation set to 8:2) to retrain the model until the model verification is passed.

[0095] Below, the present application provides an illustrative description of the scheme described in the present application in combination with specific data.

[0096] First, collect environmental perception information, including:

[0097] Image information: The image of the road ahead collected by the on-board camera, including lane lines, pedestrians, other vehicles, etc.

[0098] Obstacle distance information: The distance to the obstacle ahead is measured by the millimeter-wave radar, for example, the distance to the vehicle ahead is 10 meters.

[0099] Geographic location information: The current location of the vehicle obtained through the GPS positioning system, such as longitude 121.4737° and latitude 31.2304°.

[0100] Acceleration and angular velocity information: Vehicle acceleration and angular velocity data measured by IMU (Inertial Measurement Unit), for example, acceleration is 2m / s 2 , the angular velocity is 0.1rad / s.

[0101] Secondly, the above environmental perception information is preprocessed and fused and then input into the vehicle intelligent safety warning model. After model processing, the output risk identification result is:

[0102] The probability of normal driving is 0.8;

[0103] The probability of potential collision risk is 0.15;

[0104] The probability that a pedestrian is approaching is 0.05.

[0105] In order to ensure that drivers can notice potential risks in time, this application comprehensively considers the probability of all risks and formulates a comprehensive early warning strategy as follows:

[0106] 1. Calculate the weighted average of all risk probabilities.

[0107] Total risk = 0.15 × 0.8 + 0.15 × 0.15 + 0.05 × 0.05 = 0.1225

[0108] If the combined risk exceeds a certain threshold (e.g. 0.1), a higher level warning is triggered.

[0109] 2. Issue a warning.

[0110] Since the comprehensive risk is 0.1225, which is higher than the threshold of 0.1, the following early warning strategy can be implemented:

[0111] Strategy 1: Make a continuous "beep" sound for 2 seconds.

[0112] Strategy 2: Display a red warning icon on the dashboard, prompting "There is potential danger ahead, please slow down immediately."

[0113] Strategy 3: Automatically activate the auxiliary braking system to significantly slow down the vehicle to increase the safe distance.

[0114] Strategy 4: If the driver still does not take action, the system can further reduce the vehicle speed or even perform emergency braking.

[0115] On the contrary, if the comprehensive risk is lower than the threshold of 0.1, the following early warning strategies can be implemented:

[0116] Strategy 1: emit a slight "beep" sound with a short duration (for example, 0.5 seconds).

[0117] Strategy 2: Display a yellow warning icon on the dashboard, prompting "Watch out ahead, keep safe."

[0118] Strategy 3: If a pedestrian is detected approaching the vehicle, the auxiliary braking system can be automatically activated to slightly slow down the vehicle to increase the safe distance.

[0119] Strategy 4: Use subtle prompts on the dashboard (such as a green indicator light) to remind the driver that the current environment is relatively safe, but the driver still needs to stay focused.

[0120] In summary, this application can effectively improve the driver's alertness through comprehensive evaluation and graded warning strategies, thereby reducing the risk of potential traffic accidents.

[0121] In another exemplary embodiment, the present application also provides a vehicle intelligent safety warning device, such as Figure 3 As shown, the device includes: an acquisition module 100, which is used to acquire environmental perception information around the vehicle, and the environmental perception information includes vehicle surrounding image information, vehicle front obstacle distance information, vehicle geographic location information, and vehicle acceleration and angular velocity information; a preprocessing module 200, which is used to preprocess the environmental perception information; a fusion module 300, which is used to fuse the preprocessed environmental perception information to obtain fused environmental perception information; a model building and training module 400, which is used to build a vehicle intelligent safety warning model and train the model; an identification and execution module 500, which is used to input the fused environmental perception information into the trained vehicle intelligent safety warning model to identify the current safety risk of the vehicle and execute corresponding warning strategies based on different safety risks.

[0122] Optionally, the preprocessing module 200 includes: a cleaning submodule for cleaning the environmental perception information; a correction submodule for performing data correction on the cleaned environmental perception information; a conversion submodule for performing format conversion on the corrected environmental perception information; a normalization submodule for performing data normalization on the environmental perception information after format conversion; and an enhancement submodule for performing data enhancement on the environmental perception information after data normalization.

[0123] Based on the above embodiments, Figure 4 , for an explanation of the computer-readable storage medium of the exemplary embodiment of the present application, please refer to Figure 4 , the computer-readable storage medium shown is a CD 40, on which a computer program (i.e., a program product) is stored. When the computer program is executed by the processor, each step recorded in the above method implementation will be implemented, for example, collecting environmental perception information around the vehicle, the environmental perception information includes vehicle surrounding image information, vehicle front obstacle distance information, vehicle geographic location information, and vehicle acceleration and angular velocity information; preprocessing the environmental perception information; fusing the preprocessed environmental perception information to obtain fused environmental perception information; constructing a vehicle intelligent safety warning model and training the model; inputting the fused environmental perception information into the trained vehicle intelligent safety warning model to identify the current safety risk of the vehicle and execute corresponding warning strategies based on different safety risks. The specific implementation methods of each step will not be repeated here.

[0124] It should be noted that the computer-readable storage medium includes but is not limited to phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.

[0125] Based on the above embodiments, the present application also provides an electronic device, as shown below: Figure 5 An electronic device for downloading a file according to an exemplary embodiment of the present application is described.

[0126] Figure 5 A block diagram of an exemplary electronic device 50 suitable for implementing the embodiments of the present application is shown, and the electronic device 50 may be a computer system or a cloud server. Figure 5 The electronic device 50 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0127] like Figure 5 As shown, the electronic device 50 includes but is not limited to: one or more processors or processing units 501, a system memory 502, and a bus 503 connecting different system components (including the system memory 502 and the processing unit 501).

[0128] The electronic device 50 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 50, including volatile and non-volatile media, removable and non-removable media.

[0129] The system memory 502 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 5021 and / or cache memory 5022. The electronic device 50 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the ROM 5023 may be used to read and write non-removable, non-volatile magnetic media ( Figure 5 is not shown in the Figure 5 As shown in FIG. 5 , a disk drive for reading and writing a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) can be provided. In these cases, each drive can be connected to bus 503 via one or more data medium interfaces. System memory 502 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of each embodiment of the present application.

[0130] A program / utility 5025 having a set (at least one) of program modules 5024 may be stored, for example, in system memory 502, and such program modules 5024 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 5024 generally perform the functions and / or methods of the embodiments described herein.

[0131] The electronic device 50 may also communicate with one or more external devices 504 (e.g., a keyboard, a pointing device, a display, etc.). Such communication may be performed via an input / output (I / O) interface 505. Furthermore, the electronic device 50 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 506. Figure 5 As shown, the network adapter 506 communicates with other modules (such as the processing unit 501, etc.) of the electronic device 50 via the bus 503. It should be understood that although Figure 5 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 50 .

[0132] The processing unit 501 executes various functional applications and data processing by running the program stored in the system memory 502, for example, collecting environmental perception information around the vehicle, the environmental perception information includes vehicle surrounding image information, vehicle front obstacle distance information, vehicle geographic location information and vehicle acceleration and angular velocity information; preprocessing the environmental perception information; fusing the preprocessed environmental perception information to obtain fused environmental perception information; building a vehicle intelligent safety warning model and training the model; inputting the fused environmental perception information into the trained vehicle intelligent safety warning model to identify the current safety risk of the vehicle and execute corresponding warning strategies based on different safety risks. The specific implementation methods of each step are not repeated here. It should be noted that although several units / modules or sub-units / sub-modules of the file concurrent download device are mentioned in the above detailed description, this division is only exemplary and not mandatory. In fact, according to the implementation of the present application, the features and functions of two or more units / modules described above can be concretized in one unit / module. On the contrary, the features and functions of one unit / module described above may be further divided into multiple units / modules to be embodied.

[0133] In the description of the present application, it should be noted that the terms "first", "second" and "third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0134] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0135] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0136] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0137] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0138] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that can be executed by a processor. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a cloud server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0139] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A vehicle intelligent safety warning method, characterized in that: The method comprises: Collecting environmental perception information around the vehicle, the environmental perception information includes image information around the vehicle, distance information of obstacles in front of the vehicle, geographic location information of the vehicle, and acceleration and angular velocity information of the vehicle; Preprocessing the environmental perception information; fusing the preprocessed environmental perception information to obtain fused environmental perception information; Build a vehicle intelligent safety warning model and train the model; The fused environmental perception information is input into the trained vehicle intelligent safety warning model to identify the current safety risks of the vehicle and execute corresponding warning strategies based on different safety risks.

2. A vehicle intelligent safety warning method according to claim 1, characterized in that: Preprocessing the environmental perception information includes: Cleaning the environmental perception information; Performing data correction on the cleaned environmental perception information; Performing format conversion on the corrected environmental perception information; Performing data normalization on the environment perception information after format conversion; The environmental perception information after data normalization is subjected to data enhancement.

3. A vehicle intelligent safety warning method according to claim 1, characterized in that: The pre-processed environmental perception information is integrated, including: Representing the environmental perception information as a multimodal feature vector; Dynamically adjusting the attention weight of the multimodal feature vector through an attention mechanism; The multimodal feature vectors are weighted averaged according to the calculated attention weights to obtain a fused feature vector.

4. A vehicle intelligent safety warning method according to claim 1, characterized in that: The vehicle intelligent safety warning model includes: An input layer, used for inputting the fused environmental perception information; A feature extraction layer, used for extracting features from the fused environmental perception information; Feature fusion layer, used to fuse the extracted features; The output layer is used to output the predicted probabilities of different security risks based on the fused features.

5. A vehicle intelligent safety warning method according to claim 4, characterized in that: The feature extraction layer comprises: A first feature extraction module, used to extract image features from the fused environmental perception information; A second feature extraction module is used to extract time series features from the fused environmental perception information; The third feature extraction module is used to extract geographic location features and acceleration and angular velocity features from the fused environmental perception information.

6. A vehicle intelligent safety warning method according to claim 1, characterized in that: The vehicle intelligent safety warning model is trained through the following steps: Collect historical environmental perception information to obtain a data set, and annotate each data sample in the data set with a corresponding security risk label; Divide the labeled data set into training set and validation set in proportion; Setting training parameters, training the vehicle intelligent safety warning model through the training set, and during the training process, when the loss function converges, the model training is completed; The trained model is verified through the validation set, and the accuracy, precision, recall and F1 score are used as indicators to evaluate the model performance. When the accuracy, precision, recall and F1 score are all greater than or equal to the threshold, the model verification passes; otherwise, the training parameters are adjusted or the training samples are expanded to retrain the model until the model verification passes.

7. A vehicle intelligent safety warning device, characterized in that: The device comprises: A collection module is used to collect environmental perception information around the vehicle, wherein the environmental perception information includes image information around the vehicle, distance information of obstacles in front of the vehicle, geographic location information of the vehicle, and acceleration and angular velocity information of the vehicle; A preprocessing module, used for preprocessing the environmental perception information; A fusion module, used for fusing the preprocessed environmental perception information to obtain fused environmental perception information; Model building and training module, used to build vehicle intelligent safety warning model and train the model; The identification and execution module is used to input the fused environmental perception information into the trained vehicle intelligent safety warning model to identify the current safety risks of the vehicle and execute corresponding warning strategies based on different safety risks.

8. The vehicle intelligent safety warning device according to claim 7, characterized in that: The preprocessing module comprises: A cleaning submodule, used for cleaning the environmental perception information; A correction submodule, used for performing data correction on the cleaned environmental perception information; A conversion submodule, used for converting the format of the corrected environmental perception information; A normalization submodule, used for performing data normalization on the environmental perception information after format conversion; The enhancement submodule is used to perform data enhancement on the environmental perception information after data normalization.

9. A storage medium, characterized in that: The method comprises instructions, which, when executed on a computer, enable the computer to execute the method according to any one of claims 1 to 6.

10. An electronic device, characterized in that: The electronic device comprises: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

Images