Low-visibility driving risk identification method based on fusion mechanism and image deep features

Abstract

The invention relates to the technical field of intelligent traffic, and particularly discloses a low-visibility driving risk identification method fusing a mechanism and an image deep feature, which comprises the following steps: constructing a multi-dimensional visibility mechanism feature index system, and extracting index values of an edge feature, a depth feature and a medium transmission coefficient feature based on vehicle-mounted image data; building a convolutional neural network model, introducing cross-domain knowledge by finely tuning a pre-training model on the large data set, and obtaining visibility deep features of the vehicle-mounted image; establishing a low-visibility driving risk identification model fusing mechanism features and deep features, and adaptively allocating feature importance degrees in different scenes by using a self-attention mechanism; and setting a focus loss function, endowing a low-visibility difficult-to-identify sample with a larger loss weight, and updating model parameters through a stochastic gradient descent algorithm to obtain a low-visibility driving risk identification model.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation technology, and more specifically, to a low-visibility driving risk identification method that integrates a mechanism and deep image features. Background Art

[0002] In low-visibility scenarios such as fog, heavy rainfall, and sandstorms, the driver's vision is obstructed, making it difficult to identify the road conditions and obstacles ahead, resulting in a high driving risk and a high risk of traffic accidents. To address this type of driving risk, roadside traffic weather stations or roadside video surveillance are mainly used to monitor atmospheric visibility in order to issue corresponding traffic control measures and reduce the risk of vehicles driving in low-visibility conditions. However, the technical means of roadside monitoring are limited by the scope of equipment deployment and the coverage is very limited. There is an urgent need to develop a low-cost, high-coverage low-visibility driving risk identification method.

[0003] At present, the penetration rate of new cars with L2+ autonomous driving functions has exceeded 50%. Such cars are equipped with multiple sensing devices such as cameras and millimeter-wave radars, which enable vehicles to have rich and multiple perception and data collection capabilities. Building a low-visibility driving risk identification algorithm based on the perception capabilities of on-board cameras can achieve low-cost coverage of the entire road network. However, it is difficult to collect low-visibility image data from the vehicle's perspective, the data is limited, and the real traffic scenes are complex and changing, resulting in the existing deep learning methods. The generalization performance of a large number of unknown scenes under limited data conditions is weak, and its application in real traffic scenes is limited.

[0004] In view of this, the present invention provides a low-visibility driving risk identification method that integrates mechanism and deep image features to solve the above problems. Summary of the invention

[0005] In order to overcome the problems in the prior art, the present invention proposes a low-visibility driving risk identification method that integrates mechanism and deep image features. By integrating multi-dimensional features such as visibility mechanism features and deep image features, the generalization performance of the low-visibility driving risk identification algorithm is improved under limited data conditions.

[0006] In a first aspect, the present invention provides a method for identifying low-visibility driving risks by integrating mechanism and deep image features, comprising the following steps:

[0007] S1. Construct a multi-dimensional visibility mechanism characteristic index system, and extract the index values ​​of edge characteristics, depth characteristics and medium transmission coefficient characteristics based on vehicle-mounted image data;

[0008] S2. Build a convolutional neural network model, introduce cross-domain knowledge by fine-tuning the pre-trained model on a large dataset, and obtain the deep visibility features of vehicle images;

[0009] S3. Establish a low-visibility driving risk identification model that integrates mechanism features and deep features, and use the self-attention mechanism to adaptively allocate the importance of features in different scenarios;

[0010] S4. Set a focal loss function for the vehicle image data, assign a larger loss weight to low-visibility difficult-to-identify samples, and then update the model parameters through the stochastic gradient descent algorithm to obtain a low-visibility driving risk identification model.

[0011] As a preferred technical solution of the first aspect of the present invention, step S1 specifically includes the following steps:

[0012] S11, using an edge detection operator to calculate the gradient of pixel intensity in the vehicle-mounted image to identify edge information and obtain an edge feature map;

[0013] S12, using a monocular depth estimation algorithm, calculating the pixel-level depth value of the vehicle image to obtain a depth feature map;

[0014] S13, based on the dark channel prior, calculating the medium transmission coefficient of the vehicle-mounted image to obtain a medium transmission coefficient characteristic map;

[0015] S14, performing edge index analysis on the edge feature map based on a multi-dimensional index system to extract index values ​​of edge mechanism features; the edge indexes include edge density, average gradient amplitude of non-zero pixels, sum of edge features, maximum value of edge features, average value of edge features, center of gravity of edge features and center of edge features;

[0016] S15, by analyzing the depth value range of the depth feature map in the data set, performing depth index analysis on the depth feature map based on a multi-dimensional index system to extract the index value of the depth mechanism feature; setting a depth fixed threshold in advance, selecting a fixed depth area in the vehicle-mounted image data based on the depth fixed threshold, and marking the depth feature map area where the fixed depth area is not selected as a non-fixed depth area;

[0017] S16, based on the divided areas of the image depth map in step S15, extract the index value t_features of the medium transmission coefficient mechanism characteristics from the medium transmission coefficient feature map;

[0018] As a preferred technical solution of the first aspect of the present invention, the depth index includes a global depth index and a local depth index, and the application logic of the global depth index and the local depth index is:

[0019] The depth feature map extracted from the vehicle image data, constructing a global depth index for all areas of the depth feature map, and extracting the index value of the depth mechanism feature based on the global depth index;

[0020] A fixed depth threshold is set in advance, and a fixed depth area in the vehicle image data is selected based on the fixed depth threshold. The depth feature map area where the fixed depth area is not selected is marked as a non-fixed depth area. A local depth index is constructed based on the non-fixed depth area, and the index value of the depth mechanism feature is extracted based on the local depth index.

[0021] As a preferred technical solution of the first aspect of the present invention, the calculation formula based on the Sobel edge detection operator used in step S11 includes:

[0022] e(x)=Edge(I(x))

[0023] Where I is any vehicle-mounted image, x is a pixel on the image; Edge(·) is an edge detection operator;

[0024] As a preferred technical solution of the first aspect of the present invention, the calculation formula based on the use of the Metric3D algorithm in step S12 includes:

[0025] d(x)=Depth(I(x))

[0026] Where I is any vehicle-mounted image, x is a pixel on the image; Depth(·) is a monocular depth estimation algorithm;

[0027] As a preferred technical solution of the first aspect of the present invention, the formula for calculating the medium transmission coefficient characteristic diagram in step S13 includes:

[0028]

[0029] Among them, I is an arbitrary vehicle-mounted image, c is the RGB three color channels of the image; x is a certain area on the image, y is the pixel point in the x area; A is the average brightness of the atmosphere on the corresponding color channel; ω (0<ω≤1) is a fixed parameter.

[0030] As a preferred technical solution of the first aspect of the present invention, step S2 specifically includes the following steps:

[0031] S21. Build a convolutional neural network, load the pre-trained model on a large image classification dataset as the initial weight, and extract the vehicle image features after fine-tuning;

[0032] S22, inputting the vehicle image features into the average pooling layer to obtain the visibility deep features;

[0033] As a preferred technical solution of the first aspect of the present invention, step S3 comprises the following steps:

[0034] S31, establish four linear mapping layers, respectively map edge features, depth features, medium transmission coefficient features and deep features to the same dimension, and perform feature concatenation in the sequence dimension to obtain sequence features;

[0035] S32, build a feature fusion module based on the Transformer encoder, and use the self-attention mechanism to adaptively model the feature importance in different scenarios after assigning sequence feature position encoding information;

[0036] S33. Use the mechanism feature calculation method in step S1, the convolutional neural network model in S2 and the linear mapping layer in S31 as a feature extraction module, and then combine the feature fusion module. Finally, use the linear output layer and the Softmax function to establish an output module to establish a low-visibility driving risk identification model that integrates the mechanism features and deep features of the vehicle-mounted image.

[0037] As a preferred technical solution of the first aspect of the present invention, step S4 comprises the following steps:

[0038] S41, dividing the vehicle image visibility dataset into a training set and a test set based on scenes, and ensuring that the scenes in the training set and the test set are independent of each other through manual inspection;

[0039] S42, preprocessing the data, wherein the vehicle-mounted images are scaled and normalized, and the mechanism characteristic index values ​​are normalized using maximum and minimum;

[0040] S43. Set the focal loss function and assign a larger loss weight to low-visibility difficult-to-identify samples. Then iteratively optimize the identification model through the stochastic gradient descent algorithm. When the loss value converges, the training ends and the low-visibility driving risk identification model is obtained.

[0041] In a second aspect, the present invention provides a computer-readable storage medium storing instructions, which, when executed on a computer, enable the computer to execute the low-visibility driving risk identification method based on the above-mentioned fusion mechanism and deep image features.

[0042] The specific advantages of the present invention are as follows:

[0043] The present invention utilizes image data of real-time dynamic traffic scenes collected by on-board cameras to accurately identify low-visibility driving risks, thereby realizing low-cost, real-time dynamic identification of low-visibility driving risks for the entire road network, which is conducive to formulating real-time traffic control measures or conducting vehicle safety warnings to reduce the occurrence of accidents.

[0044] The low-visibility driving risk identification model of the present invention takes into account the advantages of both mechanism characteristics and deep characteristics under limited data conditions. It not only has stronger robustness against environmental noise or interference, but also can automatically learn complex visibility high-dimensional feature space, and adaptively model the feature importance in different scenarios through the self-attention mechanism, overcoming the defects of inaccurate prediction and inability to adapt to scene changes in traditional single feature methods, and improving the model recognition performance and generalization ability in unknown scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a structural schematic diagram of the low-visibility driving risk identification system of the present invention;

[0046] Figure 2 It is a schematic diagram of the low-visibility driving risk identification model architecture of the present invention;

[0047] Figure 3 It is a schematic diagram of the model loss value of the low visibility driving risk identification model in the training and testing process in the example of the present invention;

[0048] Figure 4 It is a schematic diagram of model test indicators during the training and testing process of the low-visibility driving risk identification model in an example of the present invention;

[0049] Figure 5 It is a schematic diagram of the identification stability of low-visibility driving risk by the low-visibility driving risk identification model and the comparison model in the example of the present invention in the test video. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] Example 1: Please refer to Figure 1 The present invention provides a low-visibility driving risk identification method integrating mechanism and deep image features, comprising the following steps:

[0052] S1. Extract visibility mechanism characteristics based on vehicle-mounted image data, wherein the visibility mechanism characteristics include edge characteristics, depth characteristics, and medium transmission coefficient characteristics, construct a multi-dimensional index system, and obtain index values ​​of edge characteristics, depth characteristics, and medium transmission coefficient characteristics for the visibility mechanism characteristics;

[0053] Specifically, step S1 includes the following steps:

[0054] S11, using the Sobel edge detection operator to calculate the gradient of pixel intensity in the vehicle image data, identifying edge information through the gradient of pixel intensity, and obtaining an edge feature map e(x);

[0055] S12, using the Metric3D real-time monocular depth estimation algorithm to calculate the image pixel-level depth value in the vehicle image data to obtain a depth feature map d(x);

[0056] S13, calculating the medium transmission coefficient in the vehicle image data based on the dark channel prior, and obtaining a medium transmission coefficient characteristic graph t(x);

[0057] S14, performing edge index analysis on the edge feature map e(x) based on a multi-dimensional index system to extract an index value e_fratures of edge mechanism characteristics; the edge index includes edge density, average gradient amplitude of non-zero pixels, sum of edge features, maximum value of edge features, average value of edge features, center of gravity of edge features and center of edge features;

[0058] S15, by analyzing the depth value range of the depth feature map d(x) in the data set, performing depth index analysis on the depth feature map d(x) based on a multi-dimensional index system to extract the index value d_features of the depth mechanism feature;

[0059] More specifically, the depth index includes a global depth index and a local depth index, wherein: the global depth index includes a global average depth, a global maximum depth, an entropy of a global depth map, and a global depth map density; the local depth index includes a local average depth, a local maximum depth, a local depth standard deviation, and an area ratio of a local fixed depth area;

[0060] The application logic of the global depth index and the local depth index is:

[0061] The depth feature map d(x) extracted from the vehicle image data is used to construct a global depth index for all regions of the depth feature map, and the index value d_features of the depth mechanism feature is extracted based on the global depth index;

[0062] A fixed depth threshold is set in advance, a fixed depth area in the vehicle image data is selected based on the fixed depth threshold, the depth feature map area where the fixed depth area is not selected is marked as a non-fixed depth area, a local depth index is constructed based on the non-fixed depth area, and an index value d_features of the depth mechanism feature is extracted based on the local depth index;

[0063] It should be noted that: in this embodiment, a depth value of 121 is set as a fixed threshold, and the depth value equal to 121 in each depth map is used as a fixed depth area, and the remaining positions are non-fixed depth areas. Then, four global depth indicators, including average depth, maximum depth, entropy of the depth map, and depth map density, are constructed based on all areas of the depth map. Four local depth indicators, including average depth, maximum depth, depth standard deviation, and area ratio of the fixed depth area, are constructed based on the non-fixed depth area to extract the indicator value of the depth mechanism feature;

[0064] S16, based on the divided areas of the image depth map in step S15, extract the index value t_features of the medium transmission coefficient mechanism characteristics from the medium transmission coefficient feature map;

[0065] More specifically, the medium transmission index includes a global medium transmission index and a local medium transmission index; the global medium transmission index includes a global average transmission coefficient, a global minimum transmission coefficient and a global transmission coefficient density; the local medium transmission index includes a local average transmission coefficient, a local minimum transmission coefficient and a local transmission coefficient standard deviation;

[0066] The global medium transmission index is used to extract the index value t_features of the medium transmission coefficient mechanism characteristics for all areas of the depth feature map;

[0067] For the non-fixed depth region of the depth feature map, the local medium transmission index is used to extract the index value t_features of the medium transmission coefficient mechanism characteristics.

[0068] More specifically, the edge feature is a visibility mechanism feature that can characterize the contours of objects in the vehicle-mounted image. Usually, there are more object contours in high-visibility scenes. The calculation formula based on the Sobel edge detection operator in step S11 includes:

[0069] e(x)=Sobel(I(x))

[0070] Where I is any vehicle image, x is a pixel on the image, Sobel(·) is an edge detection operator;

[0071] More specifically, the depth feature is a visibility mechanism feature that can characterize the distance from each pixel in the vehicle-mounted image to the camera. Usually, the distance is smaller in low visibility scenes. The calculation formula based on the Metric3D algorithm in step S12 includes:

[0072] d(x)=Metric3D(I(x))

[0073] Where I is any vehicle-mounted image, x is a pixel on the image, Metric3D(·) is a popular real-time monocular depth estimation algorithm;

[0074] More specifically, the medium transmission coefficient characteristic is a visibility mechanism characteristic, which is related to the turbidity of the air in the driving environment. The more turbid the air is (the lower the visibility is), the smaller the medium transmission coefficient is. The formula for calculating the medium transmission coefficient characteristic graph t(x) in step S13 includes:

[0075]

[0076] Wherein, I is any vehicle-mounted image, c is the RGB three color channels of the image; x is a local area on the image, and in this embodiment, the size is set to 15*15 pixels, and y is the pixel point in the x area; A is the average atmospheric brightness on the corresponding color channel, and in this embodiment, the average pixel brightness on the color channel corresponding to the position with the maximum depth value of 0.1% in the corresponding image d(x) is selected as the atmospheric brightness A value; ω (0<ω≤1) is a fixed parameter, and in this embodiment, ω=0.95 is set to simulate the aerial perspective phenomenon (aerial perspective).

[0077] S2. Build a convolutional neural network model, introduce cross-domain knowledge by fine-tuning the pre-trained model on a large dataset, and obtain the deep visibility features of vehicle images;

[0078] It should be noted that: In order to address the problem of limited low-visibility training data, cross-domain knowledge is introduced by fine-tuning the pre-trained model on a large dataset to enhance the model's feature extraction capability and accurately capture the deep features of low visibility.

[0079] Specifically, step S2 includes the following steps:

[0080] S21, use the full convolution layer of the standard MobileNetV2 model as the convolutional neural network model, load the pre-trained model on the COCO dataset as the initial weight, and extract the vehicle image features CNN_features after fine-tuning;

[0081] S22, set the average pooling layer window size to 7×7, step size S=1, input CNN_features to obtain visibility deep features deep_features, and the feature dimension is 1280 dimensions;

[0082] The average pooling layer helps smooth CNN_features and reduce the impact of noise by applying a sliding window (size F×F, step size S) on the feature map and selecting the average value in the window as the output. The specific calculation formula is as follows:

[0083]

[0084] Among them, deep_features i,j,k Represents the (i, j)th element of the deep feature on the kth channel; the feature dimension of CNN_features is h×w×c=7×7×1280 (h is the height, w is the width, c is the number of channels), i and j are the row and column indices of the feature, and k is the channel index;

[0085] S3. Establish a low-visibility driving risk identification model that integrates mechanism features and deep features, and use the self-attention mechanism to adaptively model the importance of features in different scenarios to improve the generalization ability of the model;

[0086] Specifically, step S3 includes the following steps:

[0087] S31. Establish four linear mapping layers Linear deep 、Linear e 、Linear d and Linear t , respectively map the three types of mechanism features e_fratures, d_features, t_features and deep features deep_features to 32 dimensions, and concatenate the features in the sequence dimension to obtain the sequence features seq_features;

[0088] S32. In this embodiment, a 2-layer Transformer encoder is used to construct a feature fusion module, and after assigning sequence feature position encoding information, a self-attention mechanism is used to adaptively model the feature importance in different scenarios;

[0089] S33, the mechanism feature calculation method in step S1, the convolutional neural network model in S2 and the linear mapping layer in S31 are used as feature extraction modules, and then combined with the feature fusion module, and finally the linear output layer Linear output The output module is established by using the Softmax function, and the low-visibility driving risk identification model FusionNet is established by integrating the mechanism features and deep features of vehicle-mounted images, such as Figure 2 shown.

[0090] The specific calculation formula of the identification model FusionNet is as follows:

[0091] FusionNet=Softmax(Linear output (Transformer_Encoder(seq_features+Positional_Encoding)),

[0092]

[0093] Among them, Transformer_Encoder is the encoder layer of Transformer, which includes batch normalization, self-attention mechanism and feedforward neural network, etc., to fuse multiple categories of features; Positional_Encoding is position encoding; Softmax function is used to map the output of the linear output layer (i={0,1}, j=2) into probability, where the class with the largest probability is the visibility recognition result.

[0094] S4. Set a focal loss function for the vehicle image data, assign a larger loss weight to low-visibility difficult-to-identify samples, and then update the model parameters through the stochastic gradient descent algorithm to obtain a low-visibility driving risk identification model.

[0095] It should be noted that: in order to address the imbalance problem of low\high visibility categories caused by limited low-visibility data, a focal loss function is set to suppress the influence of high-visibility and easy-to-identify samples, focus on low-visibility and difficult-to-identify samples, and improve the model recognition ability. Finally, the model parameters are updated through the stochastic gradient descent algorithm to obtain a low-visibility driving risk identification model.

[0096] Specifically, step S4 includes the following steps:

[0097] S41. In this embodiment, a vehicle image visibility dataset collected from 35 scenes is used, and the dataset is divided into a training set and a test set based on scenes according to the ratio of training set samples: test set samples = 3:1. Manual inspection is performed to ensure that the scenes in the training set and the test set are independent of each other. The data is shown in Table 1:

[0098] Table 1 Visibility dataset

[0099]

[0100] S42, preprocessing the input data of the recognition model FusionNet, wherein the vehicle-mounted images use image scaling and Z-Score normalization, and the mechanism features use maximum and minimum normalization;

[0101] S43, setting focus loss function l FL , and give a larger loss weight to low-visibility difficult-to-identify samples. The fusion model is iteratively optimized through the stochastic gradient descent algorithm. When the loss value converges, the training ends and the low-visibility driving risk identification model from the vehicle's perspective is obtained.

[0102] In this embodiment, all input images are scaled to the same size (224×224×3) by image scaling to ensure that the model can be applied to low-visibility images of different sizes collected by any vehicle-mounted camera. The specific calculation formula is as follows:

[0103] I H',W',C =Resize(I H,W,C ,(H',W'))

[0104] Where H, W and H'=224, W'=224 are the original height, width and height and width after scaling of the image, respectively; C is the RGB three channels of the image; I H,W,C is any vehicle image, I H',W',C is the image scaling result; Resize(·) is the image scaling operation, and the linear difference method is adopted in this embodiment.

[0105] Z-Score normalization is a commonly used image preprocessing technology. It scales the vehicle image to a standard normal distribution on the three RGB channels, which can accelerate model convergence and improve performance. The specific calculation formula is as follows:

[0106]

[0107] Among them, x i is the input on channel i, is the standardized output; in this embodiment, the mean value μ of the brightness value on the RGB channel is set i ∈{0.471,0.448,0.408}, the average value of the standard deviation of the brightness value σ i ∈{0.234,0.239,0.242}.

[0108] Maximum and minimum normalization is to use the maximum and minimum values ​​to standardize the value of each mechanism characteristic index. The standardized value is between [0,1], which can reduce the scale difference between the indicators. The specific calculation formula is as follows:

[0109]

[0110] Among them, x belongs to the three types of mechanism characteristic index values ​​calculated in step S2, x max and x min is the maximum and minimum value of the indicator in all data; x i and x′ i is the eigenvalue of the i-th input x and the normalized output value.

[0111] In this embodiment, the ratio of positive and negative samples, i.e., low-visibility samples: high-visibility samples, is equal to 1:3. Therefore, in order to solve the problem of imbalance in the number of positive and negative samples, the focal loss function l is used. FL , the specific calculation formula is as follows:

[0112] l FL =-α c (1-p c ) γ log(pc )

[0113] Among them, γ is a hyperparameter, which represents the degree of loss attenuation of easy-to-classify samples and is set to 2 in this embodiment; α c represents the weight of the c-th type of sample. In this embodiment, the weight of the low-visibility sample is set to 2, and the weight of the high-visibility sample is set to 1; p c Represents the probability value of the cth category of Softmax output; since the dataset only contains two types of samples, high visibility and low visibility, c takes the value of 0 and 1.

[0114] The stochastic gradient descent algorithm is a parameter optimization algorithm. The specific calculation formula is as follows:

[0115]

[0116] Among them, θ is the identification model parameter; t is the current iteration number; η is the learning rate, which controls the size of the step; is the loss value of the ith data point or data batch.

[0117] In this implementation, if Figure 3 As shown in the figure, after 20 epochs of iterative training, the low-visibility driving risk identification model that integrates mechanism features and deep features has a continuous decrease in model training loss and model test loss, and finally reaches convergence. At the same time, the prediction accuracy Precision and recall rate Recall of low-visibility samples on the test set, as well as the F0.5score that emphasizes the importance of accurate low-visibility identification, are used as model evaluation indicators, such as Figure 4 As shown in the figure, with the increase of the number of training iterations, the three indicators finally converged and achieved excellent performance.

[0118] In this embodiment, the popular MobileNetV2 model is selected as the comparison model for generalization performance. The test results on the test set are shown in Table 2. The present invention is superior to the comparison model in all three test indicators, indicating that the low-visibility driving risk identification model that integrates mechanism features and deep features proposed in the present invention has stronger generalization ability in unknown scenarios.

[0119] Table 2 Comparison of model generalization performance

[0120]

[0121] like Figure 5As shown, in this embodiment, time-varying volatility is used to evaluate the stability of the model prediction performance. In the test video, the prediction results of the present invention have fewer mutations, and the time-varying volatility is only 0.131, which is 64.9% lower than the time-varying volatility of 0.373 of the comparison model, indicating that the present invention has a more stable low-visibility driving risk identification capability under the vehicle-mounted perspective.

[0122] In addition, it should be noted that the specific embodiments described in this specification may be named differently, and the above content described in this specification is only an example of the structure of the present invention. All equivalent changes or simple changes made based on the structure, features and principles of the present invention are included in the protection scope of the present invention. Those skilled in the art of the present invention may make various modifications or supplements to the specific examples described or adopt similar methods, as long as they do not deviate from the structure of the present invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

[0123] Embodiment 2: This embodiment provides a computer-readable storage medium storing instructions. When the instructions are executed on a computer, the computer executes the low-visibility driving risk identification method based on the above-mentioned fusion mechanism and deep image features.

[0124] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including at least one computer program, and the at least one computer program is executed by a processor to complete the low-visibility driving risk identification method of the fusion mechanism and the deep features of the image in the above embodiment. For example, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0125] In an exemplary embodiment, a computer program product or a computer program is also provided, the computer program product or the computer program includes one or more program codes, and the one or more program codes are stored in a computer-readable storage medium. One or more processors of an electronic device can read the one or more program codes from the computer-readable storage medium, and the one or more processors execute the one or more program codes, so that the electronic device can execute the low-visibility driving risk identification method based on the above-mentioned fusion mechanism and deep image features.

[0126] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0127] It should be understood that determining B based on A does not mean determining B only based on A. B can also be determined based on A and / or other information.

[0128] A person of ordinary skill in the art will understand that all or part of the steps to implement the above embodiments can be completed by hardware, or can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. The above-mentioned storage medium can be a read-only memory, a disk or an optical disk, etc.

[0129] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

[0130] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only one, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0131] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0132] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0133] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0134] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A low-visibility driving risk identification method based on the fusion mechanism and deep image features, characterized in that: The following steps are involved: S1. Construct a multi-dimensional visibility mechanism characteristic index system, and extract the index values ​​of edge characteristics, depth characteristics and medium transmission coefficient characteristics based on vehicle-mounted image data; S2. Build a convolutional neural network model, introduce cross-domain knowledge by fine-tuning the pre-trained model on a large dataset, and obtain the deep visibility features of vehicle images; S3. Establish a low-visibility driving risk identification model that integrates mechanism features and deep features, and use the self-attention mechanism to adaptively allocate the importance of features in different scenarios; S4. Set a focal loss function for the vehicle image data, assign a larger loss weight to low-visibility difficult-to-identify samples, and then update the model parameters through the stochastic gradient descent algorithm to obtain a low-visibility driving risk identification model.

2. The low-visibility driving risk identification method based on the fusion mechanism and deep image features according to claim 1 is characterized in that: The step S1 specifically includes the following steps: S11, using an edge detection operator to calculate the gradient of pixel intensity in the vehicle-mounted image to identify edge information and obtain an edge feature map; S12, using a monocular depth estimation algorithm, calculating the pixel-level depth value of the vehicle image to obtain a depth feature map; S13, based on the dark channel prior, calculating the medium transmission coefficient of the vehicle-mounted image to obtain a medium transmission coefficient characteristic map; S14, performing edge index analysis on the edge feature map based on a multi-dimensional index system to extract index values ​​of edge mechanism features; the edge indexes include edge density, average gradient amplitude of non-zero pixels, sum of edge features, maximum value of edge features, average value of edge features, center of gravity of edge features and center of edge features; S15, by analyzing the depth value range of the depth feature map in the data set, performing depth index analysis on the depth feature map based on a multi-dimensional index system to extract the index value of the depth mechanism feature; setting a depth fixed threshold in advance, selecting a fixed depth area in the vehicle-mounted image data based on the depth fixed threshold, and marking the depth feature map area where the fixed depth area is not selected as a non-fixed depth area; S16, based on the divided areas of the image depth map in step S15, extract the index value t_features of the medium transmission coefficient mechanism characteristics from the medium transmission coefficient feature map; 3. The low-visibility driving risk identification method based on the fusion mechanism and deep image features according to claim 2 is characterized in that: The depth index includes a global depth index and a local depth index. The application logic of the global depth index and the local depth index is: The depth feature map extracted from the vehicle image data, constructing a global depth index for all areas of the depth feature map, and extracting the index value of the depth mechanism feature based on the global depth index; A fixed depth threshold is set in advance, and a fixed depth area in the vehicle image data is selected based on the fixed depth threshold. The depth feature map area where the fixed depth area is not selected is marked as a non-fixed depth area. A local depth index is constructed based on the non-fixed depth area, and the index value of the depth mechanism feature is extracted based on the local depth index.

4. The low-visibility driving risk identification method based on the fusion mechanism and deep image features according to claim 2 is characterized in that: The calculation formula based on the Sobel edge detection operator used in step S11 includes: e(x)=Edge(I(x)) Where I is any vehicle-mounted image, x is a pixel on the image; Edge(·) is an edge detection operator; 5. The low-visibility driving risk identification method based on the fusion mechanism and deep image features according to claim 2 is characterized in that: The calculation formula based on the Metric3D algorithm used in step S12 includes: d(x)=Depth(I(x)) Where I is any vehicle-mounted image, x is a pixel on the image; Depth(·) is a monocular depth estimation algorithm; 6. The low-visibility driving risk identification method based on the fusion mechanism and deep image features according to claim 5 is characterized in that: The formula for calculating the medium transmission coefficient characteristic diagram in step S13 includes: Among them, I is an arbitrary vehicle-mounted image, c is the RGB three color channels of the image; x is a certain area on the image, y is the pixel point in the x area; A is the average brightness of the atmosphere on the corresponding color channel; ω (0<ω≤1) is a fixed parameter.

7. The low-visibility driving risk identification method based on the fusion mechanism and deep image features according to claim 2 is characterized in that: The step S2 specifically includes the following steps: S21. Build a convolutional neural network, load the pre-trained model on a large image classification dataset as the initial weight, and extract the vehicle image features after fine-tuning; S22, inputting the vehicle image features into the average pooling layer to obtain the visibility deep features; 8. The low-visibility driving risk identification method based on the fusion mechanism and deep image features according to claim 7 is characterized in that: The step S3 comprises the following steps: S31, establish four linear mapping layers, respectively map edge features, depth features, medium transmission coefficient features and deep features to the same dimension, and perform feature concatenation in the sequence dimension to obtain sequence features; S32, build a feature fusion module based on the Transformer encoder, and use the self-attention mechanism to adaptively model the feature importance in different scenarios after assigning sequence feature position encoding information; S33. Use the mechanism feature calculation method in step S1, the convolutional neural network model in S2 and the linear mapping layer in S31 as a feature extraction module, and then combine the feature fusion module. Finally, use the linear output layer and the Softmax function to establish an output module to establish a low-visibility driving risk identification model that integrates the mechanism features and deep features of the vehicle-mounted image.

9. The low-visibility driving risk identification method based on the fusion mechanism and deep image features according to claim 8 is characterized in that: The step S4 comprises the following steps: S41, dividing the vehicle image visibility dataset into a training set and a test set based on scenes, and ensuring that the scenes in the training set and the test set are independent of each other through manual inspection; S42, preprocessing the data, wherein the vehicle-mounted images are scaled and normalized, and the mechanism characteristic index values ​​are normalized using maximum and minimum; S43. Set the focal loss function and assign a larger loss weight to low-visibility difficult-to-identify samples. Then iteratively optimize the identification model through the stochastic gradient descent algorithm. When the loss value converges, the training ends and the low-visibility driving risk identification model is obtained.

10. A computer-readable storage medium storing instructions, which, when executed on a computer, enables the computer to execute the low-visibility driving risk identification method based on the fusion mechanism and deep image features as described in any one of claims 1 to 9.

Images