A driving scenario risk assessment method and system

Abstract

The present invention provides a driving scenario risk assessment method and system. The method includes: extracting features from the input original image through an image feature extraction module, and extracting features from the input 3D point cloud through a 3D point cloud feature extraction module to respectively generate corresponding image features and 3D point cloud features; performing feature fusion processing on the image features and 3D point cloud features through a feature fusion module based on a cross-attention algorithm, and making the features from different domains obey approximately the same distribution in the feature space through a central vector constraint module to correspondingly generate several different categories of samples; adding an attachment angle margin penalty in the risk level assessment module to form corresponding angle intervals between several different categories of samples, and evaluating the corresponding risk level according to the size of the angle intervals. The present invention can objectively and accurately evaluate the corresponding driving scenario risk level, correspondingly improving the user experience.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving, and particularly relates to a driving scenario risk assessment method and system. Background Art

[0002] With the progress of technology and the rapid development of productivity, automobiles have been popularized in people's daily lives and have become one of the essential means of transportation for people's daily travel, greatly facilitating people's lives.

[0003] Among them, the autonomous driving technology of automobiles has also achieved rapid development in recent years. In the process of actual application, the dynamic uncertainty of complex driving scenarios is an important factor affecting the safety and reliability of autonomous driving. Therefore, existing autonomous driving technologies must be able to accurately perceive the surrounding environment and make reasonable decisions under complex and changeable traffic conditions.

[0004] Furthermore, existing driving scenario risk assessment methods mainly rely on risk modeling or deep learning methods. However, methods relying on risk modeling require experts to define corresponding risk indicators for assessment. Although this method has a certain degree of interpretability, it is difficult to adapt to the changes in complex dynamic scenarios, and at the same time, its generalization ability is limited. In addition, methods relying on deep learning require training a neural network model and learning implicit risk features from multi-modal sensor data. However, due to the relatively single sensor data obtained, the accuracy of the assessment results is correspondingly reduced. Therefore, in view of the deficiencies of the existing technology, it is necessary to provide a method that can comprehensively and accurately capture driving environment information and objectively and accurately complete driving scenario risk assessment. Summary of the Invention

[0005] Based on this, the purpose of the present invention is to provide a driving scenario risk assessment method and system to provide a method that can comprehensively and accurately capture driving environment information and objectively and accurately complete driving scenario risk assessment.

[0006] The first aspect of the embodiment of the present invention proposes:

[0007] A driving scenario risk assessment method, which is applied to an image feature extraction module, a 3D point cloud feature extraction module, a feature fusion module, a center vector constraint module, and a risk level assessment module. The method includes:

[0008] Performing feature extraction on the input original image through the image feature extraction module, and performing feature extraction on the input 3D point cloud through the 3D point cloud feature extraction module to respectively generate corresponding image features and 3D point cloud features;

[0009] Based on the cross-attention algorithm, the feature fusion module performs feature fusion processing on the image features and the 3D point cloud features, and the center vector constraint module makes the features from different domains follow approximately the same distribution in the feature space to correspondingly generate several different categories of samples;

[0010] In the risk level evaluation module, an attachment angle margin penalty is added to form a corresponding angle interval between several different categories of samples, and the corresponding risk level is evaluated according to the size of the angle interval.

[0011] The beneficial effects of the present invention are as follows: Through the cross-attention mechanism, the deep fusion of images and 3D point cloud features can be effectively achieved, thereby effectively eliminating the spatio-temporal alignment error. Based on this, by using the center vector constraint module for feature distribution alignment, the adaptability of the model to different driving scenario domains can be significantly improved. In addition, by introducing an additional angle margin penalty mechanism, the decision boundary of risk categories can be correspondingly expanded, fundamentally improving the discrimination accuracy of risk assessment, and thus being able to objectively and accurately evaluate the driving scenario risk level and correspondingly enhancing the user experience.

[0012] Further, the steps of the feature fusion module performing feature fusion processing on the image features and the 3D point cloud features based on the cross-attention algorithm include:

[0013] Obtain a training set, which contains a number of training samples, and each training sample respectively includes a driving scenario image and corresponding 3D point cloud data;

[0014] Extract the image feature vector in the driving scenario image through the image feature extraction module, and extract the point cloud feature vector in the 3D point cloud data through the 3D point cloud feature extraction module;

[0015] Based on the cross-attention algorithm, the feature fusion module performs fusion processing on the image feature vector and the point cloud feature vector.

[0016] Further, the steps of the center vector constraint module making the features from different domains follow approximately the same distribution in the feature space to correspondingly generate several different categories of samples include:

[0017] Randomly add a corresponding center vector to each training sample through the center vector constraint module;

[0018] By minimizing the center loss, the same category of samples in multiple different scenarios converge to the same region in the feature space;

[0019] During each training iteration, control the center vector to update in the direction of reducing the center loss at a preset learning rate, so as to correspondingly generate a number of samples of the different categories.

[0020] Further, the expression of the algorithm for minimizing the center loss is:

[0021]

[0022] Where, C represents the number of categories included, B represents the batch size during the training process, F i represents the feature vector of the i -th sample in a batch of samples, ci represents the category of the i -th sample, cen ci represents the center vector of the samples with the category of ci , L cen represents the center loss.

[0023] Further, the expression of the algorithm for controlling the center vector to update in the direction of reducing the center loss at a preset learning rate is:

[0024]

[0025] Where, , and ∆cen j t represents the average value of the distances between the samples with the category of t in a batch during the j -th iteration and the corresponding center vector, cen j t and cen j t＋1 respectively represent the center vectors before and after update during the t -th iteration, α represents the learning rate of the center vector update process, c i represents the category of the i -th sample, F i represents the i -th sample in the samples, and B represents the batch size during the training process.

[0026] Further, the step of adding an attachment angle margin penalty in the risk level evaluation module to form corresponding angle intervals between several samples of different categories includes:

[0027] Mapping the feature vectors of the training samples and the output weights of the risk level evaluation module to a preset high-dimensional hypersphere through a preset normalization algorithm;

[0028] Outputting a weight vector corresponding to the feature vector through the risk level evaluation module, and adding a corresponding angle interval penalty term to the angle between the feature vector and its corresponding weight vector, so as to correspondingly obtain the angle intervals formed between several samples of different categories.

[0029] Further, the expression of the preset normalization algorithm is:

[0030]

[0031] where F i represents the feature vector of the i th sample in a batch, f i represents the feature vector obtained after standardization by F i , B represents the size of the batch, W j represents the weight vector of the risk level evaluation module corresponding to the sample with a risk level of j , w i represents the weight vector obtained after normalization by W j , and C represents the number of categories included.

[0032] The second aspect of the embodiment of the present invention proposes:

[0033] A driving scenario risk assessment system, which is applied to an image feature extraction module, a 3D point cloud feature extraction module, a feature fusion module, a central vector constraint module, and a risk level evaluation module. The system includes:

[0034] An extraction module, configured to extract features from the input original image through the image feature extraction module, and extract features from the input 3D point cloud through the 3D point cloud feature extraction module, so as to generate corresponding image features and 3D point cloud features respectively;

[0035] A processing module, configured to perform feature fusion processing on the image features and the 3D point cloud features through the feature fusion module based on the cross-attention algorithm, and make the features from different domains follow approximately the same distribution in the feature space through the central vector constraint module, so as to correspondingly generate several different categories of samples;

[0036] An evaluation module, configured to add an attachment angle margin penalty in the risk level evaluation module, so as to form a corresponding angle interval between several different categories of samples, and evaluate the corresponding risk level according to the size of the angle interval.

[0037] Further, the processing module is specifically configured to:

[0038] Obtain a training set, where the training set contains several training samples, and each training sample respectively includes a driving scene image and corresponding 3D point cloud data;

[0039] Extract an image feature vector from the driving scene image through the image feature extraction module, and extract a point cloud feature vector from the 3D point cloud data through the 3D point cloud feature extraction module;

[0040] Perform fusion processing on the image feature vector and the point cloud feature vector through the feature fusion module based on the cross-attention algorithm.

[0041] Further, the processing module is specifically configured to:

[0042] Randomly add a corresponding central vector to each training sample through the central vector constraint module;

[0043] Minimize the central loss, so that the same category of samples in multiple different scenarios converge to the same region in the feature space;

[0044] During each training iteration, control the central vector to update in the direction of reducing the central loss at a preset learning rate, so as to correspondingly generate several different categories of samples.

[0045] Further, the expression of the algorithm for minimizing the central loss is:

[0046]

[0047] Among them, C represents the number of categories included, B represents the batch size during the training process, F i represents the feature vector of the i th sample in a batch of samples, ci represents the iThe category of a sample, cen ci indicating that the category is ci the central vector of the sample, L cen representing the said center loss.

[0048] Furthermore, the expression of the algorithm for controlling the central vector to be updated in the direction of reducing the center loss at a preset learning rate is:

[0049]

[0050] wherein, , and ∆cen j t representing in the process of the t th iteration, the average value of the distances between the samples of the category j in a batch and the corresponding central vector, cen j t and cen j t＋1 respectively represent the central vectors before and after update in the process of the t th iteration, α represents the learning rate of the central vector update process, c i representing the i th category of the sample, F i representing the i th feature vector of the sample in the sample, and B represents the batch size in the training process.

[0051] Furthermore, the evaluation module is specifically used for:

[0052] mapping the feature vectors of the training samples and the output weights of the risk level evaluation module to a preset high-dimensional hypersphere through a preset normalization algorithm;

[0053] outputting a weight vector corresponding to the feature vector through the risk level evaluation module, and adding a corresponding angular interval penalty term to the angle between the feature vector and its corresponding weight vector to correspondingly obtain the angular intervals formed between several samples of different categories.

[0054] Furthermore, the expression of the preset normalization algorithm is:

[0055]

[0056] where F i representing thei The feature vector of a sample f i denotes the feature vector obtained after being standardized by F i The batch size is denoted by B denotes the batch size W j denotes the weight vector of the risk level evaluation module corresponding to the sample with a risk level of j The weight vector of the risk level evaluation module corresponding to the sample with a risk level of w i denotes the weight vector obtained after being normalized by W j C represents the number of categories included. The weight vector obtained after being normalized by

[0057] In the third aspect of the embodiments of the present invention, it is proposed that

[0058] A computer includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the driving scenario risk assessment method described above is implemented

[0059] In the fourth aspect of the embodiments of the present invention, it is proposed that

[0060] A readable storage medium stores a computer program. When the program is executed by a processor, the driving scenario risk assessment method described above is implemented

[0061] The additional aspects and advantages of the present invention will be partially given in the following description, partially become apparent from the following description, or be understood through the practice of the present invention BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 is a flowchart of the driving scenario risk assessment method provided by the first embodiment of the present invention

[0063] Figure 2 is a schematic structural diagram of the driving scenario risk assessment model in the driving scenario risk assessment method provided by the second embodiment of the present invention

[0064] Figure 3 is a schematic structural diagram of the 3D point cloud feature fusion module in the driving scenario risk assessment method provided by the second embodiment of the present invention

[0065] Figure 4 is a schematic structural diagram of the risk assessment module in the driving scenario risk assessment method provided by the second embodiment of the present invention

[0066] Figure 5 is a schematic structural diagram of the actual application in the driving scenario risk assessment method provided by the second embodiment of the present invention

[0067] Figure 6 It is a structural block diagram of a driving scenario risk assessment system provided for the third embodiment of the present invention.

[0068] The following specific embodiments will further illustrate the present invention in conjunction with the above-mentioned drawings. Specific Embodiments

[0069] To facilitate the understanding of the present invention, the present invention will be described more comprehensively below with reference to the relevant drawings. Several embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, these embodiments are provided to make the disclosure of the present invention more thorough and comprehensive.

[0070] It should be noted that when an element is referred to as "fixedly provided on" another element, it can be directly on the other element or there may also be an intermediate element. When an element is considered to be "connected" to another element, it can be directly connected to the other element or there may be an intermediate element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are only for the purpose of illustration.

[0071] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0072] Please refer to Figure 1 , which shows a driving scenario risk assessment method provided for the first embodiment of the present invention. The driving scenario risk assessment method provided in this embodiment can objectively, accurately and comprehensively complete the driving scenario risk assessment, correspondingly improving the user experience.

[0073] Specifically, this embodiment provides:

[0074] A driving scenario risk assessment method, which is applied to an image feature extraction module, a 3D point cloud feature extraction module, a feature fusion module, a central vector constraint module and a risk level assessment module. The method includes:

[0075] Step S10, extracting features from the input original image through the image feature extraction module, and extracting features from the input 3D point cloud through the 3D point cloud feature extraction module to respectively generate corresponding image features and 3D point cloud features;

[0076] Step S20: Based on the cross-attention algorithm, perform feature fusion processing on the image features and the 3D point cloud features through the feature fusion module, and make the features from different domains follow approximately the same distribution in the feature space through the central vector constraint module to correspondingly generate several different categories of samples;

[0077] Step S30: Add an attachment angle margin penalty in the risk level assessment module to form a corresponding angle interval between several different categories of samples, and evaluate the corresponding risk level according to the size of the angle interval.

[0078] Second Embodiment

[0079] Further, the step of performing feature fusion processing on the image features and the 3D point cloud features through the feature fusion module based on the cross-attention algorithm includes:

[0080] Obtain a training set, which contains several training samples, and each training sample respectively includes a driving scene image and corresponding 3D point cloud data;

[0081] Extract the image feature vector in the driving scene image through the image feature extraction module, and extract the point cloud feature vector in the 3D point cloud data through the 3D point cloud feature extraction module;

[0082] Based on the cross-attention algorithm, perform fusion processing on the image feature vector and the point cloud feature vector through the feature fusion module.

[0083] Further, the step of making the features from different domains follow approximately the same distribution in the feature space through the central vector constraint module to correspondingly generate several different categories of samples includes:

[0084] Randomly add a corresponding central vector to each training sample through the central vector constraint module;

[0085] Minimize the central loss to make the same category of samples in multiple different scenarios converge to the same region in the feature space;

[0086] During each training iteration, control the central vector to update in the direction of reducing the central loss at a preset learning rate to correspondingly generate several different categories of samples.

[0087] Further, the expression of the algorithm for minimizing the central loss is:

[0088]

[0089] where, CIndicates the number of included categories, B Indicates the batch size during the training process, F i Indicates the i th feature vector of a sample in a batch, ci Indicates the i th category of a sample, cen ci Indicates that the category is ci center vector of the sample, L cen Indicates the said center loss.

[0090] Furthermore, the expression of the algorithm for controlling the update of the center vector in the direction of reducing the center loss at a preset learning rate is:

[0091]

[0092] Wherein, , and ∆cen j t Indicates the average value of the distances between the samples of category t in a batch and the corresponding center vector during the j th iteration, cen j t and cen j t＋1 respectively indicate the center vectors before and after update during the t th iteration process, α Indicates the learning rate of the center vector update process, c i Indicates the i th category of a sample, F i Indicates the i th feature vector of a sample in the sample, and B indicates the batch size during the training process.

[0093] Furthermore, the steps of adding an attachment angle margin penalty in the risk level evaluation module to form corresponding angular intervals between several samples of different categories include:

[0094] Mapping the feature vectors of the training samples and the output weights of the risk level evaluation module to a preset high-dimensional hypersphere through a preset normalization algorithm;

[0095] Output a weight vector corresponding to the feature vector through the risk level evaluation module, and add a corresponding angular interval penalty term to the angle between the feature vector and its corresponding weight vector, so as to correspondingly obtain the angular intervals formed between several samples of different categories.

[0096] Further, the expression of the preset normalization algorithm is:

[0097]

[0098] Where F i represents the feature vector of the i th sample in a batch, f i represents the feature vector obtained after standardization by F i , B represents the size of the batch, W j represents the weight vector of the risk level evaluation module corresponding to the sample with a risk level of j , w i represents the weight vector obtained after normalization by W j , and C represents the number of categories included.

[0099] In addition, in this embodiment, it should be noted that the present invention constructs a driving scenario risk assessment model based on the fusion of image and 3D point cloud data. Its network structure is as Figure 2 shown, including an image feature extraction module, a 3D point cloud feature extraction module, a feature fusion module, a central vector constraint module, and a driving scenario risk level evaluation module with an additional angular margin penalty. First, the input image and 3D point cloud are respectively subjected to feature extraction by the image feature extraction module and the 3D point cloud feature extraction module, and the image features and 3D point cloud features are fused through a cross-attention mechanism.

[0100] Before training the model, a dataset needs to be created to evaluate the accuracy and domain generalization ability of the proposed driving scenario risk assessment model based on the fusion of images and 3D point clouds. During data collection, images and 3D point cloud data of various different types of scenarios, such as urban scenarios, rural scenarios, highway scenarios, and night scenarios, are collected respectively to construct the dataset. During data collation, the images and point cloud data at the same moment are put together to construct an input sample, and the data from different scenarios are stored in different folders respectively. Then, a data annotation system is developed, and multiple drivers from different regions are invited to annotate the risk levels of the samples in this dataset. In this example, the risks of driving scenarios are divided into four levels: "very high risk", "relatively high risk", "relatively low risk", and "very low risk".

[0101] During training, first, the data from different scenarios are mixed together as the training set of the model. Each time a batch of samples is read in, and each training sample contains an image of a driving scenario and the corresponding 3D point cloud data. In this example, a batch of samples contains 24 samples in total.

[0102] Then, the image in the sample is feature-extracted through the image feature extraction module to obtain the image feature vector of the driving scenario. In this example, we use the image encoder of the CLIP model as the image feature extraction module.

[0103] Meanwhile, the 3D point cloud data in the sample is feature-extracted through the 3D point cloud feature extraction module to obtain the point cloud feature vector of the driving scenario. Since compared with traditional 3D point cloud feature extraction methods (such as PointNet, PointNet++, DGCNN, etc.), Point Transformer significantly improves the 3D point cloud feature extraction ability by introducing the self-attention mechanism of Transformer, and has advantages such as global relationship modeling, dynamic feature aggregation, multi-scale feature extraction, and robustness to disorder, so Point Transformer is used to extract features from the 3D point cloud in this example.

[0104] Next, the image feature vector and the 3D point cloud feature vector of the driving scenario are simultaneously input into the cross-attention-based feature fusion module to achieve the fusion of images and 3D point cloud data through cross-attention. In this example, the structure of the cross-attention-based feature fusion module is as Figure 3 shown, where the number of heads of the multi-head attention mechanism is 4, and the dimension of the feature embedding is 128. In addition, q represents the length dimension of the image, v represents the width dimension of the image, and k represents the pixel value of the image.

[0105] By the constraint of the center loss, samples from different domains are made to follow approximately the same distribution in the feature space, thereby overcoming the domain shift problem and improving the performance of the algorithm in actual application scenarios;

[0106] This method first randomly initializes a center vector for samples of each category.

[0107] Next, during the training process of the model, by minimizing the center loss, samples of the same category from multiple different scenarios are prompted to converge to the same region in the feature space. The center loss is defined as the average Euclidean distance between the feature vector and its corresponding center vector, and its calculation formula is as follows:

[0108]

[0109] Among them, C represents the number of categories included, B represents the batch size during the training process, F i represents the i th sample's feature vector in a batch of samples, ci represents the i th sample's category, cen ci represents the center vector of samples with the category of ci ; L cen represents the said center loss;

[0110] To make the center vector better reflect the distribution of the corresponding category samples, in each training iteration, the center vector will be updated in the direction of reducing the center loss at the learning rate α . The update process of the center vector is as follows:

[0111]

[0112] Among them, , and ∆cen j t represents the average value of the distances between samples of the category of t and the corresponding center vector in a batch during the j th iteration, cen j t and cen j t＋1 respectively represent the center vectors before and after update during the t th iteration, α represents the learning rate of the center vector update process, c i represents thei The category of a sample F i denotes the i th sample's feature vector in the sample, and B represents the batch size during the training process. In this example, α the value of

[0113] Finally, by adding an additional angular margin penalty in the risk level evaluation module, a larger angular separation is promoted between samples of different categories, thereby enhancing the robustness of the model and its adaptability to new scenarios. In this example, the specific process with the additional angular margin penalty is as Figure 4 shown

[0114] First, the feature vectors of the input samples and the weights of the risk level evaluation module are mapped onto a high-dimensional hypersphere through normalization. The normalization process is as follows:

[0115]

[0116] where F i denotes the i th sample's feature vector in a batch, f i denotes the feature vector obtained after standardizing by F i , B denotes the size of the batch, W j denotes the weight vector of the risk level evaluation module corresponding to the sample with a risk level of j , w i denotes the weight vector obtained after normalizing by W j , and C represents the number of categories included.

[0117] Next, in order to increase the angular separation between the feature vectors of samples of different categories on the feature hypersphere, an angular separation penalty term m is added to the angle between the feature vector and its corresponding weight vector. The calculation formula of the loss function after adding the additional angular margin penalty on the basis of the Softmax loss is:

[0118]

[0119] where, B denotes the size of the batch, m denotes the angular separation penalty term, C denotes the number of categories, s denotes a constant, θ denotes the additional angle, eRepresents a vector, y i Represents a margin, L AAMP Represents the finally calculated loss function.

[0120] Finally, during the training process of the model, the linear combination of the center loss and the Softmax loss with an additional angular margin penalty is used as the total loss function Loss of the model, and its calculation formula is as follows:

[0121] LOSS = λ·L cen +L AAMP

[0122] Where λ is the weight of the center loss in the total optimization objective function. In this example, the value of λ is 1.

[0123] Please refer to Figure 5 , in the actual application process, first, the images and 3D point cloud data of the driving scene are collected in real time through the camera and lidar, and then the collected images and 3D point cloud data are respectively subjected to feature extraction through the image feature extraction module and the 3D point cloud feature extraction module. Then, the obtained image feature vectors and 3D point cloud feature vectors are input into the feature fusion module based on cross-attention for feature fusion. Finally, the fused features are input into the risk level evaluation module with an additional angular margin penalty to realize the risk level evaluation of the current scene.

[0124] Please refer to Figure 6 , the third embodiment of the present invention provides:

[0125] A driving scene risk assessment system, which is applied to an image feature extraction module, a 3D point cloud feature extraction module, a feature fusion module, a center vector constraint module, and a risk level evaluation module. The system includes:

[0126] An extraction module for extracting features from the input original image through the image feature extraction module and extracting features from the input 3D point cloud through the 3D point cloud feature extraction module to respectively generate corresponding image features and 3D point cloud features;

[0127] A processing module for performing feature fusion processing on the image features and the 3D point cloud features through the feature fusion module based on the cross-attention algorithm, and making the features from different domains follow approximately the same distribution in the feature space through the center vector constraint module to correspondingly generate several different categories of samples;

[0128] An evaluation module, configured to add an attachment angle margin penalty in the risk level evaluation module, so as to form corresponding angle intervals between several different categories of samples, and evaluate corresponding risk levels according to the sizes of the angle intervals.

[0129] Further, the processing module is specifically configured to:

[0130] Obtain a training set, where the training set contains a number of training samples, and each training sample respectively includes a driving scene image and corresponding 3D point cloud data;

[0131] Extract an image feature vector from the driving scene image through the image feature extraction module, and extract a point cloud feature vector from the 3D point cloud data through the 3D point cloud feature extraction module;

[0132] Fuse the image feature vector and the point cloud feature vector through the feature fusion module based on the cross-attention algorithm.

[0133] Further, the processing module is specifically configured to:

[0134] Randomly add a corresponding center vector to each training sample through the center vector constraint module;

[0135] Minimize the center loss so that samples of the same category in multiple different scenes converge to the same region in the feature space;

[0136] During each training iteration, control the center vector to update in the direction of reducing the center loss at a preset learning rate to correspondingly generate several different categories of samples.

[0137] Further, the expression of the algorithm for minimizing the center loss is:

[0138]

[0139] Wherein, C represents the number of categories included, B represents the batch size during the training process, F i represents the feature vector of the i th sample in a batch of samples, ci represents the category of the i th sample, cen ci represents the center vector of samples with the category of ci , L cen represents the center loss.

[0140] Further, the expression of the algorithm for controlling the update of the central vector in the direction of reducing the central loss at a preset learning rate is as follows:

[0141]

[0142] where , and ∆cen j t represents the average distance between the samples of class j in a batch and the corresponding central vector during the t-th iteration, cen j t and cen j t＋1 represent the central vectors before and after update during the t-th iteration respectively, α represents the learning rate in the process of updating the central vector, c i represents the class of the i-th sample, F i represents the feature vector of the i-th sample in the sample, and B represents the batch size during the training process.

[0143] Further, the evaluation module is specifically used for:

[0144] mapping both the feature vectors of the training samples and the output weights of the risk level evaluation module to a preset high-dimensional hypersphere through a preset normalization algorithm;

[0145] outputting a weight vector corresponding to the feature vector through the risk level evaluation module, and adding a corresponding angular margin penalty term to the angle between the feature vector and its corresponding weight vector to correspondingly obtain the angular margins formed between several samples of different classes.

[0146] Further, the expression of the preset normalization algorithm is:

[0147]

[0148] where F i represents the feature vector of the i-th sample in a batch, f i represents the feature vector obtained after normalizing F i , B represents the batch size, W j represents the weight vector of the risk level evaluation module corresponding to the samples with risk level j, w i represents the weight vector obtained after normalizing W j , and C represents the number of classes included.

[0149] The fourth embodiment of the present invention provides a computer, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the driving scenario risk assessment method described above is implemented.

[0150] The fifth embodiment of the present invention provides a readable storage medium, on which a computer program is stored. When the program is executed by a processor, the driving scenario risk assessment method described above is implemented.

[0151] In summary, the driving scenario risk assessment method and system provided by the above embodiments of the present invention can objectively, accurately, and comprehensively complete the driving scenario risk assessment, correspondingly improving the user experience.

[0152] It should be noted that the above-mentioned each module can be a functional module or a program module, and can be implemented either by software or by hardware. For the modules implemented by hardware, the above-mentioned each module can be located in the same processor; or the above-mentioned each module can also be located in different processors in any combined form.

[0153] The logic and / or steps represented in the flowchart or described in other ways herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or in combination with these instruction execution systems, apparatus, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

[0154] More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, then editing, interpreting, or otherwise processing it as appropriate, and then storing it in a computer memory.

[0155] It should be understood that each part of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0156] In the description of this specification, the description referring to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0157] The above-described embodiments merely represent several implementation manners of the present invention. Their descriptions are relatively specific and detailed, but should not be construed as limiting the scope of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the appended claims.

Claims

1. A driving scenario risk assessment method, characterized in that Applied to an image feature extraction module, a 3D point cloud feature extraction module, a feature fusion module, a central vector constraint module, and a risk level assessment module, the method includes: Performing feature extraction on the input original image through the image feature extraction module, and performing feature extraction on the input 3D point cloud through the 3D point cloud feature extraction module to respectively generate corresponding image features and 3D point cloud features; Performing feature fusion processing on the image features and the 3D point cloud features through the feature fusion module based on the cross-attention algorithm, and making the features from different domains obey approximately the same distribution in the feature space through the central vector constraint module to correspondingly generate several different categories of samples; Adding an attachment angle margin penalty in the risk level assessment module to form a corresponding angle interval between several different categories of samples, and evaluating the corresponding risk level according to the size of the angle interval.

2. The driving scenario risk assessment method according to claim 1, wherein: The step of performing feature fusion processing on the image features and the 3D point cloud features through the feature fusion module based on the cross-attention algorithm includes: Obtaining a training set, which contains several training samples, and each training sample respectively contains a driving scene image and corresponding 3D point cloud data; Extracting an image feature vector from the driving scene image through the image feature extraction module, and extracting a point cloud feature vector from the 3D point cloud data through the 3D point cloud feature extraction module; Performing fusion processing on the image feature vector and the point cloud feature vector through the feature fusion module based on the cross-attention algorithm.

3. The driving scenario risk assessment method according to claim 2, wherein: The step of making the features from different domains obey approximately the same distribution in the feature space through the central vector constraint module to correspondingly generate several different categories of samples includes: Randomly adding a corresponding central vector to each training sample through the central vector constraint module; Minimizing the central loss to make the samples of the same category in multiple different scenes converge to the same region in the feature space; During each training iteration, controlling the central vector to update in the direction of reducing the central loss at a preset learning rate to correspondingly generate several different categories of samples.

4. The driving scenario risk assessment method according to claim 3, wherein: The expression of the algorithm for minimizing the central loss is: Among them, C represents the number of included categories, B represents the batch size during the training process, F i represents the feature vector of the i th sample in a batch of samples, ci represents the category of the i th sample, cen ci represents the center vector of the samples with the category ci , L cen represents the said center loss.

5. The driving scenario risk assessment method according to claim 3, wherein: The expression of the algorithm for controlling the central vector to update in the direction of reducing the central loss at a preset learning rate is: Among them, , and ∆cen j t represents the average distance between the samples of class t in a batch and the corresponding center vector during the j -th iteration, cen j t and cen j t＋1 represent the center vectors before and after update during the t -th iteration respectively, α represents the learning rate of the center vector update process, c i represents the class of the i -th sample, F i represents the feature vector of the i -th sample in the sample, and B represents the batch size during the training process.

6. The driving scenario risk assessment method according to claim 2, wherein: The step of adding an attachment angle margin penalty in the risk level assessment module to form a corresponding angle interval between several different categories of samples includes: Mapping the feature vector of the training sample and the output weight of the risk level assessment module to a preset high-dimensional hypersphere through a preset normalization algorithm; Outputting a weight vector corresponding to the feature vector through the risk level assessment module, and adding a corresponding angle interval penalty term to the angle between the feature vector and its corresponding weight vector to correspondingly obtain the angle interval formed between several different categories of samples.

7. The driving scenario risk assessment method according to claim 6, wherein: The expression of the preset normalization algorithm is as follows: Among them F i represents the feature vector of the i th sample in a batch, f i represents the feature vector obtained after F i standardization, B represents the size of the batch, W j represents the weight vector of the risk level evaluation module corresponding to the samples with the risk level of j , w i represents the weight vector obtained after W j normalization, and C represents the number of categories included.

8. A driving scenario risk assessment system, characterized in that, Applied to the image feature extraction module, 3D point cloud feature extraction module, feature fusion module, central vector constraint module, and risk level assessment module, the system includes: An extraction module, configured to extract features from the input original image through the image feature extraction module, and extract features from the input 3D point cloud through the 3D point cloud feature extraction module, so as to generate corresponding image features and 3D point cloud features respectively; A processing module, configured to perform feature fusion processing on the image features and the 3D point cloud features through the feature fusion module based on the cross-attention algorithm, and make the features from different domains follow approximately the same distribution in the feature space through the central vector constraint module, so as to correspondingly generate several different categories of samples; An evaluation module, configured to add an attachment angle margin penalty in the risk level assessment module, so as to form a corresponding angle interval between several different categories of samples, and evaluate the corresponding risk level according to the size of the angle interval.

9. A computer, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the driving scenario risk assessment method described in any one of claims 1 to 7.

10. A readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the driving scenario risk assessment method described in any one of claims 1 to 7.

Images