Driving scene risk assessment method and system

Abstract

The invention provides a driving scene risk assessment method and system, and the method comprises the steps: carrying out the feature extraction of an input original image through an image feature extraction module, and carrying out the feature extraction of an input 3D point cloud through a 3D point cloud feature extraction module, so as to generate corresponding image features and 3D point cloud features; carrying out feature fusion processing on the image features and the 3D point cloud features through a feature fusion module based on a cross attention algorithm, and carrying out approximately same distribution on the features from different domains in a feature space through a central vector constraint module so as to correspondingly generate a plurality of different types of samples; and accessory angle margin punishment is added in the risk level assessment module, so that corresponding angle intervals are formed among the samples of different types, and corresponding risk levels are assessed according to the sizes of the angle intervals. According to the invention, the corresponding driving scene risk level can be objectively and accurately evaluated, and the user experience is correspondingly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving, and particularly to a method and system for risk assessment of driving scenarios. Background Art

[0002] With the progress of technology and the rapid development of productivity, automobiles have been popularized in people's daily lives and have become one of the essential means of transportation for people's daily travel, greatly facilitating people's lives.

[0003] Among them, the autonomous driving technology of automobiles has also achieved rapid development in recent years. In the process of actual application, the dynamic uncertainty of complex driving scenarios is an important factor affecting the safety and reliability of autonomous driving. Therefore, existing autonomous driving technologies must be able to accurately perceive the surrounding environment and make reasonable decisions under complex and changeable traffic conditions.

[0004] Furthermore, existing methods for risk assessment of driving scenarios mainly rely on risk modeling or deep learning methods. However, methods relying on risk modeling require experts to define corresponding risk indicators for assessment. Although this method has a certain degree of interpretability, it is difficult to adapt to the changes in complex dynamic scenarios, and at the same time, its generalization ability is limited. In addition, methods relying on deep learning require training neural network models and learning implicit risk features from multi-modal sensor data. However, due to the relatively single sensor data obtained, the accuracy of the assessment results is correspondingly reduced. Therefore, in view of the deficiencies of the existing technology, it is necessary to provide a method that can comprehensively and accurately capture driving environment information and objectively and accurately complete the risk assessment of driving scenarios. Summary of the Invention

[0005] Based on this, the objective of the present invention is to provide a method and system for risk assessment of driving scenarios, so as to provide a method that can comprehensively and accurately capture driving environment information and objectively and accurately complete the risk assessment of driving scenarios.

[0006] The first aspect of the embodiment of the present invention proposes: A method for risk assessment of driving scenarios, which is applied to an image feature extraction module, a 3D point cloud feature extraction module, a feature fusion module, a central vector constraint module, and a risk level assessment module. The method includes: Performing feature extraction on the input original image through the image feature extraction module, and performing feature extraction on the input 3D point cloud through the 3D point cloud feature extraction module to respectively generate corresponding image features and 3D point cloud features; Based on the cross-attention algorithm, the feature fusion module performs feature fusion processing on the image features and the 3D point cloud features, and the central vector constraint module enables the features from different domains to follow approximately the same distribution in the feature space, so as to correspondingly generate several different categories of samples; An attachment angle margin penalty is added to the risk level evaluation module, so as to form a corresponding angle interval between several different categories of samples, and the corresponding risk level is evaluated according to the size of the angle interval.

[0007] The beneficial effects of the present invention are as follows: Through the cross-attention mechanism, the deep fusion of images and 3D point cloud features can be effectively realized, thereby effectively eliminating the spatio-temporal alignment error. Based on this, the central vector constraint module is used for feature distribution alignment, which can significantly improve the adaptability of the model to different driving scenario domains. In addition, by introducing an additional angle margin penalty mechanism, the decision boundary of risk categories can be correspondingly expanded, fundamentally improving the discrimination accuracy of risk assessment, and then objectively and accurately evaluating the driving scenario risk level, correspondingly enhancing the user experience.

[0008] Further, the steps of performing feature fusion processing on the image features and the 3D point cloud features by the feature fusion module based on the cross-attention algorithm include: Obtain a training set, where the training set contains several training samples, and each training sample respectively contains a driving scenario image and corresponding 3D point cloud data; Extract the image feature vector in the driving scenario image through the image feature extraction module, and extract the point cloud feature vector in the 3D point cloud data through the 3D point cloud feature extraction module; Based on the cross-attention algorithm, the feature fusion module performs fusion processing on the image feature vector and the point cloud feature vector.

[0009] Further, the steps of enabling the features from different domains to follow approximately the same distribution in the feature space through the central vector constraint module to correspondingly generate several different categories of samples include: Randomly add a corresponding central vector to each training sample through the central vector constraint module; Minimize the central loss, so that the same category of samples in multiple different scenarios converge to the same region in the feature space; During each training iteration, control the central vector to update in the direction of reducing the central loss at a preset learning rate, so as to correspondingly generate several different categories of samples.

[0010] Further, the expression of the algorithm for minimizing the central loss is:

[0011] Among them, C represents the number of included categories, B represents the batch size during the training process, F i represents the feature vector of the i th sample in a batch of samples, ci represents the i th sample's category, cen ci represents the center vector of the samples with the category ci ; L cen represents the said center loss.

[0012] Furthermore, the expression of the algorithm for controlling the center vector to be updated in the direction of reducing the center loss at a preset learning rate is:

[0013] Among them, , and ∆cen j t represents the average value of the distances between the samples with the category t in a batch during the j th iteration and the corresponding center vector, cen j t and cen j t＋1 respectively represent the center vectors before and after the update during the t th iteration process, α represents the learning rate of the center vector update process, c i represents the i th sample's category, F i represents the feature vector of the i th sample in the sample, and B represents the batch size during the training process.

[0014] Furthermore, the steps of adding an attachment angle margin penalty in the risk level evaluation module to form corresponding angular intervals between several samples of different categories include: Mapping the feature vectors of the training samples and the output weights of the risk level evaluation module to a preset high-dimensional hypersphere through a preset normalization algorithm; Output a weight vector corresponding to the feature vector through the risk level evaluation module, and add a corresponding angular interval penalty term to the angle between the feature vector and its corresponding weight vector, so as to correspondingly obtain the angular intervals formed between several samples of different categories.

[0015] Further, the expression of the preset normalization algorithm is:

[0016] where F i represents the feature vector of the i th sample in a batch, f i represents the feature vector obtained after standardization by F i B represents the size of the batch, W j represents the weight vector of the risk level evaluation module corresponding to the sample with a risk level of j ; w i represents the weight vector obtained after normalization by W j C represents the number of categories included.

[0017] The second aspect of the embodiments of the present invention proposes: A driving scenario risk assessment system, which is applied to an image feature extraction module, a 3D point cloud feature extraction module, a feature fusion module, a central vector constraint module, and a risk level evaluation module. The system includes: An extraction module, configured to extract features from the input original image through the image feature extraction module, and extract features from the input 3D point cloud through the 3D point cloud feature extraction module, so as to respectively generate corresponding image features and 3D point cloud features; A processing module, configured to perform feature fusion processing on the image features and the 3D point cloud features through the feature fusion module based on the cross-attention algorithm, and make the features from different domains follow approximately the same distribution in the feature space through the central vector constraint module, so as to correspondingly generate several samples of different categories; An evaluation module, configured to add an additional angular margin penalty in the risk level evaluation module, so as to form corresponding angular intervals between several samples of different categories, and evaluate the corresponding risk level according to the size of the angular intervals.

[0018] Further, the processing module is specifically configured to: Obtain a training set, where the training set contains a number of training samples, and each training sample respectively includes a driving scene image and corresponding 3D point cloud data; Extract the image feature vector in the driving scene image through the image feature extraction module, and extract the point cloud feature vector in the 3D point cloud data through the 3D point cloud feature extraction module; Based on the cross-attention algorithm, fuse the image feature vector and the point cloud feature vector through the feature fusion module.

[0019] Furthermore, the processing module is specifically configured to: Randomly add a corresponding center vector to each training sample through the center vector constraint module; Minimize the center loss to make the samples of the same category in multiple different scenes converge to the same region in the feature space; During each training iteration, control the center vector to update in the direction of reducing the center loss at a preset learning rate to correspondingly generate a number of samples of different categories.

[0020] Furthermore, the expression of the algorithm for minimizing the center loss is:

[0021] Where, C represents the number of categories included, B represents the batch size during the training process, F i represents the i th sample feature vector in a batch of samples, ci represents the i th sample category, cen ci represents the center vector of samples with category ci , L cen represents the center loss.

[0022] Furthermore, the expression of the algorithm for controlling the center vector to update in the direction of reducing the center loss at a preset learning rate is:

[0023] Where, , and ∆cen j t represents the average value of the distances between samples with category t in a batch during the j th iteration and the corresponding center vector, cenj t and cen j t＋1 respectively represent the center vectors before and after update in the t th iteration process, α represents the learning rate of the center vector update process, c i represents the i th sample's category, F i represents the i th sample's feature vector in the sample, and B represents the batch size during the training process.

[0024] Furthermore, the evaluation module is specifically configured to: map both the feature vectors of the training samples and the output weights of the risk level evaluation module to a preset high-dimensional hypersphere through a preset normalization algorithm; output a weight vector corresponding to the feature vector through the risk level evaluation module, and add a corresponding angular margin penalty term to the angle between the feature vector and its corresponding weight vector to correspondingly obtain the angular margins formed between several samples of different categories.

[0025] Furthermore, the expression of the preset normalization algorithm is:

[0026] where F i represents the i th sample's feature vector in a batch, f i represents the feature vector obtained after standardizing F i , B represents the size of the batch, W j represents the weight vector of the risk level evaluation module corresponding to the sample with the risk level of j , w i represents the weight vector obtained after normalizing W j , and C represents the number of categories included.

[0027] The third aspect of the embodiments of the present invention proposes: A computer, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the computer program, the driving scenario risk assessment method as described above is implemented.

[0028] In the fourth aspect of the embodiments of the present invention, it is proposed that: A readable storage medium stores a computer program thereon. When the program is executed by a processor, the driving scenario risk assessment method as described above is implemented.

[0029] Additional aspects and advantages of the present invention will be given in part in the following description, become apparent in part from the following description, or be understood through the practice of the present invention. Description of the Drawings

[0030] Figure 1 It is a flowchart of the driving scenario risk assessment method provided by the first embodiment of the present invention; Figure 2 It is a schematic structural diagram of the driving scenario risk assessment model in the driving scenario risk assessment method provided by the second embodiment of the present invention; Figure 3 It is a schematic structural diagram of the 3D point cloud feature fusion module in the driving scenario risk assessment method provided by the second embodiment of the present invention; Figure 4 It is a schematic structural diagram of the risk assessment module in the driving scenario risk assessment method provided by the second embodiment of the present invention; Figure 5 It is a schematic structural diagram of the actual application in the driving scenario risk assessment method provided by the second embodiment of the present invention; Figure 6 It is a structural block diagram of the driving scenario risk assessment system provided by the third embodiment of the present invention.

[0031] The following specific embodiments will further illustrate the present invention in conjunction with the above-mentioned drawings. Specific Embodiments

[0032] To facilitate the understanding of the present invention, the present invention will be described more comprehensively below with reference to the relevant drawings. Several embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, these embodiments are provided to make the disclosure of the present invention more thorough and comprehensive.

[0033] It should be noted that when an element is referred to as being "fixedly provided on" another element, it can be directly on the other element or there may also be an intermediate element. When an element is considered to be "connected" to another element, it can be directly connected to the other element or there may be an intermediate element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are only for the purpose of illustration.

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this invention belongs. The terms used in the description of this invention herein are for the purpose of describing specific embodiments only and are not intended to limit the invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0035] Please refer to Figure 1 , which shows the driving scenario risk assessment method provided by the first embodiment of the present invention. The driving scenario risk assessment method provided by this embodiment can objectively, accurately and comprehensively complete the driving scenario risk assessment, correspondingly improving the user experience.

[0036] Specifically, this embodiment provides: A driving scenario risk assessment method, which is applied to an image feature extraction module, a 3D point cloud feature extraction module, a feature fusion module, a central vector constraint module, and a risk level assessment module. The method includes: Step S10, extracting features from the input original image through the image feature extraction module, and extracting features from the input 3D point cloud through the 3D point cloud feature extraction module to respectively generate corresponding image features and 3D point cloud features; Step S20, based on the cross-attention algorithm, performing feature fusion processing on the image features and the 3D point cloud features through the feature fusion module, and making the features from different domains follow an approximately the same distribution in the feature space through the central vector constraint module to correspondingly generate several different categories of samples; Step S30, adding an attachment angle margin penalty in the risk level assessment module to form a corresponding angle interval between several different categories of samples, and evaluating the corresponding risk level according to the size of the angle interval.

[0037] Second Embodiment Furthermore, the step of performing feature fusion processing on the image features and the 3D point cloud features through the feature fusion module based on the cross-attention algorithm includes: Obtaining a training set, which contains several training samples, and each training sample respectively contains a driving scenario image and corresponding 3D point cloud data; Extracting an image feature vector from the driving scenario image through the image feature extraction module, and extracting a point cloud feature vector from the 3D point cloud data through the 3D point cloud feature extraction module; Performing fusion processing on the image feature vector and the point cloud feature vector through the feature fusion module based on the cross-attention algorithm.

[0038] Further, the step of making the features from different domains follow an approximately the same distribution in the feature space through the central vector constraint module to correspondingly generate several samples of different categories includes: Randomly adding a corresponding central vector to each of the training samples through the central vector constraint module; By minimizing the central loss, making the samples of the same category in multiple different scenarios converge to the same region in the feature space; During each training iteration, controlling the central vector to be updated in the direction of reducing the central loss at a preset learning rate to correspondingly generate several samples of different categories.

[0039] Further, the expression of the algorithm for minimizing the central loss is:

[0040] where C represents the number of categories included, B represents the batch size during the training process, F i represents the feature vector of the i th sample in a batch of samples, ci represents the category of the i th sample, cen ci represents the central vector of the samples with the category ci , L cen represents the central loss.

[0041] Further, the expression of the algorithm for controlling the central vector to be updated in the direction of reducing the central loss at a preset learning rate is:

[0042] where , and ∆cen j t represents the average distance between the samples with the category t in a batch during the j th iteration and the corresponding central vector, cen j t and cen j t＋1 respectively represent the central vectors before and after the update during the t th iteration, α represents the learning rate of the central vector update process, c i represents thei The category of a sample F i indicating the i th sample's feature vector in the sample, and B represents the batch size during the training process.

[0043] Furthermore, the step of adding an attachment angle margin penalty in the risk level evaluation module to form a corresponding angle interval between several samples of different categories includes: Mapping both the feature vector of the training sample and the output weight of the risk level evaluation module to a preset high-dimensional hypersphere through a preset normalization algorithm; Outputting a weight vector corresponding to the feature vector through the risk level evaluation module, and adding a corresponding angle interval penalty term to the angle between the feature vector and its corresponding weight vector to correspondingly obtain the angle intervals formed between several samples of different categories.

[0044] Furthermore, the expression of the preset normalization algorithm is:

[0045] where F i represents the i th sample's feature vector in a batch, f i represents the F i feature vector obtained after standardizing by B represents the size of the batch, W j represents the risk level j of the sample corresponding to the weight vector of the risk level evaluation module, w i represents the W j weight vector obtained after normalizing by , and C represents the number of categories included.

[0046] In addition, in this embodiment, it should be noted that the present invention constructs a driving scenario risk assessment model based on the fusion of image and 3D point cloud data, and its network structure is as Figure 2 shown, including an image feature extraction module, a 3D point cloud feature extraction module, a feature fusion module, a center vector constraint module, and a driving scenario risk level evaluation module with an additional angle margin penalty. First, the input image and 3D point cloud are respectively subjected to feature extraction by the image feature extraction module and the 3D point cloud feature extraction module, and the image features and 3D point cloud features are fused through a cross-attention mechanism.

[0047] Before training the model, a dataset needs to be created to evaluate the accuracy and domain generalization ability of the proposed driving scenario risk assessment model based on the fusion of images and 3D point clouds. During data collection, images and 3D point cloud data of various different types of scenarios, such as urban scenarios, rural scenarios, highway scenarios, and night scenarios, are collected respectively to construct the dataset. During data collation, the images and point cloud data at the same moment are put together to construct an input sample, and the data from different scenarios are stored in different folders respectively. Then, a data annotation system is developed, and multiple drivers from different regions are invited to annotate the risk levels of the samples in this dataset. In this example, the risks of driving scenarios are divided into four levels: "very high risk", "relatively high risk", "relatively low risk", and "very low risk".

[0048] During training, first, the data from different scenarios are mixed together as the training set of the model. Each time a batch of samples is read in, and each training sample contains an image of a driving scenario and the corresponding 3D point cloud data. In this example, a batch of samples contains 24 samples in total.

[0049] Then, the image in the sample is feature-extracted through the image feature extraction module to obtain the image feature vector of the driving scenario. In this example, we use the image encoder of the CLIP model as the image feature extraction module.

[0050] Meanwhile, the 3D point cloud data in the sample is feature-extracted through the 3D point cloud feature extraction module to obtain the point cloud feature vector of the driving scenario. Since compared with traditional 3D point cloud feature extraction methods (such as PointNet, PointNet++, DGCNN, etc.), Point Transformer significantly improves the 3D point cloud feature extraction ability by introducing the self-attention mechanism of Transformer and has advantages such as global relationship modeling, dynamic feature aggregation, multi-scale feature extraction, and robustness to disorder, so Point Transformer is used to extract features from the 3D point cloud in this example.

[0051] Next, the image feature vector and the 3D point cloud feature vector of the driving scenario are simultaneously input into the cross-attention-based feature fusion module to realize the fusion of images and 3D point cloud data through cross-attention. In this example, the structure of the cross-attention-based feature fusion module is as Figure 3 shown, where the number of heads of the multi-head attention mechanism is 4, and the dimension of the feature embedding is 128. Additionally, q represents the length dimension of the image, v represents the width dimension of the image, and k represents the pixel value of the image.

[0052] By constraining the central loss, samples from different domains are made to follow an approximately identical distribution in the feature space, thereby overcoming the domain shift problem and enhancing the performance of the algorithm in practical application scenarios; This method first randomly initializes a central vector for samples of each category.

[0053] Next, during the training process of the model, by minimizing the central loss, samples of the same category from multiple different scenarios are urged to converge to the same region in the feature space. The central loss is defined as the average Euclidean distance between the feature vector and its corresponding central vector, and its calculation formula is as follows:

[0054] Among them, C represents the number of categories included, B represents the batch size during the training process, F i represents the i th sample's feature vector in a batch of samples, ci represents the i th sample's category, cen ci represents the central vector of samples with the category ci ; L cen represents the said central loss; To enable the central vector to better reflect the distribution of corresponding category samples, in each training iteration, the central vector is updated in the direction of reducing the central loss at a learning rate α . The update process of the central vector is as follows:

[0055] Among them, , and ∆cen j t represents the average distance between samples of category t in a batch and the corresponding central vector during the j th iteration, cen j t and cen j t＋1 respectively represent the central vectors before and after update during the t th iteration, α represents the learning rate of the central vector update process, c i represents the i th sample's category, F i represents thei The feature vector of a sample, and B represents the batch size during the training process. In this example, α The value of is 0.0001.

[0056] Finally, by adding an additional angular margin penalty in the risk level evaluation module, a larger angular separation is promoted between samples of different classes, thereby enhancing the robustness of the model and its adaptability to new scenarios. In this example, the specific process with the additional angular margin penalty is as Figure 4 shown.

[0057] First, the feature vector of the input sample and the weights of the risk level evaluation module are mapped to a high-dimensional hypersphere through normalization. The normalization process is as follows:

[0058] where F i represents the feature vector of the i -th sample in a batch, f i represents the feature vector obtained after standardization by F i , B represents the batch size, W j represents the weight vector of the risk level evaluation module corresponding to the sample with a risk level of j , w i represents the weight vector obtained after normalization by W j , and C represents the number of classes included.

[0059] Next, in order to increase the angular separation between the feature vectors of samples of different classes on the feature hypersphere, an angular separation penalty term m is added to the angle between the feature vector and its corresponding weight vector. The calculation formula of the loss function after adding the additional angular margin penalty on the basis of the Softmax loss is:

[0060] where, B represents the batch size, m represents the angular separation penalty term, C represents the number of classes, s represents a constant, θ represents the additional angle, e represents a vector, y i represents the margin, L AAMP represents the finally calculated loss function.

[0061] Finally, during the training process of the model, a linear combination of the center loss and the Softmax loss with an additional angular margin penalty is used as the total loss function Loss of the model, and its calculation formula is as follows: LOSS = λ·L cen +L AAMP where λ is the weight of the center loss in the total optimization objective function. In this example, the value of λ is 1.

[0062] Please refer to Figure 5 , in the actual application process, first, the images and 3D point cloud data of the driving scene are collected in real time through the camera and lidar, and then the collected images and 3D point cloud data are respectively subjected to feature extraction through the image feature extraction module and the 3D point cloud feature extraction module. Then, the obtained image feature vectors and 3D point cloud feature vectors are input into the feature fusion module based on cross-attention for feature fusion. Finally, the fused features are input into the risk level evaluation module with an additional angular margin penalty to realize the risk level evaluation of the current scene.

[0063] Please refer to Figure 6 , the third embodiment of the present invention provides: A driving scene risk assessment system, which is applied to an image feature extraction module, a 3D point cloud feature extraction module, a feature fusion module, a center vector constraint module, and a risk level evaluation module. The system includes: An extraction module, configured to perform feature extraction on the input original image through the image feature extraction module, and perform feature extraction on the input 3D point cloud through the 3D point cloud feature extraction module to respectively generate corresponding image features and 3D point cloud features; A processing module, configured to perform feature fusion processing on the image features and the 3D point cloud features through the feature fusion module based on the cross-attention algorithm, and make the features from different domains follow an approximately the same distribution in the feature space through the center vector constraint module to correspondingly generate several different categories of samples; An evaluation module, configured to add an attachment angular margin penalty in the risk level evaluation module to form a corresponding angular interval between several different categories of samples, and evaluate the corresponding risk level according to the size of the angular interval.

[0064] Further, the processing module is specifically configured to: Obtain a training set, where the training set contains several training samples, and each training sample respectively contains a driving scene image and corresponding 3D point cloud data; Extract the image feature vector in the driving scene image through the image feature extraction module, and extract the point cloud feature vector in the 3D point cloud data through the 3D point cloud feature extraction module; Based on the cross-attention algorithm, fuse the image feature vector and the point cloud feature vector through the feature fusion module.

[0065] Furthermore, the processing module is specifically configured to: Randomly add a corresponding center vector to each of the training samples through the center vector constraint module; Minimize the center loss so that samples of the same category in multiple different scenarios converge to the same region in the feature space; During each training iteration, control the center vector to update in the direction of reducing the center loss at a preset learning rate to correspondingly generate several samples of different categories.

[0066] Furthermore, the expression of the algorithm for minimizing the center loss is:

[0067] Where, C represents the number of categories included, B represents the batch size during the training process, F i represents the feature vector of the i th sample in a batch of samples, ci represents the i th sample's category, cen ci represents the center vector of the samples with the category of ci , L cen represents the center loss.

[0068] Furthermore, the expression of the algorithm for controlling the center vector to update in the direction of reducing the center loss at a preset learning rate is:

[0069] Where, , and ∆cen j t represents the average distance between the samples of category j in a batch and the corresponding center vector during the t-th iteration, cen j t and cen j t＋1 respectively represent the center vector before and after the update during the t-th iteration, α represents the learning rate of the center vector update process, c irepresents the category of the i-th sample, F i represents the feature vector of the i-th sample in the sample, and B represents the batch size during the training process.

[0070] Furthermore, the evaluation module is specifically configured to: map the feature vector of the training sample and the output weight of the risk level evaluation module to a preset high-dimensional hypersphere through a preset normalization algorithm; output a weight vector corresponding to the feature vector through the risk level evaluation module, and add a corresponding angular margin penalty term to the angle between the feature vector and its corresponding weight vector, so as to correspondingly obtain the angular margins formed between several samples of different categories.

[0071] Furthermore, the expression of the preset normalization algorithm is:

[0072] where F i represents the feature vector of the i-th sample in a batch, and f i represents the feature vector obtained after normalizing F i , B represents the size of the batch, W j represents the weight vector of the risk level evaluation module corresponding to the sample with risk level j, and w i represents the weight vector obtained after normalizing W j , and C represents the number of categories included.

[0073] The fourth embodiment of the present invention provides a computer, including a memory, a processor, and a computer program stored on the memory and executable on the processor. Wherein, when the processor executes the computer program, the driving scenario risk assessment method described above is implemented.

[0074] The fifth embodiment of the present invention provides a readable storage medium, on which a computer program is stored. Wherein, when the program is executed by a processor, the driving scenario risk assessment method described above is implemented.

[0075] In summary, the driving scenario risk assessment method and system provided by the above embodiments of the present invention can objectively, accurately, and comprehensively complete the driving scenario risk assessment, correspondingly improving the user experience.

[0076] It should be noted that the above-mentioned various modules can be functional modules or program modules, and can be implemented either by software or by hardware. For the modules implemented by hardware, the above-mentioned various modules can be located in the same processor; or the above-mentioned various modules can also be located in different processors in any combination form.

[0077] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a definitional sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch instructions from the instruction execution system, apparatus, or device and execute the instructions), or used in conjunction with these instruction execution systems, apparatus, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0078] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection part having one or more wirings (electronic device), a portable computer disk cartridge (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or otherwise processing as appropriate, and then storing it in a computer memory.

[0079] It should be understood that various parts of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0080] In the description of this specification, the descriptions referring to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0081] The embodiments described above merely represent several implementation manners of the present invention. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all fall within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the appended claims.

Claims

1. A driving scenario risk assessment method, characterized in that: Applied to an image feature extraction module, a 3D point cloud feature extraction module, a feature fusion module, a center vector constraint module, and a risk level assessment module, the method includes: The image feature extraction module performs feature extraction on the input original image, and the 3D point cloud feature extraction module performs feature extraction on the input 3D point cloud, so as to generate corresponding image features and 3D point cloud features respectively; Based on the cross attention algorithm, the feature fusion module performs feature fusion processing on the image features and the 3D point cloud features, and the center vector constraint module makes the features from different domains obey approximately the same distribution in the feature space, so as to generate a number of samples of different categories accordingly; An additional angle margin penalty is added to the risk level assessment module to form corresponding angle intervals between the samples of different categories, and the corresponding risk level is assessed according to the size of the angle interval.

2. The driving scenario risk assessment method according to claim 1, characterized in that: The step of performing feature fusion processing on the image features and the 3D point cloud features through the feature fusion module based on the cross attention algorithm includes: Obtaining a training set, wherein the training set includes a plurality of training samples, each of the training samples includes a driving scene image and corresponding 3D point cloud data; Extracting an image feature vector from the driving scene image by means of the image feature extraction module, and extracting a point cloud feature vector from the 3D point cloud data by means of the 3D point cloud feature extraction module; Based on the cross-attention algorithm, the image feature vector and the point cloud feature vector are fused through the feature fusion module.

3. The driving scenario risk assessment method according to claim 2, characterized in that: The step of using the center vector constraint module to make features from different domains obey approximately the same distribution in the feature space to generate samples of several different categories accordingly includes: A corresponding center vector is randomly added to each of the training samples through the center vector constraint module; By minimizing the center loss, samples of the same category in multiple different scenes are made to converge to the same area in the feature space; During each training iteration, the center vector is controlled to be updated in a direction of reducing the center loss at a preset learning rate, so as to generate a number of samples of different categories accordingly.

4. The driving scenario risk assessment method according to claim 3, characterized in that: The expression of the algorithm for minimizing the center loss is: in, C Indicates the number of categories included. B represents the batch size during training, F i Indicates the first i The feature vector of the samples, ci Indicates i The category of samples, cen ci Indicates the category ci The center vector of the sample, L cen represents the center loss.

5. The driving scenario risk assessment method according to claim 3, characterized in that: The expression of the algorithm for controlling the center vector to be updated in the direction of reducing the center loss at a preset learning rate is: in, , and ∆cen j t Indicated in t In the process of iterations, the categories in a batch are j The average distance between the samples and the corresponding center vector, cen j t and cen j t＋1 Respectively expressed in t The center vectors before and after updating in the iteration process are: α represents the learning rate of the center vector update process, c i Indicates i The category of samples, F i Indicates the sample i The feature vector of samples, B represents the batch size during training.

6. The driving scenario risk assessment method according to claim 2, characterized in that: The step of adding an additional angle margin penalty in the risk level assessment module to form corresponding angle intervals between the samples of different categories comprises: Mapping the feature vector of the training sample and the output weight of the risk level assessment module onto a preset high-dimensional hypersphere through a preset normalization algorithm; The risk level assessment module outputs a weight vector corresponding to the feature vector, and adds a corresponding angle interval penalty term to the angle between the feature vector and the corresponding weight vector, so as to obtain the corresponding angle intervals formed between the samples of the different categories.

7. The driving scenario risk assessment method according to claim 6, characterized in that: The expression of the preset normalization algorithm is: in F i Indicates the first i The feature vector of the samples, f i Indicated by F i The feature vector obtained after normalization is B Indicates the batch size, W j Indicates the risk level is j The weight vector of the risk level assessment module corresponding to the sample, w i Indicated by W j The normalized weight vector, C represents the number of categories included.

8. A driving scenario risk assessment system, characterized in that: Applied to image feature extraction module, 3D point cloud feature extraction module, feature fusion module, center vector constraint module and risk level assessment module, the system includes: An extraction module, used to perform feature extraction on the input original image through the image feature extraction module, and to perform feature extraction on the input 3D point cloud through the 3D point cloud feature extraction module, so as to generate corresponding image features and 3D point cloud features respectively; A processing module, configured to perform feature fusion processing on the image features and the 3D point cloud features through the feature fusion module based on a cross attention algorithm, and to make the features from different domains obey approximately the same distribution in the feature space through the center vector constraint module, so as to generate a number of samples of different categories accordingly; The evaluation module is used to add an additional angle margin penalty in the risk level evaluation module to form corresponding angle intervals between the samples of different categories, and evaluate the corresponding risk level according to the size of the angle interval.

9. A computer comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the driving scenario risk assessment method as described in any one of claims 1 to 7 is implemented.

10. A readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the driving scenario risk assessment method as described in any one of claims 1 to 7 is implemented.

Images