Behavior decision optimization method and system based on intelligent driving

Abstract

The invention relates to the technical field of intelligent driving, in particular to a behavior decision optimization method and system based on intelligent driving, and the method comprises the following steps: taking a multi-modal feature map for behavior decision of an intelligent driving vehicle as input, taking a behavior decision parameter as output, training a deep neural network, and obtaining a behavior decision model; and training the behavior decision model by taking the multi-modal feature map for behavior decision optimization of the intelligent driving vehicle as input and the behavior decision optimization parameters as output to obtain a decision optimization model. According to the invention, the behavior decision model is constructed, and the universal driving behavior decision is optimized according to the real-time driving environment data of the intelligent driving vehicle, so that the real-time driving behavior decision adaptive to the real-time driving environment of the intelligent driving vehicle is obtained. Therefore, the intelligent driving vehicle can quickly obtain the optimal matching to adapt to the driving behavior of the real-time driving scene in the driving process, and the accuracy and timeliness of the decision making of the intelligent driving behavior are ensured.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving technology, and in particular to a behavior decision optimization method and system based on intelligent driving. Background Art

[0002] Intelligent driving obtains the surrounding environment through identification, and then makes full use of this information to understand and analyze it, and decide how to take the next step, that is, to achieve behavioral decision-making for intelligent driving. There are currently two ways to deal with behavioral decisions for intelligent driving: expert rule-based and AI-based. Expert rule-based means that the rules are written in advance, and these rules must be strictly followed when decisions need to be made. For example, when preparing to overtake and change lanes, the following conditions must be met: the road radius is greater than 500R (no lane change on a curve); the distance to the front and rear vehicles in the target lane is more than 20m; the speed is no slower than that of the rear vehicle by no more than 5km / h, then you can overtake and change lanes. AI-based (Artificial Intelligence) imitates the human brain, understands the scene through AI algorithms, and plans intelligent driving behaviors to adapt to the driving path environment.

[0003] The intelligent driving behavior decision made in the prior art belongs to advanced behavior decision planning, or in other words, the scene information of the driving path is collected in advance, the scene information is processed to plan the driving behavior, and the driving behavior of the intelligent driving vehicle is guided by the driving behavior obtained through advance planning. However, since the scene information collected by the driving behavior planning occurs before the intelligent driving behavior, the driving behavior decision obtained by the plan cannot fully match and adapt to the real-time driving scene, resulting in the failure of the behavior decision. Summary of the invention

[0004] The purpose of the present invention is to provide a behavior decision optimization method and system based on intelligent driving, so as to solve the technical problem that the driving behavior decision obtained by planning in the prior art cannot fully match and adapt to the real-time driving scene, resulting in the failure of the behavior decision.

[0005] In order to solve the above technical problems, the present invention specifically provides the following technical solutions:

[0006] A behavior decision optimization method based on intelligent driving includes the following steps:

[0007] Using the laser radar and camera mounted on the data collection vehicle, the environmental image data and environmental point cloud data of each position on the driving path are collected one by one, and the corresponding marks are marked as environmental image prior data and environmental point cloud prior data;

[0008] Based on the environmental image prior data and the environmental point cloud prior data, a multimodal feature map for intelligent driving vehicle behavior decision-making is extracted;

[0009] Taking the multimodal feature map used for intelligent driving vehicle behavior decision as input and the behavior decision parameter as output, the deep neural network is trained to obtain the behavior decision model;

[0010] Using the laser radar and camera mounted on the intelligent driving vehicle on the driving path, the environmental image data and environmental point cloud data of the real-time position during driving are collected, and the data are marked as environmental image real-time data and environmental point cloud real-time data respectively;

[0011] Extract multimodal feature maps for intelligent driving vehicle behavior decision optimization based on real-time environmental image data and real-time environmental point cloud data;

[0012] Taking the multimodal feature map used for optimizing the behavior decision of intelligent driving vehicles as input and the behavior decision optimization parameters as output, the behavior decision model is trained to obtain a decision optimization model.

[0013] As a preferred solution of the present invention, the environmental point cloud data is constructed from driving path scanning data acquired by a laser radar.

[0014] As a preferred solution of the present invention, a method for extracting a multimodal feature map for intelligent driving vehicle behavior decision-making includes:

[0015] The environment image prior data and the environment point cloud prior data are subjected to feature extraction using a multi-scale feature extraction network to obtain an environment image prior feature map and an environment point cloud prior feature map;

[0016] The prior feature map of the environment image is divided into a prior image consistency feature and a prior image difference feature, and the prior feature map of the environment point cloud is divided into a prior point cloud consistency feature and a prior point cloud difference feature;

[0017] The prior image consistency feature and the prior point cloud consistency feature are multimodally fused to obtain a priori consistency multimodal feature map;

[0018] The prior image difference features and the prior point cloud difference features are multimodally fused to obtain a prior difference multimodal feature map.

[0019] As a preferred solution of the present invention, the method for constructing the behavior decision model includes:

[0020] Based on the environmental image prior data and environmental point cloud prior data at each position on the driving path, the behavior decision parameters of the intelligent driving vehicle at each position on the driving path are manually calculated;

[0021] The priori consistent multimodal feature map is used as an input of a first deep neural network, and the first deep neural network outputs a consistent behavior decision parameter of the intelligent driving vehicle;

[0022] The prior difference multimodal feature map is used as the input of the second deep neural network, and the second deep neural network outputs the difference behavior decision parameters of the intelligent driving vehicle;

[0023] The similarity loss between the sum of the consistent behavior decision parameters and the differential behavior decision parameters and the behavior decision parameters of the intelligent driving vehicle at each position on the driving path is used as a loss function for training the multi-scale feature extraction network, the first deep neural network and the second deep neural network to obtain the behavior decision model;

[0024] The trained multi-scale feature extraction network, the first deep neural network and the second deep neural network are used as the behavior decision model;

[0025] The behavioral decision model is:

[0026] [Hs+Hz]=ResNet50(Hdata_history);

[0027] [Gz+Gz]=ResNet50(Gdata_history);

[0028] Ms = BP1([Hs+Gs]);

[0029] Mz=BP2([Hz+Gz]);

[0030] Where Ms is the consistent behavior decision parameter, Mz is the differential behavior decision parameter, [Hs+Gs] is the prior consistent multimodal feature map, [Hz+Gz] is the prior differential multimodal feature map, Hs is the prior point cloud consistency feature, Gs is the prior image consistency feature, Hz is the prior point cloud difference feature, Gz is the prior image difference feature, BP1 is the first deep neural network, BP2 is the second deep neural network, Hdata_history is the environmental point cloud prior data, Gdata_history is the environmental image prior data, and ResNet50 is a multi-scale feature extraction network;

[0031] The loss function of the behavior decision model is: Where L is the loss value of the behavior decision model, Ms is the consistent behavior decision parameter, Mz is the differential behavior decision parameter, and Mr is the behavior decision parameter of the intelligent driving vehicle at each position on the driving path calculated manually. is the L2 norm.

[0032] As a preferred solution of the present invention, a method for extracting a multimodal feature map for optimizing intelligent driving vehicle behavior decision-making includes:

[0033] The real-time data of the environment image and the real-time data of the environment point cloud are subjected to feature extraction using a multi-scale feature extraction network to obtain a real-time feature map of the environment image and a real-time feature map of the environment point cloud;

[0034] The real-time feature map of the environment image is divided into a real-time image consistency feature and a real-time image difference feature, and the real-time feature map of the environment point cloud is divided into a real-time point cloud consistency feature and a real-time point cloud difference feature;

[0035] The real-time image consistency feature and the real-time point cloud consistency feature are multimodally fused to obtain a real-time consistency multimodal feature map;

[0036] The real-time image difference features and the real-time point cloud difference features are multimodally fused to obtain a real-time difference multimodal feature map.

[0037] As a preferred solution of the present invention, the method for constructing the decision optimization model includes:

[0038] Based on the real-time data of the environment image at the real-time position during driving and the prior data of the environment point cloud, the behavior decision parameters of the intelligent driving vehicle at the real-time position during driving are artificially calculated;

[0039] The real-time differential multimodal feature map is used as the input of the second deep neural network in the behavior decision model, and the second deep neural network in the behavior decision model outputs the differential behavior decision optimization parameter;

[0040] The similarity loss between the sum of the consistent behavior decision parameter and the differential behavior decision optimization parameter and the behavior decision parameter of the intelligent driving vehicle at the real-time position during driving, the similarity loss between the real-time consistent multimodal feature map and the prior consistent multimodal feature map, the cross loss between the real-time consistent multimodal feature map and the real-time differential multimodal feature map, and the cross loss between the prior consistent multimodal feature map and the prior differential multimodal feature map are used as the loss function for retraining the behavior decision model to obtain the behavior decision optimization model;

[0041] The behavior decision model after training is used as a behavior decision optimization model;

[0042] The behavior decision optimization model is:

[0043] [Hs_now+Hz_now]=ResNet50(Hdata_now);

[0044] [Gz_now+Gz_now]=ResNet50(Gdata_now);

[0045] Ms = BP1([Hs+Gs]);

[0046] Mz_now=BP2([Hz_now+Gz_now]);

[0047] In the formula, Ms is the consistent behavior decision parameter, Mz_now is the difference behavior decision optimization parameter,

[0048] [Hs+Gs] is a priori consistent multimodal feature map, [Hz_now+Gz_now] is a real-time difference multimodal feature map, Hs is a priori point cloud consistency feature, Gs is a priori image consistency feature, Hs_now is a real-time point cloud consistency feature, Gs_now is a real-time image consistency feature, Hz_now is a real-time point cloud difference feature, Gz_now is a real-time image difference feature, BP1 is the first deep neural network, BP2 is the second deep neural network, Hdata_now is the real-time data of the environmental point cloud, Gdata_now is the real-time data of the environmental image, and ResNet50 is a multi-scale feature extraction network;

[0049] The loss function of the behavior decision optimization model is: Lnow=Lu+Lv+Lw+Le;

[0050] Wherein, Lnow is the loss value of the behavior decision optimization model, Lu is the similarity loss between the sum of the consistent behavior decision parameters and the differential behavior decision optimization parameters and the behavior decision parameters of the intelligent driving vehicle at the real-time position during driving, Lv is the similarity loss between the real-time consistent multimodal feature map and the prior consistent multimodal feature map, Lw is the cross loss between the real-time consistent multimodal feature map and the real-time differential multimodal feature map, and Le is the cross loss between the prior consistent multimodal feature map and the prior differential multimodal feature map;

[0051] in, In the formula, Ms is the consistent behavior decision parameter, Mz_now is the differential behavior decision optimization parameter, and Mr_now is the behavior decision parameter of the intelligent driving vehicle at the real-time position during driving calculated by humans. is the L2 norm;

[0052] Where [Hs_now+Gs_now] is the real-time consistency multimodal feature map, [Hs+Gs] is the prior consistency multimodal feature map, Hs is the prior point cloud consistency feature, Gs is the prior image consistency feature, Hs_now is the real-time point cloud consistency feature, Gs_now is the real-time image consistency feature, is the L2 norm;

[0053] Lw=1 / {1+D KL ([Hs_now+Gs_now]||[Hz now +Gznow ]), where

[0054] [Hs_now+Gs_now] is a real-time consistency multimodal feature map, [Hz_now+Gz_now] is a real-time difference multimodal feature map, Hs_now is a real-time point cloud consistency feature, Gs_now is a real-time image consistency feature, Hz_now is a real-time point cloud difference feature, Gz_now is a real-time image difference feature, D KL ([Hs_now+Gs_now]||[Hz_now+Gz_now]) is the KL divergence between [Hs_now+Gs_now] and [Hz_now+Gz_now];

[0055] Le=1 / {1+D KL ([Hs+Gs]||[Hz+Gz])}, where [Hs+Gs] is the prior consistency multimodal feature map, [Hz+Gz] is the prior difference multimodal feature map, Hs is the prior point cloud consistency feature, Gs is the prior image consistency feature, Hz is the prior point cloud difference feature, Gz is the prior image difference feature, D KL ([Hs+Gs]||[Hz+Gz]) is the KL divergence between [Hs+Gs] and [Hz+Gz].

[0056] As a preferred solution of the present invention, the behavior decision parameters include yaw rate and acceleration, and the behavior decision optimization parameters include an optimized value of yaw rate and an optimized value of acceleration.

[0057] As a preferred solution of the present invention, the evaluation indicators of the behavior decision model and the behavior decision planning model are recall rate and average precision.

[0058] As a preferred solution of the present invention, the environmental image priori data and the environmental point cloud priori data, the environmental image real-time data and the environmental point cloud real-time data are normalized.

[0059] As a preferred solution of the present invention, the present invention provides a behavior decision optimization system based on intelligent driving, which is applied to a behavior decision optimization method based on intelligent driving. The system includes:

[0060] A data acquisition unit is used to collect environmental image data and environmental point cloud data at each position of the driving path one by one on the driving path by using a laser radar and a camera mounted on the data acquisition vehicle, and the environmental image data and environmental point cloud data are marked as environmental image prior data and environmental point cloud prior data respectively;

[0061] and for collecting environmental image data and environmental point cloud data of real-time position during driving on the driving path using the laser radar and camera mounted on the intelligent driving vehicle, which are marked as environmental image real-time data and environmental point cloud real-time data respectively;

[0062] A feature processing unit, used to extract a multimodal feature map for intelligent driving vehicle behavior decision-making based on environmental image prior data and environmental point cloud prior data;

[0063] and for extracting a multimodal feature map for optimizing intelligent driving vehicle behavior decisions based on real-time environmental image data and real-time environmental point cloud data;

[0064] A model building unit, for training a deep neural network with a multimodal feature map for intelligent driving vehicle behavior decision as input and a behavior decision parameter as output to obtain a behavior decision model;

[0065] And it is used to train the behavior decision model by taking the multimodal feature map used for optimizing the behavior decision of intelligent driving vehicles as input and the behavior decision optimization parameter as output to obtain the decision optimization model.

[0066] Compared with the prior art, the present invention has the following beneficial effects:

[0067] The present invention utilizes a data collection vehicle to collect data on the driving environment of a driving path before the driving behavior occurs, and pre-plans the driving behavior of intelligent driving in the driving environment of the driving path by constructing a behavior decision model, and obtains in advance a universal driving behavior decision to guide the intelligent driving vehicle on the driving path, and then optimizes the universal driving behavior decision according to the real-time driving environment data of the intelligent driving vehicle by constructing a behavior decision model, and obtains a real-time driving behavior decision that adapts to the real-time driving environment of the intelligent driving vehicle, so that the intelligent driving vehicle can quickly obtain the optimal matching driving behavior that adapts to the real-time driving scene during the driving process, and ensure the accuracy and timeliness of the intelligent driving behavior decision. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0069] Figure 1 A flow chart of a behavior decision optimization method based on intelligent driving provided by an embodiment of the present invention;

[0070] Figure 2A block diagram of a behavior decision optimization system based on intelligent driving provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0071] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] like Figure 1 As shown, the present invention provides a behavior decision optimization method based on intelligent driving, comprising the following steps:

[0073] Using the laser radar and camera mounted on the data collection vehicle, the environmental image data and environmental point cloud data of each position on the driving path are collected one by one, and the corresponding marks are marked as environmental image prior data and environmental point cloud prior data;

[0074] Based on the environmental image prior data and the environmental point cloud prior data, a multimodal feature map for intelligent driving vehicle behavior decision-making is extracted;

[0075] Taking the multimodal feature map used for intelligent driving vehicle behavior decision as input and the behavior decision parameters (yaw angular velocity and acceleration) as output, the deep neural network is trained to obtain a behavior decision model;

[0076] Using the laser radar and camera mounted on the intelligent driving vehicle on the driving path, the environmental image data and environmental point cloud data of the real-time position during driving are collected, and the data are marked as environmental image real-time data and environmental point cloud real-time data respectively;

[0077] Extract multimodal feature maps for intelligent driving vehicle behavior decision optimization based on real-time environmental image data and real-time environmental point cloud data;

[0078] Taking the multimodal feature map used for optimizing the behavior decision of intelligent driving vehicles as input and the behavior decision optimization parameters as output, the behavior decision model is trained to obtain a decision optimization model.

[0079] The present invention first establishes a behavior decision model, and uses a data acquisition vehicle to pre-collect environmental data on a driving path, including a three-dimensional point cloud of the environmental scene and a two-dimensional image of the environmental scene, and respectively obtains the morphological position, color texture and other information of objects at various positions on the driving path. After mastering multi-source environmental information, the multi-source information is associated and mapped with the vehicle's behavior decision, so as to achieve the calculation of the vehicle's behavior decision (i.e., yaw angular velocity and acceleration) based on the environmental information on the driving path, and further achieve the calculation of the vehicle's hardware parameters such as the driving motor power required by the vehicle based on the environmental information on the driving path, so that the vehicle can automatically and accurately make decisions and control the environmental information on the driving path, and smoothly perform obstacle avoidance driving on the driving path.

[0080] Since the behavior decision model relies on multi-source environmental information collected in advance, it belongs to the general planning before driving and cannot be fully adapted to the real scene during driving (for example, pedestrians and animals that suddenly appear during driving do not exist in the pre-collected environmental information). At this time, it is difficult for the behavior decision model to achieve accurate vehicle decision control. Therefore, the present invention establishes a behavior decision optimization model to optimize the behavior decision parameters obtained by the general planning before driving, adapt to the environmental information during driving, and enable the vehicle to travel smoothly during driving.

[0081] The behavior decision optimization model in the present invention is obtained by transfer learning based on the behavior decision model, so as to optimize the pre-planned behavior decisions according to the environmental information during driving, that is, the pre-planned behavior decisions are still retained during driving, and only the behavior decisions that are not applicable to the environmental information during driving are optimized and corrected, and local real-time adjustments are quickly made based on the prior planning, without the need to re-plan the whole process during driving, and the available parts of the pre-planning are retained as much as possible, and the behavior decisions planned before driving are quickly transferred to the driving process. The behavior decision optimization model improves the efficiency of behavior decision-making during driving, and the pre-driving plan is partially corrected according to the actual driving environment to adapt to the actual driving environment. Compared with directly using the pre-driving behavior decision plan to guide intelligent driving behavior during driving, the behavior decision optimization model improves the accuracy of behavior decision-making during driving.

[0082] In order to quickly transfer the behavioral decision-making planned before driving to the driving process, the present invention retains the available parts of the pre-planning as much as possible. When constructing the behavioral decision model and the behavioral decision optimization model, the multi-source environmental information pre-acquired before driving and the multi-source environmental information acquired in real time during driving are divided into the feature extraction of consistency features and difference features. The consistent multi-source environmental information features before driving and during driving are extracted, and the difference features between the two are simultaneously distinguished. In this regard, when transferring learning between the behavioral decision model and the behavioral decision optimization model, it is only necessary to transfer the difference features of the two, so as to calculate the behavioral decision parameters according to the environmental information that changes during driving and before driving, and correspondingly realize the optimization and correction of the behavioral decisions that are still retained in the pre-planned behavior decisions during driving, and only the behavioral decisions that are not applicable to the environmental information during driving are optimized and corrected, and local real-time adjustments are quickly made based on the priori planning.

[0083] In order to achieve feature migration only for differential features before driving and during driving, the present invention designs a model structure into a dual network structure, one network structure processes consistent features, and the other network structure processes differential features, which is convenient for the subsequent migration of the behavior decision optimization model. The differential features during driving are used to retrain the network structure that processes the differential features before driving in the behavior decision model, so that the behavior decision model is migrated to the network structure that processes the differential features during driving, and the behavior decision parameters are calculated according to the environmental information that changes during driving and before driving. The network structure that retains the consistent features is used to achieve the corresponding behavior decision that is still reserved in advance during driving.

[0084] Correspondingly, in the transfer learning from the behavior decision model to the behavior decision optimization model, the present invention divides the loss function into four parts. The first part is the similarity loss between the sum of the consistent behavior decision parameter and the differential behavior decision optimization parameter and the behavior decision parameter of the intelligent driving vehicle at the real-time position during driving. It is a measure of the difference between the output of the behavior decision optimization model and the actual output. It can ensure that the behavior decision parameters output by the behavior decision optimization model are most accurately adapted to the driving environment information. The second part is the similarity loss between the real-time consistent multimodal feature map and the prior consistent multimodal feature map. It quantifies the difference between the consistent features of the environmental information before driving and during driving. It can ensure that the consistent features of the environmental information before driving and during driving can be extracted to the greatest extent, and ensure that the pre-planned behavior decision is still retained in driving after transfer learning. The third part It is the cross loss between the real-time consistent multimodal feature map and the real-time differential multimodal feature map, and it quantifies the dependence between the consistent features and the differential features of the environmental information during driving. It can separate the consistent features and the differential features of the environmental information during driving to the greatest extent, avoid the intersection of the two, affect the extraction of the environmental information features that change during driving and before driving, and cause the error of transfer learning to increase. The fourth part is the cross loss between the prior consistent multimodal feature map and the prior differential multimodal feature map, and it can quantify the dependence between the consistent features and the differential features of the environmental information before driving. It can separate the consistent features and the differential features of the environmental information before driving to the greatest extent, avoid the intersection of the two, affect the extraction of the environmental information features that change during driving and before driving, and cause the error of transfer learning to increase.

[0085] In summary, the present invention can optimize the planning parameters on the basis of advance planning, adapt to the real scene during driving, and realize the optimization of behavioral decisions of intelligent driving.

[0086] The present invention uses a three-dimensional point cloud and a two-dimensional image of the environment scene to obtain the intelligent driving environment, and obtains the shape, position, color, texture and other information of the objects at various positions on the driving path, as follows:

[0087] The environmental point cloud data is constructed from the driving path scanning data obtained by the laser radar; the method for constructing the environmental point cloud data includes:

[0088] Using a laser radar to perform laser scanning on the environment at each position in the driving path, and obtaining scanning data of the environment at each position in the driving path, wherein the scanning data includes a laser ranging value, a horizontal angle value, a vertical angle value, and a horizontal offset angle of a radar channel;

[0089] The three-dimensional coordinate data of each position environment in the driving environment is converted through the scanning data of each position environment in the driving environment;

[0090] The conversion formula of the three-dimensional coordinate data is:

[0091]

[0092] In the formula, x kj is the x-axis coordinate of the k-th position environment in the driving path corresponding to the j-th channel of the laser radar, and y kj is the y-axis coordinate of the k-th position environment in the driving path corresponding to the j-th channel of the laser radar, z kj is the z-axis coordinate of the k-th position environment in the driving path corresponding to the j-th channel of the laser radar, r kj is the laser ranging value obtained by the jth channel of the laser radar at the kth position environment in the driving path, α kj is the horizontal angle value obtained by the jth channel of the laser radar at the kth position environment in the driving path, δ kj is the horizontal offset angle value of the jth channel of the laser radar at the kth position environment in the driving path, β kj is the vertical angle value obtained by the j-th channel of the laser radar at the k-th position environment in the driving path;

[0093] Among them, the three-dimensional coordinate data of the kth position environment in the driving path is {(x kj ,y kj ,z kj )|j∈[1,n]}, where (x kj ,y kj ,z kj ) is the three-dimensional coordinate of the kth position environment in the driving path corresponding to the jth channel of the laser radar, and n is the number of channels of the laser radar.

[0094] The present invention uses laser radar and camera to obtain radar scanning data for sensing the shape and position of obstacles in the driving environment and visual image data of the color and texture of obstacles, so as to achieve environmental perception by fusion of multi-source information. Compared with target detection of a single information source, multi-source information enhances the identifiable feature quantity of obstacle perception, thereby improving the accuracy of environmental perception. Therefore, multi-source information enhances the environmental perception accuracy of intelligent driving.

[0095] The method for extracting multimodal feature maps for intelligent driving vehicle behavior decision-making includes:

[0096] The environment image prior data and the environment point cloud prior data are subjected to feature extraction using a multi-scale feature extraction network to obtain an environment image prior feature map and an environment point cloud prior feature map;

[0097] The prior feature map of the environment image is divided into a prior image consistency feature and a prior image difference feature, and the prior feature map of the environment point cloud is divided into a prior point cloud consistency feature and a prior point cloud difference feature;

[0098] The prior image consistency feature and the prior point cloud consistency feature are multimodally fused to obtain a priori consistency multimodal feature map;

[0099] The prior image difference features and the prior point cloud difference features are multimodally fused to obtain a prior difference multimodal feature map.

[0100] The construction methods of behavioral decision-making models include:

[0101] Based on the environmental image prior data and environmental point cloud prior data at each position on the driving path, the behavior decision parameters of the intelligent driving vehicle at each position on the driving path are manually calculated;

[0102] The priori consistent multimodal feature map is used as an input of a first deep neural network, and the first deep neural network outputs a consistent behavior decision parameter of the intelligent driving vehicle;

[0103] The prior difference multimodal feature map is used as the input of the second deep neural network, and the second deep neural network outputs the difference behavior decision parameters of the intelligent driving vehicle;

[0104] The similarity loss between the sum of the consistent behavior decision parameters and the differential behavior decision parameters and the behavior decision parameters of the intelligent driving vehicle at each position on the driving path is used as a loss function for training the multi-scale feature extraction network, the first deep neural network and the second deep neural network to obtain the behavior decision model;

[0105] The trained multi-scale feature extraction network, the first deep neural network and the second deep neural network are used as the behavior decision model;

[0106] The behavioral decision model is:

[0107] [Hs+Hz]=ResNet50(Hdata_history);

[0108] [Gz+Gz]=ResNet50(Gdata_history);

[0109] Ms = BP1([Hs+Gs]);

[0110] Mz=BP2([Hz+Gz]);

[0111] Where Ms is the consistent behavior decision parameter, Mz is the differential behavior decision parameter, [Hs+Gs] is the prior consistent multimodal feature map, [Hz+Gz] is the prior differential multimodal feature map, Hs is the prior point cloud consistency feature, Gs is the prior image consistency feature, Hz is the prior point cloud difference feature, Gz is the prior image difference feature, BP1 is the first deep neural network, BP2 is the second deep neural network, Hdata_history is the environmental point cloud prior data, Gdata_history is the environmental image prior data, and ResNet50 is a multi-scale feature extraction network;

[0112] The loss function of the behavior decision model is: Where L is the loss value of the behavior decision model, Ms is the consistent behavior decision parameter, Mz is the differential behavior decision parameter, and Mr is the behavior decision parameter of the intelligent driving vehicle at each position on the driving path calculated manually. is the L2 norm.

[0113] The present invention first establishes a behavior decision model, and uses a data acquisition vehicle to pre-collect environmental data on a driving path, including a three-dimensional point cloud of the environmental scene and a two-dimensional image of the environmental scene, and respectively obtains the morphological position, color texture and other information of objects at various positions on the driving path. After mastering multi-source environmental information, the multi-source information is associated and mapped with the vehicle's behavior decision, so as to achieve the calculation of the vehicle's behavior decision (i.e., yaw angular velocity and acceleration) based on the environmental information on the driving path, and further achieve the calculation of the vehicle's hardware parameters such as the driving motor power required by the vehicle based on the environmental information on the driving path, so that the vehicle can automatically and accurately make decisions and control the environmental information on the driving path, and smoothly perform obstacle avoidance driving on the driving path.

[0114] The behavior decision optimization model in the present invention is obtained by transfer learning based on the behavior decision model, so as to optimize the pre-planned behavior decision according to the environmental information in driving, that is, the pre-planned behavior decision is still retained in driving, and only the behavior decision that is not applicable to the environmental information in driving is optimized and corrected, and local real-time adjustment is quickly made based on the priori planning, without the need to re-plan the whole process during driving, and the available part of the pre-plan is retained as much as possible, and the behavior decision planned before driving is quickly transferred to the driving process. The behavior decision optimization model improves the efficiency of behavior decision in driving, and the pre-driving plan is partially corrected according to the actual driving environment to adapt to the actual driving environment. Compared with directly using the pre-driving behavior decision plan for intelligent driving behavior guidance in driving, the behavior decision optimization model improves the accuracy of behavior decision in driving, as follows:

[0115] The method for extracting multimodal feature maps for optimizing intelligent driving vehicle behavior decision-making includes:

[0116] The real-time data of the environment image and the real-time data of the environment point cloud are subjected to feature extraction using a multi-scale feature extraction network to obtain a real-time feature map of the environment image and a real-time feature map of the environment point cloud;

[0117] The real-time feature map of the environment image is divided into a real-time image consistency feature and a real-time image difference feature, and the real-time feature map of the environment point cloud is divided into a real-time point cloud consistency feature and a real-time point cloud difference feature;

[0118] The real-time image consistency feature and the real-time point cloud consistency feature are multimodally fused to obtain a real-time consistency multimodal feature map;

[0119] The real-time image difference features and the real-time point cloud difference features are multimodally fused to obtain a real-time difference multimodal feature map.

[0120] The method for constructing the decision optimization model includes:

[0121] Based on the real-time data of the environment image at the real-time position during driving and the prior data of the environment point cloud, the behavior decision parameters of the intelligent driving vehicle at the real-time position during driving are artificially calculated;

[0122] The real-time differential multimodal feature map is used as the input of the second deep neural network in the behavior decision model, and the second deep neural network in the behavior decision model outputs the differential behavior decision optimization parameter;

[0123] The similarity loss between the sum of the consistent behavior decision parameter and the differential behavior decision optimization parameter and the behavior decision parameter of the intelligent driving vehicle at the real-time position during driving, the similarity loss between the real-time consistent multimodal feature map and the prior consistent multimodal feature map, the cross loss between the real-time consistent multimodal feature map and the real-time differential multimodal feature map, and the cross loss between the prior consistent multimodal feature map and the prior differential multimodal feature map are used as the loss function for retraining the behavior decision model to obtain the behavior decision optimization model;

[0124] The behavior decision model after training is used as a behavior decision optimization model;

[0125] The behavior decision optimization model is:

[0126] [Hs_now+Hz_now]=ResNet50(Hdata_now);

[0127] [Gz_now+Gz_now]=ResNet50(Gdata_now);

[0128] Ms = BP1([Hs+Gs]);

[0129] Mz_now=BP2([Hz_now+Gz_now]);

[0130] In the formula, Ms is the consistent behavior decision parameter, Mz_now is the difference behavior decision optimization parameter,

[0131] [Hs+Gs] is a priori consistent multimodal feature map, [Hz_now+Gz_now] is a real-time difference multimodal feature map, Hs is a priori point cloud consistency feature, Gs is a priori image consistency feature, Hs_now is a real-time point cloud consistency feature, Gs_now is a real-time image consistency feature, Hz_now is a real-time point cloud difference feature, Gz_now is a real-time image difference feature, BP1 is the first deep neural network, BP2 is the second deep neural network, Hdata_now is the real-time data of the environmental point cloud, Gdata_now is the real-time data of the environmental image, and ResNet50 is a multi-scale feature extraction network;

[0132] The loss function of the behavior decision optimization model is: Lnow=Lu+Lv+Lw+Le;

[0133] Wherein, Lnow is the loss value of the behavior decision optimization model, Lu is the similarity loss between the sum of the consistent behavior decision parameters and the differential behavior decision optimization parameters and the behavior decision parameters of the intelligent driving vehicle at the real-time position during driving, Lv is the similarity loss between the real-time consistent multimodal feature map and the prior consistent multimodal feature map, Lw is the cross loss between the real-time consistent multimodal feature map and the real-time differential multimodal feature map, and Le is the cross loss between the prior consistent multimodal feature map and the prior differential multimodal feature map;

[0134] in, In the formula, Ms is the consistent behavior decision parameter, Mz_now is the differential behavior decision optimization parameter, and Mr_now is the behavior decision parameter of the intelligent driving vehicle at the real-time position during driving calculated by humans. is the L2 norm;

[0135] Where [Hs_now+Gs_now] is the real-time consistency multimodal feature map, [Hs+Gs] is the prior consistency multimodal feature map, Hs is the prior point cloud consistency feature, Gs is the prior image consistency feature, Hs_now is the real-time point cloud consistency feature, Gs_now is the real-time image consistency feature, is the L2 norm;

[0136] Lw=1 / {1+D KL([Hs_now+Gs_now]||[Hz now +Gz now ]), where

[0137] [Hs_now+Gs_now] is a real-time consistency multimodal feature map, [Hz_now+Gz_now] is a real-time difference multimodal feature map, Hs_now is a real-time point cloud consistency feature, Gs_now is a real-time image consistency feature, Hz_now is a real-time point cloud difference feature, Gz_now is a real-time image difference feature, D KL ([Hs_now+Gs_now]||[Hz_now+Gz_now]) is the KL divergence between [Hs_now+Gs_now] and [Hz_now+Gz_now];

[0138] Le=1 / {1+D KL ([Hs+Gs]||[Hz+Gz])}, where [Hs+Gs] is the prior consistency multimodal feature map, [Hz+Gz] is the prior difference multimodal feature map, Hs is the prior point cloud consistency feature, Gs is the prior image consistency feature, Hz is the prior point cloud difference feature, Gz is the prior image difference feature, D KL ([Hs+Gs]||[Hz+Gz]) is the KL divergence between [Hs+Gs] and [Hz+Gz].

[0139] In order to quickly transfer the behavioral decision-making planned before driving to the driving process, the present invention retains the available parts of the pre-planning as much as possible. When constructing the behavioral decision model and the behavioral decision optimization model, the multi-source environmental information pre-acquired before driving and the multi-source environmental information acquired in real time during driving are divided into the feature extraction of consistency features and difference features. The consistent multi-source environmental information features before driving and during driving are extracted, and the difference features between the two are simultaneously distinguished. In this regard, when transferring learning between the behavioral decision model and the behavioral decision optimization model, it is only necessary to transfer the difference features of the two, so as to calculate the behavioral decision parameters according to the environmental information that changes during driving and before driving, and correspondingly realize the optimization and correction of the behavioral decisions that are still retained in the pre-planned behavior decisions during driving, and only the behavioral decisions that are not applicable to the environmental information during driving are optimized and corrected, and local real-time adjustments are quickly made based on the priori planning.

[0140] In order to achieve feature migration only for differential features before driving and during driving, the present invention designs a model structure into a dual network structure, one network structure processes consistent features, and the other network structure processes differential features, which is convenient for the subsequent migration of the behavior decision optimization model. The differential features during driving are used to retrain the network structure that processes the differential features before driving in the behavior decision model, so that the behavior decision model is migrated to the network structure that processes the differential features during driving, and the behavior decision parameters are calculated according to the environmental information that changes during driving and before driving. The network structure that retains the consistent features is used to achieve the corresponding behavior decision that is still reserved in advance during driving.

[0141] Correspondingly, in the transfer learning from the behavior decision model to the behavior decision optimization model, the present invention divides the loss function into four parts. The first part is the similarity loss between the sum of the consistent behavior decision parameter and the differential behavior decision optimization parameter and the behavior decision parameter of the intelligent driving vehicle at the real-time position during driving. It is a measure of the difference between the output of the behavior decision optimization model and the actual output. It can ensure that the behavior decision parameters output by the behavior decision optimization model are most accurately adapted to the driving environment information. The second part is the similarity loss between the real-time consistent multimodal feature map and the prior consistent multimodal feature map. It quantifies the difference between the consistent features of the environmental information before driving and during driving. It can ensure that the consistent features of the environmental information before driving and during driving can be extracted to the greatest extent, and ensure that the pre-planned behavior decision is still retained in driving after transfer learning. The third part It is the cross loss between the real-time consistent multimodal feature map and the real-time differential multimodal feature map, and it quantifies the dependence between the consistent features and the differential features of the environmental information during driving. It can separate the consistent features and the differential features of the environmental information during driving to the greatest extent, avoid the intersection of the two, affect the extraction of the environmental information features that change during driving and before driving, and cause the error of transfer learning to increase. The fourth part is the cross loss between the prior consistent multimodal feature map and the prior differential multimodal feature map, and it can quantify the dependence between the consistent features and the differential features of the environmental information before driving. It can separate the consistent features and the differential features of the environmental information before driving to the greatest extent, avoid the intersection of the two, affect the extraction of the environmental information features that change during driving and before driving, and cause the error of transfer learning to increase.

[0142] The behavior decision parameters include yaw rate and acceleration, and the behavior decision optimization parameters include the optimized value of yaw rate and the optimized value of acceleration.

[0143] The evaluation indicators of the behavior decision model and the behavior decision planning model are recall rate and average precision.

[0144] The environmental image prior data, environmental point cloud prior data, environmental image real-time data and environmental point cloud real-time data are normalized.

[0145] The behavior decision parameters in the present invention are yaw rate and acceleration, which are universal parameters for driving. The following provides a method for converting the motion index for each vehicle, which can be adapted to various models of intelligent driving vehicles and realize the individualization of universal decision-making, specifically:

[0146]

[0147] In the formula, yawRate(t), a veh (t) are the yaw rate and acceleration input at the t-th step in the real-time position during driving, v veh (t) is the vehicle speed at the t-th step in the real-time position during driving, vx veh (t), vx veh (t) is the lateral and longitudinal components of the vehicle speed, x veh (t), y veh (t) is the horizontal and vertical coordinates of the real-time position during driving, Δt is the step length of each interaction, which is set to 0.1s in the present invention.

[0148] When the vehicle receives the yaw angular velocity and acceleration output by the behavior decision optimization module, it can update the motion state according to the above formula, including the update of state information such as acceleration, lateral and longitudinal position and speed. At the same time, it can also obtain the current lane of the vehicle from the relative position relationship between the vehicle's position coordinates and the lane line.

[0149] The resistance that a vehicle needs to overcome when driving mainly includes rolling resistance F f , road slope resistance F i , acceleration resistance F j and air resistance F w ,in,

[0150]

[0151] Where m is the mass of the car, g is the acceleration of gravity, f is the rolling resistance coefficient, α is the road slope, δ is the rotational mass conversion factor, and a veh is the vehicle acceleration, C D is the air resistance coefficient, A f is the frontal area, and ρ is the air density.

[0152] The driving force F required by the car d =F f +F i +F j +F wThe driving force of the car is transmitted to the tires by the driving motor through the main reducer and other transmission mechanisms. Therefore, the torque required by the driving motor can be calculated. In the formula, i g Represents the transmission ratio of the main reducer, η T represents the efficiency of the transmission system, r w is the tire radius.

[0153] Similarly, the angular velocity ω that the drive motor needs to maintain can be calculated from the vehicle speed m , speed n m ,as follows:

[0154]

[0155] The power P required to drive the motor m The angular velocity ω m With torque T m The result is as follows:

[0156]

[0157] Where η m is the efficiency of the drive motor.

[0158] In summary, the driving power of each intelligent driving vehicle can be determined according to the behavioral decision parameters (yaw angular velocity and acceleration) to achieve intelligent control of driving at the real-time position during driving.

[0159] like Figure 2 As shown, the present invention provides a behavior decision optimization system based on intelligent driving, which is applied to a behavior decision optimization method based on intelligent driving. The system includes:

[0160] A data acquisition unit is used to collect environmental image data and environmental point cloud data at each position of the driving path one by one on the driving path by using a laser radar and a camera mounted on the data acquisition vehicle, and the environmental image data and environmental point cloud data are marked as environmental image prior data and environmental point cloud prior data respectively;

[0161] and for collecting environmental image data and environmental point cloud data of real-time position during driving on the driving path using the laser radar and camera mounted on the intelligent driving vehicle, which are marked as environmental image real-time data and environmental point cloud real-time data respectively;

[0162] A feature processing unit, used to extract a multimodal feature map for intelligent driving vehicle behavior decision-making based on environmental image prior data and environmental point cloud prior data;

[0163] and for extracting a multimodal feature map for optimizing intelligent driving vehicle behavior decisions based on real-time environmental image data and real-time environmental point cloud data;

[0164] A model building unit, for training a deep neural network with a multimodal feature map for intelligent driving vehicle behavior decision as input and a behavior decision parameter as output to obtain a behavior decision model;

[0165] And it is used to train the behavior decision model by taking the multimodal feature map used for optimizing the behavior decision of intelligent driving vehicles as input and the behavior decision optimization parameter as output to obtain the decision optimization model.

[0166] The present invention utilizes a data collection vehicle to collect data on the driving environment of a driving path before the driving behavior occurs, and pre-plans the driving behavior of intelligent driving in the driving environment of the driving path by constructing a behavior decision model, and obtains in advance a universal driving behavior decision to guide the intelligent driving vehicle on the driving path, and then optimizes the universal driving behavior decision according to the real-time driving environment data of the intelligent driving vehicle by constructing a behavior decision model, and obtains a real-time driving behavior decision that adapts to the real-time driving environment of the intelligent driving vehicle, so that the intelligent driving vehicle can quickly obtain the optimal matching driving behavior that adapts to the real-time driving scene during the driving process, and ensure the accuracy and timeliness of the intelligent driving behavior decision.

[0167] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.

Claims

1. A behavior decision optimization method based on intelligent driving, characterized in that: The following steps are involved: Using the laser radar and camera mounted on the data collection vehicle, the environmental image data and environmental point cloud data of each position on the driving path are collected one by one, and the corresponding marks are marked as environmental image prior data and environmental point cloud prior data; Based on the environmental image prior data and the environmental point cloud prior data, a multimodal feature map for intelligent driving vehicle behavior decision-making is extracted; Taking the multimodal feature map used for intelligent driving vehicle behavior decision as input and the behavior decision parameter as output, the deep neural network is trained to obtain the behavior decision model; Using the laser radar and camera mounted on the intelligent driving vehicle on the driving path, the environmental image data and environmental point cloud data of the real-time position during driving are collected, and the data are marked as environmental image real-time data and environmental point cloud real-time data respectively; Extract multimodal feature maps for intelligent driving vehicle behavior decision optimization based on real-time environmental image data and real-time environmental point cloud data; Taking the multimodal feature map used for optimizing the behavior decision of intelligent driving vehicles as input and the behavior decision optimization parameters as output, the behavior decision model is trained to obtain a decision optimization model.

2. The behavior decision optimization method based on intelligent driving according to claim 1, characterized in that: The environmental point cloud data is constructed from the driving path scanning data acquired by the laser radar.

3. The behavior decision optimization method based on intelligent driving according to claim 2, characterized in that: The method for extracting multimodal feature maps for intelligent driving vehicle behavior decision-making includes: The environment image prior data and the environment point cloud prior data are subjected to feature extraction using a multi-scale feature extraction network to obtain an environment image prior feature map and an environment point cloud prior feature map; The prior feature map of the environment image is divided into a prior image consistency feature and a prior image difference feature, and the prior feature map of the environment point cloud is divided into a prior point cloud consistency feature and a prior point cloud difference feature; The prior image consistency feature and the prior point cloud consistency feature are multimodally fused to obtain a priori consistency multimodal feature map; The prior image difference features and the prior point cloud difference features are multimodally fused to obtain a prior difference multimodal feature map.

4. The behavior decision optimization method based on intelligent driving according to claim 3 is characterized in that: The method for constructing the behavior decision model includes: Based on the environmental image prior data and environmental point cloud prior data at each position on the driving path, the behavior decision parameters of the intelligent driving vehicle at each position on the driving path are manually calculated; The priori consistent multimodal feature map is used as an input of a first deep neural network, and the first deep neural network outputs a consistent behavior decision parameter of the intelligent driving vehicle; The prior difference multimodal feature map is used as the input of the second deep neural network, and the second deep neural network outputs the difference behavior decision parameters of the intelligent driving vehicle; The similarity loss between the sum of the consistent behavior decision parameters and the differential behavior decision parameters and the behavior decision parameters of the intelligent driving vehicle at each position on the driving path is used as a loss function for training the multi-scale feature extraction network, the first deep neural network and the second deep neural network to obtain the behavior decision model; The trained multi-scale feature extraction network, the first deep neural network and the second deep neural network are used as the behavior decision model; The behavior decision model is: [Hs+Hz]=ResNet50(Hdata_history); [Gz+Gz]=ResNet50(Gdata_history); Ms = BP1([Hs+Gs]); Mz=BP2([Hz+Gz]); Where Ms is the consistent behavior decision parameter, Mz is the differential behavior decision parameter, [Hs+Gs] is the prior consistent multimodal feature map, [Hz+Gz] is the prior differential multimodal feature map, Hs is the prior point cloud consistency feature, Gs is the prior image consistency feature, Hz is the prior point cloud difference feature, Gz is the prior image difference feature, BP1 is the first deep neural network, BP2 is the second deep neural network, Hdata_history is the environmental point cloud prior data, Gdata_history is the environmental image prior data, and ResNet50 is a multi-scale feature extraction network; The loss function of the behavior decision model is: Where L is the loss value of the behavior decision model, Ms is the consistent behavior decision parameter, Mz is the differential behavior decision parameter, and Mr is the behavior decision parameter of the intelligent driving vehicle at each position on the driving path calculated manually. is the L2 norm.

5. The behavior decision optimization method based on intelligent driving according to claim 4 is characterized in that: The method for extracting multimodal feature maps for optimizing intelligent driving vehicle behavior decision-making includes: The real-time data of the environment image and the real-time data of the environment point cloud are subjected to feature extraction using a multi-scale feature extraction network to obtain a real-time feature map of the environment image and a real-time feature map of the environment point cloud; The real-time feature map of the environment image is divided into a real-time image consistency feature and a real-time image difference feature, and the real-time feature map of the environment point cloud is divided into a real-time point cloud consistency feature and a real-time point cloud difference feature; The real-time image consistency feature and the real-time point cloud consistency feature are multimodally fused to obtain a real-time consistency multimodal feature map; The real-time image difference features and the real-time point cloud difference features are multimodally fused to obtain a real-time difference multimodal feature map.

6. The behavior decision optimization method based on intelligent driving according to claim 5, characterized in that: The method for constructing the decision optimization model includes: Based on the real-time data of the environment image at the real-time position during driving and the prior data of the environment point cloud, the behavior decision parameters of the intelligent driving vehicle at the real-time position during driving are artificially calculated; The real-time differential multimodal feature map is used as the input of the second deep neural network in the behavior decision model, and the second deep neural network in the behavior decision model outputs the differential behavior decision optimization parameter; The similarity loss between the sum of the consistent behavior decision parameter and the differential behavior decision optimization parameter and the behavior decision parameter of the intelligent driving vehicle at the real-time position during driving, the similarity loss between the real-time consistent multimodal feature map and the prior consistent multimodal feature map, the cross loss between the real-time consistent multimodal feature map and the real-time differential multimodal feature map, and the cross loss between the prior consistent multimodal feature map and the prior differential multimodal feature map are used as the loss function for retraining the behavior decision model to obtain the behavior decision optimization model; The behavior decision model after training is used as a behavior decision optimization model; The behavior decision optimization model is: [Hs_now+Hz_now]=ResNet50(Hdata_now); [Gz_now+Gz_now]=ResNet50(Gdata_now); Ms = BP1([Hs+Gs]); Mz_now=BP2([Hz_now+Gz_now]); In the formula, Ms is the consistent behavior decision parameter, Mz_now is the difference behavior decision optimization parameter, [Hs+Gs] is a priori consistent multimodal feature map, [Hz_now+Gz_now] is a real-time difference multimodal feature map, Hs is a priori point cloud consistency feature, Gs is a priori image consistency feature, Hs_now is a real-time point cloud consistency feature, Gs_now is a real-time image consistency feature, Hz_now is a real-time point cloud difference feature, Gz_now is a real-time image difference feature, BP1 is the first deep neural network, BP2 is the second deep neural network, Hdata_now is the real-time data of the environmental point cloud, Gdata_now is the real-time data of the environmental image, and ResNet50 is a multi-scale feature extraction network; The loss function of the behavior decision optimization model is: Lnow=Lu+Lv+Lw+Le; Wherein, Lnow is the loss value of the behavior decision optimization model, Lu is the similarity loss between the sum of the consistent behavior decision parameters and the differential behavior decision optimization parameters and the behavior decision parameters of the intelligent driving vehicle at the real-time position during driving, Lv is the similarity loss between the real-time consistent multimodal feature map and the prior consistent multimodal feature map, Lw is the cross loss between the real-time consistent multimodal feature map and the real-time differential multimodal feature map, and Le is the cross loss between the prior consistent multimodal feature map and the prior differential multimodal feature map; in, In the formula, Ms is the consistent behavior decision parameter, Mz_now is the differential behavior decision optimization parameter, and Mr_now is the behavior decision parameter of the intelligent driving vehicle at the real-time position during driving calculated by humans. is the L2 norm; Where [Hs_now+Gs_now] is the real-time consistency multimodal feature map, [Hs+Gs] is the prior consistency multimodal feature map, Hs is the prior point cloud consistency feature, Gs is the prior image consistency feature, Hs_now is the real-time point cloud consistency feature, Gs_now is the real-time image consistency feature, is the L2 norm; Lw = 1 / {1 + D KL ([Hs_now + Gs_now] || [Hz now + Gz now ),where [Hs_now+Gs_now] is a real-time consistency multimodal feature map, [Hz_now+Gz_now] is a real-time difference multimodal feature map, Hs_now is a real-time point cloud consistency feature, Gs_now is a real-time image consistency feature, Hz_now is a real-time point cloud difference feature, Gz_now is a real-time image difference feature, D KL ([Hs_now+Gs_now]||[Hz_now+Gz_now]) is the KL divergence between [Hs_now+Gs_now] and [Hz_now+Gz_now]; Le=1 / {1+D KL ([Hs+Gs]||[Hz+Gz])}, where [Hs+Gs] is the prior consistency multimodal feature map, [Hz+Gz] is the prior difference multimodal feature map, Hs is the prior point cloud consistency feature, Gs is the prior image consistency feature, Hz is the prior point cloud difference feature, Gz is the prior image difference feature, D KL ([Hs+Gs]||[Hz+Gz]) is the KL divergence between [Hs+Gs] and [Hz+Gz].

7. The behavior decision optimization method based on intelligent driving according to claim 1, characterized in that: The behavior decision parameters include yaw rate and acceleration, and the behavior decision optimization parameters include the optimized value of yaw rate and the optimized value of acceleration.

8. The behavior decision optimization method based on intelligent driving according to claim 6, characterized in that: The evaluation indicators of the behavior decision model and the behavior decision planning model are recall rate and average precision.

9. The behavior decision optimization method based on intelligent driving according to claim 8, characterized in that: The environmental image prior data, environmental point cloud prior data, environmental image real-time data and environmental point cloud real-time data are normalized.

10. A behavior decision optimization system based on intelligent driving, characterized in that: A behavior decision optimization method based on intelligent driving applied to any one of claims 1 to 9, the system comprising: A data acquisition unit is used to collect environmental image data and environmental point cloud data at each position of the driving path one by one on the driving path by using a laser radar and a camera mounted on the data acquisition vehicle, and the data are marked as environmental image prior data and environmental point cloud prior data respectively; And for collecting environmental image data and environmental point cloud data of real-time position during driving on the driving path by using the laser radar and camera loaded on the intelligent driving vehicle, which are marked as environmental image real-time data and environmental point cloud real-time data respectively; A feature processing unit, used to extract a multimodal feature map for intelligent driving vehicle behavior decision-making based on environmental image prior data and environmental point cloud prior data; and for extracting a multimodal feature map for optimizing intelligent driving vehicle behavior decisions based on real-time environmental image data and real-time environmental point cloud data; A model building unit, for training a deep neural network with a multimodal feature map for intelligent driving vehicle behavior decision as input and a behavior decision parameter as output to obtain a behavior decision model; And it is used to train the behavior decision model by taking the multimodal feature map used for optimizing the behavior decision of intelligent driving vehicles as input and the behavior decision optimization parameter as output to obtain the decision optimization model.

Images