A behavior decision optimization method and system based on intelligent driving

Abstract

The present application relates to the technical field of intelligent driving, and particularly relates to a behavior decision optimization method and system based on intelligent driving, comprising the following steps: taking a multi-modal feature map for intelligent driving vehicle behavior decision as input, and taking a behavior decision parameter as output, training a deep neural network to obtain a behavior decision model; taking a multi-modal feature map for intelligent driving vehicle behavior decision optimization as input, and taking a behavior decision optimization parameter as output, training the behavior decision model to obtain a decision optimization model. The present application optimizes general driving behavior decision according to real-time driving environment data of the intelligent driving vehicle by constructing the behavior decision model, obtains real-time driving behavior decision that adapts to the real-time driving environment of the intelligent driving vehicle, so that the intelligent driving vehicle quickly obtains optimal matching adaptive real-time driving behavior in the driving process, and ensures the accuracy and timeliness of the intelligent driving behavior decision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent driving, in particular to a behavior decision optimization method and system based on intelligent driving. BACKGROUND

[0002] Intelligent driving recognizes the surrounding environment, and then fully utilizes the information to understand and analyze how to proceed, that is, to realize the behavior decision of intelligent driving. There are two ways to handle the behavior decision of intelligent driving at present: expert rule type and AI type. The expert rule type, called rule-based in English, means that rules are written in advance, and these rules must be strictly followed when a decision needs to be made. For example, when preparing to overtake and change lanes, the following conditions need to be met: the road radius is greater than 500R (no lane changing on a curve); the distance to the front and rear vehicles on the target lane is more than 20m; and the speed is not more than 5km / h slower than the rear vehicle, which can overtake and change lanes. The AI type (Artificial Intelligence) simulates the human brain and understands the scene through AI algorithms to plan the behavior of intelligent driving to adapt to the driving path environment.

[0003] The behavior decision of intelligent driving in the prior art belongs to advanced behavior decision planning, or the scene information of the driving path is collected in advance, the driving behavior is planned based on the scene information, and the driving behavior obtained by the advanced planning guides the driving behavior of the intelligent driving vehicle. However, since the scene information collected by the driving behavior planning occurs before the intelligent driving behavior, the driving behavior decision obtained by the planning cannot completely match and adapt to the real-time driving scene, resulting in the phenomenon of invalid behavior decision. SUMMARY

[0004] The present application aims to provide a behavior decision optimization method and system based on intelligent driving to solve the technical problem that the driving behavior decision obtained by the planning in the prior art cannot completely match and adapt to the real-time driving scene, resulting in invalid behavior decision.

[0005] To solve the above technical problems, the present application specifically provides the following technical solutions:

[0006] A behavior decision optimization method based on intelligent driving, comprising the following steps:

[0007] Using the laser radar and camera loaded on the data collection vehicle to collect environmental image data and environmental point cloud data of each position on the driving path one by one, and corresponding to mark as environmental image prior data and environmental point cloud prior data;

[0008] Extracting a multi-modal feature map for behavior decision of an intelligent driving vehicle based on the environmental image prior data and the environmental point cloud prior data;

[0009] A deep neural network is trained by taking the multi-modal feature map for intelligent driving vehicle behavior decision as input and taking the behavior decision parameter as output, to obtain a behavior decision model;

[0010] The laser radar and the camera loaded on the intelligent driving vehicle are used to collect environment image data and environment point cloud data at real-time positions on a driving path, which are marked as environment image real-time data and environment point cloud real-time data;

[0011] A multi-modal feature map for intelligent driving vehicle behavior decision optimization is extracted based on the environment image real-time data and the environment point cloud real-time data;

[0012] The behavior decision model is trained by taking the multi-modal feature map for intelligent driving vehicle behavior decision optimization as input and taking the behavior decision optimization parameter as output, to obtain a decision optimization model.

[0013] As a preferred scheme of the present application, the environment point cloud data is constructed from driving path scanning data obtained by the laser radar.

[0014] As a preferred scheme of the present application, the extraction method of the multi-modal feature map for intelligent driving vehicle behavior decision includes:

[0015] The environment image prior data and the environment point cloud prior data are subjected to feature extraction by a multi-scale feature extraction network, to obtain environment image prior feature maps and environment point cloud prior feature maps;

[0016] The environment image prior feature maps are divided into prior image consistency features and prior image difference features, and the environment point cloud prior feature maps are divided into prior point cloud consistency features and prior point cloud difference features;

[0017] The prior image consistency features and the prior point cloud consistency features are subjected to multi-modal fusion, to obtain prior consistency multi-modal feature maps;

[0018] The prior image difference features and the prior point cloud difference features are subjected to multi-modal fusion, to obtain prior difference multi-modal feature maps.

[0019] As a preferred scheme of the present application, the construction method of the behavior decision model includes:

[0020] The behavior decision parameters of the intelligent driving vehicle at each position on the driving path are artificially calculated according to the environment image prior data and the environment point cloud prior data at each position on the driving path;

[0021] The prior consistency multi-modal feature maps are taken as input of the first deep neural network, and the consistency behavior decision parameters of the intelligent driving vehicle are output by the first deep neural network;

[0022] the prior difference multi-modal feature map as an input of the second deep neural network, and outputting a difference behavior decision variable of the intelligent driving vehicle by the second deep neural network;

[0023] a similarity loss between the sum of the consistency behavior decision variable and the difference behavior decision variable and the behavior decision variable of the intelligent driving vehicle at each position on the driving path as a loss function of the behavior decision model obtained by training the multi-scale feature extraction network, the first deep neural network and the second deep neural network;

[0024] the multi-scale feature extraction network, the first deep neural network and the second deep neural network after training as the behavior decision model;

[0025] the behavior decision model is:

[0026] [Hs+Hz]=ResNet50(Hdata_history);

[0027] [Gz+Gz]=ResNet50(Gdata_history);

[0028] Ms=BP1([Hs+Gs]);

[0029] Mz=BP2([Hz+Gz]);

[0030] In the formula, Ms is a consistency behavior decision variable, Mz is a difference behavior decision variable, [Hs+Gs] is a prior consistency multi-modal feature map, [Hz+Gz] is a prior difference multi-modal feature map, Hs is a prior point cloud consistency feature, Gs is a prior image consistency feature, Hz is a prior point cloud difference feature, Gz is a prior image difference feature, BP1 is the first deep neural network, BP2 is the second deep neural network, Hdata_history is environment point cloud prior data, Gdata_history is environment image prior data, and ResNet50 is a multi-scale feature extraction network.

[0031] the loss function of the behavior decision model is: In the formula, L is a loss value of the behavior decision model, Ms is a consistency behavior decision variable, Mz is a difference behavior decision variable, and Mr is a behavior decision variable of the intelligent driving vehicle at each position on the driving path measured by a person, is an L2 norm.

[0032] As a preferred scheme of the present application, the method for extracting a multi-modal feature map for intelligent driving vehicle behavior decision optimization comprises:

[0033] The environment image real-time data and the environment point cloud real-time data are subjected to feature extraction by using a multi-scale feature extraction network to obtain environment image real-time feature maps and environment point cloud real-time feature maps;

[0034] The environment image real-time feature maps are divided into real-time image consistency features and real-time image difference features, and the environment point cloud real-time feature maps are divided into real-time point cloud consistency features and real-time point cloud difference features;

[0035] The real-time image consistency features and the real-time point cloud consistency features are subjected to multi-modal fusion to obtain real-time consistency multi-modal feature maps;

[0036] The real-time image difference features and the real-time point cloud difference features are subjected to multi-modal fusion to obtain real-time difference multi-modal feature maps.

[0037] As a preferred scheme of the application, the construction method of the decision optimization model comprises:

[0038] According to the environment image real-time data and the environment point cloud prior data at the real-time position in driving, the behavior decision parameter of the intelligent driving vehicle at the real-time position in driving is artificially calculated;

[0039] The real-time difference multi-modal feature maps are taken as the input of the second deep neural network in the behavior decision model, and the difference behavior decision optimization parameter is output by the second deep neural network in the behavior decision model;

[0040] The similarity loss between the sum of the consistency behavior decision parameter and the difference behavior decision optimization parameter and the behavior decision parameter of the intelligent driving vehicle at the real-time position in driving, the similarity loss between the real-time consistency multi-modal feature maps and the prior consistency multi-modal feature maps, the cross loss between the real-time consistency multi-modal feature maps and the real-time difference multi-modal feature maps, and the cross loss between the prior consistency multi-modal feature maps and the prior difference multi-modal feature maps are taken as the loss function of the behavior decision optimization model obtained by retraining the behavior decision model;

[0041] The trained behavior decision model is taken as the behavior decision optimization model;

[0042] The behavior decision optimization model is:

[0043] [Hs_now+Hz_now]=ResNet50(Hdata_now);

[0044] [Gz_now+Gz_now]=ResNet50(Gdata_now);

[0045] Ms=BP1([Hs+Gs]);

[0046] Mz_now = BP2([Hz_now + Gz_now]);

[0047] wherein Ms is a consistency behavior decision parameter, Mz_now is a difference behavior decision optimization parameter,

[0048] [Hs+Gs] is a prior consistency multimodal feature map, [Hz_now+Gz_now] is a real-time difference multimodal feature map, Hs is a prior point cloud consistency feature, Gs is a prior image consistency feature, Hs_now is a real-time point cloud consistency feature, Gs_now is a real-time image consistency feature, Hz_now is a real-time point cloud difference feature, Gz_now is a real-time image difference feature, BP1 is a first deep neural network, BP2 is a second deep neural network, Hdata_now is real-time environment point cloud data, Gdata_now is real-time environment image data, and ResNet50 is a multi-scale feature extraction network;

[0049] The loss function of the behavior decision optimization model is: Lnow = Lu + Lv + Lw + Le;

[0050] wherein Lnow is a loss value of the behavior decision optimization model, Lu is a similarity loss between a sum of the consistency behavior decision parameter and the difference behavior decision optimization parameter and a behavior decision parameter of the intelligent driving vehicle at a real-time position in driving, Lv is a similarity loss between a real-time consistency multimodal feature map and a prior consistency multimodal feature map, Lw is a cross loss between the real-time consistency multimodal feature map and a real-time difference multimodal feature map, and Le is a cross loss between the prior consistency multimodal feature map and a prior difference multimodal feature map;

[0051] wherein, wherein Ms is a consistency behavior decision parameter, Mz_now is a difference behavior decision optimization parameter, and Mr_now is a behavior decision parameter of the intelligent driving vehicle at a real-time position in driving measured by a human, is an L2 norm formula;

[0052] wherein [Hs_now+Gs_now] is a real-time consistency multimodal feature map, [Hs+Gs] is a prior consistency multimodal feature map, Hs is a prior point cloud consistency feature, Gs is a prior image consistency feature, Hs_now is a real-time point cloud consistency feature, and Gs_now is a real-time image consistency feature, is an L2 norm formula;

[0053] Lw = 1 / {1+D KL ([Hs_now+Gs_now]||[Hz now +Gznow ]), wherein,

[0054] [Hs_now+Gs_now] is a real-time consistent multi-modal feature map, [Hz_now+Gz_now] is a real-time difference multi-modal feature map, Hs_now is a real-time point cloud consistent feature, Gs_now is a real-time image consistent feature, Hz_now is a real-time point cloud difference feature, and Gz_now is a real-time image difference feature, D KL ([Hs_now+Gs_now]||[Hz_now+Gz_now]) is the KL divergence between [Hs_now+Gs_now] and [Hz_now+Gz_now] ;

[0055] Le=1 / {1+D KL ([Hs+Gs]||[Hz+Gz])}, wherein, [Hs+Gs] is a prior consistent multi-modal feature map, [Hz+Gz] is a prior difference multi-modal feature map, Hs is a prior point cloud consistent feature, Gs is a prior image consistent feature, Hz is a prior point cloud difference feature, and Gz is a prior image difference feature, D KL ([Hs+Gs]||[Hz+Gz]) is the KL divergence between [Hs+Gs] and [Hz+Gz].

[0056] As a preferred scheme of the present application, the behavior decision variable includes a yaw angular velocity and an acceleration, and the behavior decision optimization variable includes an optimized value of the yaw angular velocity and an optimized value of the acceleration.

[0057] As a preferred scheme of the present application, the evaluation index of the behavior decision model and the behavior decision planning model is a recall rate and an average precision.

[0058] As a preferred scheme of the present application, the environment image prior data and the environment point cloud prior data, the environment image real-time data, and the environment point cloud real-time data are normalized.

[0059] As a preferred scheme of the present application, the present application provides a behavior decision optimization system based on intelligent driving, which is applied to a behavior decision optimization method based on intelligent driving, and the system comprises:

[0060] A data acquisition unit is configured to acquire environment image data and environment point cloud data of each position on a driving path by using a laser radar and a camera loaded on a data acquisition vehicle, and the data is marked as environment image prior data and environment point cloud prior data.

[0061] and for collecting environment image data and environment point cloud data corresponding to real-time position in driving on a driving path by using laser radar and camera loaded on the intelligent driving vehicle;

[0062] a feature processing unit configured to extract a multi-modal feature map for behavior decision of the intelligent driving vehicle based on the environment image prior data and the environment point cloud prior data;

[0063] and extract a multi-modal feature map for behavior decision optimization of the intelligent driving vehicle based on the environment image real-time data and the environment point cloud real-time data;

[0064] a model construction unit configured to train a deep neural network by taking the multi-modal feature map for behavior decision of the intelligent driving vehicle as input and taking a behavior decision parameter as output, to obtain a behavior decision model;

[0065] and train the behavior decision model by taking the multi-modal feature map for behavior decision optimization of the intelligent driving vehicle as input and taking a behavior decision optimization parameter as output, to obtain a decision optimization model.

[0066] Compared with the prior art, the present application has the following beneficial effects:

[0067] The present application collects driving environment data of the driving path before driving behavior occurs by using a data collection vehicle, and pre-plans driving behavior of intelligent driving in the driving environment of the driving path by constructing a behavior decision model, to obtain a general driving behavior decision for guiding the intelligent driving vehicle on the driving path. BRIEF DESCRIPTION OF DRAWINGS

[0068] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only exemplary, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of the provided drawings.

[0069] Figure 1 a behavior decision optimization method based on intelligent driving provided by the embodiment of the present application;

[0070] Figure 2A system block diagram of behavior decision optimization based on intelligent driving is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0071] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0072] As shown in Figure 1 The present application provides a behavior decision optimization method based on intelligent driving, comprising the following steps:

[0073] The laser radar and the camera loaded on the data collection vehicle are used to collect the environmental image data and the environmental point cloud data of each position on the driving path one by one, which are marked as environmental image prior data and environmental point cloud prior data respectively;

[0074] The multi-modal feature map used for behavior decision of the intelligent driving vehicle is extracted based on the environmental image prior data and the environmental point cloud prior data;

[0075] The deep neural network is trained with the multi-modal feature map used for behavior decision of the intelligent driving vehicle as input and the behavior decision parameters (yaw rate and acceleration) as output, to obtain a behavior decision model;

[0076] The laser radar and the camera loaded on the intelligent driving vehicle are used to collect the environmental image data and the environmental point cloud data of the real-time position during driving on the driving path, which are marked as environmental image real-time data and environmental point cloud real-time data respectively;

[0077] The multi-modal feature map used for behavior decision optimization of the intelligent driving vehicle is extracted based on the environmental image real-time data and the environmental point cloud real-time data;

[0078] The behavior decision model is trained with the multi-modal feature map used for behavior decision optimization of the intelligent driving vehicle as input and the behavior decision optimization parameters as output, to obtain a decision optimization model.

[0079] Firstly, the application establishes a behavior decision model, uses a data collection vehicle to collect environmental data on a driving path in advance, including a three-dimensional point cloud of an environmental scene and a two-dimensional image of the environmental scene, respectively corresponding to the shape, position, color, texture and other information of objects at each position on the driving path, after mastering the multi-source environmental information, the multi-source information is associated with the behavior decision of the vehicle, so that the behavior decision (i.e. yaw rate and acceleration) of the vehicle is calculated according to the environmental information on the driving path, and further, the driving motor power required by the vehicle and other vehicle hardware parameters are calculated according to the environmental information on the driving path, so that the vehicle can make automatic decision control on the environmental information on the driving path and smoothly avoid obstacles on the driving path.

[0080] Since the behavior decision model depends on the pre-collected multi-source environmental information, it belongs to the general planning before driving and cannot be completely adapted to the real scene during driving (such as pedestrians and animals suddenly appearing during driving, which do not exist in the pre-collected environmental information), at this time, the behavior decision model is difficult to achieve accurate vehicle decision control, therefore, the application further establishes a behavior decision optimization model, which optimizes the behavior decision parameters obtained by the general planning before driving to adapt to the environmental information during driving, so that the vehicle can smoothly drive during driving.

[0081] The behavior decision optimization model in the application is obtained by transfer learning based on the behavior decision model, so as to optimize the pre-planned behavior decision according to the environmental information during driving, that is, the pre-planned behavior decision is still retained during driving, only the behavior decision which is not suitable for the environmental information during driving is optimized and corrected, the local real-time adjustment is quickly made on the basis of the prior planning, the global planning is not needed during driving, the usable part of the pre-planning is retained as much as possible, the behavior decision planned before driving is quickly transferred to the driving process, the behavior decision optimization model improves the behavior decision efficiency during driving, and the pre-planning is locally corrected according to the actual driving environment, adapts to the actual driving environment, compared with directly using the pre-driving behavior decision planning to guide the intelligent driving behavior during driving, the behavior decision optimization model improves the behavior decision accuracy during driving.

[0082] The application is to realize the fast migration of the pre-driving planning behavior decision to the driving process, to retain the available part of the pre-planning as much as possible, to divide the pre-acquired multi-source environment information and the real-time acquired multi-source environment information in the driving into the consistency feature and the difference feature in the feature extraction when constructing the behavior decision model and the behavior decision optimization model, to extract the consistent multi-source environment information feature before and during the driving, and to distinguish the difference feature between the two simultaneously. In this regard, only the difference feature between the two needs to be migrated in the migration learning between the behavior decision model and the behavior decision optimization model, so as to calculate the behavior decision parameters according to the changed environment information before and during the driving, to realize the pre-planned behavior decision during the driving, and to only optimize and correct the behavior decision which is not applicable to the environment information in the driving, to realize the local real-time adjustment on the basis of the prior planning.

[0083] The application is to realize the fast migration of the pre-driving planning behavior decision to the driving process, to retain the available part of the pre-planning as much as possible, to divide the pre-acquired multi-source environment information and the real-time acquired multi-source environment information in the driving into the consistency feature and the difference feature in the feature extraction when constructing the behavior decision model and the behavior decision optimization model, to extract the consistent multi-source environment information feature before and during the driving, and to distinguish the difference feature between the two simultaneously. In this regard, only the difference feature between the two needs to be migrated in the migration learning between the behavior decision model and the behavior decision optimization model, so as to calculate the behavior decision parameters according to the changed environment information before and during the driving, to realize the pre-planned behavior decision during the driving, and to only optimize and correct the behavior decision which is not applicable to the environment information in the driving, to realize the local real-time adjustment on the basis of the prior planning.

[0084] Correspondingly, in the transfer learning of the behavior decision model to the behavior decision optimization model, the loss function is divided into four parts, the first part is the similarity loss between the sum of the consistent behavior decision parameters and the differential behavior decision optimization parameters and the behavior decision parameters of the intelligent driving vehicle at the real-time position in driving, which is the difference between the output of the behavior decision optimization model and the real output, and can play a role in ensuring that the behavior decision parameters output by the behavior decision optimization model are most accurately adapted to the driving environment information, the second part is the similarity loss between the real-time consistent multi-modal feature map and the prior consistent multi-modal feature map, which quantifies the difference between the consistent features of the pre-driving and driving environment information, and can play a role in ensuring that the consistent features of the pre-driving and driving environment information are extracted to the greatest extent, and ensuring that the pre-planned behavior decision is still retained after transfer learning, the third part is the cross loss between the real-time consistent multi-modal feature map and the real-time differential multi-modal feature map, which quantifies the dependence between the consistent features and the differential features of the driving environment information, and can play a role in separating the consistent features and the differential features of the driving environment information to the greatest extent, avoiding mutual intersection, affecting the extraction of the changed environment information features between driving and pre-driving, and causing the transfer learning error to increase, and the fourth part is the cross loss between the prior consistent multi-modal feature map and the prior differential multi-modal feature map, which can play a role in quantifying the dependence between the consistent features and the differential features of the pre-driving environment information, and can play a role in separating the consistent features and the differential features of the pre-driving environment information to the greatest extent, avoiding mutual intersection, affecting the extraction of the changed environment information features between driving and pre-driving, and causing the transfer learning error to increase.

[0085] In summary, the present application can optimize the planning parameters on the basis of pre-planning, adapt to the real scene in driving, and realize the behavior decision optimization of intelligent driving.

[0086] In the present application, the intelligent driving environment is acquired by using three-dimensional point cloud of the environment scene and two-dimensional image of the environment scene, which correspondingly acquire the shape position, color texture and other information of objects at each position on the driving path, as follows:

[0087] The environment point cloud data is constructed from the driving path scanning data acquired by the laser radar; the construction method of the environment point cloud data comprises:

[0088] The laser radar is used to scan the environment at each position in the driving path to obtain scanning data of the environment at each position in the driving path, and the scanning data comprises laser ranging value, horizontal angle value, vertical angle value and horizontal offset angle of the radar channel;

[0089] The three-dimensional coordinate data of each position environment in the driving environment is obtained by converting the scanning data of each position environment in the driving environment.

[0090] The conversion formula of the three-dimensional coordinate data is:

[0091]

[0092] In the formula, x kj is the x-axis coordinate of the kth position environment in the driving path corresponding to the jth channel of the laser radar, y kj is the y-axis coordinate of the kth position environment in the driving path corresponding to the jth channel of the laser radar, z kj is the z-axis coordinate of the kth position environment in the driving path corresponding to the jth channel of the laser radar, r kj is the laser ranging value obtained by the jth channel of the laser radar at the kth position environment in the driving path, alpha kj is the horizontal angle value obtained by the jth channel of the laser radar at the kth position environment in the driving path, delta kj is the horizontal offset angle value existing in the kth position environment in the driving path of the jth channel of the laser radar, beta kj is the vertical angle value obtained by the jth channel of the laser radar at the kth position environment in the driving path.

[0093] The three-dimensional coordinate data of the kth position environment in the driving path is {(x kj ,y kj ,z kj )|j∈[1,n]}. kj kj kj is the three-dimensional coordinate of the kth position environment in the driving path corresponding to the jth channel of the laser radar, and n is the number of channels of the laser radar.

[0094] The laser radar and the camera are used to obtain radar scanning data for perceiving the shape and position of obstacles in the driving environment and visual image data of the color and texture of the obstacles, so that multi-source information fusion is achieved for environmental perception.

[0095] The method for extracting a multi-modal feature map for intelligent driving vehicle behavior decision-making comprises:

[0096] The environmental image prior data and the environmental point cloud prior data are subjected to feature extraction by using a multi-scale feature extraction network to obtain an environmental image prior feature map and an environmental point cloud prior feature map. ​​

[0097] diversity features of the environment image prior feature map and the prior point cloud diversity features of the environment point cloud prior feature map;

[0098] The prior consistency multimodal feature map is obtained by fusing the prior image consistency features and the prior point cloud consistency features in a multi-modal manner.

[0099] The prior diversity multimodal feature map is obtained by fusing the prior image diversity features and the prior point cloud diversity features in a multi-modal manner.

[0100] The method for constructing the behavior decision model comprises:

[0101] According to the environment image prior data and the environment point cloud prior data at each position on the driving path, the behavior decision parameters of the intelligent driving vehicle at each position on the driving path are artificially calculated;

[0102] The prior consistency multimodal feature map is taken as the input of the first deep neural network, and the consistency behavior decision parameters of the intelligent driving vehicle are output by the first deep neural network.

[0103] The prior diversity multimodal feature map is taken as the input of the second deep neural network, and the diversity behavior decision parameters of the intelligent driving vehicle are output by the second deep neural network.

[0104] The similarity loss between the sum of the consistency behavior decision parameters and the diversity behavior decision parameters and the behavior decision parameters of the intelligent driving vehicle at each position on the driving path is taken as the loss function for training the multi-scale feature extraction network, the first deep neural network and the second deep neural network to obtain the behavior decision model.

[0105] The multi-scale feature extraction network, the first deep neural network and the second deep neural network after being trained are taken as the behavior decision model.

[0106] The behavior decision model is:

[0107] [Hs+Hz]=ResNet50(Hdata_history);

[0108] [Gz+Gz]=ResNet50(Gdata_history);

[0109] Ms=BP1([Hs+Gs]);

[0110] Mz=BP2([Hz+Gz]);

[0111] In the formula, Ms is a consistent behavior decision parameter, Mz is a difference behavior decision parameter, [Hs+Gs] is a prior consistent multi-modal feature map, [Hz+Gz] is a prior difference multi-modal feature map, Hs is a prior point cloud consistency feature, Gs is a prior image consistency feature, Hz is a prior point cloud difference feature, Gz is a prior image difference feature, BP1 is a first deep neural network, BP2 is a second deep neural network, Hdata_history is prior data of an environment point cloud, Gdata_history is prior data of an environment image, and ResNet50 is a multi-scale feature extraction network.

[0112] The loss function of the behavior decision model is: In the formula, L is a loss value of the behavior decision model, Ms is a consistent behavior decision parameter, Mz is a difference behavior decision parameter, Mr is a behavior decision parameter of an intelligent driving vehicle at each position on a driving path calculated by a person, is an L2 norm formula.

[0113] The application first establishes a behavior decision model, uses a data collection vehicle to pre-collect environment data on a driving path, includes three-dimensional point clouds of an environment scene and two-dimensional images of the environment scene, and respectively obtains the shape position, color texture and other information of objects at each position on the driving path, after mastering the multi-source environment information, the multi-source information is associated and mapped with the behavior decision of the vehicle, so that the behavior decision (i.e., yaw angular velocity and acceleration) of the vehicle is calculated according to the environment information on the driving path, and further, the driving motor power required by the vehicle and other vehicle hardware parameters are calculated according to the environment information on the driving path, so that the vehicle makes automatic decision control on the environment information on the driving path and smoothly avoids obstacles on the driving path.

[0114] The behavior decision optimization model in the application is obtained by transfer learning based on the behavior decision model, so as to optimize the pre-planned behavior decision according to the environment information in driving, that is, the pre-planned behavior decision is still retained in driving, only the behavior decision which is not suitable for the environment information in driving is optimized and corrected, the local real-time adjustment is quickly made on the basis of the prior planning, the global planning is not needed in the driving process, the usable part of the pre-planned planning is retained as much as possible, the behavior decision planned before driving is quickly transferred to the driving process, the behavior decision optimization model improves the behavior decision efficiency in driving, and the pre-planning is locally corrected according to the actual driving environment, adapts to the actual driving environment, compared with directly using the pre-driving behavior decision planning for driving intelligent driving behavior guidance, the behavior decision optimization model improves the behavior decision accuracy in driving, and the specific process is as follows:

[0115] The method for extracting multi-modal feature maps for intelligent driving vehicle behavior decision optimization comprises:

[0116] Real-time environmental image data and real-time environmental point cloud data are subjected to feature extraction using a multi-scale feature extraction network to obtain real-time environmental image feature maps and real-time environmental point cloud feature maps;

[0117] The real-time environmental image feature maps are divided into real-time image consistency features and real-time image difference features, and the real-time environmental point cloud feature maps are divided into real-time point cloud consistency features and real-time point cloud difference features;

[0118] The real-time image consistency features and the real-time point cloud consistency features are subjected to multi-modal fusion to obtain real-time consistency multi-modal feature maps;

[0119] The real-time image difference features and the real-time point cloud difference features are subjected to multi-modal fusion to obtain real-time difference multi-modal feature maps.

[0120] The method for constructing the decision optimization model comprises:

[0121] The behavior decision parameters of the intelligent driving vehicle at the real-time location during driving are artificially calculated according to the real-time environmental image data and the environmental point cloud prior data at the real-time location during driving;

[0122] The real-time difference multi-modal feature maps are input into the second deep neural network in the behavior decision model, and the difference behavior decision optimization parameters are output by the second deep neural network in the behavior decision model;

[0123] The similarity loss between the sum of the consistency behavior decision parameters and the difference behavior decision optimization parameters and the behavior decision parameters of the intelligent driving vehicle at the real-time location during driving, the similarity loss between the real-time consistency multi-modal feature maps and the prior consistency multi-modal feature maps, the cross loss between the real-time consistency multi-modal feature maps and the real-time difference multi-modal feature maps, and the cross loss between the prior consistency multi-modal feature maps and the prior difference multi-modal feature maps are used as the loss function for retraining the behavior decision model to obtain the behavior decision optimization model;

[0124] The trained behavior decision model is used as the behavior decision optimization model;

[0125] The behavior decision optimization model is:

[0126] [Hs_now+Hz_now]=ResNet50(Hdata_now);

[0127] [Gz_now+Gz_now]=ResNet50(Gdata_now);

[0128] Ms = BP1([Hs + Gs]);

[0129] Mz_now = BP2([Hz_now + Gz_now]);

[0130] wherein Ms is a consistency behavior decision parameter, Mz_now is a difference behavior decision optimization parameter,

[0131] [Hs + Gs] is a prior consistency multimodal feature map, [Hz_now + Gz_now] is a real-time difference multimodal feature map, Hs is a prior point cloud consistency feature, Gs is a prior image consistency feature, Hs_now is a real-time point cloud consistency feature, Gs_now is a real-time image consistency feature, Hz_now is a real-time point cloud difference feature, Gz_now is a real-time image difference feature, BP1 is a first deep neural network, BP2 is a second deep neural network, Hdata_now is real-time environment point cloud data, Gdata_now is real-time environment image data, and ResNet50 is a multi-scale feature extraction network;

[0132] The loss function of the behavior decision optimization model is: Lnow = Lu + Lv + Lw + Le;

[0133] wherein Lnow is a loss value of the behavior decision optimization model, Lu is a similarity loss between a sum of the consistency behavior decision parameter and the difference behavior decision optimization parameter and a behavior decision parameter of the intelligent driving vehicle at a real-time position in driving, Lv is a similarity loss between a real-time consistency multimodal feature map and a prior consistency multimodal feature map, Lw is a cross loss between the real-time consistency multimodal feature map and a real-time difference multimodal feature map, and Le is a cross loss between the prior consistency multimodal feature map and a prior difference multimodal feature map;

[0134] wherein, wherein Ms is a consistency behavior decision parameter, Mz_now is a difference behavior decision optimization parameter, and Mr_now is a behavior decision parameter of the intelligent driving vehicle at a real-time position in driving measured by a human, is an L2 norm formula;

[0135] wherein [Hs_now + Gs_now] is a real-time consistency multimodal feature map, [Hs + Gs] is a prior consistency multimodal feature map, Hs is a prior point cloud consistency feature, Gs is a prior image consistency feature, Hs_now is a real-time point cloud consistency feature, and Gs_now is a real-time image consistency feature, is an L2 norm formula;

[0136] Lw = 1 / {1 + D KL([Hs_now+Gs_now]||[Hz now +Gz now ]), wherein,

[0137] [Hs_now+Gs_now] is a real-time consistent multimodal feature map, [Hz_now+Gz_now] is a real-time difference multimodal feature map, Hs_now is a real-time point cloud consistency feature, Gs_now is a real-time image consistency feature, Hz_now is a real-time point cloud difference feature, Gz_now is a real-time image difference feature, D KL ([Hs_now+Gs_now]||[Hz_now+Gz_now]) is the KL divergence between [Hs_now+Gs_now] and [Hz_now+Gz_now];

[0138] Le=1 / {1+D KL ([Hs+Gs]||[Hz+Gz])}, wherein, [Hs+Gs] is a prior consistent multimodal feature map, [Hz+Gz] is a prior difference multimodal feature map, Hs is a prior point cloud consistency feature, Gs is a prior image consistency feature, Hz is a prior point cloud difference feature, Gz is a prior image difference feature, D KL ([Hs+Gs]||[Hz+Gz]) is the KL divergence between [Hs+Gs] and [Hz+Gz].

[0139] The present application is to realize the rapid migration of pre-driving planning behavior decision to driving process, and to retain the available part of pre-planning as much as possible. In the construction of behavior decision model and behavior decision optimization model, the multi-source environment information obtained in advance before driving and the multi-source environment information obtained in real time during driving are divided into two parts of consistent features and difference features. The consistent multi-source environment information features before driving and during driving are extracted, and the difference features between them are distinguished. Therefore, only the difference features need to be transferred between the behavior decision model and the behavior decision optimization model during transfer learning, so that the behavior decision parameters are calculated according to the changed environment information during driving and before driving, and the behavior decision planned in advance is still retained during driving. Only the behavior decision which is not suitable for the environment information during driving is optimized and corrected, and local real-time adjustment is realized on the basis of prior planning.

[0140] In order to correspondingly realize feature migration of only the difference features before driving and during driving, the model structure is designed into a double network structure, one network structure processes consistent features, and the other network structure processes difference features, so as to facilitate the migration of the subsequent behavior decision optimization model, the network structure processing the difference features before driving in the behavior decision model is retrained by using the difference features during driving, so as to migrate the behavior decision model into the network structure processing the difference features during driving, realize the behavior decision parameter calculation according to the changed environment information during driving and before driving, and retain the network structure processing the consistent features to realize the corresponding implementation of the behavior decision which still retains the pre-planned behavior decision during driving.

[0141] Correspondingly, in the migration learning of the behavior decision model to the behavior decision optimization model, the loss function is divided into four parts, the first part is the similarity loss between the sum of the consistent behavior decision parameters and the difference behavior decision optimization parameters and the behavior decision parameters of the intelligent driving vehicle at the real-time position during driving, which is the difference between the output of the behavior decision optimization model and the real output, and can play a role in ensuring that the behavior decision parameters output by the behavior decision optimization model are most accurately adapted to the environment information during driving, the second part is the similarity loss between the real-time consistent multi-modal feature map and the prior consistent multi-modal feature map, which is the difference quantization between the consistent features of the environment information before driving and during driving, and can play a role in ensuring that the consistent features of the environment information before driving and during driving are extracted to the greatest extent, so that the behavior decision during driving still retains the pre-planned behavior decision after migration learning, the third part is the cross loss between the real-time consistent multi-modal feature map and the real-time difference multi-modal feature map, which quantizes the dependence between the consistent features and the difference features of the environment information during driving, and can play a role in separating the consistent features and the difference features of the environment information during driving to the greatest extent, avoiding mutual intersection, affecting the extraction of the features of the changed environment information during driving and before driving, and causing the error of migration learning to increase, and the fourth part is the cross loss between the prior consistent multi-modal feature map and the prior difference multi-modal feature map, which can play a role in quantizing the dependence between the consistent features and the difference features of the environment information before driving, and can play a role in separating the consistent features and the difference features of the environment information before driving to the greatest extent, avoiding mutual intersection, affecting the extraction of the features of the changed environment information during driving and before driving, and causing the error of migration learning to increase.

[0142] The behavior decision parameters include yaw angular velocity and acceleration, and the behavior decision optimization parameters include the optimized value of the yaw angular velocity and the optimized value of the acceleration.

[0143] The evaluation indexes of the behavior decision model and the behavior decision planning model are recall rate and average precision.

[0144] The environmental image prior data and the environmental point cloud prior data, the environmental image real-time data and the environmental point cloud real-time data are normalized.

[0145] The behavior decision variable in the application is yaw rate and acceleration, which are general variables for driving, and the following provides a conversion to motion indicators for each vehicle, which can adapt to various types of intelligent driving vehicles, realize individual landing of general decisions, and specifically comprises:

[0146]

[0147] In the formula, yawRate(t) and a veh (t) are the yaw rate and acceleration of the tth step input at the real-time position during driving, v veh (t) is the vehicle speed of the tth step at the real-time position during driving, vx veh (t) and vx veh (t) are the components of the vehicle speed in the longitudinal and lateral directions, x veh (t) and y veh (t) are the longitudinal and lateral coordinates of the real-time position during driving, and Δt is the step length of each interaction, which is set to 0.1s in the application.

[0148] When the vehicle receives the yaw rate and acceleration output by the behavior decision optimization module, the motion state can be updated according to the above formula, including the update of state information such as acceleration, longitudinal and lateral positions, and speed, and the current lane of the vehicle can also be obtained from the relative position relationship between the position coordinates of the vehicle and the lane lines.

[0149] The resistance that needs to be overcome by the vehicle during driving mainly includes rolling resistance F f , road slope resistance F i , acceleration resistance F j , and air resistance F w , wherein,

[0150]

[0151] In the formula, m is the mass of the vehicle, g is the acceleration of gravity, f is the rolling resistance coefficient, α is the road slope, δ is the rotational mass conversion coefficient, a veh is the vehicle acceleration, C D is the air resistance coefficient, A f is the wind area, and ρ is the air density.

[0152] The driving force F d required by the vehicle = F f +F i +F j +F wThe driving force of the vehicle is transmitted to the tire by the driving motor through a transmission mechanism such as a final drive, and thus the torque required for the driving motor to output can be calculated where i g represents the transmission ratio of the final drive, and η T represents the efficiency of the drive train, and r w is the radius of the tire.

[0153] Similarly, the angular velocity ω m required for the driving motor to maintain can be calculated from the vehicle speed m , and the rotational speed n m , as follows:

[0154]

[0155] The power P m required for the driving motor can be calculated from the angular velocity ω m and the torque T m , as follows:

[0156]

[0157] where η m is the efficiency of the driving motor.

[0158] In summary, the driving power of each intelligent driving vehicle can be determined according to the behavior decision parameters (yaw rate and acceleration) to achieve intelligent control of driving at real-time positions during driving.

[0159] As shown in Figure 2 , the present application provides a behavior decision optimization system based on intelligent driving, which is applied to a behavior decision optimization method based on intelligent driving, and the system comprises:

[0160] a data acquisition unit for acquiring environmental image data and environmental point cloud data of each position on the driving path one by one using the laser radar and camera loaded on the data acquisition vehicle on the driving path, which are marked as environmental image prior data and environmental point cloud prior data;

[0161] and for acquiring environmental image data and environmental point cloud data of real-time positions during driving using the laser radar and camera loaded on the intelligent driving vehicle on the driving path, which are marked as environmental image real-time data and environmental point cloud real-time data;

[0162] a feature processing unit for extracting a multi-modal feature map for behavior decision of the intelligent driving vehicle based on the environmental image prior data and the environmental point cloud prior data;

[0163] and for extracting a multi-modal feature map for behavior decision optimization of the intelligent driving vehicle based on the environmental image real-time data and the environmental point cloud real-time data;

[0164] a model construction unit configured to train a deep neural network by taking a multi-modal feature map for intelligent driving vehicle behavior decision as input and taking a behavior decision parameter as output, to obtain a behavior decision model;

[0165] and train the behavior decision model by taking a multi-modal feature map for intelligent driving vehicle behavior decision optimization as input and taking a behavior decision optimization parameter as output, to obtain a decision optimization model.

[0166] The application collects driving environment data of a driving path before driving behavior occurs by using a data collection vehicle, pre-plans driving behavior of intelligent driving in the driving environment of the driving path by constructing a behavior decision model, obtains a general driving behavior decision for guiding intelligent driving vehicles on the driving path in advance, optimizes the general driving behavior decision according to real-time driving environment data of the intelligent driving vehicle by constructing a behavior decision model, and obtains a real-time driving behavior decision that is adapted to the real-time driving environment of the intelligent driving vehicle, so that the intelligent driving vehicle quickly obtains an optimal matching adaptive real-time driving behavior in the driving process, and ensures the accuracy and timeliness of the intelligent driving behavior decision.

[0167] The above examples are only exemplary embodiments of the application and are not used to limit the application, and the protection scope of the application is defined by the claims. Those skilled in the art can make various modifications or equivalent replacements to the application within the spirit and protection scope of the application, and such modifications or equivalent replacements are also regarded as falling within the protection scope of the application.

Claims

1. A behavior decision optimization method based on intelligent driving, characterized in that, Includes the following steps: Using lidar and cameras mounted on a data acquisition vehicle, environmental image data and environmental point cloud data are collected one by one at each location along the driving path, and are correspondingly labeled as environmental image prior data and environmental point cloud prior data. Multimodal feature maps for intelligent driving vehicle behavior decision-making are extracted based on prior environmental image data and prior environmental point cloud data. Using multimodal feature maps for intelligent driving vehicle behavior decision-making as input and behavior decision parameters as output, a deep neural network is trained to obtain a behavior decision-making model. Using lidar and cameras mounted on intelligent driving vehicles, environmental image data and environmental point cloud data of the real-time location during driving are collected along the driving path and labeled as real-time environmental image data and real-time environmental point cloud data. Multimodal feature maps for optimizing intelligent driving vehicle behavior decisions are extracted based on real-time environmental image data and real-time environmental point cloud data. Using multimodal feature maps for optimizing the behavior decisions of intelligent driving vehicles as input and behavior decision optimization parameters as output, the behavior decision model is trained to obtain the behavior decision optimization model. The behavioral decision optimization model is obtained by transfer learning based on the behavioral decision model. When constructing the behavioral decision-making model and the behavioral decision-making optimization model, the multi-source environmental information acquired before driving and the multi-source environmental information acquired in real time during driving are divided into two parts for feature extraction: consistent features and differential features. The consistent multi-source environmental information features before driving and during driving are extracted, and the differential features between the two are distinguished simultaneously. Furthermore, when transferring learning between the behavioral decision-making model and the behavioral decision-making optimization model, only the differential features need to be transferred. Thus, behavioral decision parameters are calculated based on the environmental information that changes during driving and before driving. This enables the optimization and correction of behavioral decisions that are not applicable to the environmental information during driving, while still retaining the pre-planned behavioral decisions during driving. This allows for rapid local real-time adjustments based on prior planning.

2. The behavior decision optimization method based on intelligent driving according to claim 1, characterized in that: The environmental point cloud data is constructed from the driving path scanning data acquired by LiDAR.

3. The behavior decision optimization method based on intelligent driving according to claim 2, characterized in that: Methods for extracting multimodal feature maps for intelligent driving vehicle behavior decision-making include: The environmental image prior data and the environmental point cloud prior data are used to extract features using a multi-scale feature extraction network to obtain environmental image prior feature maps and environmental point cloud prior feature maps. The prior feature map of the environment image is divided into prior image consistency features and prior image difference features, and the prior feature map of the environment point cloud is divided into prior point cloud consistency features and prior point cloud difference features. The prior image consistency features and the prior point cloud consistency features are fused in a multimodal manner to obtain a prior consistency multimodal feature map. The prior image difference features and the prior point cloud difference features are fused in a multimodal manner to obtain a prior difference multimodal feature map.

4. The behavior decision optimization method based on intelligent driving according to claim 3, characterized in that: The method for constructing the behavioral decision-making model includes: Based on prior environmental image data and prior environmental point cloud data at various locations along the driving path, the behavioral decision-making parameters of the intelligent driving vehicle at various locations along the driving path are manually calculated. The prior consistent multimodal feature map is used as the input of the first deep neural network, and the first deep neural network outputs the consistent behavior decision parameters of the intelligent driving vehicle. The prior differential multimodal feature map is used as the input of the second deep neural network, and the second deep neural network outputs the differential behavior decision parameters of the intelligent driving vehicle. The similarity loss between the sum of consistent and differential behavioral decision parameters and the behavioral decision parameters of the intelligent driving vehicle at each location on the driving path is used as the loss function for training the multi-scale feature extraction network, the first deep neural network, and the second deep neural network to obtain the behavioral decision model. The trained multi-scale feature extraction network, the first deep neural network, and the second deep neural network are used as the behavior decision model. The behavioral decision-making model is as follows: [Hs+Hz]=ResNet50(Hdata_history); [Gz+Gz]= ResNet50(Gdata_history); Ms = BP1([Hs + Gs]); Mz = BP2([Hz + Gz]); In the formula, Ms is the consistent behavior decision parameter, Mz is the differential behavior decision parameter, [Hs+Gs] is the prior consistent multimodal feature map, [Hz+Gz] is the prior differential multimodal feature map, Hs is the prior point cloud consistency feature, Gs is the prior image consistency feature, Hz is the prior point cloud differential feature, Gz is the prior image differential feature, BP1 is the first deep neural network, BP2 is the second deep neural network, Hdata_history is the prior environmental point cloud data, Gdata_history is the prior environmental image data, and ResNet50 is the multi-scale feature extraction network. The loss function of the behavioral decision-making model is: In the formula, L is the loss value of the behavior decision model, Ms is the consistent behavior decision parameter, Mz is the differential behavior decision parameter, and Mr is the behavior decision parameter of the intelligent driving vehicle at each location on the driving path calculated by humans. It is an L2 norm.

5. The behavior decision optimization method based on intelligent driving according to claim 4, characterized in that: Methods for extracting multimodal feature maps for optimizing the behavior decisions of intelligent driving vehicles include: The real-time environmental image data and real-time environmental point cloud data are used to extract features using a multi-scale feature extraction network to obtain real-time feature maps of environmental images and real-time feature maps of environmental point clouds. The real-time feature map of the environment image is divided into real-time image consistency features and real-time image difference features, and the real-time feature map of the environment point cloud is divided into real-time point cloud consistency features and real-time point cloud difference features. Real-time image consistency features and real-time point cloud consistency features are fused in a multimodal manner to obtain a real-time consistent multimodal feature map. Multimodal fusion of real-time image differential features and real-time point cloud differential features yields a real-time differential multimodal feature map.

6. The behavior decision optimization method based on intelligent driving according to claim 5, characterized in that: The method for constructing the behavioral decision optimization model includes: Based on real-time environmental image data and prior environmental point cloud data at the real-time location during driving, the behavioral decision parameters of the intelligent driving vehicle at the real-time location during driving are manually calculated. The real-time differential multimodal feature map is used as the input to the second deep neural network in the behavior decision model, and the differential behavior decision optimization parameters are output by the second deep neural network in the behavior decision model. The similarity loss between the sum of consistent behavior decision parameters and differential behavior decision optimization parameters and the behavior decision parameters of the intelligent driving vehicle at the real-time location during driving, the similarity loss between the real-time consistent multimodal feature map and the prior consistent multimodal feature map, the cross loss between the real-time consistent multimodal feature map and the real-time differential multimodal feature map, and the cross loss between the prior consistent multimodal feature map and the prior differential multimodal feature map are used as the loss function for retraining the behavior decision model to obtain the behavior decision optimization model. The trained behavioral decision-making model is used as the behavioral decision-making optimization model. The behavioral decision optimization model is as follows: [Hs_now+Hz_now]=ResNet50(Hdata_now); [Gz_now +Gz_now]= ResNet50(Gdata_now); Ms = BP1([Hs + Gs]); Mz_now =BP2([Hz_now +Gz_now]); In the formula, Ms is the consistency behavior decision parameter, Mz_now is the difference behavior decision optimization parameter, [Hs+Gs] is the prior consistency multimodal feature map, [Hz_now +Gz_now] is the real-time difference multimodal feature map, Hs is the prior point cloud consistency feature, Gs is the prior image consistency feature, Hs_now is the real-time point cloud consistency feature, Gs_now is the real-time image consistency feature, Hz_now is the real-time point cloud difference feature, Gz_now is the real-time image difference feature, BP1 is the first deep neural network, BP2 is the second deep neural network, Hdata_now is the real-time environmental point cloud data, Gdata_now is the real-time environmental image data, and ResNet50 is the multi-scale feature extraction network. The loss function of the behavioral decision optimization model is: Linw = Lu + Lv + Lw + Le; In the formula, Lenw is the loss value of the behavior decision optimization model, Lu is the similarity loss between the sum of consistent behavior decision parameters and differential behavior decision optimization parameters and the behavior decision parameters of the intelligent driving vehicle at the real-time location during driving, Lv is the similarity loss between the real-time consistent multimodal feature map and the prior consistent multimodal feature map, Lw is the cross loss between the real-time consistent multimodal feature map and the real-time differential multimodal feature map, and Le is the cross loss between the prior consistent multimodal feature map and the prior differential multimodal feature map. in, In the formula, Ms is the consistent behavior decision parameter, Mz_now is the differential behavior decision optimization parameter, and Mr_now is the behavior decision parameter of the intelligent driving vehicle at the real-time location during driving, calculated by the driver. It is an L2 norm form; In the formula, [Hs_now+Gs_now] represents the real-time consistent multimodal feature map, [Hs+Gs] represents the prior consistent multimodal feature map, Hs represents the prior point cloud consistency feature, Gs represents the prior image consistency feature, Hs_now represents the real-time point cloud consistency feature, and Gs_now represents the real-time image consistency feature. It is an L2 norm form; In the formula, [Hs_now+Gs_now] is the real-time consistency multimodal feature map, [Hz_now+Gz_now] is the real-time difference multimodal feature map, Hs_now is the real-time point cloud consistency feature, Gs_now is the real-time image consistency feature, Hz_now is the real-time point cloud difference feature, and Gz_now is the real-time image difference feature. The KL divergence between [Hs_now+Gs_now] and [Hz_now+Gz_now]; In the formula, [Hs + Gs] represents the prior consistent multimodal feature map, [Hz + Gz] represents the prior dissimilar multimodal feature map, Hs represents the prior point cloud consistency feature, Gs represents the prior image consistency feature, Hz represents the prior point cloud dissimilar feature, and Gz represents the prior image dissimilar feature. Let KL divergence be between [Hs+Gs] and [Hz+Gz].

7. The behavior decision optimization method based on intelligent driving according to claim 1, characterized in that: Behavioral decision parameters include yaw rate and acceleration, while behavioral decision optimization parameters include optimized values ​​for yaw rate and acceleration.

8. The behavior decision optimization method based on intelligent driving according to claim 6, characterized in that: The evaluation metrics for the behavioral decision-making model and the behavioral decision-making optimization model are recall and average precision.

9. The behavior decision optimization method based on intelligent driving according to claim 8, characterized in that: The prior data of environmental images and environmental point clouds, as well as the real-time data of environmental images and environmental point clouds, are normalized.

10. A behavior decision optimization system based on intelligent driving, characterized in that, The system, which is applied to the behavior decision optimization method based on intelligent driving according to any one of claims 1-9, comprises: The data acquisition unit is used to collect environmental image data and environmental point cloud data at each location along the driving path using the lidar and camera mounted on the data acquisition vehicle, and the corresponding data are labeled as environmental image prior data and environmental point cloud prior data. And environmental image data and environmental point cloud data of the real-time location during driving are collected by using lidar and cameras mounted on intelligent driving vehicles along the driving path, and are correspondingly labeled as real-time environmental image data and real-time environmental point cloud data. The feature processing unit is used to extract multimodal feature maps for intelligent driving vehicle behavior decision-making based on prior environmental image data and prior environmental point cloud data. And multimodal feature maps for optimizing intelligent driving vehicle behavior decisions based on real-time environmental image data and real-time environmental point cloud data; The model building unit is used to train a deep neural network with multimodal feature maps for intelligent driving vehicle behavior decision-making as input and behavior decision parameters as output to obtain a behavior decision-making model. And a behavior decision optimization model is obtained by training a behavior decision model with multimodal feature maps for optimizing the behavior decision of intelligent driving vehicles as input and behavior decision optimization parameters as output.

Images