Training method and device for driving strategy model of vehicle and terminal equipment

Abstract

The invention is suitable for the technical field of autonomous driving, and provides a vehicle driving strategy model training method comprising the following steps: obtaining fusion features of driving strategy sample data of a plurality of vehicles, a current iteration frequency of a global driving strategy model and a current model parameter, the fusion features of the driving strategy sample data are obtained by fusion after residual features and accumulated features are extracted from the driving strategy sample data through a global driving strategy model, a current updating mode of the global driving strategy model is determined according to the current number of iterations, and the current updating mode comprises local gradient updating and global aggregation updating; and based on the current updating mode, the driving strategy model of the vehicle is updated according to the driving strategy sample data of each vehicle and the current model parameters of the global driving strategy model, and the updated driving strategy model of the vehicle is obtained. According to the invention, the problems of long training time and low efficiency caused by training the driving strategy model through the existing federal learning method can be solved.

Description

Technical Field

[0001] The present application belongs to the field of autonomous driving technology, and in particular, relates to a training method, apparatus and terminal device for a vehicle driving strategy model. Background Art

[0002] Traditional autonomous vehicles often use centralized large data sets to train traditional machine learning models, and the trained traditional machine learning models are used to decide the driving plans of autonomous vehicles. Although this driving strategy model trained with centralized large data sets improves the accuracy of autonomous vehicles' driving strategies, the collection of centralized big data also brings the risk of user data privacy leakage.

[0003] As an emerging distributed machine learning framework, federated learning allows multiple devices to jointly train a model through the coordination of a central server without sharing original data. This can avoid the direct exposure of private data and effectively protect the privacy of user data.

[0004] However, the existing federated learning method focuses on cross-data center training, and still relies on a central server to coordinate all parties. The model parameters of the driving policy model are transmitted through the central server, and the model training efficiency is low. Once a problem occurs in the central server, the entire training process will be affected, increasing the probability of single point failure risk of the entire system. In addition, data from different data centers may have heterogeneity, such as data distribution, feature space, or label space. This heterogeneity may make it difficult for the driving policy model to converge globally or degrade performance, making the model training time longer. The model parameters of the driving policy model are transmitted through a third party, resulting in a technical problem of low model training efficiency. Summary of the invention

[0005] The embodiments of the present application provide a method, apparatus, and terminal device for a vehicle driving strategy model, which can solve the problem of long training time and low efficiency caused by training the driving strategy model through the existing federated learning method.

[0006] In a first aspect, an embodiment of the present application provides a method for training a driving strategy model of a vehicle, comprising: Acquire fusion features of driving strategy sample data of multiple vehicles, and the current iteration number and current model parameters of a pre-built global driving strategy model; wherein the fusion features of the driving strategy sample data are obtained by fusing residual features and cumulative features extracted from the driving strategy sample data by the global driving strategy model, the current iteration number is used to indicate the training update round currently being conducted by the global driving strategy model, and the current model parameters include: model parameters corresponding to the driving strategy model of each of the vehicles in the global driving strategy model; Determining a current update mode of the global driving strategy model according to the current number of iterations, wherein the current update mode includes: local gradient update and global aggregation update; Based on the current updating method, the driving strategy model of the vehicle in the global driving strategy model is updated according to the fusion features of the driving strategy sample data of each vehicle and the current model parameters of the global driving strategy model to obtain an updated driving strategy model of the vehicle.

[0007] In a possible implementation manner of the first aspect, determining a current update manner of the global driving strategy model according to the current number of iterations includes: Determining whether the current number of iterations meets a preset condition determined based on a preset aggregation period; When the current number of iterations does not meet a preset condition, determining that the current updating mode of the global driving strategy model is a local gradient update; When the current number of iterations meets a preset condition, it is determined that the current updating mode of the global driving strategy model is global aggregation updating.

[0008] In a possible implementation manner of the first aspect, when the current update mode of the global driving strategy model is local gradient update, updating the driving strategy model of the vehicle in the global driving strategy model based on the current update mode according to the fusion features of the driving strategy sample data of each vehicle and the current model parameters of the global driving strategy model to obtain the updated driving strategy model of the vehicle includes: The driving strategy of the vehicle is predicted by using the driving strategy model of the vehicle according to the fusion characteristics of the driving strategy sample data of the vehicle, and a driving strategy prediction value corresponding to the driving strategy sample data is obtained; wherein the driving strategy sample data includes an actual driving strategy value; According to the actual driving strategy value in the driving strategy sample data and the predicted driving strategy value corresponding to the driving strategy sample data, based on a preset local loss function, a current gradient of the driving strategy model of the vehicle is calculated; According to the current gradient of the driving strategy model of the vehicle, the model parameters corresponding to the driving strategy model of the vehicle are updated to obtain the updated model parameters corresponding to the driving strategy model of the vehicle; An updated driving strategy model of the vehicle is determined according to the updated model parameters corresponding to the driving strategy model of the vehicle.

[0009] In a possible implementation manner of the first aspect, when the current update mode of the global driving strategy model is global aggregate update, updating the driving strategy model of the vehicle in the global driving strategy model based on the current update mode according to the fusion features of the driving strategy sample data of each vehicle and the current model parameters of the global driving strategy model to obtain the updated driving strategy model of the vehicle includes: Acquiring a communication resource of each of the vehicles and an initial communication path between a plurality of the vehicles; Optimizing the initial communication paths between the plurality of vehicles according to the communication resources of each of the vehicles to obtain the optimal communication paths between the plurality of vehicles; Determining a neighbor vehicle of each of the vehicles according to an optimal communication path between the plurality of vehicles; According to the model parameters corresponding to the driving strategy models of the neighboring vehicles of the vehicle, the model parameters corresponding to the driving strategy model of the vehicle are aggregated and updated to obtain the updated model parameters corresponding to the driving strategy model of the vehicle; An updated driving strategy model of the vehicle is determined according to the updated model parameters corresponding to the driving strategy model of the vehicle.

[0010] In a possible implementation manner of the first aspect, the step of aggregating and updating the model parameters corresponding to the driving strategy model of the vehicle according to the model parameters corresponding to the driving strategy models of neighboring vehicles of the vehicle to obtain the updated model parameters corresponding to the driving strategy model of the vehicle includes: Constructing a consensus matrix for each of the vehicles according to the number of neighboring vehicles of each of the vehicles; According to the consensus matrix of the vehicle, a weighted sum is performed on the model parameters corresponding to the driving strategy model of each of the neighboring vehicles of the vehicle to obtain updated model parameters corresponding to the driving strategy model of the vehicle.

[0011] In a possible implementation manner of the first aspect, obtaining fusion features of driving strategy sample data of multiple vehicles includes: Inputting the driving strategy sample data of each vehicle into the driving strategy model of the vehicle, performing normalization preprocessing on the driving strategy sample data, and obtaining preprocessed driving strategy sample data; Performing multiple feature extractions on the preprocessed driving strategy sample data to obtain fusion features of the driving strategy sample data.

[0012] In a possible implementation manner of the first aspect, performing multiple feature extractions on the preprocessed driving strategy sample data to obtain fusion features of the driving strategy sample data includes: Extracting local features of the preprocessed driving strategy sample data through the convolution layer and the pooling layer in the driving strategy model to obtain local features of the driving strategy sample data; Extracting residual block features of local features of the driving strategy sample data through multiple residual blocks in the driving strategy model to obtain multiple residual features of the driving strategy sample data; Through multiple global average pooling layers in the driving strategy model, multiple residual features of the driving strategy sample data are respectively pooled and summarized to obtain multiple global pooling features of the driving strategy sample data; Performing weighted calculation on multiple global pooling features of the driving strategy sample data through an accumulation layer in the driving strategy model to obtain accumulation features of the driving strategy sample data; The accumulated features of the driving strategy sample data and the multiple residual features of the driving strategy sample data are fused through the aggregation layer in the driving strategy model to obtain the fused features of the driving strategy sample data.

[0013] In a second aspect, an embodiment of the present application provides a training device for a vehicle driving strategy model, comprising: A data acquisition module, used to obtain fusion features of driving strategy sample data of multiple vehicles, and the current iteration number and current model parameters of a pre-built global driving strategy model; wherein the fusion features of the driving strategy sample data are obtained by fusing residual features and cumulative features extracted from the driving strategy sample data by the global driving strategy model, the current iteration number is used to indicate the training update round currently being conducted by the global driving strategy model, and the current model parameters include: model parameters corresponding to the driving strategy model of each of the vehicles in the global driving strategy model; A model update mode determination module, used to determine a current update mode of the global driving strategy model according to the current number of iterations, wherein the current update mode includes: local gradient update and global aggregation update; A model updating module is used to update the driving strategy model of the vehicle in the global driving strategy model based on the current updating mode and according to the fusion features of the driving strategy sample data of each vehicle and the current model parameters of the global driving strategy model to obtain an updated driving strategy model of the vehicle.

[0014] In a third aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the methods described above when executing the computer program.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements any of the methods described above.

[0016] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute any of the methods described above.

[0017] Compared with the prior art, the embodiments of the present application have the following beneficial effects: by obtaining the fusion features of driving strategy sample data of multiple vehicles, and the current iteration number and current model parameters of a pre-built global driving strategy model; according to the current iteration number, determining the current update mode of the global driving strategy model; and the current update mode includes local gradient update and global aggregation update, the current iteration number is used to indicate the training update round currently being performed by the global driving strategy model, based on the current update mode, according to the fusion features of the driving strategy sample data of each vehicle and the current model parameters of the global driving strategy model, the driving strategy model of the vehicle in the global driving strategy model is updated to obtain the updated driving strategy model of the vehicle; since the current model parameters include the model parameters corresponding to the driving strategy model of each vehicle in the global driving strategy model, the fusion features of the driving strategy sample data The features are obtained by fusing residual features and cumulative features extracted from driving strategy sample data by a global driving strategy model, so that the updated driving strategy model of each vehicle is obtained by updating the model parameters according to the driving strategy sample data of the vehicle and the model parameters corresponding to the driving strategy models of multiple vehicles. The training process does not need to obtain the driving strategy sample data of other vehicles, thereby protecting the data privacy between vehicles and eliminating the need to set up a third-party server for communication. In addition, the fused features are obtained by fusing the residual features and the cumulative features, thereby saving the time for updating the model parameters, making the update efficiency of the driving strategy model of each vehicle higher, the training time shorter, and the recognition accuracy of the trained driving strategy model higher, thereby solving the problem of long training time and low efficiency caused by training the driving strategy model through the existing federated learning method. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 It is a flowchart of a method for training a vehicle driving strategy model provided by an embodiment of the present application; Figure 2 is a flow chart of a method for updating model parameters corresponding to a driving strategy model of a vehicle provided in another embodiment of the present application; Figure 3 is a flow chart of a method for updating model parameters corresponding to another vehicle driving strategy model provided by another embodiment of the present application; Figure 4It is a structural schematic diagram of a communication path of a vehicle provided by an embodiment of the present application; Figure 5 is a flow chart of another method for training a driving strategy model of a vehicle provided by another embodiment of the present application; Figure 6 It is a structural schematic diagram of a vehicle driving strategy model training device provided by an embodiment of the present application; Figure 7 It is a structural diagram of a terminal device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0020] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0021] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0022] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0023] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0024] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0025] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0026] The Deep Federated Learning (DFL) framework is a decentralized deep federated learning framework that can use overlay networks to optimize communication between nodes, making the model more convergent and protecting data privacy between nodes.

[0027] The Federated autonomous driving network (FAD Net) is a deep network that can utilize the deep federated learning framework. It introduces a residual structure and multi-scale feature fusion to make model training easier. Multi-scale feature fusion can use information at different scales to improve the accuracy of disparity estimation, thereby improving model stability, ensuring model convergence, and dealing with the problem of unbalanced data distribution.

[0028] The present application embodiment provides a method for training a vehicle driving strategy model. Figure 1 , Figure 1 It is a flowchart of a method for training a driving strategy model of a vehicle provided by an embodiment of the present application, including: Step S11, obtaining fusion features of driving strategy sample data of multiple vehicles, and the current iteration number and current model parameters of a pre-built global driving strategy model; wherein the fusion features of the driving strategy sample data are obtained by fusing residual features and cumulative features extracted from the driving strategy sample data by the global driving strategy model, the current iteration number is used to indicate the training update round currently being conducted by the global driving strategy model, and the current model parameters include: model parameters corresponding to the driving strategy model of each vehicle in the global driving strategy model; Step S12, determining a current update mode of the global driving strategy model according to the current number of iterations, wherein the current update mode includes: local gradient update and global aggregation update; Step S13, based on the current updating mode, according to the fusion features of the driving strategy sample data of each vehicle and the current model parameters of the global driving strategy model, the driving strategy model of the vehicle in the global driving strategy model is updated to obtain the updated driving strategy model of the vehicle.

[0029] Specifically, for autonomous driving vehicles, they form a federal autonomous driving network with other surrounding vehicles. Each vehicle serves as a data island and corresponds to a pre-built driving strategy model. The driving strategy models of all vehicles constitute a global driving strategy model.

[0030] It should be noted that the network of the driving strategy model can be built using a federated autonomous driving network, and this embodiment does not impose any specific limitation.

[0031] Specifically, for vehicles that serve as data islands, each vehicle corresponds to multiple driving strategy sample data, and each driving strategy sample data includes the vehicle surround view image of the vehicle and the annotated driving strategy actual value. Among them, the vehicle surround view image is an image collected by the camera, radar sensor and other acquisition devices around the vehicle to identify the surrounding environment, such as the vehicle front view image, vehicle rear view image, etc. The vehicle surround view image includes road information, such as surrounding obstacles, road conditions, vehicle motion information or movable objects, etc. The annotated driving strategy actual value is the decision made by the vehicle, such as the steering angle of the vehicle, the speed of the vehicle, etc.

[0032] The vehicle driving strategy model is used to extract features from the vehicle surround image in the driving strategy sample data by inputting driving strategy sample data, extracting residual features and cumulative features of the driving strategy sample data, and then fusing the extracted residual features and cumulative features to obtain the fused features of the driving strategy sample data. Then, based on the fused features of the driving strategy sample data, a decision is made on the vehicle's driving strategy. For example, a vehicle surround image is input into the driving strategy model, and after prediction and recognition by the vehicle's driving strategy model, driving strategies such as turning 10 degrees, accelerating the vehicle, and braking the vehicle are output.

[0033] For each vehicle's driving strategy model in the global driving strategy model, there are corresponding model parameters at the beginning. When the global driving strategy model is trained, after the global driving strategy model is updated through multiple rounds of training, the model parameters corresponding to the driving strategy model of each vehicle are updated through multiple rounds, and the driving strategy model after the model parameters are updated is obtained, that is, the updated driving strategy model of the vehicle. Among them, the model parameters corresponding to the driving strategy model of each vehicle at the beginning are pre-set when the network model is constructed. The multiple rounds of training of the global driving strategy model can be distributed training, that is, the driving strategy models of multiple vehicles can be trained simultaneously, thereby saving the model training time and improving the model training efficiency.

[0034] The global driving strategy model is updated in different ways during each round of training and can be either local gradient update or global aggregate update. Local gradient update is to perform local training on the driving strategy model of the vehicle based on the fusion features extracted from the driving strategy sample data of each vehicle in the federated autonomous driving network, and update the model parameters corresponding to the driving strategy model to obtain the updated driving strategy model of the vehicle. Global aggregate update is to aggregate and exchange the model parameters of the driving strategy models of all vehicles in the federated autonomous driving network through network communication, weighting, etc., and aggregate and update to obtain the updated driving strategy model of each vehicle.

[0035] At present, when the global driving policy model is trained for this round, the current update mode of the global driving policy model can be determined as local gradient update or global aggregate update according to the current iteration number. The current iteration number indicates the training update round that the global driving policy model is currently undergoing. For example, initially, the training update round that the global driving policy model is currently undergoing is 1, and the current iteration number is 1. The current update mode of the global driving policy model is determined as local gradient update or global aggregate update according to the current iteration number.

[0036] After determining the current update method, the global driving strategy model is updated for this round of training: based on the current update method, for each vehicle in the global driving strategy model, the model parameters of the driving strategy model of the vehicle are updated according to the driving strategy sample data of the vehicle and the current model parameters of the global driving strategy model, to obtain the updated model parameters corresponding to the driving strategy model of the vehicle and the updated driving strategy model of the vehicle.

[0037] It should be noted that, when the global driving policy model has completed multiple training update rounds, the training of the global driving policy model can be terminated according to a pre-set termination condition. The pre-set termination condition can be any one of the following: reaching a pre-set training update round, the loss function of the updated global driving policy model converges, or the performance of the updated global driving policy model meets user requirements.

[0038] Specifically, after repeatedly executing the above steps S12 and S13 for multiple times, the finally trained driving strategy models corresponding to each vehicle are obtained.

[0039] It should be noted that the above driving strategy model can not only be used in the field of autonomous driving to predict and identify driving strategies, but can also be used in fields with different user needs according to the different input driving strategy sample data, especially in scenarios with strict requirements on user data privacy. For example: unmanned vehicles in public transportation systems, logistics distribution vehicles, etc., can efficiently train models with excellent performance while protecting user data privacy.

[0040] It can be understood that the technical solution provided by the present embodiment obtains the fusion features of driving strategy sample data of multiple vehicles, and the current iteration number and current model parameters of the pre-built global driving strategy model; determines the current update mode of the global driving strategy model according to the current iteration number; and the current update mode includes local gradient update and global aggregation update, the current iteration number is used to indicate the training update round that the global driving strategy model has currently completed, based on the current update mode, according to the fusion features of the driving strategy sample data of each vehicle and the current model parameters of the global driving strategy model, updates the driving strategy model of the vehicle in the global driving strategy model to obtain the updated driving strategy model of the vehicle; since the current model parameters include the model parameters corresponding to the driving strategy model of each vehicle in the global driving strategy model, the fusion features of the driving strategy sample data The global driving strategy model is used to extract residual features and cumulative features from driving strategy sample data and then fuse them, so that the updated driving strategy model of each vehicle is obtained by updating the model parameters based on the driving strategy sample data of the vehicle and the model parameters corresponding to the driving strategy models of multiple vehicles. The training process does not need to obtain driving strategy sample data of other vehicles, thereby protecting the data privacy between vehicles and saving time for updating model parameters. In addition, the fused features are obtained by fusing the residual features and the cumulative features, so that the update efficiency of the driving strategy model of each vehicle is higher, the training time is shorter, and the recognition accuracy of the trained driving strategy model is higher, thereby solving the problems of long training time and low efficiency caused by training driving strategy models through existing federated learning methods.

[0041] In a possible implementation, step S12, determining the current update mode of the global driving strategy model according to the current iteration number, includes: Step S121, determining whether the current number of iterations meets a preset condition determined based on a preset aggregation period; Step S122, when the current number of iterations does not meet the preset condition, determining that the current update mode of the global driving strategy model is local gradient update; Step S123: When the current number of iterations meets a preset condition, it is determined that the current update mode of the global driving strategy model is a global aggregation update.

[0042] Specifically, when determining the current update mode of the global driving strategy model according to the current number of iterations, the method includes: Determine whether the current number of iterations meets the preset condition determined based on the preset aggregation period. The preset condition is an integer multiple of the preset aggregation period, and the aggregation period is the number of rounds of training and updating required between two consecutive aggregations of the global driving strategy model. For example, assuming that the aggregation period is s+1, the preset condition is an integer multiple of s+1.

[0043] When the current number of iterations does not meet the preset conditions, that is, when the current number of iterations is not an integer multiple of the aggregation period, the current update mode of the global driving strategy model is determined to be local gradient update, that is, the driving strategy model of each vehicle in the global driving strategy model is locally gradient updated according to its corresponding driving strategy sample data to obtain the global driving strategy model after training and update in this round.

[0044] When the current number of iterations meets the preset conditions, that is, when the current number of iterations is an integer multiple of the aggregation period, the current update mode of the global driving strategy model is determined to be a global aggregation update, that is, the driving strategy model of each vehicle in the global driving strategy model is aggregated, exchanged, weighted, etc. according to the model parameters of the driving strategy models of all vehicles connected through network communication, and aggregated and updated to obtain the global driving strategy model after training and update in this round.

[0045] In one possible implementation, Figure 2 As shown, Figure 2 1 is a flow chart of a method for updating model parameters corresponding to a driving strategy model of a vehicle provided by another embodiment of the present application. When the current update mode of the global driving strategy model is local gradient update, in step S13, based on the current update mode, according to the fusion features of the driving strategy sample data of each vehicle and the current model parameters of the global driving strategy model, the driving strategy model of the vehicle in the global driving strategy model is updated to obtain the updated driving strategy model of the vehicle, including: Step S1311, predicting the driving strategy of the vehicle according to the fusion characteristics of the driving strategy sample data of the vehicle through the driving strategy model of the vehicle, and obtaining a driving strategy prediction value corresponding to the driving strategy sample data; wherein the driving strategy sample data includes an actual driving strategy value; Step S1312, calculating the current gradient of the driving strategy model of the vehicle based on the actual driving strategy value in the driving strategy sample data and the driving strategy prediction value corresponding to the driving strategy sample data and a preset local loss function; Step S1313, updating the model parameters corresponding to the driving strategy model of the vehicle according to the current gradient of the driving strategy model of the vehicle, to obtain updated model parameters corresponding to the driving strategy model of the vehicle; Step S1314: determining an updated driving strategy model of the vehicle according to the updated model parameters corresponding to the driving strategy model of the vehicle.

[0046] Specifically, when the current update mode of the global driving strategy model is local gradient update, in step S13, based on the current update mode, according to the driving strategy sample data of each vehicle and the current model parameters of the global driving strategy model, the driving strategy model of the vehicle in the global driving strategy model is updated to obtain the updated driving strategy model of the vehicle, including: For each vehicle in the federated autonomous driving network, the vehicle corresponds to multiple driving strategy sample data. The multiple driving strategy sample data of the vehicle are input into the driving strategy model of the vehicle for training. Feature extraction is first performed to obtain the fusion feature of each driving strategy sample data. Then, based on the fusion feature of each driving strategy sample data, driving strategy prediction is performed to obtain the driving strategy prediction value corresponding to each driving strategy sample data. The driving strategy sample data includes the actual driving strategy value that has been marked. The driving strategy prediction value is obtained after the driving strategy model of the vehicle predicts the driving strategy of the vehicle based on the input driving strategy sample data.

[0047] In an optional example, the fusion features of the driving strategy sample data obtained above are input into the fully connected layer of the driving strategy model to predict the driving strategy of the vehicle and obtain the driving strategy prediction value corresponding to the driving strategy sample data. After the fusion features of the vehicle surround view images corresponding to all the driving strategy sample data corresponding to the vehicle are input into the fully connected layer of the driving strategy model, the driving strategy prediction value corresponding to the corresponding driving strategy sample data can be obtained.

[0048] It should be noted that the vehicle's driving strategy model can adopt the Federal Autonomous Driving Network (FAD Net), such as Figure 5 As shown, Figure 5 It is a flowchart of another method for training a driving strategy model of a vehicle provided by another embodiment of the present application, by setting structures such as an input layer, a convolution layer, a maximum pooling layer, a residual block, a global average pooling layer, an accumulation layer, an aggregation layer and an output layer, configuring a loss function (such as Figure 5The Loss in the algorithm and the optimizer are used to uniformly process the unbalanced data of different vehicles, extract the features of the vehicle surround image in each input driving strategy sample data, perform driving strategy recognition and prediction based on the extracted features, and obtain the driving strategy prediction value corresponding to the input driving strategy sample data.

[0049] In an optional example, assuming that the actual driving strategy value in the driving strategy sample data is the steering angle, multiple driving strategy sample data of the vehicle are input into the driving strategy model of the vehicle for training, and the driving strategy prediction value corresponding to each driving strategy sample data is also the steering angle. Wherein, the steering angle is a continuous value, which is used to indicate the steering angle that the vehicle should take, so as to control the driving direction of the vehicle during autonomous driving. Assuming that the actual driving strategy value in the driving strategy sample data is the adjustment speed, multiple driving strategy sample data of the vehicle are input into the driving strategy model of the vehicle for training, and the collision probability corresponding to each driving strategy sample data is obtained. The driving strategy prediction value calculated based on the collision probability is also the adjustment speed. Wherein, the collision probability is a continuous value between 0-1, which indicates the possibility of a collision at present, and is used to calculate whether the vehicle needs to take emergency braking or adjust the speed.

[0050] A small batch of sample data is selected from multiple driving strategy sample data of the vehicle to obtain the fusion features of the small batch of sample data. According to the actual value of the driving strategy in the selected small batch of sample data and the predicted value of the driving strategy corresponding to the small batch of sample data, based on the pre-set local loss function, the current gradient of the driving strategy model of the vehicle in the current training update round is calculated. In the training process of the driving strategy model, the data set composed of multiple driving strategy sample data is divided into multiple smaller subsets, each subset contains a certain number of randomly selected driving strategy sample data, and the randomly selected driving strategy sample data in each subset is the small batch sample data.

[0051] According to the current gradient of the driving strategy model of the vehicle at the current training update round, the updated model parameters corresponding to the driving strategy model of the vehicle at the current training update round are calculated according to the following formula: : ; in, Indicates the number of small batch sample data; Represents small batch sample data; Indicates the number of iterations; It means the number of iterations is When the vehicle Model parameters of the driving strategy model; It means the number of iterations is The learning rate at time ; It means the number of iterations is When the vehicle The first of the mini-batch sample data of the driving policy model Sample data; It means the number of iterations is When the vehicle The driving strategy model The loss function calculated from the sample data The current gradient with respect to the model parameters.

[0052] It should be noted that the learning rate can change with the number of iterations.

[0053] The updated model parameters corresponding to the current training update round are the updated model parameters corresponding to the driving strategy model of the vehicle. The updated driving strategy model of the vehicle is determined based on the updated model parameters corresponding to the driving strategy model of the vehicle.

[0054] It is understandable that when training the driving policy model, small batch sample data is used to calculate the gradient or update the model parameters in each iteration, which can significantly reduce the amount of calculation in each iteration, thereby speeding up the training. At the same time, since only part of the data is used in each iteration, it can also reduce memory consumption and reduce the risk of overfitting to a certain extent.

[0055] In one possible implementation, Figure 3 As shown, Figure 3 1 is a flow chart of a method for updating model parameters corresponding to another driving strategy model of a vehicle provided by another embodiment of the present application. When the current update mode of the global driving strategy model is global aggregate update, in step S13, based on the current update mode, according to the fusion features of the driving strategy sample data of each vehicle and the current model parameters of the global driving strategy model, the driving strategy model of the vehicle in the global driving strategy model is updated to obtain the updated driving strategy model of the vehicle, including: Step S1321, obtaining the communication resources of each vehicle and the initial communication paths between multiple vehicles; Step S1322, optimizing the initial communication paths between the multiple vehicles according to the communication resources of each vehicle to obtain the optimal communication paths between the multiple vehicles; Step S1323, determining neighbor vehicles of each vehicle according to the optimal communication path between the multiple vehicles; Step S1324, performing aggregate update on the model parameters corresponding to the driving strategy model of the vehicle according to the model parameters corresponding to the driving strategy models of the neighboring vehicles of the vehicle, to obtain updated model parameters corresponding to the driving strategy model of the vehicle; Step S1325, determining the updated driving strategy model of the vehicle according to the updated model parameters corresponding to the driving strategy model of the vehicle.

[0056] Specifically, when the current update mode of the global driving strategy model is global aggregate update, based on the current update mode, according to the driving strategy sample data of each vehicle and the current model parameters of the global driving strategy model, the driving strategy model of the vehicle in the global driving strategy model is updated to obtain the updated driving strategy model of the vehicle, each vehicle has a certain communication resource corresponding to it, and all vehicles communicate with each other through a pre-set initial communication path. One of the purposes of communication includes sending each other's model parameters so that other vehicles can update the model parameters of their own driving strategy models according to the received model parameters. Among them, the communication resources may include resource information such as the link bandwidth between two vehicles, the size of the model parameters, the time for the vehicle's driving strategy model to perform local training and update, and the communication delay.

[0057] However, since the pre-set communication path may not be in line with reality, the communication delay time between vehicles is increased, which affects the communication efficiency and thus affects the training time of the vehicle's driving strategy model. In order to improve the training efficiency of the vehicle's driving strategy model, it is necessary to select the optimal communication path between vehicles.

[0058] The above methods include: Each vehicle has a certain amount of communication resources. All vehicles communicate with each other through a pre-set initial communication path. All vehicles and the initial communication path together constitute a connected graph. Assumption: Connected graph for: .in, Indicates vehicle, Represents the initial communication path between vehicles.

[0059] Using the Christofides algorithm or other optimization algorithms, based on the communication resources of all vehicles, optimize the pre-set initial communication paths between all vehicles and generate a subgraph from the connected graph. , , as the communication coverage graph trained by DFL (Deep Federated Learning Framework), the edge set in the communication coverage graph As the optimal communication path, it is a strongly connected subgraph with the minimum cycle time, that is, the communication delay between all vehicles in the graph is the minimum. The communication coverage graph can communicate without a central server, ensuring that the communication resources of each vehicle can be used evenly, avoiding overload of some vehicles, optimizing the initial communication path, so that effective model training can be carried out, reducing communication time, speeding up the aggregation of model parameters of the driving strategy model, and the updating process of model parameters, and improving the efficiency of model training.

[0060] In an alternative example, assuming that the vehicle , and the initial communication path between vehicles , forming a connected graph For a connected graph , first use Kruskal algorithm (Kruskal algorithm) or Prim algorithm (minimum spanning tree algorithm) to build a minimum spanning tree , then traverse Among all the vehicles in , find the set of vehicles with odd degrees After that, use the Edmonds algorithm or other matching algorithms to find the vehicle set Perfect match for vehicles . After that, and Merge to form a new connected graph . Use Hierholzer's algorithm (algorithm for finding Euler circuits) to find a new connected graph An Euler circuit in By skipping vehicles that visit repeatedly, the Eulerian circuit Convert to Hamiltonian circuit . The Hamiltonian circuit As a strongly connected graph The edge set , thus obtaining the optimal communication path between vehicles .

[0061] For any vehicle, the vehicle that is directly connected to it can be regarded as its neighbor vehicle, or all vehicles that can communicate with it directly or indirectly can be regarded as its neighbor vehicle. The neighbor vehicles of each vehicle are determined according to the optimal communication path between multiple vehicles.

[0062] The model parameters corresponding to the driving strategy model of the vehicle are aggregated and updated according to the model parameters corresponding to the driving strategy models of the neighboring vehicles of the vehicle to obtain the updated model parameters corresponding to the driving strategy model of the vehicle.

[0063] In an optional example, if Figure 4 As shown, Figure 4This is a structural schematic diagram of a vehicle communication path provided by an embodiment of the present application. The neighboring vehicles of vehicle 1 (Silo 1 in the figure) are vehicle 2 (Silo 2 in the figure), vehicle 3 (Silo 3 in the figure) and vehicle 4 (Silo 4 in the figure). When the model parameters corresponding to the driving strategy model of vehicle 1 are aggregated and updated, the model parameters corresponding to the driving strategy model of vehicle 2 are transmitted to vehicle 3, the model parameters corresponding to the driving strategy models of vehicles 2 and 3 are transmitted to vehicle 4, and the model parameters corresponding to the driving strategy models of vehicles 2, 3 and 4 are transmitted to vehicle 1. The model parameters corresponding to the driving strategy models of vehicles 2, 3 and 4 are aggregated to obtain the updated model parameters corresponding to the driving strategy model of vehicle 1. The model parameters corresponding to the driving strategy models of vehicles 2, 3 and 4 are the updated model parameters after one or more rounds of local training, and are the model parameters corresponding to the driving strategy model under the current number of iterations.

[0064] An updated driving strategy model of the vehicle is determined according to the updated model parameters corresponding to the driving strategy model of the vehicle.

[0065] It should be noted that this embodiment does not specifically limit the optimization method of the initial communication path, and it may be the Christofides algorithm or other algorithms.

[0066] In a possible implementation, in step S1324, the model parameters corresponding to the driving strategy model of the vehicle are aggregated and updated according to the model parameters corresponding to the driving strategy models of the neighboring vehicles of the vehicle to obtain the updated model parameters corresponding to the driving strategy model of the vehicle, including: Construct a consensus matrix for each vehicle based on the number of its neighboring vehicles; According to the consensus matrix of the vehicle, the model parameters corresponding to the driving strategy model of each neighboring vehicle of the vehicle are weighted summed to obtain the updated model parameters corresponding to the driving strategy model of the vehicle.

[0067] Specifically, for any vehicle, based on the model parameters corresponding to the driving strategy models of neighboring vehicles of the vehicle, the model parameters corresponding to the driving strategy model of the vehicle are aggregated and updated to obtain the updated model parameters corresponding to the driving strategy model of the vehicle, the method includes: For any vehicle, the consensus matrix of the vehicle is constructed according to the number of its neighboring vehicles. The consensus matrix of the vehicle is a The matrix of is the total number of vehicles. The elements of the consensus matrix Indicates vehicle From vehicle The weights of the model parameters received and fused in the driving policy model.

[0068] In an optional example, assuming that there are a total of 4 vehicles, for vehicle 1, its connection relationship with other vehicles is: vehicle 1 is connected to vehicle 2 and vehicle 3 respectively, vehicle 2 is connected to vehicle 1 and vehicle 4 respectively, vehicle 3 is connected to vehicle 1 and vehicle 4 respectively, and vehicle 4 is connected to vehicle 2 and vehicle 3 respectively. The consensus matrix is ​​constructed using a simple average method based on neighbor nodes. : .

[0069] According to the consensus matrix of the vehicle, the vehicle is The model parameters corresponding to the driving strategy model of each neighboring vehicle are weighted summed to obtain the updated model parameters corresponding to the driving strategy model of the vehicle : ; in, Indicates vehicle Neighbor's vehicle , Indicates that the vehicle The consensus matrix Line The element value of the column; Indicates that the number of iterations is When the vehicle Model parameters of the driving strategy model; Indicates vehicle The number of neighboring vehicles.

[0070] The model parameters of all neighboring vehicles of the vehicle including the model parameters of the driving strategy model of the vehicle are weighted summed to calculate the updated model parameters corresponding to the driving strategy model of the vehicle. .

[0071] When the current number of iterations is an integer multiple of the aggregation period, the current update mode of the global driving strategy model is global aggregation update. The updated model parameters corresponding to the driving strategy model of each vehicle are calculated in the same way as in step S1324. .

[0072] In a possible implementation, in step S11, obtaining fusion features of driving strategy sample data of multiple vehicles includes: Step S111, inputting the driving strategy sample data of each vehicle into the driving strategy model of the vehicle, performing normalization preprocessing on the driving strategy sample data, and obtaining preprocessed driving strategy sample data; Step S112, performing multiple feature extractions on the preprocessed driving strategy sample data to obtain fusion features of the driving strategy sample data.

[0073] Specifically, the driving strategy sample data includes a vehicle surround view image, and the driving strategy sample data of each vehicle is input into the driving strategy model of the vehicle for training, and the fusion features of the driving strategy sample data of multiple vehicles are obtained, including: The driving strategy model of the vehicle in this embodiment is a FAD Net (Federated autonomous driving network), which includes a normalization unit, a feature extraction unit, and a prediction unit. Among them, the normalization unit includes a normalization layer, which is used to normalize the input image to solve the problem of uneven distribution of the input image. The feature extraction unit is used to extract features from the input image, so that the driving strategy model can effectively identify the features of the input image and perform subsequent predictions. The prediction unit is used to predict the results of the image based on the input features.

[0074] For each driving strategy model corresponding to a vehicle, and any driving strategy sample data corresponding to the vehicle, the driving strategy sample data is input into the driving strategy model of the vehicle, and the vehicle surround view image in the driving strategy sample data is normalized and preprocessed through the normalization layer to obtain the preprocessed driving strategy sample data, that is, the preprocessed vehicle surround view image. Among them, the preprocessed vehicle forward view is the normalized vehicle data. After all the driving strategy sample data corresponding to the vehicle are input into the normalization layer in the driving strategy model, the corresponding preprocessed vehicle surround view image can be obtained.

[0075] The preprocessed driving strategy sample data obtained above is input into the feature extraction unit for multiple feature extractions to obtain the fusion features of the driving strategy sample data corresponding to the vehicle surround view image in the driving strategy sample data. The feature extraction unit may include multiple feature extraction layers to perform feature extraction at different levels on the preprocessed vehicle surround view image, so that the fusion features of the obtained driving strategy sample data are more comprehensive. After the preprocessed vehicle surround view images corresponding to all the driving strategy sample data corresponding to the vehicle are input into the normalization layer in the driving strategy model, the fusion features of the corresponding driving strategy sample data can be obtained.

[0076] For any vehicle, after its corresponding driving strategy sample data is processed by the method of the above-mentioned steps S111 to S112, the fusion features of the corresponding driving strategy sample data can be obtained.

[0077] In a possible implementation, in step S112, multiple feature extractions are performed on the preprocessed driving strategy sample data to obtain fusion features of the driving strategy sample data, including: Through the convolution layer and pooling layer in the driving strategy model, local features of the preprocessed driving strategy sample data are extracted to obtain local features of the driving strategy sample data; Extracting residual block features of local features of driving strategy sample data through multiple residual blocks in the driving strategy model to obtain multiple residual features of the driving strategy sample data; Through multiple global average pooling layers in the driving strategy model, multiple residual features of the driving strategy sample data are pooled and summarized respectively to obtain multiple global pooling features of the driving strategy sample data; Through the accumulation layer in the driving strategy model, multiple global pooling features of the driving strategy sample data are weighted and calculated to obtain the accumulated features of the driving strategy sample data; Through the aggregation layer in the driving strategy model, the accumulated features of the driving strategy sample data and the multiple residual features of the vehicle surround view image are fused to obtain the fused features of the driving strategy sample data.

[0078] Specifically, when performing multiple feature extractions on the preprocessed driving strategy sample data to obtain fusion features of the driving strategy sample data, the method includes: like Figure 5 As shown in the figure, the feature extraction unit in the driving strategy model includes a convolution layer, a pooling layer, a residual block (ResBlock 1, ResBlock 2 and ResBlock 3, 3 residual blocks), a fully connected layer, a global average pooling layer (GAP), an accumulation layer, an aggregation layer, etc. Among them, there can be one or more convolution layers, which are connected to a normalization unit in front and a pooling layer in the back. There can also be one or more pooling layers. The convolution layer and the pooling layer can encode the preprocessed vehicle surround image. There are multiple residual blocks connected after the pooling layer, and each residual block is connected to a global average pooling layer. The multiple residual blocks can also be connected to a fully connected layer, and then connected to an accumulation layer. After the accumulation layer, there is an aggregation layer to aggregate all the features, so that more abundant information can be extracted for the input vehicle surround image. For example, there are 3 residual blocks and 3 global average pooling layers.

[0079] The vehicle surround view image in the preprocessed driving strategy sample data is input into the convolution layer and the pooling layer, and the local features of the vehicle surround view image in the preprocessed driving strategy sample data are extracted through the convolution layer and the pooling layer in the driving strategy model to obtain the local features of the vehicle surround view image in the driving strategy sample data. Among them, the pooling layer outputs a feature map after dimensionality reduction, which can reduce the spatial dimension of the vehicle surround view image, thereby reducing the amount of calculation and increasing translation invariance.

[0080] The local features of the vehicle surround image in the driving strategy sample data extracted above are extracted and input into multiple residual blocks in the driving strategy model. The local features of the vehicle surround image in the driving strategy sample data are extracted by residual block features through multiple residual blocks to obtain multiple residual features of the driving strategy sample data, that is, multiple residual features of the vehicle surround image. After passing through a fully connected layer (FC), multiple residual features can be combined. Among them, each residual block contains multiple convolutional layers and ReLU activation functions (rectified linear unit activation function), and a jump connection. Multiple residual blocks help to effectively transfer gradients in deep network structures. By introducing jump connections to solve the gradient vanishing problem, deep networks are easier to train. The fully connected layer further extracts useful features from the high-dimensional features extracted by the residual blocks, thereby playing an effective role in updating the model parameters.

[0081] However, since the residual block easily causes the weight variance of the vehicle's driving strategy model to increase and become difficult to converge, the residual features obtained after each residual block are input into the global average pooling layer connected to the residual block. Each global average pooling layer pools and summarizes the residual features of the input driving strategy sample data to obtain the global pooling features of the driving strategy sample data, that is, the corresponding global pooling features of the vehicle surround view image. The global average pooling layer is a non-weighted pooling layer that is not affected by the weight variance and can summarize the spatial information of the vehicle surround view image from each residual block, thereby reducing the spatial dimension of the feature map while retaining the overall features of each channel. For all global average pooling layers, multiple global pooling features of the vehicle surround view image are output, that is, multiple global pooling features of the driving strategy sample data.

[0082] The multiple global pooling features obtained above are input into the accumulation layer connected to the back of the pooling layer. The accumulation layer in the driving strategy model performs weighted calculation on the multiple global pooling features of the vehicle surround image to obtain the accumulated features of the driving strategy sample data. The accumulation layer is a variant of the fully connected layer, but it does not weight the contribution of the input nodes like the fully connected layer, but weights the contribution of multiple features of the input layer. Among them, the accumulated features of the vehicle surround image , calculated according to the following formula: ; in, , For vehicles Driving strategy model A collection of global pooling features from different global average pooling layers; is the global pooling feature of the first global average pooling layer, is the global pooling feature of the second global average pooling layer, For the Global pooling features of the global average pooling layer; is the number of global average pooling layers; For the The learnable weights of the global average pooling layer are used to weight the global pooling features; For the Global pooling features of a global average pooling layer.

[0083] The accumulated features and multiple residual features extracted above are input into the aggregation layer in the driving strategy model. The accumulated features of the driving strategy sample data and the multiple residual features of the driving strategy sample data are fused by the aggregation layer according to the following formula to obtain the fused features of the driving strategy sample data: : ; in, represents the residual features of the vehicle surround view image, represents the accumulated features of the vehicle surround view image, represents the Hadamard product (element-wise product), Represents a mean operation, usually taking the average over a specific dimension.

[0084] The aggregation layer can comprehensively consider multiple features, thereby improving the accuracy and robustness of the driving strategy model.

[0085] It can be understood that the driving policy model enhances the expressiveness of the model by fusing features from different layers, which helps to improve the understanding and prediction accuracy of complex driving environments.

[0086] It can be understood that the technical solution provided by the present embodiment obtains the fusion features of the driving strategy sample data of multiple vehicles, and the current iteration number and current model parameters of the pre-built global driving strategy model; determines the current update mode of the global driving strategy model according to the current iteration number; and the current update mode includes local gradient update and global aggregation update, the current iteration number is used to indicate the training update round that the global driving strategy model has currently completed, based on the current update mode, according to the fusion features of the driving strategy sample data of each vehicle and the current model parameters of the global driving strategy model, the driving strategy model of the vehicle in the global driving strategy model is updated to obtain the updated driving strategy model of the vehicle; since the current model parameters include the model parameters corresponding to the driving strategy model of each vehicle in the global driving strategy model, the fusion features of the driving strategy sample data The feature is obtained by fusing the residual features and cumulative features extracted from the driving strategy sample data by the global driving strategy model, so that the updated driving strategy model of each vehicle is obtained by updating the model parameters according to the driving strategy sample data of the vehicle and the model parameters corresponding to the driving strategy models of multiple vehicles. The training process does not need to obtain the driving strategy sample data of other vehicles, thereby protecting the data privacy between vehicles, and does not need to set up a third-party server for communication, thereby saving the time for updating the model parameters. In addition, the fused feature is obtained by fusing the residual feature and the cumulative feature, so that the update efficiency of the driving strategy model of each vehicle is higher, the training time is shorter, and the recognition accuracy of the trained driving strategy model is higher, thereby solving the problem of long training time and low efficiency caused by training the driving strategy model through the existing federated learning method.

[0087] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0088] Corresponding to the method described in the above embodiment, Figure 6 A schematic structural diagram of a vehicle driving strategy model training device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0089] Reference Figure 3 , the device comprises: The data acquisition module 31 is used to obtain the fusion features of the driving strategy sample data of multiple vehicles, and the current iteration number and current model parameters of the pre-built global driving strategy model; wherein the fusion features of the driving strategy sample data are obtained by fusing the residual features and the cumulative features extracted from the driving strategy sample data by the global driving strategy model, the current iteration number is used to indicate the training update round currently being conducted by the global driving strategy model, and the current model parameters include: the model parameters corresponding to the driving strategy model of each vehicle in the global driving strategy model; A model update mode determination module 32 is used to determine a current update mode of the global driving strategy model according to a current number of iterations, wherein the current update mode includes: local gradient update and global aggregation update; The model updating module 33 is used to update the driving strategy model of the vehicle in the global driving strategy model based on the current updating mode, according to the fusion characteristics of the driving strategy sample data of each vehicle and the current model parameters of the global driving strategy model, to obtain the updated driving strategy model of the vehicle.

[0090] In a possible implementation, when determining the current update mode of the global driving strategy model according to the current number of iterations, the model update mode determination module 32 is specifically used to determine whether the current number of iterations satisfies a preset condition determined based on a preset aggregation period; When the current number of iterations does not meet the preset condition, determining that the current update mode of the global driving strategy model is local gradient update; When the current number of iterations meets the preset conditions, it is determined that the current update mode of the global driving strategy model is global aggregation update.

[0091] In a possible implementation, based on the current update mode, according to the fusion features of the driving strategy sample data of each vehicle and the current model parameters of the global driving strategy model, the driving strategy model of the vehicle in the global driving strategy model is updated to obtain the updated driving strategy model of the vehicle. The model update module 33 is specifically used to predict the driving strategy of the vehicle according to the fusion features of the driving strategy sample data of the vehicle through the driving strategy model of the vehicle when the current update mode of the global driving strategy model is local gradient update, and obtain the driving strategy prediction value corresponding to the driving strategy sample data; wherein the driving strategy sample data includes the actual value of the driving strategy; According to the actual driving strategy value in the driving strategy sample data and the driving strategy prediction value corresponding to the driving strategy sample data, based on a preset local loss function, a current gradient of the driving strategy model of the vehicle is calculated; According to the current gradient of the driving strategy model of the vehicle, the model parameters corresponding to the driving strategy model of the vehicle are updated to obtain the updated model parameters corresponding to the driving strategy model of the vehicle; An updated driving strategy model of the vehicle is determined according to the updated model parameters corresponding to the driving strategy model of the vehicle.

[0092] In a possible implementation, the driving strategy model of the vehicle in the global driving strategy model is updated according to the fusion features of the driving strategy sample data of each vehicle and the current model parameters of the global driving strategy model to obtain the updated driving strategy model of the vehicle. The model updating module 33 is specifically used to obtain the communication resources of each vehicle and the initial communication paths between the multiple vehicles when the current update mode of the global driving strategy model is global aggregation update; According to the communication resources of each vehicle, the initial communication paths between the multiple vehicles are optimized to obtain the optimal communication paths between the multiple vehicles; Determine neighbor vehicles of each vehicle according to an optimal communication path between multiple vehicles; According to the model parameters corresponding to the driving strategy models of the neighboring vehicles of the vehicle, the model parameters corresponding to the driving strategy model of the vehicle are aggregated and updated to obtain the updated model parameters corresponding to the driving strategy model of the vehicle; An updated driving strategy model of the vehicle is determined according to the updated model parameters corresponding to the driving strategy model of the vehicle.

[0093] In a possible implementation, when the model parameters corresponding to the driving strategy model of the vehicle are aggregated and updated according to the model parameters corresponding to the driving strategy model of the neighboring vehicles of the vehicle, and the updated model parameters corresponding to the driving strategy model of the vehicle are obtained, the model updating module 33 is specifically used to construct a consensus matrix for each vehicle according to the number of neighboring vehicles of each vehicle; According to the consensus matrix of the vehicle, the model parameters corresponding to the driving strategy model of each neighboring vehicle of the vehicle are weighted summed to obtain the updated model parameters corresponding to the driving strategy model of each vehicle.

[0094] In a possible implementation, when acquiring fusion features of driving strategy sample data of multiple vehicles, the data acquisition module 31 is specifically used to input the driving strategy sample data of each vehicle into the driving strategy model of the vehicle, perform normalization preprocessing on the driving strategy sample data, and obtain preprocessed driving strategy sample data; The preprocessed driving strategy sample data is subjected to multiple feature extractions to obtain fused features of the driving strategy sample data.

[0095] In a possible implementation, when multiple feature extractions are performed on the preprocessed driving strategy sample data to obtain fusion features of the driving strategy sample data, the model updating module 33 is specifically used to extract local features of the preprocessed driving strategy sample data through the convolution layer and the pooling layer in the driving strategy model to obtain local features of the driving strategy sample data; Extracting residual block features of local features of driving strategy sample data through multiple residual blocks in the driving strategy model to obtain multiple residual features of the driving strategy sample data; Through multiple global average pooling layers in the driving strategy model, multiple residual features of the driving strategy sample data are pooled and summarized respectively to obtain multiple global pooling features of the driving strategy sample data; Through the accumulation layer in the driving strategy model, multiple global pooling features of the driving strategy sample data are weighted and calculated to obtain the accumulated features of the driving strategy sample data; Through the aggregation layer in the driving strategy model, the accumulated features of the driving strategy sample data and the multiple residual features of the driving strategy sample data are fused to obtain the fused features of the driving strategy sample data.

[0096] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0097] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0098] like Figure 7 As shown, Figure 7It is a structural diagram of a terminal device provided by an embodiment of the present application. The embodiment of the present application also provides a terminal device 6, which includes: at least one processor 62, a memory 61, and a computer program stored in the memory and executable on the at least one processor, and when the processor executes the computer program, it implements the steps in any of the above-mentioned method embodiments.

[0099] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0100] An embodiment of the present application provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0101] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0102] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0103] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0104] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0105] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0106] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for training a vehicle driving strategy model, characterized in that: include: Acquire fusion features of driving strategy sample data of multiple vehicles, and the current iteration number and current model parameters of a pre-built global driving strategy model; wherein the fusion features of the driving strategy sample data are obtained by fusing residual features and cumulative features extracted from the driving strategy sample data by the global driving strategy model, the current iteration number is used to indicate the training update round currently being conducted by the global driving strategy model, and the current model parameters include: model parameters corresponding to the driving strategy model of each of the vehicles in the global driving strategy model; Determining a current update mode of the global driving strategy model according to the current number of iterations, wherein the current update mode includes: local gradient update and global aggregation update; Based on the current updating method, the driving strategy model of the vehicle in the global driving strategy model is updated according to the fusion features of the driving strategy sample data of each vehicle and the current model parameters of the global driving strategy model to obtain an updated driving strategy model of the vehicle.

2. The method for training a driving strategy model of a vehicle as claimed in claim 1, characterized in that: Determining the current update mode of the global driving strategy model according to the current number of iterations includes: Determining whether the current number of iterations meets a preset condition determined based on a preset aggregation period; When the current number of iterations does not meet a preset condition, determining that the current updating mode of the global driving strategy model is a local gradient update; When the current number of iterations meets a preset condition, it is determined that the current updating mode of the global driving strategy model is global aggregation updating.

3. The method for training a driving strategy model of a vehicle as claimed in claim 2, characterized in that: When the current update mode of the global driving strategy model is local gradient update, the driving strategy model of the vehicle in the global driving strategy model is updated based on the current update mode according to the fusion features of the driving strategy sample data of each vehicle and the current model parameters of the global driving strategy model to obtain the updated driving strategy model of the vehicle, including: The driving strategy of the vehicle is predicted by using the driving strategy model of the vehicle and according to the fusion characteristics of the driving strategy sample data of the vehicle, so as to obtain a driving strategy prediction value corresponding to the driving strategy sample data; wherein the driving strategy sample data includes an actual driving strategy value; According to the actual driving strategy value in the driving strategy sample data and the predicted driving strategy value corresponding to the driving strategy sample data, based on a preset local loss function, a current gradient of the driving strategy model of the vehicle is calculated; According to the current gradient of the driving strategy model of the vehicle, the model parameters corresponding to the driving strategy model of the vehicle are updated to obtain the updated model parameters corresponding to the driving strategy model of the vehicle; An updated driving strategy model of the vehicle is determined according to the updated model parameters corresponding to the driving strategy model of the vehicle.

4. The method for training a driving strategy model of a vehicle as claimed in claim 2, characterized in that: When the current update mode of the global driving strategy model is global aggregate update, the driving strategy model of the vehicle in the global driving strategy model is updated based on the current update mode according to the fusion features of the driving strategy sample data of each vehicle and the current model parameters of the global driving strategy model to obtain the updated driving strategy model of the vehicle, including: Acquiring a communication resource of each of the vehicles and an initial communication path between a plurality of the vehicles; Optimizing the initial communication paths between the plurality of vehicles according to the communication resources of each of the vehicles to obtain the optimal communication paths between the plurality of vehicles; Determining a neighbor vehicle of each of the vehicles according to an optimal communication path between the plurality of vehicles; According to the model parameters corresponding to the driving strategy models of the neighboring vehicles of the vehicle, the model parameters corresponding to the driving strategy model of the vehicle are aggregated and updated to obtain the updated model parameters corresponding to the driving strategy model of the vehicle; An updated driving strategy model of the vehicle is determined according to the updated model parameters corresponding to the driving strategy model of the vehicle.

5. The method for training a driving strategy model of a vehicle as claimed in claim 4, characterized in that: The step of aggregating and updating the model parameters corresponding to the driving strategy model of the vehicle according to the model parameters corresponding to the driving strategy models of the neighboring vehicles of the vehicle to obtain the updated model parameters corresponding to the driving strategy model of the vehicle includes: Constructing a consensus matrix for each of the vehicles according to the number of neighboring vehicles of each of the vehicles; According to the consensus matrix of the vehicle, a weighted sum is performed on the model parameters corresponding to the driving strategy model of each of the neighboring vehicles of the vehicle to obtain updated model parameters corresponding to the driving strategy model of the vehicle.

6. The method for training a driving strategy model of a vehicle as claimed in claim 1, characterized in that: The step of obtaining fusion features of driving strategy sample data of multiple vehicles includes: Inputting the driving strategy sample data of each vehicle into the driving strategy model of the vehicle, performing normalization preprocessing on the driving strategy sample data, and obtaining preprocessed driving strategy sample data; Performing multiple feature extractions on the preprocessed driving strategy sample data to obtain fusion features of the driving strategy sample data.

7. The method for training a driving strategy model of a vehicle as claimed in claim 6, characterized in that: The performing multiple feature extractions on the preprocessed driving strategy sample data to obtain fusion features of the driving strategy sample data includes: Extracting local features of the preprocessed driving strategy sample data through the convolution layer and the pooling layer in the driving strategy model to obtain local features of the driving strategy sample data; Extracting residual block features of local features of the driving strategy sample data through multiple residual blocks in the driving strategy model to obtain multiple residual features of the driving strategy sample data; Through multiple global average pooling layers in the driving strategy model, multiple residual features of the driving strategy sample data are respectively pooled and summarized to obtain multiple global pooling features of the driving strategy sample data; Performing weighted calculation on multiple global pooling features of the driving strategy sample data through an accumulation layer in the driving strategy model to obtain accumulation features of the driving strategy sample data; The accumulated features of the driving strategy sample data and the multiple residual features of the driving strategy sample data are fused through the aggregation layer in the driving strategy model to obtain the fused features of the driving strategy sample data.

8. A training device for a vehicle driving strategy model, characterized in that: include: A data acquisition module, used to obtain fusion features of driving strategy sample data of multiple vehicles, and the current iteration number and current model parameters of a pre-built global driving strategy model; wherein the fusion features of the driving strategy sample data are obtained by fusing residual features and cumulative features extracted from the driving strategy sample data by the global driving strategy model, the current iteration number is used to indicate the training update round currently being conducted by the global driving strategy model, and the current model parameters include: model parameters corresponding to the driving strategy model of each of the vehicles in the global driving strategy model; A model update mode determination module, used to determine a current update mode of the global driving strategy model according to the current number of iterations, wherein the current update mode includes: local gradient update and global aggregation update; A model updating module is used to update the driving strategy model of the vehicle in the global driving strategy model based on the current updating mode and according to the fusion features of the driving strategy sample data of each vehicle and the current model parameters of the global driving strategy model to obtain an updated driving strategy model of the vehicle.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images