Machine learning model training method, unmanned vehicle control method, and unmanned vehicle

Abstract

The present disclosure relates to a training method of a machine learning model, a control method of an unmanned vehicle and the unmanned vehicle, and relates to the field of artificial intelligence. The training method comprises: sending, by a first vehicle, a first gradient calculated by a machine learning model issued by a server to a second vehicle, the channel quality of the second vehicle being better than that of the first vehicle; and training the machine learning model by using a first fusion gradient issued by the server, the first fusion gradient being generated according to the first gradient uploaded by the second vehicle and a second gradient calculated by the second vehicle according to the machine learning model. The technical solution of the present disclosure can solve the problem of increased time delay caused by poor channel quality of part of the vehicles, thereby improving the response speed of the system.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of artificial intelligence, in particular to the field of unmanned driving, and more particularly to a machine learning model training method, a machine learning model training device, a control method of an unmanned vehicle, a control device of an unmanned vehicle, a control system of an unmanned vehicle, an electronic device, and an unmanned vehicle. BACKGROUND

[0002] At present, an unmanned driving device is used to automatically transport a person or an object from one location to another location. The unmanned driving device collects environmental information through sensors on the device and completes automatic transportation. An unmanned delivery vehicle controlled based on unmanned driving technology greatly improves the convenience of production and life and saves labor costs.

[0003] For the unmanned driving device, deep learning is widely used in multiple tasks of automatic driving. This makes the scale of training and deployment of neural networks for automatic driving continue to increase, thereby further increasing the challenges of user privacy, data collection network overhead, training and deployment overhead, and the like.

[0004] However, the deep learning training process needs to upload user privacy data to a data center, so the user's privacy may be leaked. Therefore, the user may refuse to provide his own data to participate in model training and updating. Federated learning can jointly perform machine learning training tasks without leaking underlying data by each participant.

[0005] In related technologies, each autonomous vehicle uploads the calculated gradient information to the center server through a V2R (Vehicle to Road) uplink. SUMMARY

[0006] The present inventors have found that the above related technology has the following problem: vehicles with poor V2R channel quality can increase the latency of the entire system.

[0007] In view of this, the present disclosure provides a machine learning model training technical solution, which can reduce latency.

[0008] According to some embodiments of the present disclosure, a machine learning model training method is provided, including: sending, by a first vehicle, a first gradient calculated by the first vehicle using a machine learning model issued by a server to a second vehicle, the channel quality of the second vehicle being better than that of the first vehicle; and training the machine learning model using a first fusion gradient issued by the server, the first fusion gradient being generated according to the first gradient uploaded by the second vehicle and a second gradient calculated by the second vehicle according to the machine learning model.

[0009] In some embodiments, the training method further comprises: calculating a first time delay of the first vehicle sending the estimated data amount to each candidate vehicle, and calculating a second time delay of each candidate vehicle uploading the estimated data amount to the server, the channel quality of each candidate vehicle being better than that of the first vehicle; determining the second vehicle from each candidate vehicle according to the first time delay and the second time delay.

[0010] In some embodiments, calculating the first time delay of the first vehicle sending the estimated data amount to each candidate vehicle comprises: calculating the first time delay of the first vehicle sending the estimated data amount to each candidate vehicle through each candidate resource; determining the sending resource from each candidate resource according to the first time delay and the second time delay; wherein sending the first gradient calculated by the first vehicle using the machine learning model issued by the server to the second vehicle comprises: sending the first gradient to the second vehicle using the sending resource.

[0011] In some embodiments, determining the second vehicle from each candidate vehicle and determining the sending resource from each candidate resource comprises: taking each candidate resource and each candidate vehicle as a variable, determining a time delay function according to the maximum value of the sum of the first time delay and the second time delay; determining a target function according to the time delay function; solving the target function under the condition that the target function reaches a minimum value to determine the sending resource and the second vehicle.

[0012] In some embodiments, determining the target function comprises: determining the target function according to the time delay function and the weighted average of the loss functions of the machine learning models corresponding to each candidate vehicle, the loss function of each candidate vehicle being determined according to the gradient uploaded by each candidate vehicle to the server.

[0013] In some embodiments, determining the target function comprises: updating the machine learning model using the candidate fusion gradient calculated by the server according to the gradient uploaded by each candidate vehicle, the uploaded gradient being determined according to the first gradient and the gradient calculated by each candidate vehicle; processing test data using the updated machine learning model to obtain a test result; determining the loss function corresponding to each candidate vehicle according to the test result and the label of the test data.

[0014] In some embodiments, determining the target function comprises: determining the weight value at the current moment according to the initial weight value and the weight value decay factor at the current moment; determining the weighted average of the time delay function and the loss function according to the weight value at the current moment.

[0015] In some embodiments, the weight value at the current moment is positively correlated with the initial weight value and negatively correlated with the current moment.

[0016] In some embodiments, the determining the target function according to the latency function and the weighted average of the loss functions of the machine learning models corresponding to the candidate vehicles comprises: in a first time period, determining the target function according to the latency function; in a second time period, determining the target function according to the latency function and the weighted average of the loss functions of the machine learning models corresponding to the candidate vehicles, the second time period being after the first time period.

[0017] In some embodiments, the calculating the first latency of the first vehicle sending the estimated data amount to the candidate vehicles via the candidate resources comprises: calculating a transmission rate of the first vehicle on the candidate resources according to a signal-to-noise ratio of the first vehicle on the candidate resources and a channel bandwidth of the candidate resources; and determining the first latency according to the transmission rate of the first vehicle on the candidate resources and the estimated data amount.

[0018] In some embodiments, the calculating the transmission rate of the first vehicle on the candidate resources comprises: determining a signal-to-noise ratio of the first vehicle on the candidate resources according to a signal power of the first vehicle on the candidate resources and a signal power of the candidate vehicles on the candidate resources.

[0019] In some embodiments, the determining the first latency comprises: determining the first latency according to a latency of the first vehicle on the candidate resources when the transmission rate of the first vehicle on the candidate resources is greater than or equal to the estimated data amount; and the calculating the second latency of the candidate vehicles uploading the estimated data amount to the server comprises: determining the second latency according to a latency of the second vehicle when the transmission rate is greater than or equal to the estimated data amount.

[0020] In some embodiments, the first fusion gradient is calculated according to second fusion gradients uploaded by the plurality of second vehicles, and the second fusion gradient is calculated according to the first gradient and the second gradient uploaded by the plurality of first vehicles.

[0021] In some embodiments, the second fusion gradient is calculated according to a plurality of gradients in a plurality of third fusion gradients, and each third fusion gradient is calculated according to the first gradient uploaded by the plurality of first vehicles at each time and the second gradient at each time.

[0022] In some embodiments, the sending the first gradient calculated by the first vehicle using the machine learning model issued by the server to the second vehicle comprises: calculating a plurality of gradients of the machine learning model at a plurality of times according to local data of the first vehicle; and determining the first gradient according to a plurality of gradients in the plurality of gradients at the plurality of times.

[0023] According to some embodiments of the present disclosure, a training method of a machine learning model is provided, comprising: receiving, by a second vehicle, a first gradient calculated by a first vehicle using a machine learning model distributed by a server, the channel quality of the second vehicle being better than that of the first vehicle; calculating, by the second vehicle, a second gradient using the machine learning model distributed by the server; uploading, by the second vehicle, the first gradient and the second gradient to the server; and training, by the second vehicle, the machine learning model using a first fusion gradient distributed by the server, the first fusion gradient being generated according to the first gradient and the second gradient.

[0024] In some embodiments, the uploading, by the second vehicle, of the first gradient and the second gradient to the server comprises: calculating a second fusion gradient according to the first gradient and the second gradient uploaded by a plurality of first vehicles; and uploading the second fusion gradient to the server, the first fusion gradient being calculated according to a plurality of second fusion gradients uploaded by a plurality of second vehicles.

[0025] In some embodiments, the calculating of the second fusion gradient according to the first gradient and the second gradient uploaded by a plurality of first vehicles comprises: calculating a third fusion gradient at each time according to the first gradient uploaded by a plurality of first vehicles at each time and the second gradient at each time; and calculating the second fusion gradient according to a plurality of gradients in the third fusion gradient at each time with the largest value.

[0026] According to some other embodiments of the present disclosure, a control method of an unmanned vehicle is provided, comprising: controlling the unmanned vehicle using a trained machine learning model, the machine learning model being trained by the following steps: sending, by a first vehicle, a first gradient calculated using a machine learning model distributed by a server to a second vehicle, the channel quality of the second vehicle being better than that of the first vehicle; and training, using a first fusion gradient distributed by the server, the machine learning model, the first fusion gradient being generated according to a first gradient uploaded by the second vehicle and a second gradient calculated by the second vehicle according to the machine learning model.

[0027] According to some other embodiments of the present disclosure, a training device of a machine learning model is provided, comprising: a sending unit configured to send, to a second vehicle, a first gradient calculated by a first vehicle using a machine learning model distributed by a server, the channel quality of the second vehicle being better than that of the first vehicle; and a training unit configured to train, using a first fusion gradient distributed by the server, the machine learning model, the first fusion gradient being generated according to a first gradient uploaded by the second vehicle and a second gradient calculated by the second vehicle according to the machine learning model.

[0028] In some embodiments, the training device further comprises a computing unit configured to calculate a first time delay for the first vehicle to send the estimated data amount to each candidate vehicle, and calculate a second time delay for each candidate vehicle to upload the estimated data amount to the server, each candidate vehicle has a better channel quality than the first vehicle, and determine the second vehicle from the candidate vehicles according to the first time delay and the second time delay.

[0029] In some embodiments, the computing unit calculates the first time delay for the first vehicle to send the estimated data amount to each candidate vehicle through each candidate resource, and determines the sending resource from the candidate resources according to the first time delay and the second time delay, and the sending unit sends the first gradient to the second vehicle using the sending resource.

[0030] In some embodiments, the computing unit determines a time delay function according to a maximum value of a sum of the first time delay and the second time delay, and determines a target function according to the time delay function, and determines the sending resource and the second vehicle by solving the target function under the condition that the target function reaches a minimum value.

[0031] In some embodiments, the computing unit determines the target function according to the time delay function and a weighted mean of loss functions of the machine learning models corresponding to the candidate vehicles, and the loss function corresponding to each candidate vehicle is determined according to a gradient uploaded to the server by the candidate vehicle.

[0032] In some embodiments, the computing unit updates the machine learning model using a candidate fusion gradient calculated by the server according to the gradients uploaded by the candidate vehicles, the uploaded gradient is determined according to the first gradient and a gradient calculated by each candidate vehicle, processes test data using the updated machine learning model to obtain a test result, and determines the loss function corresponding to each candidate vehicle according to the test result and a label of the test data.

[0033] In some embodiments, the computing unit determines a weight value at a current time according to an initial weight value and a weight decay factor at the current time, and determines the time delay function and the weighted mean of the loss functions according to the weight value at the current time.

[0034] In some embodiments, the weight value at the current time is positively correlated with the initial weight value and negatively correlated with the current time.

[0035] In some embodiments, the computing unit determines the target function according to the time delay function in a first time period, and determines the target function according to the time delay function and the weighted mean of the loss functions of the machine learning models corresponding to the candidate vehicles in a second time period, the second time period is after the first time period.

[0036] In some embodiments, the computing unit calculates the transmission rate of the first vehicle on each candidate resource according to the signal-to-noise ratio of the first vehicle on each candidate resource and the channel bandwidth of each candidate resource, and determines the first delay according to the transmission rate of the first vehicle on each candidate resource and the estimated data volume.

[0037] In some embodiments, the computing unit determines the signal-to-noise ratio of the first vehicle on each candidate resource according to the signal power of the first vehicle on each candidate resource and the signal power of each candidate vehicle on each candidate resource.

[0038] In some embodiments, the computing unit determines the first delay according to the delay of the first vehicle when the transmission rate is greater than or equal to the estimated data volume, and determines the second delay according to the delay of the second vehicle when the transmission rate is greater than or equal to the estimated data volume.

[0039] In some embodiments, the first fusion gradient is calculated according to the second fusion gradients uploaded by the plurality of second vehicles, and the second fusion gradient is calculated according to the first gradient and the second gradient uploaded by the plurality of first vehicles.

[0040] In some embodiments, the second fusion gradient is calculated according to the plurality of gradients with the largest value in the plurality of third fusion gradients, and each third fusion gradient is calculated according to the first gradient uploaded by the plurality of first vehicles at each time and the second gradient at each time.

[0041] In some embodiments, the sending unit calculates the gradient of the machine learning model at each time according to the local data of the first vehicle, and determines the first gradient according to the plurality of gradients with the largest value in the plurality of times.

[0042] According to still another embodiment of the present disclosure, a training device of a machine learning model is provided, comprising: a receiving unit configured to receive a first gradient sent by a first vehicle through a second vehicle, the first gradient being calculated by the first vehicle using a machine learning model issued by a server, and the channel quality of the second vehicle being better than that of the first vehicle; a computing unit configured to calculate a second gradient by the second vehicle using the machine learning model issued by the server; an uploading unit configured to upload the first gradient and the second gradient to the server by the second vehicle; and a training unit configured to train the machine learning model by the second vehicle using a first fusion gradient issued by the server, the first fusion gradient being generated according to the first gradient and the second gradient.

[0043] In some embodiments, the uploading unit calculates a second fusion gradient according to the first gradient and the second gradient uploaded by the plurality of first vehicles, and uploads the second fusion gradient to the server, and the first fusion gradient is calculated according to the second fusion gradient uploaded by the plurality of second vehicles.

[0044] In some embodiments, the computing unit calculates a third fusion gradient at each time point according to the first gradient uploaded by the plurality of first vehicles at each time point and the second gradient at each time point; and calculates the second fusion gradient according to a plurality of gradients with the largest value in the third fusion gradient at each time point.

[0045] According to still another embodiment of the present disclosure, a control device of an unmanned vehicle is provided, comprising: a control unit configured to control the unmanned vehicle by using a trained machine learning model, and the machine learning model is trained by the following steps: sending a first gradient calculated by a first vehicle using a machine learning model issued by a server to a second vehicle, and the channel quality of the second vehicle is better than that of the first vehicle; training the machine learning model by using a first fusion gradient issued by the server, and the first fusion gradient is generated according to a first gradient uploaded by the second vehicle and a second gradient calculated by the second vehicle according to the machine learning model.

[0046] According to still another embodiment of the present disclosure, an unmanned vehicle is provided, comprising: a control device configured to perform the training method of the machine learning model or the control method of the unmanned vehicle according to any one of the above embodiments.

[0047] According to still another embodiment of the present disclosure, a control system of an unmanned vehicle is provided, comprising: an unmanned vehicle according to any one of the above embodiments; and a server configured to issue a machine learning model to the unmanned vehicle.

[0048] According to still another embodiment of the present disclosure, an electronic device is provided, comprising: a memory; and a processor coupled to the memory, and the processor is configured to perform the training method of the machine learning model or the control method of the unmanned vehicle according to any one of the above embodiments based on instructions stored in the memory.

[0049] According to still another embodiment of the present disclosure, a non-volatile computer readable storage medium is provided, and the medium stores a computer program, and the program is executed by a processor to implement the training method of the machine learning model or the control method of the unmanned vehicle according to any one of the above embodiments.

[0050] In the above embodiments, the vehicle with poor channel quality transmits the gradient to the vehicle with good channel quality through V2V (Vehicle to Vehicle) connection, and then the vehicle with good channel quality uploads the gradient to the server. In this way, the problem of increased time delay caused by poor channel quality of some vehicles is solved, thereby improving the response speed of the system. BRIEF DESCRIPTION OF DRAWINGS

[0051] The accompanying drawings, which form a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0052] The present disclosure can be more clearly understood and appreciated from the following detailed description, taken in conjunction with the following drawings of which:

[0053] Figure 1 A flowchart illustrating some embodiments of a method of training a machine learning model of the present disclosure;

[0054] Figure 2a A schematic diagram illustrating some embodiments of a method of training a machine learning model of the present disclosure;

[0055] Figure 2b A schematic diagram illustrating some embodiments of a method of training a machine learning model of the present disclosure;

[0056] Figure 3 A flowchart illustrating some embodiments of a method of training a machine learning model of the present disclosure;

[0057] Figure 4 A flowchart illustrating some embodiments of a method of controlling an unmanned vehicle of the present disclosure;

[0058] Figure 5a A block diagram illustrating some embodiments of a training apparatus of a machine learning model of the present disclosure;

[0059] Figure 5b A block diagram illustrating some embodiments of a training apparatus of a machine learning model of the present disclosure;

[0060] Figure 6 A block diagram illustrating some embodiments of a control apparatus of an unmanned vehicle of the present disclosure;

[0061] Figure 7 A block diagram illustrating some embodiments of an unmanned vehicle of the present disclosure;

[0062] Figure 8 A block diagram illustrating some embodiments of a control system of an unmanned vehicle of the present disclosure;

[0063] Figure 9 A block diagram illustrating some embodiments of an electronic device of the present disclosure;

[0064] Figure 10 A block diagram illustrating some embodiments of an electronic device of the present disclosure;

[0065] Figures 11a-11b A block diagram illustrating some embodiments of an unmanned vehicle of the present disclosure. DETAILED DESCRIPTION

[0066] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of the components and steps set forth in these embodiments, numerical expressions, and numerical values, unless specifically stated otherwise, do not limit the scope of the present disclosure.

[0067] It should be understood, of course, that the dimensions of the various elements shown in the figures are chosen primarily for convenience and clarity of presentation and are not necessarily to scale.

[0068] The following description of at least one exemplary embodiment is merely exemplary in nature and is in no way intended to limit the disclosure, its application, or uses.

[0069] Techniques, methods, and apparatus known to those of ordinary skill in the relevant art can not be discussed in detail herein, but should be considered as part of the specification.

[0070] In all of the examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not as a limitation. Other examples of the exemplary embodiments can have different values.

[0071] It should be noted that like reference numerals and letters refer to like items throughout the attached drawings, and thus, once certain items are defined in one drawing, further discussion of such items in subsequent drawings need not be repeated.

[0072] As mentioned before, in the process of deep learning, the procedure of training and updating neural network is divided into the following steps: first, a large amount of data collected by vehicle users is collected through online data transmission or offline collection; then, the collected data is preprocessed in the center server to organize the training data set; then, the model is iterated by using gradient descent and other optimization methods to make the model loss function as small as possible to improve the performance of the model; finally, the updated model is deployed to each vehicle user end.

[0073] This process may result in the leakage of user privacy data uploaded to the data center, and thus, the user's privacy may be compromised. Therefore, the user may refuse to provide their own data for model training and updating.

[0074] Federated learning is an encrypted distributed machine learning training technology that can jointly perform machine learning training tasks without leaking underlying data. The core idea is that the client calculates the gradient to be updated using local data; then, compared to the mode of uploading data to the center, federated learning uploads the gradient to be updated.

[0075] In order to protect user privacy while using user data to improve the model, a federated learning training mode can be used. For example, by introducing a heterogeneous deep learning training framework, user privacy data is saved locally and does not need to be uploaded to the data center to complete model training iteration.

[0076] The process of federated learning can be divided into the following steps: first, the user uses his own data locally to calculate the gradient needed for gradient descent of the model through forward propagation and back propagation; then, the calculated gradient (not the user data) is uploaded to the server; then, the server processes the collected gradient data to update the previous neural network model; finally, the updated model is deployed to the vehicle user end.

[0077] Federated learning changes the paradigm of uploading data to the paradigm of uploading variables (such as model backpropagation gradients) in the model training process, thereby effectively protecting user privacy.

[0078] In order to reduce the time-consuming of gradient uploading while ensuring certain model training performance, the processing methods are mainly divided into three categories: data pruning compression, communication structure design, and asynchronous model updating.

[0079] Data pruning compression aims to reduce the amount of data that needs to be transmitted by compressing the updated part of the data. For example, in order to reduce the communication demand in federated learning, TopK element sparse update and non-zero element quantization processing can be used for non-independent and identically distributed data.

[0080] Communication structure design aims to optimize the communication energy and bandwidth overhead in the transmission process. For example, in order to balance the communication cost and model iteration accuracy, the core problem is to minimize the communication energy consumption of the client and the iteration time of federated learning to solve the loss function descent problem under the condition of limited resources.

[0081] As can be seen, since the server needs to collect all the gradients uploaded by the users in the federated learning process and then perform fusion update operations, the time delay of the model iteration process is greatly limited by the vehicle users with poor uplink channel quality. That is, the data transmission in the federated learning training mode has more stringent requirements on the latency and capacity of the Internet of Vehicles.

[0082] In view of the technical problem that in the gradient uploading process of federated learning, the communication quality between some users and the base station under the cellular network architecture is poor, and the gradients cannot be returned to the center, resulting in a decrease in the overall training quality, a channel resource and computing resource allocation scheme can be formulated using a heterogeneous network architecture such as inter-device multi-hop communication and clustering communication. In this way, the training convergence delay and model loss function can be jointly reduced.

[0083] Compared with the cellular network, due to the high mobility of vehicles, the network topology of the Internet of Vehicles changes rapidly over time. This forms a more rapidly changing heterogeneous network structure, which brings great challenges to the inter-device multi-hop and clustering type gradient uploading scheme.

[0084] To combat the mobility of vehicles, a clustering communication solution can be adopted for mapping and positioning and trajectory prediction tasks in autonomous driving. That is, by means of platooning vehicles traveling in the same direction, the surrounding three-dimensional environment is jointly reconstructed, and multi-route trajectory joint prediction is performed; with the help of clustering communication mode, the gradient is uploaded. In this way, the technical problem of poor channel of some vehicle users can be effectively addressed.

[0085] In addition to improving the upload communication rate, it is also necessary to reduce the amount of uploaded data while ensuring that the model performance is not affected. The gradient in the federal learning process is a special type of data, and the optimization of its training process is closely related to the network.

[0086] For example, TopK sparse processing can be performed on the gradient. The method includes: each user end sorts the gradients of each node of the neural network according to the absolute value size; the largest K gradients (i.e., sparse gradients) are selected for uploading; and the sparse gradients uploaded by all user ends are fused and updated at the center end.

[0087] The sparsity introduced by TopK sparse processing helps to reduce the model overfitting phenomenon caused by the non-identically distributed characteristics of local data, both reducing the amount of data uploaded and improving the generalization of the model.

[0088] Most deep learning tasks are mainly classification, and the number of categories can directly reflect the degree of non-identically distributed data. Tasks such as detection and time series prediction cannot intuitively reflect the non-identically distributed characteristics of the task. Therefore, the assumption of data non-identically distributed is mostly measured by the distribution of the data categories of each user.

[0089] For the special scenario of Internet of Vehicles, considering the correlation between road zoning and data identically distributed, the gradients collected by the cluster center vehicle (a vehicle with good channel quality in a vehicle group) can be regarded as gradients trained using non-identically distributed data.

[0090] The cluster center vehicle can effectively improve the overall identically distributed characteristics by fusing the gradient data it collects. Therefore, in addition to TopK sparse processing of ordinary vehicles, the data processing capability of the cluster center vehicle can be utilized to perform fusion and TopK sparse processing at the cluster center vehicle, further reducing the amount of data uploaded while ensuring the training performance of the model. Thus, the overall uplink time is further reduced.

[0091] That is, the core problem of federated learning in training effect is to resist the non-independent and identically distributed characteristics of data. For the special scene of Internet of Vehicles, the correlation between road zoning and data independent and identically distributed needs to be considered. And the optimization strategy of federated learning in the scene of Internet of Vehicles is studied according to this characteristic; the amount of gradient backpropagation and model update data is reduced in the scene of federated learning in Internet of Vehicles; the uplink and downlink content distribution mode is studied; the training architecture is improved to resist non-independent and identically distributed data; and the performance gain brought by gradient sparsity is studied.

[0092] In view of the above technical problems, the present disclosure jointly considers model training performance and upload delay, and studies a heterogeneous model update scheme. The cluster center vehicle with good uplink channel quality collects gradient information of surrounding vehicles with poor uplink channel quality by using V2V connection; considering the non-independent and identically distributed characteristics of local data, the cluster center vehicle performs fusion and sparsification locally, and then transmits the data to the server through V2R uplink.

[0093] In this way, the overall training time is reduced by improving the transmission rate and reducing the amount of uploaded data; and the model generalization ability is introduced by local non-independent and identically distributed feature fusion, so as to ensure that the training quality does not decrease due to the reduction of the amount of uploaded data. For example, the technical scheme of the present disclosure can be implemented by the following embodiments.

[0094] Figure 1 A flowchart showing some embodiments of the training method of the machine learning model of the present disclosure is shown.

[0095] As shown in Figure 1 , in step 110, the first gradient calculated by the first vehicle using the machine learning model downloaded by the server is sent to the second vehicle. The channel quality of the second vehicle is better than that of the first vehicle. For example, the channel quality includes the channel quality of V2R connection.

[0096] In some embodiments, the first gradient and the second gradient are uploaded to the server by the second vehicle.

[0097] For example, the first gradient can be sent by the Figure 2a embodiment.

[0098] Figure 2a A schematic diagram showing some embodiments of the training method of the machine learning model of the present disclosure is shown.

[0099] As shown in Figure 2a , vehicle 2 is a cluster center vehicle with good V2R uplink channel quality among N vehicles, i.e. the second vehicle. The V2R uplink channel of vehicle 1 and vehicle N-1 is poor, i.e. the first vehicle, and the transmission rate corresponding to t time and is small.

[0100] In some embodiments, the computer machine learning model calculates gradients at multiple time instants according to local data of the first vehicle; and determines a first gradient according to a plurality of gradients with the largest magnitude among the gradients at the multiple time instants.

[0101] In some embodiments, a third fused gradient at each time instant is calculated according to the first gradient and the second gradient uploaded by the plurality of first vehicles at each time instant; and a second fused gradient is calculated according to a plurality of gradients with the largest magnitude among the third fused gradients at the multiple time instants.

[0102] For example, the vehicle 1 and the vehicle N-1 first upload their respective TopK gradients AM1 and AMN-1 to the vehicle 2 through a local fast V2V link. N-1 After local gradient fusion, the vehicle 2 obtains a third fused gradient; and performs TopK sparsification on the third fused gradient to obtain a processed second fused gradient. N-1

[0103]

[0104] For example, the vehicle 2 obtains a third fused gradient after local fusion. The third fused gradient is uploaded to the server at a faster transmission rate.

[0105] In some embodiments, some vehicles (such as the vehicle N) do not need to perform the clustering communication step described above, but directly upload their calculated gradients through a V2R connection because the V2R channel quality of the vehicles is good.

[0106] In some embodiments, the second vehicle calculates a second fused gradient according to the first gradient and the second gradient uploaded by the plurality of first vehicles; the second vehicle uploads the second fused gradient to the server; and the server calculates a first fused gradient according to the second fused gradient uploaded by the plurality of second vehicles.

[0107] For example, after the server collects gradient data of all cluster center vehicles, the server performs a model update operation:

[0108]

[0109] N' is the number of cluster center vehicles, M init is an initial gradient of the machine learning model, is the second fused gradient uploaded by the i-th cluster center vehicle, M update is the first fused gradient.

[0110] To improve the model training effect and reduce the training iteration delay, appropriate V2V channel resources need to be allocated to vehicles with poor V2R channels, and suitable cluster center vehicles around the vehicles need to be selected to upload gradients. For example, the channel resource allocation and cluster center vehicle selection problem can be modeled as​​Figure 2b Examples are shown in the text.

[0111] Figure 2b Schematic diagrams illustrating some other embodiments of the training methods for the machine learning models of this disclosure.

[0112] like Figure 2b As shown, the candidate resources include R resource blocks, N ordinary vehicles (i.e., the first vehicle) and cluster center vehicles (i.e., the second vehicle).

[0113] In some embodiments, a first delay is calculated for the first vehicle to send an estimated amount of data to each candidate vehicle, and a second delay is calculated for each candidate vehicle to upload an estimated amount of data to the server, wherein the channel quality of each candidate vehicle is better than that of the first vehicle; and a second vehicle is determined from the candidate vehicles based on the first delay and the second delay.

[0114] For example, the first delay is calculated when the first vehicle sends the estimated amount of data to each candidate vehicle through each candidate resource; based on the first delay and the second delay, a transmission resource is determined from each candidate resource; and the first gradient is sent to the second vehicle using the transmission resource.

[0115] In some embodiments, the signal-to-noise ratio of the first vehicle in each candidate resource is determined based on the signal power of the first vehicle in each candidate resource and the signal power of each candidate vehicle in each candidate resource.

[0116] For example, y n,r x represents the allocation of resource block r and vehicle n. n,c This represents the allocation of vehicle c and vehicle n at the cluster center. n,r and x n,c Both are Boolean variables that are either 0 or 1. Considering the interference of co-channel signals from surrounding vehicles, the signal-to-noise ratio of vehicle n selecting resource block r at time t is modeled as:

[0117]

[0118] This represents the signal power of vehicle n selecting resource block r at time t. y represents the signal power of vehicle c selecting resource block r at time t. c,r σ represents the allocation of resource block r and vehicle c. 2 Let C represent noise, and C be the number of vehicles at the cluster center.

[0119] In some embodiments, the transmission rate of the first vehicle on each candidate resource is calculated based on the signal-to-noise ratio of the first vehicle on each candidate resource and the channel bandwidth of each candidate resource; and the first delay is determined based on the transmission rate of the first vehicle on each candidate resource and the estimated data volume.

[0120] In some embodiments, the first latency is determined according to a latency of the first vehicle under a condition that a transmission rate of the first vehicle on each candidate resource is greater than or equal to the estimated data amount; and the second latency is determined according to a latency of the second vehicle under a condition that the transmission rate is greater than or equal to the estimated data amount.

[0121] For example, the V2V channel bandwidth corresponding to the resource block r is B r The V2V transmission rate of the vehicle n at time t is:

[0122]

[0123] For example, after the V2V local transmission, a V2R transmission process from the cluster center vehicle to the server is also needed. The sum of the overall two-stage latency (the first latency and the second latency) of the heterogeneous network is:

[0124]

[0125] represents the estimated data amount, is the V2R transmission rate of the vehicle c at time t. If a single vehicle does not use the TopK sparse processing technology for compression and pruning, then approximately equal to the size of the machine learning model; otherwise is the size of the model after sparse processing.

[0126] In some embodiments, the second vehicle calculates the second fusion gradient according to the first gradient and the second gradient uploaded by the plurality of first vehicles. For example, the third fusion gradient at each time is calculated according to the first gradient uploaded by the plurality of first vehicles at each time and the second gradient at each time; and the second fusion gradient is calculated according to a plurality of gradients with the largest value in the third fusion gradient at each time.

[0127] For example, the local fusion and the second fusion gradient after sparse processing of the cluster center vehicle c are as follows:

[0128]

[0129] is the second gradient calculated by the vehicle c at time t, is the first gradient uploaded by the vehicle n at time t.

[0130] In some embodiments, the latency function is determined according to the maximum value of the sum of the first latency and the second latency, taking each candidate resource and each candidate vehicle as a variable; the objective function is determined according to the latency function; and the sending resource and the second vehicle are determined by solving the objective function under the condition that the objective function reaches a minimum value.

[0131] In the above embodiments, the non-identically distributed characteristics of the local data for the cluster center vehicles need to be fused and sparsely processed. Therefore, the clustering communication scheme not only affects the transmission process, but also affects the performance of model training.

[0132] For example, in order to make the clustering communication scheme reduce the upload delay while ensuring the model training performance, the model loss function of the neural network can be added to the objective function to jointly optimize the model training performance.

[0133] In some embodiments, the objective function is determined according to a delay function and a weighted average of loss functions of machine learning models corresponding to each candidate vehicle, and the loss function corresponding to each candidate vehicle is determined according to the gradient uploaded by each candidate vehicle to the server.

[0134] For example, the machine learning model is updated using the candidate fusion gradient calculated by the server according to the gradient uploaded by each candidate vehicle, and the uploaded gradient is determined according to the first gradient and the gradient calculated by each candidate vehicle; the test data is processed using the updated machine learning model to obtain a test result; and the loss function corresponding to each candidate vehicle is determined according to the test result and the label of the test data.

[0135] For example, after all candidate cluster center vehicles complete gradient uploading, the server performs gradient fusion to obtain a candidate fusion gradient; the model gradient M t at time t is updated to the model gradient at time t+1:

[0136]

[0137] For example, the loss function corresponding to each candidate vehicle can be determined using a cross-entropy loss function:

[0138] L loss = f loss (M t+1 ,d,Θ)

[0139] d and Θ represent the input data and the label of the test set, respectively.

[0140] In some embodiments, in order to jointly optimize the transmission delay in the delay function and the model loss in the loss function, a harmonic factor α is introduced as a weight to linearly weight the delay function and the loss function. For example, the objective function is as follows:

[0141]

[0142] X and Y represent the allocation variable x n,c and the channel resource allocation variable y n,r of the cluster center vehicle, respectively.

[0143] In the above embodiments, for the integer linear programming optimization problem, the heuristic optimization scheme is proposed according to the uploading delay T delay the characteristics of (X, Y) and the model loss function L loss (X). For the uploading delay T delay (X, Y), the value is affected by both the clustering communication scheme and the V2V channel resource allocation scheme; for the model loss function L loss (X), since the model parameters are only related to the local fusion result, the value is only affected by the clustering scheme.

[0144] In some embodiments, in a first time period, the objective function is determined according to the delay function; in a second time period, the objective function is determined according to the delay function and the weighted average of the loss functions of the machine learning models corresponding to the candidate vehicles, the second time period being after the first time period.

[0145] For example, due to the complexity of the neural network, the loss function of the model cannot be expressed as an explicit function expression, and the specific value needs to be obtained by forward inference using the input data and the updated model. Therefore, the server can only calculate the loss function after each round of federated learning update.

[0146] Based on this calculation process, the loss function part is used as the reference optimization quantity at each iteration. Therefore, the optimization problem to be solved in the current round is degenerated to:

[0147]

[0148] In some embodiments, this channel resource allocation problem can be estimated and planned using the channel information collected by the server. For example, a heuristic greedy search scheme can be used to solve this integer linear programming problem.

[0149] For example, first, the clustering matrix X G is initialized; the signal-to-noise ratio matrix Γ is calculated according to the collected channel information Then, Y′ is fixed, and the optimal clustering matrix X After multiple iterations, the optimal clustering matrix and channel allocation matrix are obtained.

[0150] In the above embodiments, as the number of training rounds increases, the performance of the model gradually improves, and the expected value of the model loss function monotonically decreases; since the model only changes the parameter value, the structure and the number of parameters do not change, so the amount of transmission data does not decrease accordingly, and the uploading delay of each round is basically unchanged.

[0151] In some embodiments, the weight value at the current time is determined according to the initial weight value and the weight decay factor at the current time; the weighted average of the delay function and the loss function is determined according to the weight value at the current time.

[0152] For example, in order to balance the weights of the upload delay and the model loss function in the objective function between different rounds, the weight of the model loss function can be set. The dynamic weight at time t can be set as follows:

[0153] α t = α G · e -βt

[0154] α G is the initial weight, and e -βt is the weight decay factor.

[0155] For example, for the measurement of multi-round training, the overall overhead should be considered in terms of delay, while only the final training effect needs to be considered in terms of model performance. Therefore, the objective function for training multiple rounds is as follows:

[0156]

[0157] After the first vehicle transmits the first gradient, the remaining steps in the following equation can be used to continue the model training. Figure 1

[0158] In step 120, the machine learning model is trained using the first fusion gradient issued by the server. The first fusion gradient is generated according to the first gradient uploaded by the second vehicle and the second gradient calculated by the second vehicle according to the machine learning model.

[0159] In some embodiments, the first fusion gradient is calculated according to second fusion gradients uploaded by multiple second vehicles, and the second fusion gradient is calculated according to first gradients and second gradients uploaded by multiple first vehicles. For example, the second fusion gradient is calculated according to multiple gradients with the largest value in multiple third fusion gradients, and each third fusion gradient is calculated according to first gradients uploaded by multiple first vehicles at each time and second gradients at each time.

[0160] In the above embodiments, the model training performance and the upload delay are jointly considered to study the heterogeneous model updating scheme; the cluster center vehicle with good uplink channel quality collects gradient information of vehicles with poor uplink channel quality around it through V2V; considering the non-independent and identically distributed characteristics of local data, fusion and sparsification are attempted locally at the cluster center vehicle; and the operation is transmitted to the center server through the V2R uplink. By increasing the transmission rate and reducing the amount of uploaded data, the overall training time is reduced; and by fusing the local non-independent and identically distributed characteristics, a certain model generalization ability is introduced.

[0161] Figure 3 Flowcharts showing other embodiments of the method for training the machine learning model of the present disclosure.

[0162] As​Figure 3 As shown in step 310, the first gradient calculated by the first vehicle is received by the second vehicle. The first gradient is calculated by the server-deployed machine learning model, and the channel quality of the second vehicle is better than that of the first vehicle.

[0163] In step 320, the second gradient is calculated by the second vehicle using the server-deployed machine learning model.

[0164] In step 330, the first gradient and the second gradient are uploaded to the server by the second vehicle.

[0165] In some embodiments, a second fusion gradient is calculated according to the first gradient and the second gradient uploaded by the plurality of first vehicles; the second fusion gradient is uploaded to the server, and a first fusion gradient is calculated according to the second fusion gradient uploaded by the plurality of second vehicles.

[0166] For example, a third fusion gradient at each time is calculated according to the first gradient and the second gradient uploaded by the plurality of first vehicles at each time; and the second fusion gradient is calculated according to a plurality of gradients with the largest value in the third fusion gradient at each time.

[0167] In step 340, the first fusion gradient is used by the second vehicle to train the machine learning model. The first fusion gradient is generated according to the first gradient and the second gradient.

[0168] Figure 4 A flowchart showing some embodiments of the control method of the unmanned vehicle of the present disclosure.

[0169] As Figure 4 As shown in step 410, the machine learning model is trained by the following steps: the first gradient calculated by the first vehicle using the server-deployed machine learning model is sent to the second vehicle, the channel quality of the second vehicle is better than that of the first vehicle; the first fusion gradient is used to train the machine learning model, the first fusion gradient is generated according to the first gradient and the second gradient uploaded by the second vehicle, and the second gradient is calculated by the second vehicle according to the machine learning model.

[0170] In step 420, the trained machine learning model is used to control the unmanned vehicle,

[0171] Figure 5a A block diagram showing some embodiments of the training device of the machine learning model of the present disclosure.

[0172] As Figure 5aAs shown, the training device 5a of the machine learning model comprises: a sending unit 51a configured to send the first gradient calculated by the machine learning model of the first vehicle to the second vehicle, the channel quality of the second vehicle being better than that of the first vehicle; and a training unit 52a configured to train the machine learning model by using the first fusion gradient sent by the server, the first fusion gradient being generated according to the first gradient uploaded by the second vehicle and a second gradient calculated by the second vehicle according to the machine learning model.

[0173] In some embodiments, the training device 5a further comprises a calculation unit 53a configured to calculate a first time delay of the first vehicle in sending the estimated data amount to each candidate vehicle, and calculate a second time delay of each candidate vehicle in uploading the estimated data amount to the server, the channel quality of each candidate vehicle being better than that of the first vehicle, and determine the second vehicle from each candidate vehicle according to the first time delay and the second time delay.

[0174] In some embodiments, the calculation unit 53a calculates the first time delay of the first vehicle in sending the estimated data amount to each candidate vehicle through each candidate resource; determines the sending resource from each candidate resource according to the first time delay and the second time delay; and the sending unit 51a sends the first gradient to the second vehicle by using the sending resource.

[0175] In some embodiments, the calculation unit 53a determines a time delay function according to a maximum value of the sum of the first time delay and the second time delay, taking each candidate resource and each candidate vehicle as variables, determines a target function according to the time delay function, and determines the sending resource and the second vehicle by solving the target function under the condition that the target function reaches a minimum value.

[0176] In some embodiments, the calculation unit 53a determines the target function according to the time delay function and a weighted average of loss functions of the machine learning models corresponding to each candidate vehicle, and determines the loss function corresponding to each candidate vehicle according to the gradient uploaded by each candidate vehicle to the server.

[0177] In some embodiments, the calculation unit 53a updates the machine learning model by using a candidate fusion gradient calculated by the server according to the gradient uploaded by each candidate vehicle, the uploaded gradient being determined according to the first gradient and the gradient calculated by each candidate vehicle, processes test data by using the updated machine learning model to obtain a test result, and determines the loss function corresponding to each candidate vehicle according to the test result and a label of the test data.

[0178] In some embodiments, the calculation unit 53a determines the weight value at the current moment according to the initial weight value and a weight value decay factor at the current moment, and determines the weighted average of the time delay function and the loss function according to the weight value at the current moment.

[0179] In some embodiments, the weight value of the current time positively correlates with the initial weight value and negatively correlates with the current time.

[0180] In some embodiments, the computing unit 53a determines the target function according to the latency function in the first time period, and determines the target function according to the latency function and the weighted mean of the loss functions of the machine learning models corresponding to the candidate vehicles in the second time period, the second time period being after the first time period.

[0181] In some embodiments, the computing unit 53a calculates the transmission rate of the first vehicle on each candidate resource according to the signal-to-noise ratio of the first vehicle on each candidate resource and the channel bandwidth of each candidate resource, and determines the first latency according to the transmission rate of the first vehicle on each candidate resource and the estimated data volume.

[0182] In some embodiments, the computing unit 53a determines the signal-to-noise ratio of the first vehicle on each candidate resource according to the signal power of the first vehicle on each candidate resource and the signal power of each candidate vehicle on each candidate resource.

[0183] In some embodiments, the computing unit 53a determines the first latency according to the latency when the transmission rate of the first vehicle on each candidate resource is greater than or equal to the estimated data volume, and determines the second latency according to the latency when the transmission rate of the second vehicle is greater than or equal to the estimated data volume.

[0184] In some embodiments, the first fusion gradient is calculated according to the second fusion gradients uploaded by the plurality of second vehicles, and the second fusion gradient is calculated according to the first gradient and the second gradient uploaded by the plurality of first vehicles.

[0185] In some embodiments, the second fusion gradient is calculated according to a plurality of gradients with the largest value in a plurality of third fusion gradients, and each third fusion gradient is calculated according to the first gradient uploaded by the plurality of first vehicles at each time and the second gradient at each time.

[0186] In some embodiments, the sending unit 53a calculates the gradient of the machine learning model at a plurality of times according to the local data of the first vehicle, and determines the first gradient according to a plurality of gradients with the largest value in the plurality of times.

[0187] Figure 5b A block diagram showing another embodiment of the training device of the machine learning model of the present disclosure.

[0188] As Figure 5bAs shown, the training device 5b for the machine learning model includes: a receiving unit 51b, used to receive a first gradient sent by a first vehicle via a second vehicle, the first gradient being calculated using a machine learning model sent by a server, wherein the channel quality of the second vehicle is better than that of the first vehicle; a calculation unit 52b, used to calculate a second gradient via the second vehicle using the machine learning model sent by the server; an uploading unit 53b, used to upload the first gradient and the second gradient to the server via the second vehicle; and a training unit 54b, used to train the machine learning model via the second vehicle using a first fused gradient sent by the server, wherein the first fused gradient is generated based on the first gradient and the second gradient.

[0189] In some embodiments, the uploading unit 53b calculates a second fused gradient based on the first gradient and the second gradient uploaded by multiple first vehicles; and uploads the second fused gradient to the server, wherein the first fused gradient is calculated based on the second fused gradient uploaded by multiple second vehicles.

[0190] In some embodiments, the calculation unit 52b calculates the third fusion gradient at each time based on the first gradient uploaded by the plurality of first vehicles at each time and the second gradient at each time; and calculates the second fusion gradient based on the plurality of gradients with the largest third fusion gradient at each time.

[0191] Figure 6 Block diagrams illustrating some embodiments of the control device for the unmanned vehicle disclosed herein are shown.

[0192] like Figure 6 As shown, the control device 6 of the unmanned vehicle includes: a control unit 61, used to control the unmanned vehicle using a trained machine learning model. The machine learning model is trained through the following steps: the first gradient calculated by the machine learning model sent by the server to the second vehicle, the channel quality of the second vehicle is better than that of the first vehicle; the machine learning model is trained using the first fused gradient sent by the server, the first fused gradient is generated based on the first gradient and the second gradient uploaded by the second vehicle, and the second gradient is calculated by the second vehicle based on the machine learning model.

[0193] Figure 7 Block diagrams illustrating some embodiments of the driverless vehicle disclosed herein are shown.

[0194] like Figure 7 As shown, the unmanned vehicle 7 includes a control device 71, used to execute the training method of the machine learning model in any of the above embodiments, or the control method of the unmanned vehicle.

[0195] Figure 8 Block diagrams illustrating some embodiments of the control system for the unmanned vehicle disclosed herein are shown.

[0196] like Figure 8As shown, the control system 8 of the unmanned vehicle includes the unmanned vehicle 81 in any of the above embodiments; and a server 82 configured to issue a machine learning model to the unmanned vehicle.

[0197] Figure 9 A block diagram illustrating some embodiments of the electronic device of the present disclosure.

[0198] As Figure 9 As shown, the electronic device 9 of this embodiment includes a memory 91 and a processor 92 coupled to the memory 91, the processor 92 being configured to perform the training method of the machine learning model in any of the above embodiments, or the control method of the unmanned vehicle, based on instructions stored in the memory 91.

[0199] The memory 91 may, for example, include a system memory, a fixed non-volatile storage medium, etc. The system memory may, for example, store an operating system, an application program, a Boot Loader, a database, and other programs, etc.

[0200] Figure 10 A block diagram illustrating some other embodiments of the electronic device of the present disclosure.

[0201] As Figure 10 As shown, the electronic device 10 of this embodiment includes a memory 1010 and a processor 1020 coupled to the memory 1010, the processor 1020 being configured to perform the training method of the machine learning model in any of the above embodiments, or the control method of the unmanned vehicle, based on instructions stored in the memory 1010.

[0202] The memory 1010 may, for example, include a system memory, a fixed non-volatile storage medium, etc. The system memory may, for example, store an operating system, an application program, a Boot Loader, and other programs, etc.

[0203] The electronic device 10 may, for example, further include an input / output interface 1030, a network interface 1040, a storage interface 1050, etc. These interfaces 1030, 1040, 1050, and the memory 1010 and the processor 1020 may, for example, be connected through a bus 1060. The input / output interface 1030 provides a connection interface for display, mouse, keyboard, touch screen, microphone, speaker, and other input / output devices. The network interface 1040 provides a connection interface for various networking devices. The storage interface 1050 provides a connection interface for external storage devices such as SD card and U disk.

[0204] Figures 11a-11b A block diagram illustrating some other embodiments of the unmanned vehicle of the present disclosure.

[0205] As Figure 11aAs shown, it is a vehicle system framework of the unmanned vehicle. The unmanned vehicle mainly includes a chassis module, an automatic driving module, a cargo box module, and a remote monitoring and streaming module. The automatic driving module includes a core processing unit (Orin or Xavier module), a traffic light recognition camera, front and rear surround view cameras, a multi-line laser radar, a positioning module (such as Beidou, GPS, etc.), and an inertial navigation unit. The camera can communicate with the automatic driving module. In order to improve the transmission speed and reduce the wire harness, a GMSL link communication can be used. The chassis module mainly includes a battery, a power management device, a chassis controller, a motor driver, and a power motor. The battery provides power for the entire unmanned vehicle system. The power management device converts the battery output into different voltage levels that can be used by each functional module, and controls the power on and off. The chassis controller receives the motion instructions issued by the automatic driving module, and controls the steering, forward movement, backward movement, and braking of the unmanned vehicle. The remote monitoring and streaming module is composed of a front monitoring camera, a rear monitoring camera, a left monitoring camera, a right monitoring camera, and a streaming module. The module transmits the video data collected by the monitoring cameras to the background server for viewing by the background operator. The wireless communication module communicates with the background server through an antenna, and can realize remote control of the unmanned vehicle by the background operator. The cargo box module is a cargo carrying device of the unmanned vehicle. The cargo box module is also provided with a display interaction module. The display interaction module is used for interaction between the unmanned vehicle and the user. The user can perform operations such as picking up, storing, and purchasing goods through the display interaction module. The type of the cargo box can be changed according to actual needs. For example, in a logistics scenario, the cargo box can include multiple sub-boxes of different sizes, which can be used to load goods for distribution. In a retail scenario, the cargo box can be set as a transparent box to allow the user to directly view the products for sale.

[0206] As shown, Figure 11b the hardware structure of the unmanned vehicle. The unmanned vehicle includes a laser radar, a right side blind filling radar, a rear side blind filling radar, a left side blind filling radar, a right side camera, a cargo box, a chassis, a display interaction module, etc.

[0207] Those skilled in the art should understand that the embodiments of the present disclosure can be provided as a method, a system, or a computer program product. Therefore, the present disclosure can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present disclosure can take the form of a computer program product implemented on one or more computer usable non-transitory storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0208] Thus far, the training method of a machine learning model, the training apparatus of a machine learning model, the control method of an unmanned vehicle, the control apparatus of an unmanned vehicle, the control system of an unmanned vehicle, the electronic device, and the unmanned vehicle according to the present disclosure have been described in detail. In order to avoid obscuring the idea of the present disclosure, some details well-known in the art are not described. Those skilled in the art can fully understand how to implement the technical solutions disclosed herein according to the above description.

[0209] The methods and systems of the present disclosure can be implemented in a number of ways. For example, the methods and systems of the present disclosure can be implemented via software, hardware, firmware, or any combination of software, hardware, and firmware. The above described order of steps for the methods is merely for illustration, and the steps of the methods of the present disclosure are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, the present disclosure can also be implemented as programs recorded in a recording medium, which include machine-readable instructions for implementing the methods according to the present disclosure. Thus, the present disclosure also covers the recording medium storing the programs for executing the methods according to the present disclosure.

[0210] While some specific embodiments of the present disclosure have been described in detail by way of examples, those skilled in the art should understand that the above examples are only for illustration, and are not intended to limit the scope of the present disclosure. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. A method for training a machine learning model, executed on the side of a first vehicle, comprising: The first gradient of the machine learning model calculated by the first vehicle using the machine learning model issued by the server is sent to the second vehicle. The uplink channel quality between the second vehicle and the server is better than that between the first vehicle and the server. The machine learning model is trained locally on the first vehicle using the first fusion gradient issued by the server. The first fusion gradient is generated based on the first gradient uploaded by the second vehicle and the second gradient of the machine learning model. The second gradient is calculated locally on the second vehicle based on the machine learning model. The first delay for the first vehicle to send the estimated data amount to each candidate vehicle is calculated, and the second delay for each candidate vehicle to upload the estimated data amount to the server is calculated, wherein the channel quality of each candidate vehicle is better than that of the first vehicle. Based on the first delay and the second delay, the second vehicle is determined from the candidate vehicles. The first delay in calculating the estimated data amount sent by the first vehicle to each candidate vehicle includes: Calculate the first delay for the first vehicle to send the estimated data amount to each of the candidate vehicles through each candidate resource; Based on the first delay and the second delay, a transmission resource is determined from the candidate resources; The step of sending the first gradient calculated by the first vehicle using the machine learning model sent by the server to the second vehicle includes: Using the aforementioned transmission resources, the first gradient is transmitted to the second vehicle.

2. The training method according to claim 1, wherein, The step of determining the second vehicle from the candidate vehicles and determining the transmission resource from the candidate resources includes: Using each candidate resource and each candidate vehicle as variables, a delay function is determined based on the maximum value of the sum of the first delay and the second delay; Based on the time delay function, determine the objective function; The objective function is solved by taking the condition that it reaches its minimum value, and the sending resources and the second vehicle are determined.

3. The training method according to claim 2, wherein, The determination of the objective function includes: The objective function is determined based on the weighted average of the time delay function and the loss function of the machine learning model corresponding to each candidate vehicle. The loss function corresponding to each candidate vehicle is determined based on the gradient uploaded by each candidate vehicle to the server.

4. The training method according to claim 3, wherein, Determining the objective function includes: The machine learning model is updated using the candidate fusion gradient calculated by the server based on the gradients uploaded by each candidate vehicle. The uploaded gradient is determined based on the first gradient and the gradients calculated by each candidate vehicle. The updated machine learning model is used to process the test data to obtain the test results; Based on the test results and the labels of the test data, the loss function corresponding to each candidate vehicle is determined.

5. The training method according to claim 3, wherein, Determining the objective function includes: The weight at the current moment is determined based on the initial weight and the weight decay factor at the current moment; Based on the weights at the current moment, determine the weighted average of the time delay function and the loss function.

6. The training method according to claim 5, wherein, The weight at the current moment is positively correlated with the initial weight and negatively correlated with the current moment.

7. The training method according to claim 3, wherein, The step of determining the objective function based on the weighted average of the time delay function and the loss function of the machine learning model corresponding to each candidate vehicle includes: Within the first time period, the objective function is determined based on the time delay function; During the second time period, the objective function is determined based on the weighted average of the time delay function and the loss function of the machine learning model corresponding to each candidate vehicle. The second time period is located after the first time period.

8. The training method according to claim 1, wherein, The calculation of the first delay in sending the estimated data amount from the first vehicle to each candidate vehicle through each candidate resource includes: The transmission rate of the first vehicle on each candidate resource is calculated based on the signal-to-noise ratio of the first vehicle on each candidate resource and the channel bandwidth of each candidate resource. The first delay is determined based on the transmission rate of the first vehicle on each candidate resource and the estimated data volume.

9. The training method according to claim 8, wherein, The calculation of the transmission rate of the first vehicle on each candidate resource includes: The signal-to-noise ratio of the first vehicle in each candidate resource is determined based on the signal power of the first vehicle in each candidate resource and the signal power of each candidate vehicle in each candidate resource.

10. The training method according to claim 1, wherein, Determining the first time delay includes: The first delay is determined based on the delay when the transmission rate of the first vehicle on each candidate resource is greater than or equal to the estimated data volume; The second delay in calculating the estimated data amount uploaded by each candidate vehicle to the server includes: The second delay is determined based on the delay of the second vehicle when the transmission rate is greater than or equal to the estimated data amount.

11. The training method according to any one of claims 1-10, wherein, The first fusion gradient is calculated based on the second fusion gradient uploaded by multiple second vehicles, and the second fusion gradient is calculated based on the first gradient and the second gradient uploaded by multiple first vehicles.

12. The training method according to claim 11, wherein, The second fusion gradient is calculated based on the largest gradient among the third fusion gradients at multiple times. Each third fusion gradient is calculated based on the first gradient uploaded by multiple first vehicles at each time and the second gradient at each time.

13. The training method according to any one of claims 1-10, wherein, The step of sending the first gradient calculated by the first vehicle using the machine learning model sent by the server to the second vehicle includes: Based on the local data of the first vehicle, calculate the gradient of the machine learning model at multiple time points; The first gradient is determined based on the largest gradient among the gradients at the plurality of time points.

14. A method for training a machine learning model, performed on the second vehicle side, comprising: The second vehicle receives the first gradient of the machine learning model sent by the first vehicle. The first gradient is calculated using the machine learning model sent by the server. The uplink channel quality between the second vehicle and the server is better than that between the first vehicle and the server. The second vehicle is determined from candidate vehicles based on a first delay and a second delay. The first delay is determined by calculating the delay of the first vehicle sending the estimated amount of data to each candidate vehicle using each candidate resource. The second delay is determined by calculating the delay of each candidate vehicle uploading the estimated amount of data to the server. The channel quality of each candidate vehicle is better than that of the first vehicle. Each candidate resource is used to determine the transmission resource based on the first delay and the second delay. The second vehicle uses the machine learning model issued by the server to calculate the second gradient of the machine learning model; The first gradient and the second gradient are uploaded to the server via the second vehicle; The machine learning model is trained locally on the second vehicle using the first fusion gradient issued by the server. The first fusion gradient is generated based on the first gradient and the second gradient. Wherein, receiving the first gradient of the machine learning model sent by the first vehicle through the second vehicle includes: The first gradient is received by the second vehicle using the aforementioned transmission resources.

15. A control method for an unmanned vehicle, comprising: The autonomous vehicle is controlled using a trained machine learning model, which is trained through the following steps: The first vehicle sends the first gradient of the machine learning model calculated by the machine learning model issued by the server to the second vehicle, and the uplink channel quality between the second vehicle and the server is better than that between the first vehicle and the server. The machine learning model is trained locally on the first vehicle using the first fusion gradient issued by the server. The first fusion gradient is generated based on the first gradient uploaded by the second vehicle and the second gradient of the machine learning model. The second gradient is calculated locally on the second vehicle based on the machine learning model. The first delay for the first vehicle to send the estimated data amount to each candidate vehicle is calculated, and the second delay for each candidate vehicle to upload the estimated data amount to the server is calculated, wherein the channel quality of each candidate vehicle is better than that of the first vehicle. Based on the first delay and the second delay, the second vehicle is determined from the candidate vehicles. The first delay in calculating the estimated data amount sent by the first vehicle to each candidate vehicle includes: Calculate the first delay for the first vehicle to send the estimated data amount to each of the candidate vehicles through each candidate resource; Based on the first delay and the second delay, a transmission resource is determined from the candidate resources; The step of sending the first gradient calculated by the first vehicle using the machine learning model sent by the server to the second vehicle includes: Using the aforementioned transmission resources, the first gradient is transmitted to the second vehicle.

16. A training device for a machine learning model, configured on the side of a first vehicle, comprising: The sending unit is used to send the first gradient of the machine learning model calculated by the first vehicle using the machine learning model issued by the server to the second vehicle, wherein the uplink channel quality between the second vehicle and the server is better than the uplink channel quality between the first vehicle and the server. The training unit is used to train the machine learning model locally on the first vehicle using the first fusion gradient issued by the server. The first fusion gradient is generated based on the first gradient uploaded by the second vehicle and the second gradient of the machine learning model. The second gradient is calculated locally on the second vehicle based on the machine learning model. The calculation unit is configured to calculate a first delay for the first vehicle to send the estimated data amount to each candidate vehicle, and a second delay for each candidate vehicle to upload the estimated data amount to the server, wherein the channel quality of each candidate vehicle is better than that of the first vehicle. Based on the first delay and the second delay, the second vehicle is determined from the candidate vehicles. The unit also calculates the first delay for the first vehicle to send the estimated data amount to each candidate vehicle through each candidate resource, and determines the transmission resource from the candidate resources based on the first delay and the second delay. The sending unit is used to send the first gradient to the second vehicle using the sending resources.

17. A training device for a machine learning model, configured on the side of a second vehicle, comprising: The receiving unit is configured to receive, via the second vehicle, a first gradient of a machine learning model sent by the first vehicle. The first gradient is calculated using the machine learning model sent by the server. The uplink channel quality between the second vehicle and the server is better than that between the first vehicle and the server. The second vehicle is determined from candidate vehicles based on a first delay and a second delay. The first delay is determined by calculating the delay at which the first vehicle sends an estimated amount of data to each candidate vehicle using each candidate resource. The second delay is determined by calculating the delay at which each candidate vehicle uploads the estimated amount of data to the server. The channel quality of each candidate vehicle is better than that of the first vehicle. Each candidate resource is used to determine a transmission resource based on the first delay and the second delay. The computing unit is used to calculate the second gradient of the machine learning model by using the machine learning model issued by the server through the second vehicle; An upload unit is used to upload the first gradient and the second gradient to the server via the second vehicle; The training unit is configured to train the machine learning model locally on the second vehicle using the first fused gradient issued by the server. The first fused gradient is generated based on the first gradient and the second gradient. The receiving unit is configured to receive the first gradient via the second vehicle using the transmitting resources.

18. A control device for an unmanned vehicle, comprising: A control unit is used to control the autonomous vehicle using a trained machine learning model, which is trained through the following steps: The first vehicle sends the first gradient of the machine learning model calculated by the machine learning model issued by the server to the second vehicle, and the uplink channel quality between the second vehicle and the server is better than that between the first vehicle and the server. The machine learning model is trained locally on the first vehicle using the first fusion gradient issued by the server. The first fusion gradient is generated based on the first gradient uploaded by the second vehicle and the second gradient of the machine learning model. The second gradient is calculated locally on the second vehicle based on the machine learning model. The first delay for the first vehicle to send the estimated data amount to each candidate vehicle is calculated, and the second delay for each candidate vehicle to upload the estimated data amount to the server is calculated, wherein the channel quality of each candidate vehicle is better than that of the first vehicle. Based on the first delay and the second delay, the second vehicle is determined from the candidate vehicles. The first delay in calculating the estimated data amount sent by the first vehicle to each candidate vehicle includes: Calculate the first delay for the first vehicle to send the estimated data amount to each of the candidate vehicles through each candidate resource; Based on the first delay and the second delay, a transmission resource is determined from the candidate resources; The step of sending the first gradient calculated by the first vehicle using the machine learning model sent by the server to the second vehicle includes: Using the aforementioned transmission resources, the first gradient is transmitted to the second vehicle.

19. An unmanned vehicle, comprising: A control device for executing the training method of the machine learning model according to any one of claims 1 to 14, or the control method of the unmanned vehicle according to claim 15.

20. A control system for an unmanned vehicle, comprising: The unmanned vehicle as described in claim 19; The server is used to send machine learning models to the driverless vehicle.

21. An electronic device, comprising: Memory; and A processor coupled to the memory, the processor being configured to execute, based on instructions stored in the memory, the training method of the machine learning model according to any one of claims 1-14, or the control method of the unmanned vehicle according to claim 15.

22. A non-volatile computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the training method of the machine learning model according to any one of claims 1-14, or the control method of the unmanned vehicle according to claim 15.

Images