Multi-target model training and multi-target distribution method and device based on reinforcement learning

Abstract

The invention provides a multi-target model training method and device based on reinforcement learning, and relates to the technical field of artificial intelligence. A specific embodiment of the method comprises the steps of obtaining a preset sample set and a multi-target network pre-constructed based on reinforcement learning; executing the following training steps: selecting a sample task group from the sample set and the remaining working time of the vehicle group corresponding to the sample task group; the task information of the sample task group and the remaining working duration of the vehicle group corresponding to the sample task group are input into a multi-target network; based on a task vehicle distribution strategy output by the multi-target network, calculating an expected value of a reward function of the multi-target network, the reward function being at least used for representing a vehicle short-distance and a reduction amount of a vehicle number; and in response to the expected value reaching a training completion condition, obtaining a multi-target model corresponding to the multi-target network. According to the embodiment, the accuracy of sequential decision making is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, specifically to the field of artificial intelligence technology, and in particular to a multi-objective model training method and device based on reinforcement learning, a multi-objective allocation method and device, an electronic device, and a computer-readable medium. Background Art

[0002] With the rapid development of the Internet and e-commerce, logistics, as the most basic and important link in the circulation field, has also achieved unprecedented development.

[0003] In the field of e-commerce logistics and distribution, the current scheduling models and optimization strategies are mainly divided into two categories: modeling the scheduling of tasks (orders) that need to be transported and vehicles, usually using integer programming modeling with any one of the following single goals: the minimum number of vehicles, the minimum total mileage of all vehicles, and the minimum total cost, and using heuristic algorithms, local search, genetic algorithms, etc. to solve them. However, the existing scheduling models do not consider the impact of connection fees in the vehicle distribution process. Summary of the invention

[0004] The embodiments of the present disclosure propose a multi-objective model training method and device based on reinforcement learning, a multi-objective allocation method and device, an electronic device, and a computer-readable medium.

[0005] In a first aspect, an embodiment of the present disclosure provides a multi-objective model training method based on reinforcement learning, the method comprising: obtaining a preset sample set and a multi-objective network pre-constructed based on reinforcement learning, wherein the sample set comprises: task information of at least one sample task group and the remaining working time of a vehicle group corresponding to the sample task group, and the multi-objective network is used to characterize the correspondence between the task information, the remaining working time and the task vehicle allocation strategy; performing the following training steps: selecting a sample task group and the remaining working time of a vehicle group corresponding to the sample task group from the sample set; inputting the task information of the sample task group and the remaining working time of the vehicle group corresponding to the sample task group into the multi-objective network; calculating the expected value of the reward function of the multi-objective network based on the task vehicle allocation strategy output by the multi-objective network, the reward function being used to at least characterize the reduction in vehicle short-distance and vehicle quantity; in response to the expected value reaching the training completion condition, obtaining a multi-objective model corresponding to the multi-objective network.

[0006] In some embodiments, the strategy function corresponding to the above-mentioned multi-objective network is used to characterize the conditional probability of input states and output tasks; the multi-objective network includes: an encoder and a decoder, the encoder is used to convert the task information of the selected sample task group and the remaining working time of the vehicle group into a feature sequence, and the decoder is used to convert the feature sequence into a task vehicle allocation strategy.

[0007] In some embodiments, the multi-objective network is also used to characterize the correspondence between vehicle types and task vehicle allocation strategies; the sample set includes: vehicle types of the vehicle group, and the method also includes: detecting whether the vehicle type of the vehicle group matches the task information of the task group, and in response to the vehicle type matching the task information, when executing the training step, the vehicle type of the vehicle group is input into the multi-objective network.

[0008] In some embodiments, the method also includes: setting constraints for the task group and the vehicle group; inputting the constraints, the task information of the sample task group and the remaining working time of the vehicle group corresponding to the sample task group into the multi-objective network so that the multi-objective network excludes the task vehicle allocation strategy under the constraints.

[0009] In a second aspect, an embodiment of the present disclosure provides a multi-objective allocation method, the method comprising: obtaining task data of a task group to be assigned and the remaining working time of a vehicle group to be assigned; inputting the task data of the task group to be assigned and the remaining working time of the vehicle group to be assigned into a multi-objective model, and obtaining a vehicle allocation strategy corresponding to each task in the task group to be assigned output by the multi-objective model, wherein the multi-objective model is trained based on the multi-objective model training method of any embodiment in the first aspect.

[0010] In a third aspect, an embodiment of the present disclosure provides a multi-objective model training device based on reinforcement learning, the device comprising: a sample acquisition unit, configured to acquire a preset sample set and a multi-objective network pre-constructed based on reinforcement learning, wherein the sample set comprises: task information of at least one sample task group and the remaining working time of a vehicle group corresponding to the sample task group, and the multi-objective network is used to characterize the correspondence between the task information, the remaining working time and the task vehicle allocation strategy; a selection unit, configured to select a sample task group and the remaining working time of the vehicle group corresponding to the sample task group from the sample set; an input unit, configured to input the task information of the sample task group and the remaining working time of the vehicle group corresponding to the sample task group into the multi-objective network; a calculation unit, configured to calculate the expected value of the reward function of the multi-objective network based on the task vehicle allocation strategy output by the multi-objective network, the reward function being used to characterize at least the reduction in vehicle short-distance and vehicle quantity; a training unit, configured to obtain a multi-objective model corresponding to the multi-objective network in response to the expected value reaching the training completion condition.

[0011] In some embodiments, the strategy function corresponding to the above-mentioned multi-objective network is used to characterize the conditional probability of input states and output tasks; the multi-objective network includes: an encoder and a decoder, the encoder is used to convert the task information of the selected sample task group and the remaining working time of the vehicle group into a feature sequence, and the decoder is used to convert the feature sequence into a task vehicle allocation strategy.

[0012] In some embodiments, the above-mentioned multi-objective network is also used to characterize the correspondence between vehicle types and task vehicle allocation strategies; the sample set includes: the vehicle type of the vehicle group, and the device also includes: a detection unit, configured to detect whether the vehicle type of the vehicle group matches the task information of the task group, and in response to the vehicle type matching the task information, the control input unit inputs the vehicle type of the vehicle group into the multi-objective network.

[0013] In some embodiments, the device also includes: a setting unit, configured to set constraints for the task group and the vehicle group, and control the input unit to input the constraints, the task information of the sample task group and the remaining working time of the vehicle group corresponding to the sample task group into the multi-objective network, so that the multi-objective network excludes the task vehicle allocation strategy under the constraints.

[0014] In the fourth aspect, an embodiment of the present disclosure provides a multi-objective allocation device, which includes: a data acquisition unit, configured to acquire task data of the task group to be assigned and the remaining working time of the vehicle group to be assigned; an acquisition unit, configured to input the task data of the task group to be assigned and the remaining working time of the vehicle group to be assigned into a multi-objective model, and obtain the vehicle allocation strategy corresponding to each task in the task group to be assigned output by the multi-objective model, and the multi-objective model is trained based on the multi-objective model training device of any embodiment of the third aspect.

[0015] In a fifth aspect, an embodiment of the present disclosure provides an electronic device, comprising: one or more processors; a storage device on which one or more programs are stored; when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any embodiment of the first aspect or the second aspect.

[0016] In a sixth aspect, an embodiment of the present disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any embodiment of the first aspect or the second aspect.

[0017] The multi-objective model training method and device based on reinforcement learning provided by the embodiment of the present disclosure first obtains a preset sample set and a multi-objective network pre-constructed based on reinforcement learning; secondly, selects the remaining working time of the sample task group and the vehicle group corresponding to the sample task group from the sample set; thirdly, inputs the task information of the sample task group and the remaining working time of the vehicle group corresponding to the sample task group into the multi-objective network; thirdly, calculates the expected value of the reward function of the multi-objective network based on the task vehicle allocation strategy output by the multi-objective network; finally, in response to the expected value reaching the training completion condition, obtains the multi-objective model corresponding to the multi-objective network. Thus, the multi-objective network generated by reinforcement learning can realize the delivery of goods from the warehouse to each distribution point, fully considers the various matches (combinations) between each task in the task group, can reduce the use of vehicles, and at the same time reduces the cost increase caused by unreasonable vehicle connection by optimizing the reward function, thereby reducing the total transportation cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Other features, objects and advantages of the present disclosure will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0019] Figure 1 is an exemplary system architecture diagram in which an embodiment of the present disclosure may be applied;

[0020] Figure 2 is a flowchart of an embodiment of a multi-objective model training method based on reinforcement learning according to the present disclosure;

[0021] Figure 3 It is a structural schematic diagram of a multi-objective network disclosed in the present invention;

[0022] Figure 4 is a flow chart of an embodiment of a multi-objective allocation method according to the present disclosure;

[0023] Figure 5 is a structural schematic diagram of an embodiment of a multi-objective model training device based on reinforcement learning according to the present disclosure;

[0024] Figure 6 is a structural schematic diagram of an embodiment of a multi-objective allocation device according to the present disclosure;

[0025] Figure 7 It is a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure. DETAILED DESCRIPTION

[0026] The present disclosure is further described in detail below in conjunction with the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0027] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0028] Figure 1 An exemplary system architecture 100 is shown to which the reinforcement learning-based multi-objective model training method or multi-objective allocation method of the present disclosure can be applied.

[0029] like Figure 1 As shown, the system architecture 100 may include terminals 101, 102, a network 103, a database server 104, and a server 105. The network 103 is used to provide a medium for communication links between the terminals 101, 102, the database server 104, and the server 105. The network 103 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0030] The user 110 can use the terminals 101 and 102 to interact with the server 105 through the network 103 to receive or send messages, etc. Various client applications can be installed on the terminals 101 and 102, such as model training applications, image recognition applications, shopping applications, payment applications, web browsers, and instant messaging tools.

[0031] The terminals 101 and 102 here can be hardware or software. When the terminals 101 and 102 are hardware, they can be various electronic devices with display screens, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III, Dynamic Image Experts Compression Standard Audio Layer 3), laptops and desktop computers, etc. When the terminals 101 and 102 are software, they can be installed in the electronic devices listed above. It can be implemented as multiple software or software modules (for example, to provide distributed services), or it can be implemented as a single software or software module. No specific limitation is made here.

[0032] The database server 104 may be a database server that provides various services. For example, a sample set may be stored in the database server. The sample set includes a large number of sample task groups and vehicle groups corresponding to the sample task groups. In this way, the user 110 may also select a sample task group and a vehicle group corresponding to the sample task group from the sample set stored in the database server 104 through the terminals 101 and 102.

[0033] The server 105 may also be a server that provides various services, such as a background server that provides support for various applications displayed on the terminals 101 and 102. The background server may train a multi-objective model based on reinforcement learning using samples in the sample set sent by the terminals 101 and 102, and may send the trained multi-objective model to the terminals 101 and 102. In this way, the user may apply the generated multi-objective model to determine a task vehicle allocation strategy, etc.

[0034] The database server 104 and the server 105 here can also be hardware or software. When they are hardware, they can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When they are software, they can be implemented as multiple software or software modules (for example, for providing distributed services), or as a single software or software module. No specific limitation is made here.

[0035] It should be noted that the multi-objective model training method or multi-objective allocation method based on reinforcement learning provided in the embodiments of the present disclosure is generally executed by the server 105. Accordingly, the multi-objective model training device or multi-objective allocation device based on reinforcement learning is generally also provided in the server 105.

[0036] It should be pointed out that, in the case where the server 105 can implement the related functions of the database server 104 , the database server 104 may not be provided in the system architecture 100 .

[0037] It should be understood that Figure 1 The number of terminals, networks, database servers and servers in the embodiment is only for illustration. Any number of terminals, networks, database servers and servers may be provided according to the implementation requirements.

[0038] In the field of e-commerce logistics and distribution, after the order packages are produced from the warehouse, they are first picked up to different storage areas at the sorting center according to the business department corresponding to the order receipt. When the scheduled departure time arrives, the transfer vehicle will transport the goods to the corresponding business department, and then the goods will be distributed to the delivery staff, who will deliver the goods to the customers.

[0039] When using traditional scheduling models and optimization strategies for delivery vehicle scheduling, it is usually expected to reduce the number of vehicles and reduce costs. On the other hand, the reduction in vehicles will inevitably lead to more tasks being delivered by the same vehicle. At this time, it becomes particularly important to consider the connection (empty driving) costs. From a system perspective, when the number of vehicles is reduced, there will inevitably be extra connection costs. Therefore, traditional technologies solve a single goal such as the minimum vehicle size or total mileage. Under such conditions, the solution with the minimum number of vehicles often does not have the lowest connection costs.

[0040] The vehicle scheduling problem in different tasks is a sequential decision problem, that is, it is necessary to decide the assigned vehicle for each task in turn. This is a Markov decision process (MDP) that can be solved using reinforcement learning technology. To this end, the present disclosure provides a multi-objective model training method based on reinforcement learning. In the multi-objective model training process based on reinforcement learning, a reward function is used to characterize the short-haul distance and the number of vehicles. The obtained multi-objective model can be better applied to downstream tasks. Figure 2 , shows a process 200 of an embodiment of a multi-objective model training method based on reinforcement learning according to the present disclosure, the multi-objective model training method based on reinforcement learning comprises the following steps:

[0041] Step 201, obtaining a preset sample set and a multi-objective network pre-constructed based on reinforcement learning.

[0042] In this embodiment, the sample set includes: task information of at least one sample task group and the remaining working time of the vehicle group corresponding to the sample task group, each sample task group corresponds to a vehicle group, the sample task group has at least one task, the vehicle group has at least one vehicle, the task group corresponds to a vehicle group, and the vehicles in the vehicle group are used to complete the tasks in the task group.

[0043] In this embodiment, the task can be an order transportation task. After the order package is generated in the warehouse, it is picked up by the sorting center according to the order and the corresponding business department to different storage areas. When the scheduled departure time arrives, the vehicle group's vehicle transports the goods to the corresponding business department, and then distributes the goods to the delivery staff, who deliver the goods to the customer. For the order transportation task, the task information includes: task type, task duration, task vehicle model and task mileage. The task type is related to the specific task. For example, in the logistics distribution task, the task type includes: forward type and reverse type, among which the forward type is the task type transmitted from the sorting center of the goods to the business department, and the reverse type is the task type transmitted from the business department to the sorting center. The task duration refers to the specific time spent to complete a task. For example, the task duration required to complete the transportation task transmitted from the sorting center to multiple business operations is 5h. The task vehicle model refers to the specific type of vehicle used to complete the task. For example, the task vehicle models include: trams and cars. The task mileage refers to the distance actually required to complete a task.

[0044] In this embodiment, the task information of the sample task group and the remaining working time of the vehicle group are samples obtained from the sample set. For different task information, the form of the sample task group can be different. For example, for tasks with a time sequence, the sample task group is a task sequence.

[0045] Aiming at the problem that the connection cost (the waiting time between any two vehicles when they are handed over during a task) is not fully considered in the current delivery vehicle scheduling, a multi-objective network is established based on the reinforcement learning architecture to simultaneously minimize the number of vehicles and the connection cost.

[0046] In this embodiment, the multi-objective network is used to characterize the correspondence between the task information of the task group, the remaining working time of the vehicle group and the task vehicle allocation strategy, wherein the task vehicle allocation strategy is a task and vehicle allocation rule. Specifically, the task and vehicle allocation rule may include: the order of different vehicles in the vehicle group corresponding to each task in the task group (for example, the vehicles in the vehicle group used in task 1 are vehicle 1 and vehicle 4 respectively, and the vehicles in the vehicle group used in task 2 are vehicle 2 and vehicle 3 respectively), or the order of the vehicle group in each task (for example, vehicle 1 and vehicle 4 in the vehicle group respectively perform the tasks of task 1, and vehicle 2 and vehicle 3 in the vehicle group perform the tasks of task 2 in turn). In this embodiment, the task vehicle allocation strategy is a rule obtained by the multi-objective network based on the input task information and the remaining time prediction of each vehicle in the vehicle group. The rule can effectively match the vehicles in the vehicle group and the tasks in the task group.

[0047] In this embodiment, reinforcement learning is an agent that obtains input from observations in the environment, then takes corresponding actions based on the input, and then determines rewards or penalties based on whether the actions are harmful or not. The reward signal defines the goal of the reinforcement learning problem. In each time step, the scalar value sent by the environment to the reinforcement learning is the reward, which can define whether the agent performs well or not.

[0048] Reinforcement learning is a learning mechanism that learns how to map from state to behavior in order to maximize the reward. Such an agent needs to continuously experiment in the environment and continuously optimize the state-behavior correspondence through the feedback (rewards) given by the environment. Therefore, repeated experiments (trial and error) and delayed rewards (delayed reward) are the two most important features of reinforcement learning.

[0049] For the delivery vehicle call task, consider the case of m (m>1) delivery tasks and n (n>1) vehicles. This scenario is modeled as an MDP, represented by a four-tuple {X t ,A,P,R}, where X t represents the state of the system at time t, Represents the characteristics of the input i state at time t (such as the task information of the i-th task in the task group and the remaining working time of the vehicle group). The task information in this scenario represents the characteristics of the i-th task at time t, including task type, task time period, task vehicle type, task mileage, remaining working time of the vehicle group, etc. A represents a set of actions, which can be expressed as a policy function P(y|x) in this scenario. The policy function represents the conditional probability of output y (task) based on the input state x. The complete expression is shown in formula (1):

[0050]

[0051] Among them, formula (1) means that at each moment, the input condition is xt, and the output is yt+1. The whole conditional probability is the multiplication of all probability values ​​at the current moment, which is the final probability value generated by the entire task group.

[0052] In the quadruple, R represents the reward function, which in this scenario can represent the reduction in the connection distance and the number of vehicles.

[0053] In this embodiment, the execution subject of the multi-objective model training method based on reinforcement learning (for example Figure 1 The server shown in the figure) can obtain the sample set and extract samples from the sample set in a variety of ways. For example, the execution subject can obtain the sample set from the database server (for example, Figure 1For example, a user may obtain an existing sample set stored in the database server 104 as shown in FIG. Figure 1 In this way, the execution subject can receive the samples collected by the terminal and store the samples locally, thereby generating a sample set.

[0054] In this embodiment, for the delivery vehicle calling task, the samples in the sample set are statistically analyzed by the type, mileage, and time period of the real task, and then randomly generated within a reasonable range. The vehicle group can set the vehicles and the remaining working time of the vehicles based on the actual vehicle scale.

[0055] Step 202 : Select a sample task group and the remaining working time of the vehicle group corresponding to the sample task group from the sample set.

[0056] In this embodiment, a sample may be randomly selected from the sample set, and the sample is a sample task group and the remaining working time of the vehicle group corresponding to the sample task group.

[0057] Optionally, the above step 202 includes: based on the current task demand, selecting from the sample set the remaining working time of the sample task group corresponding to the current task demand and the vehicle group corresponding to the sample task group. The sample set is provided with task labels for sample task groups of different tasks, such as cargo delivery tasks, vehicle sorting tasks, etc. Based on the current task demand, the task label corresponding to the sample task group can be determined, and the sample task group corresponding to the current task demand and the remaining working time of the vehicle group corresponding to the task group can be selected through the task label.

[0058] Step 203 , inputting the task information of the sample task group and the remaining working time of the vehicle group corresponding to the sample task group into the multi-objective network.

[0059] In this embodiment, the multi-objective network is a sequence-to-sequence learning network, which can use two recurrent neural networks as an encoder and a decoder respectively, and apply the encoder and the decoder to sequence-to-sequence learning tasks.

[0060] In this embodiment, the sample task group is a sequence of multiple tasks. The task information of each sample task in the sample task group and the remaining working time of the vehicle group corresponding to the sample task are input into the multi-objective network, and a task vehicle allocation strategy for vehicle allocation prediction for all tasks in the sample task group can be obtained.

[0061] Step 204 , based on the task vehicle allocation strategy output by the multi-objective network, calculate the expected value of the reward function of the multi-objective network.

[0062] In this embodiment, the reward function is used to characterize at least the reduction in the short-distance of vehicles and the number of vehicles. The short-distance refers to the connection distance between different vehicles in the same task (one vehicle completes part of the task and is then taken over by another vehicle, and the connection distance is the distance between one vehicle and another vehicle).

[0063] Optionally, the reward function can also be used to characterize the reduction in vehicle short-distance, vehicle number, and task cost.

[0064] In this embodiment, the system status of the sample task group (task type, time period, task vehicle model, task mileage, and remaining working time of the vehicle) is input into the multi-objective network, and the multi-objective network will output a sequence. The sequence is divided by identifiers, and the identifiers are used to identify different vehicles in the vehicle group or the identifiers are used to identify different tasks in the sample task group, thus forming the final task vehicle allocation strategy.

[0065] like Figure 3 As shown, the system state zz including the task information of the sample task group and the remaining working time of the vehicle group is input into the multi-objective network, the sequence feature is extracted in the encoder of the multi-objective network, and the extracted sequence feature is predicted in the decoder to obtain the task vehicle allocation strategy output by the multi-objective network. Figure 3 In the task vehicle allocation strategy, the vehicle identification of different vehicles (such as Figure 3 The mission information of different vehicles identified by v1, v2, v3…vn).

[0066] In this embodiment, in order to effectively adjust the parameters of the multi-objective network, the reward function is used as the objective function (action value function) of the multi-objective network, the reward function is minimized when training the multi-objective network, and the expected value of the multi-objective network optimization reward function is calculated, ultimately achieving the purpose of optimizing the parameters of the multi-objective network.

[0067] In this embodiment, the reward function is set as the weighted sum of the number of vehicles and the short-haul distance of the vehicles, as shown in formula (2):

[0068] W1*numbers+W2*distance(2)

[0069] In actual scenarios, the determination of weights W1 and W2 depends on experience. In the actual operation of logistics distribution, sometimes managers need to change the goals according to the situation. For example, if the fulfillment rate needs to be guaranteed within a certain period of time, the number of vehicles may need to be adjusted. In normal operation, the number of vehicles is a more important goal, and the weights need to be adjusted accordingly based on demand.

[0070] In this embodiment, a variety of different weight pairs can be pre-set for weights W1 and W2 in the multi-objective network, such as (0.1, 0.9), (0.2, 0.8), (0.3, 0.7), (0.4, 0.6), (0.5, 0.5), (0.6, 0.4), (0.7, 0.3), (0.8, 0.2), (0.9, 0.1), and unified training data is generated according to the distribution of real tasks, and the results that meet the actual needs are finally selected from a large number of results. Finally, the user can determine the weights according to the needs, and the desired results can be quickly selected from the results output by the model.

[0071] In this embodiment, the expected value of the reward function may be an actual value calculated in each iterative training (episodic).

[0072] Optionally, when the multi-objective function has a value function value, the expected value of the reward function may also be the actual value of the value function. It should be noted that the value of the value function may be a value obtained after accumulating the values ​​of multiple reward functions.

[0073] Step 205, in response to the expected value reaching the training completion condition, a multi-objective model corresponding to the multi-objective network is obtained.

[0074] In this embodiment, the multi-objective model based on reinforcement learning is a model corresponding to the multi-objective network. When the multi-objective network meets the training completion condition in the iterative training process, it is determined that the current multi-objective network is the multi-objective model based on reinforcement learning.

[0075] In this embodiment, a multi-objective network and a multi-objective model based on reinforcement learning are used to characterize the correspondence between the task information of the task group and the remaining working time of the vehicle group and the task vehicle allocation strategy, wherein the task can be: a logistics distribution task. For example, the multi-objective model based on reinforcement learning is used to characterize the correspondence between the logistics distribution task information, the distribution vehicle and the task vehicle allocation strategy, wherein the task vehicle allocation strategy includes: the working time of each vehicle under different tasks and the vehicle connection order in each task (such as vehicle 1 connecting with vehicle 3 in task 1, etc.).

[0076] In this embodiment, the training completion condition may include: the number of parameter adjustments of the multi-objective network reaches a predetermined number of times and the expected value of the reward function reaches a predetermined value. For example, the number of parameter adjustments reaches 50,000 times and the predetermined value is less than 0.05.

[0077] In this embodiment, the above-mentioned response to the expected value reaching the training completion condition, obtaining the multi-objective model corresponding to the multi-objective network includes: in an iterative training process, using policy gradient to perform policy evaluation on the task vehicle allocation strategy output by the multi-objective network, obtaining the policy optimization result of the current iterative training, and calculating the expected value of the reward function based on the policy optimization result (the policy optimization result is a task vehicle allocation strategy, and the expected value corresponding to the reward function can be obtained by statistically analyzing the allocation of tasks and vehicles in the policy optimization result); when the number of iterative training reaches a preset number (for example, 5 times), if the expected value of the reward function does not change, the multi-objective network is determined to be a multi-objective model. Alternatively, after using policy gradient to perform policy evaluation on the task vehicle allocation strategy output by the multi-objective network, the expected value of the reward function is the minimum value, and the multi-objective network is determined to be a multi-objective model.

[0078] Optionally, in response to detecting that the expected value of the reward function of the multi-objective network cannot meet the training completion condition, steps 202 to 205 may continue to be performed until the expected value of the reward function of the multi-objective network meets the training completion condition.

[0079] The multi-objective model training method provided in this embodiment considers multiple objectives in the dispatching task of the distribution vehicle, thus overcoming the shortcoming that the traditional technology only considers a single objective; the multi-objective model based on reinforcement learning solves the problem that the weight in the reward function is difficult to determine with a unified model; the transportation capacity of the distribution task can be planned in advance to realize the distribution of goods from the warehouse to each distribution point, and various matches (combinations) between tasks are fully considered, which can reduce the use of vehicles and reduce the cost increase caused by unreasonable connection, thereby reducing the total transportation cost. At the same time, it can also meet the needs of the business party under various target weights.

[0080] The multi-objective model training method based on reinforcement learning provided by the embodiment of the present disclosure first obtains a preset sample set and a multi-objective network pre-constructed based on reinforcement learning; secondly, selects the remaining working time of the sample task group and the vehicle group corresponding to the sample task group from the sample set; thirdly, inputs the task information of the sample task group and the remaining working time of the vehicle group corresponding to the sample task group into the multi-objective network; thirdly, calculates the expected value of the reward function of the multi-objective network based on the task vehicle allocation strategy output by the multi-objective network; finally, in response to the expected value reaching the training completion condition, obtains the multi-objective model corresponding to the multi-objective network. Thus, the multi-objective network generated by reinforcement learning can realize the delivery of goods from the warehouse to each distribution point, fully considers the various matches (combinations) between each task in the task group, can reduce the use of vehicles, and at the same time reduces the cost increase caused by unreasonable vehicle connection by optimizing the reward function, thereby reducing the total transportation cost.

[0081] In some optional implementations of this embodiment, the strategy function corresponding to the multi-objective network is used to characterize the conditional probability of input states and output tasks; the multi-objective network includes: an encoder and a decoder, the encoder is used to convert the task information of the selected sample task group and the remaining working time of the vehicle group into a feature sequence, and the decoder is used to convert the feature sequence into a task vehicle allocation strategy.

[0082] In this optional implementation, the policy function is used to characterize the conditional probability of the input state and the output task, where the input state refers to: the task information of the task group and the remaining working time of the vehicle group corresponding to the task group. The policy function is used to characterize the input state sequence x t , the output is y t+1 , the entire solution is to multiply the probability values ​​at all times, which is the probability value generated by the final state sequence of the entire input.

[0083] In this embodiment, the encoder may use a neural network, such as DNN (Deep Neural Networks) and CNN (Convolutional Neural Network); the decoder uses a pointer network.

[0084] In the multi-objective network, the encoder converts the input sequence into an encoding, which is then fed into the decoder. In the training step of each task in each task group, the decoder produces a vector based on the encoding, modulating a content-based attention mechanism on the input. The output of the attention mechanism is a softmax distribution with a dictionary size equal to the length of the input, resulting in a task-vehicle assignment strategy for the task group and vehicle group.

[0085] The multi-objective network provided by this optional implementation adopts an encoder and decoder structure. The encoder converts the task information of the selected sample task group and the remaining working time of the vehicle group into a feature sequence, and the decoder converts the feature sequence into a task vehicle allocation strategy, thereby improving the reliability of multi-objective model training.

[0086] In some optional implementations of this embodiment, the multi-objective network is also used to characterize the correspondence between the vehicle type and the task vehicle allocation strategy; the sample set includes: the vehicle type of the vehicle group, and the method further includes:

[0087] It is detected whether the vehicle type of the vehicle group matches the task information of the task group, and in response to the vehicle type matching the task information, the vehicle type of the vehicle group is input into the multi-objective network when performing the training step.

[0088] In this optional implementation, vehicle types may include: trams, cars, and oil and gas hybrids. A vehicle group can be a group of cars or a group of trams. For different types of vehicle groups, the situations for completing and allocating tasks are different. Therefore, the vehicle type is matched with the task information. When the vehicle type of the vehicle group matches the task group, the vehicle type is input into the multi-target network, which can enable the multi-target network to more effectively understand the information of the current vehicle group.

[0089] In this embodiment, the task information may include a task vehicle model. The above-mentioned detection of whether the vehicle type of the vehicle group matches the task information of the task group includes: comparing the similarity between the vehicle type of the vehicle group and the task vehicle model in the task information, and in response to the similarity between the vehicle type and the task vehicle model being greater than a similarity threshold, determining that the vehicle type of the vehicle group matches the task information of the task group.

[0090] In the multi-objective model training method provided in this embodiment, the multi-objective network is also used to characterize the correspondence between vehicle types and vehicle allocation strategies. Before training the multi-objective network, different types of vehicle groups are detected and the vehicle groups of this type are allocated to tasks corresponding to the vehicle types of the vehicle groups, thereby providing a reliable basis for the effective implementation of the task vehicle allocation strategy derived by the multi-objective network.

[0091] Optionally, constraints on the vehicle type and the task vehicle model in the task information may be set, and the constraints may be sent to the multi-objective network so that the multi-objective network excludes the task vehicle allocation strategy that satisfies the vehicle type and the task vehicle model.

[0092] In this optional implementation, the constraints on the vehicle type and the mission vehicle type include: the vehicle type and the mission vehicle type are different; the vehicle type does not have the type required in the mission vehicle type.

[0093] In some optional implementations of this embodiment, the method also includes: setting constraints for task groups and vehicle groups; inputting the constraints, task information of the sample task group, and the remaining working time of the vehicle group corresponding to the sample task group into the multi-objective network, so that the multi-objective network excludes the task vehicle allocation strategy under the constraints.

[0094] In this optional implementation, setting constraints can mark the multi-objective network with tasks whose duration conflicts with the remaining working duration in the vehicle group, so that the multi-objective network can remove the task vehicle allocation strategy that meets the constraints under the guidance of the current constraints.

[0095] In this embodiment, the constraints include: vehicles with time conflicts in the vehicle group are not used in any two tasks; and the latter task of the adjacent tasks does not use vehicles with remaining working time conflicts with the vehicle of the previous task.

[0096] The multi-objective model training method provided by this optional implementation sets constraints on the task duration and the remaining working time of the vehicle group, and inputs the constraints into the multi-objective network, so that the multi-objective network can effectively exclude the task vehicle allocation strategy that meets the constraints, providing a reliable basis for the implementation of the task vehicle allocation strategy.

[0097] See also Figure 4 , which shows a process 400 of an embodiment of a multi-objective allocation method provided by the present disclosure, and the multi-objective allocation method may include the following steps:

[0098] Step 401, obtaining the task data of the task group to be assigned and the remaining working time of the vehicle group to be assigned.

[0099] In this embodiment, the execution subject on which the multi-target allocation method is running can communicate with the terminal (such as Figure 1 The terminals 101 and 102 communicate with each other to obtain the task data of the task group to be assigned and the remaining working time of the vehicles to be assigned sent by the terminals. The task data includes: the task type, task time, task vehicle type, and task mileage of each task in the task group; in this embodiment, the vehicle group to be assigned includes at least one vehicle, and the remaining working time of the vehicles to be assigned is the time that each vehicle to be assigned in the vehicle group to be assigned can participate in the task.

[0100] Step 402 , inputting the task data of the task group to be assigned and the remaining working time of the vehicle group to be assigned into the multi-objective model, and obtaining the vehicle allocation strategy corresponding to each task in the task group to be assigned output by the multi-objective model.

[0101] In this embodiment, the number of tasks in the task group to be assigned and the number of vehicles in the vehicle group to be assigned can be set based on demand. When the task data of the task group to be assigned is input into the multi-objective model, the task data of each task in the task group to be assigned is input into the multi-objective model in sequence. After the task data of all tasks in the task group are input, the multi-objective model assigns vehicles to each task in sequence, and after the vehicle is assigned to the previous task, the remaining working time of the assigned vehicle is correspondingly removed.

[0102] In this embodiment, the vehicle allocation strategy corresponding to each task may be the vehicles in the to-be-allocated vehicle group corresponding to each task in the to-be-allocated task group; the vehicle allocation strategy may also be expressed as: tasks in the to-be-allocated task group corresponding to different vehicles.

[0103] In this embodiment, the multi-objective model is a model generated by the multi-objective model training method based on reinforcement learning in the above embodiment. The specific generation process of the multi-objective model based on reinforcement learning can be found in Figure 2 The relevant description of the embodiments will not be repeated here.

[0104] It should be noted that the multi-objective allocation method of this embodiment can be used to test the multi-objective model based on reinforcement learning generated by the above embodiments. Then, the multi-objective model based on reinforcement learning can be continuously optimized according to the test results. This method can also be a practical application method of the multi-objective model based on reinforcement learning generated by the above embodiments. Using the multi-objective model based on reinforcement learning generated by the above embodiments to allocate multiple objectives helps to improve the allocation efficiency of multiple objectives.

[0105] The multi-objective allocation method provided by the embodiment of the present disclosure first obtains the task data of the task group to be allocated and the remaining working time of the vehicle group to be allocated; secondly, the task data of the task group to be allocated and the remaining working time of the vehicle group to be allocated are input into the multi-objective model to obtain the vehicle allocation strategy corresponding to each task in the task group to be allocated output by the multi-objective model. Thus, the multi-objective model based on reinforcement learning obtained by pre-training reasonably allocates the tasks of the task group to be allocated and the vehicles of the vehicle group to be allocated, while allocating tasks with the least number of vehicles, reducing the connection costs of vehicles and improving the efficiency of multi-objective allocation.

[0106] Further references Figure 5 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a multi-objective model training device based on reinforcement learning. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0107] like Figure 5As shown, an embodiment of the present disclosure provides a multi-objective model training device 500 based on reinforcement learning, and the device 500 includes: a sample acquisition unit 501, a selection unit 502, an input unit 503, a calculation unit 504, and a training unit 505. The sample acquisition unit 501 can be configured to acquire a preset sample set and a multi-objective network pre-constructed based on reinforcement learning, wherein the sample set includes: task information of at least one sample task group and the remaining working time of the vehicle group corresponding to the sample task group, and the multi-objective network is used to characterize the correspondence between the task information, the remaining working time and the task vehicle allocation strategy. The selection unit 502 can be configured to select the sample task group and the remaining working time of the vehicle group corresponding to the sample task group from the sample set. The input unit 503 can be configured to input the task information of the sample task group and the remaining working time of the vehicle group corresponding to the sample task group into the multi-objective network. The calculation unit 504 can be configured to calculate the expected value of the reward function of the multi-objective network based on the task vehicle allocation strategy output by the multi-objective network, and the reward function is used to characterize at least the reduction in the short-distance of the vehicle and the number of vehicles. The training unit 505 can be configured to obtain a multi-objective model corresponding to the multi-objective network in response to the expected value reaching the training completion condition.

[0108] In this embodiment, in the multi-objective model training device 500 based on reinforcement learning, the specific processing of the sample acquisition unit 501, the selection unit 502, the input unit 503, the calculation unit 504, and the training unit 505 and the technical effects thereof can be referred to respectively. Figure 2 Corresponding to step 201, step 202, step 203, step 204, and step 205 in the embodiment.

[0109] In some embodiments, the strategy function corresponding to the above-mentioned multi-objective network is used to characterize the conditional probability of input states and output tasks; the multi-objective network includes: an encoder and a decoder, the encoder is used to convert the task information of the selected sample task group and the remaining working time of the vehicle group into a feature sequence, and the decoder is used to convert the feature sequence into a task vehicle allocation strategy.

[0110] In some embodiments, the multi-objective network is also used to characterize the correspondence between vehicle types and task vehicle allocation strategies; the sample set includes: the vehicle type of the vehicle group, and the device also includes: a detection unit (not shown in the figure), and the detection unit can be configured to detect whether the vehicle type of the vehicle group matches the task information of the task group. In response to the vehicle type matching the task information, the control input unit inputs the vehicle type of the vehicle group into the multi-objective network.

[0111] In some embodiments, the device 500 also includes: a setting unit (not shown in the figure), which can be configured to set constraints for the task group and the vehicle group, and control the input unit to input the constraints, the task information of the sample task group and the remaining working time of the vehicle group corresponding to the sample task group into the multi-objective network, so that the multi-objective network excludes the task vehicle allocation strategy under the constraints.

[0112] The embodiment of the present disclosure provides a multi-objective model training device based on reinforcement learning. First, the sample acquisition unit 501 acquires a preset sample set and a multi-objective network pre-constructed based on reinforcement learning; secondly, the selection unit 502 selects the remaining working time of the sample task group and the vehicle group corresponding to the sample task group from the sample set; thirdly, the input unit 503 inputs the task information of the sample task group and the remaining working time of the vehicle group corresponding to the sample task group into the multi-objective network; then, the calculation unit 504 calculates the expected value of the reward function of the multi-objective network based on the task vehicle allocation strategy output by the multi-objective network; finally, the training unit 505 obtains the multi-objective model corresponding to the multi-objective network in response to the expected value reaching the training completion condition. Thus, the multi-objective network generated by reinforcement learning can realize the delivery of goods from the warehouse to each distribution point, fully considers the various matches (combinations) between each task in the task group, can reduce the use of vehicles, and at the same time reduces the cost increase caused by unreasonable vehicle connection by optimizing the reward function, thereby reducing the total transportation cost.

[0113] Further references Figure 6 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a multi-objective allocation device. Figure 4 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0114] like Figure 6 As shown, an embodiment of the present disclosure provides a multi-objective allocation device 600, and the device 600 includes: a data acquisition unit 601 and an obtaining unit 602. The data acquisition unit 601 can be configured to obtain the task data of the task group to be assigned and the remaining working time of the vehicle group to be assigned. The obtaining unit 602 can be configured to input the task data of the task group to be assigned and the remaining working time of the vehicle group to be assigned into a multi-objective model, and obtain the vehicle allocation strategy corresponding to each task in the task group to be assigned output by the multi-objective model, and the multi-objective model is trained based on the multi-objective model training device of the above embodiment.

[0115] In this embodiment, the multi-objective model based on reinforcement learning is trained using a multi-objective model training device based on reinforcement learning.

[0116] In this embodiment, the specific processing of the data acquisition unit 601 and the obtaining unit 602 and the technical effects thereof can be referred to in Figure 4 Corresponding to step 401 and step 402 in the embodiment.

[0117] The multi-objective allocation device provided by the embodiment of the present disclosure firstly obtains the task data of the task group to be allocated and the remaining working time of the vehicle group to be allocated by the data acquisition unit 601; secondly, the obtaining unit 602 inputs the task data of the task group to be allocated and the remaining working time of the vehicle group to be allocated into the multi-objective model, and obtains the vehicle allocation strategy corresponding to each task in the task group to be allocated output by the multi-objective model. Thus, the multi-objective model based on reinforcement learning obtained by pre-training reasonably allocates the tasks of the task group to be allocated and the vehicles of the vehicle group to be allocated, while allocating tasks with the least number of vehicles, reducing the connection costs of vehicles and improving the efficiency of multi-objective allocation.

[0118] Reference below Figure 7 , which shows a schematic structural diagram of an electronic device 700 suitable for implementing an embodiment of the present disclosure.

[0119] like Figure 7 As shown, the electronic device 700 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 are also stored. The processing device 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0120] Typically, the following devices may be connected to the I / O interface 705: an input device 706 including, for example, a touch screen, a touch pad, a keyboard, a mouse, etc.; an output device 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 708 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 709. The communication device 709 may allow the electronic device 700 to communicate with other devices wirelessly or by wire to exchange data. Figure 7 The electronic device 700 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 7 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0121] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 709, or installed from a storage device 708, or installed from a ROM 702. When the computer program is executed by the processing device 701, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0122] It should be noted that the computer-readable medium of the embodiment of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the embodiment of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the embodiment of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0123] The computer-readable medium may be included in the server; or it may exist independently without being assembled into the server. The computer-readable medium carries one or more programs. When the one or more programs are executed by the server, the server: obtains a preset sample set and a multi-objective network pre-constructed based on reinforcement learning, wherein the sample set includes: task information of at least one sample task group and the remaining working time of the vehicle group corresponding to the sample task group, and the multi-objective network is used to characterize the correspondence between the task information, the remaining working time and the task vehicle allocation strategy; performs the following training steps: selects the sample task group and the remaining working time of the vehicle group corresponding to the sample task group from the sample set; inputs the task information of the sample task group and the remaining working time of the vehicle group corresponding to the sample task group into the multi-objective network; calculates the expected value of the reward function of the multi-objective network based on the task vehicle allocation strategy output by the multi-objective network, and the reward function is at least used to characterize the reduction in the short-distance distance of the vehicle and the number of vehicles; in response to the expected value reaching the training completion condition, obtains the multi-objective model corresponding to the multi-objective network.

[0124] Computer program code for performing the operations of embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on a user's computer, partially on a user's computer, as a separate software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0125] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0126] The units involved in the embodiments described in the present disclosure may be implemented by software or by hardware. The described units may also be arranged in a processor, for example, may be described as: a processor comprising a sample acquisition unit, a selection unit, an input unit, a calculation unit, and a training unit. Among them, the names of these units do not constitute a limitation on the unit itself under certain circumstances. For example, the acquisition unit may also be described as a "sample acquisition unit, configured to obtain a preset sample set and a multi-objective network pre-built based on reinforcement learning;"

[0127] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) to form a technical solution.

Claims

1. A multi-objective model training method based on reinforcement learning, the method comprising: Acquire a preset sample set and a multi-objective network pre-constructed based on reinforcement learning, wherein the sample set includes: task information of at least one sample task group and the remaining working time of a vehicle group corresponding to the sample task group, and the multi-objective network is used to characterize the corresponding relationship between the task information, the remaining working time and the task vehicle allocation strategy; Perform the following training steps: Selecting a sample task group and the remaining working time of the vehicle group corresponding to the sample task group from the sample set; Inputting the task information of the sample task group and the remaining working time of the vehicle group corresponding to the sample task group into the multi-objective network; Based on the task vehicle allocation strategy output by the multi-objective network, calculating the expected value of the reward function of the multi-objective network, wherein the reward function is used to characterize at least the reduction in the short-distance vehicle transfer and the number of vehicles; In response to the expected value reaching a training completion condition, a multi-objective model corresponding to the multi-objective network is obtained.

2. The method according to claim 1, wherein: The policy function corresponding to the multi-objective network is used to characterize the conditional probability of the input state and the output task; The multi-objective network includes: an encoder and a decoder, wherein the encoder is used to convert the task information of the selected sample task group and the remaining working time of the vehicle group into a feature sequence, and the decoder is used to convert the feature sequence into a task vehicle allocation strategy.

3. According to the method of claim 1, the multi-objective network is also used to characterize the correspondence between vehicle types and task vehicle allocation strategies; the sample set includes: The vehicle type of the vehicle group, the method further comprising: Detecting whether a vehicle type of the vehicle group matches the task information of the task group, and in response to the vehicle type matching the task information, inputting the vehicle type of the vehicle group into the multi-objective network when executing the training step.

4. The method according to any one of claims 1 to 3, wherein: The method further comprises: Setting constraints for the task group and the vehicle group; The constraint conditions, the task information of the sample task group and the remaining working time of the vehicle group corresponding to the sample task group are input into the multi-objective network so that the multi-objective network excludes the task vehicle allocation strategy under the constraint conditions.

5. A multi-objective allocation method, the method comprising: Obtain the task data of the task group to be assigned and the remaining working time of the vehicle group to be assigned; The task data of the task group to be assigned and the remaining working time of the vehicle group to be assigned are input into a multi-objective model to obtain a vehicle allocation strategy corresponding to each task in the task group to be assigned output by the multi-objective model, and the multi-objective model is trained based on the multi-objective model training method described in any one of claims 1-4.

6. A multi-objective model training device based on reinforcement learning, the device comprising: A sample acquisition unit is configured to acquire a preset sample set and a multi-objective network pre-constructed based on reinforcement learning, wherein the sample set includes: task information of at least one sample task group and the remaining working time of a vehicle group corresponding to the sample task group, and the multi-objective network is used to characterize the corresponding relationship between the task information, the remaining working time and the task vehicle allocation strategy; A selection unit is configured to select a sample task group and a remaining working time of a vehicle group corresponding to the sample task group from the sample set; An input unit is configured to input the task information of the sample task group and the remaining working time of the vehicle group corresponding to the sample task group into the multi-objective network; A computing unit configured to calculate an expected value of a reward function of the multi-objective network based on a task vehicle allocation strategy output by the multi-objective network, wherein the reward function is used to characterize at least a reduction in a short-distance vehicle transfer and a reduction in the number of vehicles; The training unit is configured to obtain a multi-objective model corresponding to the multi-objective network in response to the expected value reaching a training completion condition.

7. The device according to claim 6, wherein: The policy function corresponding to the multi-objective network is used to characterize the conditional probability of the input state and the output task; The multi-objective network includes: an encoder and a decoder, wherein the encoder is used to convert the task information of the selected sample task group and the remaining working time of the vehicle group into a feature sequence, and the decoder is used to convert the feature sequence into a task vehicle allocation strategy.

8. The device according to claim 6, wherein the multi-objective network is further used to characterize the correspondence between vehicle types and task vehicle allocation strategies; and the sample set includes: The vehicle type of the vehicle group, the device also includes: The detection unit is configured to detect whether the vehicle type of the vehicle group matches the task information of the task group. In response to the vehicle type matching the task information, the control input unit inputs the vehicle type of the vehicle group into the multi-objective network.

9. The device according to any one of claims 6 to 8, wherein: The device also includes: a setting unit, configured to set constraints for the task group and the vehicle group, and control an input unit to input the constraints, task information of the sample task group, and the remaining working time of the vehicle group corresponding to the sample task group into the multi-objective network, so that the multi-objective network excludes the task vehicle allocation strategy under the constraints.

10. A multi-target allocation device, comprising: A data acquisition unit is configured to acquire task data of the task group to be assigned and the remaining working time of the vehicle group to be assigned; The obtaining unit is configured to input the task data of the task group to be assigned and the remaining working time of the vehicle group to be assigned into a multi-objective model, and obtain the vehicle allocation strategy corresponding to each task in the task group to be assigned output by the multi-objective model. The multi-objective model is trained based on the multi-objective model training device described in any one of claims 6-9.

11. An electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

12. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images