A service intelligent matching and pushing method for multi-type downstream users

Abstract

The application relates to the technical field of instant logistics, and discloses a service intelligent matching and pushing method for multiple types of downstream users, which comprises the following steps: collecting historical order data and driver data, the order data comprising order time, pickup location, delivery location and order completion time, and the driver data comprising driver position, driver familiar area and current driver state; generating order features from the historical order data, generating driver features from the driver data, combining the order features and the driver features to generate comprehensive state features, inputting the comprehensive state features into an order distribution model, and outputting an order distribution strategy from the order distribution model, wherein the order distribution strategy comprises a next order received by each driver; and pushing the next order to each driver according to the order distribution strategy output by the order distribution model; the pushing method improves the overall efficiency of goods distribution, improves the satisfaction of all participants, and reduces the operation cost.

Description

Technical Field

[0001] This invention relates to the field of instant logistics, and more specifically, to a business intelligent matching and push method for multiple types of downstream users. Background Technology

[0002] The delivery order from the freight company needs to be pushed to the truck driver through the freight platform. The driver who accepts the order will deliver the goods to the designated receiving point. The order allocation method is mainly divided into two categories: order grabbing and order dispatching. The order grabbing mode relies on the truck driver's personal experience and is prone to picking and choosing orders blindly, resulting in high randomness in delivery efficiency. Therefore, most freight platforms have begun to choose the order dispatching method.

[0003] The order dispatching process of a freight platform mainly considers the following four factors: the distance between the driver and the pickup point, the distance between the delivery address and the delivery route, the time when the freight company places the order and the estimated delivery time. By calculating these four factors, the platform selects the optimal delivery efficiency and dispatches orders to the nearest freight driver.

[0004] For a driver, delivery efficiency is highest when working in a familiar work area. The order dispatch process described above can easily cause drivers to leave their familiar work areas, which will undoubtedly reduce overall delivery efficiency. Currently, return trip orders are used to allow truck drivers to quickly return to their familiar areas, enabling them to return to their familiar areas as soon as possible after receiving the return trip order. However, due to the uncertainty of return trip order distribution and the difficulty in predicting regional order volume, the allocation of return trip orders has become complicated and difficult to be accurate. Summary of the Invention

[0005] This invention provides a business intelligent matching and push method for multiple types of downstream users, solving the technical problems mentioned in the background.

[0006] This invention provides a business intelligent matching and push method for multiple types of downstream users, including the following steps:

[0007] Step 101: Collect historical order data and driver data. Order data includes order time, pickup location, delivery location, and order completion time. Driver data includes driver location, driver's familiar area, and driver's current status. Driver's current status includes delivery status and non-delivery status.

[0008] Step 102: Generate order features through historical order data, generate driver features through driver data, combine order features and driver features to generate comprehensive status features, input them into the order allocation model, and the order allocation model outputs an order allocation strategy, which includes the next order received by each driver.

[0009] Step 103: Push the next order to each driver according to the order allocation strategy output by the order allocation model.

[0010] In a preferred embodiment, the formula for calculating the combination of order features and driver features is as follows:

[0011] X = CONCAT(D d Q s )

[0012] Where X represents the comprehensive state characteristic, D d Indicates order characteristics, Q s This represents driver characteristics, and CONCAT indicates vector concatenation.

[0013] In a preferred embodiment, the order allocation model includes an input layer, a policy network, and a value network;

[0014] The policy network includes a first hidden layer, a second hidden layer, and an output layer. The input layer inputs the comprehensive state features into the first hidden layer. The first hidden layer generates a first hidden feature, which is input into the second hidden layer. The second hidden layer generates a second hidden feature, which is input into the output layer. The output layer outputs the probability value of each possible order allocation strategy and selects the order from the order allocation strategy with the highest probability as the next order received by the driver.

[0015] In a preferred embodiment, the formula for calculating the first hidden layer is:

[0016] h1 = σ(W1·X + b1)

[0017] Where h1 represents the first hidden feature, σ represents the ReLU activation function, W1 represents the first hidden weight matrix, X represents the comprehensive state feature, and b1 represents the bias vector of the first hidden layer.

[0018] The formula for calculating the second hidden layer is:

[0019] h2=σ(W2·h1+b2)

[0020] Where h2 represents the second hidden feature, σ represents the ReLU activation function, W2 represents the second hidden weight matrix, h1 represents the first hidden feature, and b2 represents the bias vector of the second hidden layer.

[0021] The formula for calculating the output layer is:

[0022] p = softmax(W p h2+b p )

[0023] Where p represents the probability of an order in the order allocation strategy, softmax represents the softmax function, and W pLet h1 represent the policy network weight matrix of the output layer, h2 represent the second hidden feature, and b represent the weight matrix of the output layer. p This represents the bias vector of the output layer.

[0024] In a preferred embodiment, the value network includes a value processing layer and a value generation layer. The input layer inputs the comprehensive state features into the value processing layer, the value processing layer generates a value vector, the value vector is input into the value generation layer, and the value generation layer outputs the estimated value of the current state.

[0025] In a preferred embodiment, the calculation formula for the value processing layer is:

[0026] v=σ(W v1 ·X+b v1 )

[0027] Where v represents the value vector, σ represents the ReLU activation function, and W v1 This represents the weight matrix of the value processing layer, where X represents the comprehensive state features, and b v1 Indicates the bias parameters of the value processing layer;

[0028] The formula for calculating the value generation layer is:

[0029] V(s) = W v2 ·v+b v2

[0030] Where V(s) represents the estimated value in state s, and W v2 Let v represent the weight matrix of the value generation layer, and b represent the value vector. v2 This represents the bias parameters of the value generation layer.

[0031] In a preferred embodiment, the state space includes order data and driver data;

[0032] Define the action space: Combine the next orders that each driver can accept to generate the action space;

[0033] Orders include return tickets and non-return tickets;

[0034] It should be noted that each order is viewed as a vector, pointing from the merchant point to the user point, and this vector represents the order vector;

[0035] Definition of return trip: Define a return vector, pointing from the driver's current position to the center of the driver's familiar area, and use the upper and lower tangent vectors of the driver's current position and the boundary of the driver's familiar area as the upper and lower threshold ranges of the return vector. If the angle between the vector of any order and the return vector is within the upper and lower threshold ranges of the return vector, then the order is a return trip; otherwise, it is a non-return trip.

[0036] An action includes each driver receiving the next order.

[0037] In a preferred embodiment, the reward is calculated after the driver completes the daily delivery tasks pushed by the platform, and the reward calculation formula is as follows:

[0038] r=ω1R eff +ω2R sat +ω3R cust +ω4R back

[0039] Where r represents the total feedback reward value obtained, R eff R represents the delivery efficiency bonus value. sat R represents the driver satisfaction reward value. cust R represents the customer satisfaction reward value upon receiving the goods. back This represents the return time bonus value, where ω1, ω2, ω3, and ω4 represent the first, second, third, and fourth bonus parameters, respectively.

[0040] The formula for calculating delivery efficiency bonus value is:

[0041]

[0042] Among them, R eff T represents the delivery efficiency bonus value. delivery This indicates the total time from order acceptance to delivery completion;

[0043] The formula for calculating driver satisfaction bonus is:

[0044]

[0045] Among them, R sat F represents the driver satisfaction reward value. order T represents the frequency at which drivers accept orders. work Indicates the driver's working hours, I in This represents the driver's income level, and α represents a moderating factor.

[0046] The formula for calculating the customer satisfaction reward value is as follows:

[0047]

[0048] Among them, R cust This represents the reward value for customer satisfaction upon receiving the goods. This represents the feedback rating from the receiving user;

[0049] The formula for calculating the return trip time bonus is:

[0050]

[0051] Among them, R back Let ρ and τ represent the return time bonus value, where ρ and τ are both positive numbers and ρ > τ.

[0052] A business intelligent matching system for multiple types of downstream users includes the following modules:

[0053] The data acquisition module is used to collect historical order data and driver data;

[0054] The data preprocessing module is used to preprocess the collected data;

[0055] The feature generation module is used to generate order features and driver features, and combine them into comprehensive status features;

[0056] The order allocation model module is used to take comprehensive status features as input and output order allocation strategies.

[0057] The action and feedback module pushes the next order to each driver based on the output order allocation strategy.

[0058] A storage medium storing non-transitory computer-readable instructions, which, when executed by a computer, can perform the steps in a business intelligent matching and push method for multiple types of downstream users as described above.

[0059] The beneficial effects of this invention are as follows:

[0060] 1. The push method of the present invention not only improves the overall efficiency of goods delivery, but also enhances the satisfaction of all participants, while reducing operating costs, making it a comprehensively optimized solution;

[0061] 2. This invention uses an intelligent matching and push method to allocate orders to suitable drivers more efficiently, thereby reducing delivery time and improving delivery efficiency. By using the return trip priority principle, drivers are encouraged to return to their familiar areas, which can reduce the average delivery time of future orders. Attached Figure Description

[0062] Figure 1 This is a flowchart of the intelligent matching and push method for services targeting multiple types of downstream users according to the present invention.

[0063] Figure 2 This is a block diagram of the intelligent business matching system for multiple types of downstream users according to the present invention.

[0064] In the diagram: 100, Data Acquisition Module; 200, Data Preprocessing Module; 300, Feature Generation Module; 400, Order Allocation Model Module; 500, Action and Feedback Module. Detailed Implementation

[0065] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, features described in some examples may be combined in other examples.

[0066] like Figure 1 As shown, a business intelligent matching and push method for multiple types of downstream users includes the following steps:

[0067] Step 101: Collect historical order data and driver data. Order data includes order time, pickup location, delivery location, and order completion time. Driver data includes driver location, driver's familiar area, and driver's current status. Driver's current status includes delivery status and non-delivery status.

[0068] Historical order data can be obtained from the backend storage data of the goods platform;

[0069] Driver data can also be obtained from the backend storage data of the cargo platform;

[0070] The driver's location can be represented using real-time GPS coordinates. The dynamically updated order list in the cargo platform includes the location and time information of orders to be delivered in the order data. The driver's familiar area can be determined based on the driver's historical delivery records using clustering algorithms (such as K-means) to generate the driver's main activity area. The driver's current status includes whether there are any orders in progress and whether the driver is in the familiar area. The time information in the order data and driver data includes the current time, the predicted pickup and delivery time, and any time window restrictions.

[0071] It should be noted that historical order data and driver data are preprocessed, and the preprocessing methods include:

[0072] Outlier detection and handling: Remove or correct obviously erroneous data points, such as unreasonable coordinates or negative delivery times.

[0073] Data standardization involves normalizing or standardizing numerical features to reduce the impact of scale differences.

[0074] It should be further explained that new features can also be constructed, such as distance calculation, speed estimation, and the definition of familiar areas (based on the areas where the driver has made the most deliveries in the past).

[0075] Step 102: Generate order features through historical order data, generate driver features through driver data, combine order features and driver features to generate comprehensive status features, input them into the order allocation model, and the order allocation model outputs an order allocation strategy, which includes the next order received by each driver.

[0076] The formula for calculating the combination of order characteristics and driver characteristics is as follows:

[0077] X = CONCAT(D d Q s )

[0078] Where X represents the comprehensive state characteristic, D d Indicates order characteristics, Q s Represents driver characteristics; CONCAT indicates vector concatenation.

[0079] The order allocation model includes an input layer, a policy network, and a value network;

[0080] The policy network includes a first hidden layer, a second hidden layer, and an output layer. The input layer inputs the comprehensive state features into the first hidden layer. The first hidden layer generates a first hidden feature, which is input into the second hidden layer. The second hidden layer generates a second hidden feature, which is input into the output layer. The output layer outputs the probability value of each possible order allocation strategy and selects the order from the order allocation strategy with the highest probability as the next order received by the driver.

[0081] The formula for calculating the first hidden layer is:

[0082] h1 = σ(W1·X + b1)

[0083] Where h1 represents the first hidden feature, σ represents the ReLU activation function, W1 represents the first hidden weight matrix, X represents the comprehensive state feature, and b1 represents the bias vector of the first hidden layer.

[0084] The formula for calculating the second hidden layer is:

[0085] h2=σ(W2·h1+b2)

[0086] Where h2 represents the second hidden feature, σ represents the ReLU activation function, W2 represents the second hidden weight matrix, h1 represents the first hidden feature, and b2 represents the bias vector of the second hidden layer.

[0087] The formula for calculating the output layer is:

[0088] p = softmax(W p h2+b p )

[0089] Where p represents the probability of an order in the order allocation strategy, softmax represents the softmax function, and W p Let h1 represent the policy network weight matrix of the output layer, h2 represent the second hidden feature, and b represent the weight matrix of the output layer. p This represents the bias vector of the output layer;

[0090] The value network consists of a value processing layer and a value generation layer. The input layer inputs the comprehensive state features into the value processing layer, the value processing layer generates a value vector, the value vector is input into the value generation layer, and the value generation layer outputs the estimated value of the current state.

[0091] The calculation formula for the value processing layer is:

[0092] v=σ(W v1 ·X+b v1 )

[0093] Where v represents the value vector, σ represents the ReLU activation function, and W v1 This represents the weight matrix of the value processing layer, where X represents the comprehensive state features, and b v1 Indicates the bias parameters of the value processing layer;

[0094] The formula for calculating the value generation layer is:

[0095] V(s) = W v2 ·v+b v2

[0096] Where V(s) represents the estimated value in state s, and W v2 Let v represent the weight matrix of the value generation layer, and b represent the value vector. v2 Indicates the bias parameters of the value generation layer;

[0097] The state space includes order data and driver data;

[0098] Define the action space: Combine the next orders that each driver can accept to generate the action space;

[0099] Orders include return tickets and non-return tickets;

[0100] It should be noted that each order is viewed as a vector, pointing from the merchant point to the user point, and this vector represents the order vector;

[0101] Definition of a return trip order: Define a return vector, pointing from the driver's current position to the center of the driver's familiar area. The upper and lower thresholds of the return vector are based on the upper and lower tangent vectors of the driver's current position and the boundary of the driver's familiar area. If the angle between the vector of any order and the return vector is within the upper and lower thresholds of the return vector, then the order is considered a return trip order; otherwise, it is not a return trip order. In other words, for any order, the angle between the order vector and the return vector can be calculated to determine whether it is a return trip order.

[0102] An action includes each driver receiving the next order;

[0103] It should be noted that the next order assigned to a driver is based on the principle of prioritizing return trips, unless a non-return trip can significantly improve overall efficiency.

[0104] The reward is calculated after the driver completes the daily delivery tasks pushed by the platform. The reward calculation formula is as follows:

[0105] r=ω1R eff +ω2R sat +ω3R cust +ω4R back

[0106] Where r represents the total feedback reward value obtained, R eff R represents the delivery efficiency bonus value. sat R represents the driver satisfaction reward value. cust R represents the customer satisfaction reward value upon receiving the goods. back This represents the return time bonus value, where ω1, ω2, ω3, and ω4 represent the first, second, third, and fourth bonus parameters, respectively.

[0107] The formula for calculating delivery efficiency bonus value is:

[0108]

[0109] Among them, R eff T represents the delivery efficiency bonus value. delivery This indicates the total time from order acceptance to delivery completion;

[0110] The formula for calculating driver satisfaction bonus is:

[0111]

[0112] Among them, R sat F represents the driver satisfaction reward value. order T represents the frequency at which drivers accept orders. work Indicates the driver's working hours, I in This represents the driver's income level, and α represents a moderating factor.

[0113] The formula for calculating the customer satisfaction reward value is as follows:

[0114]

[0115] Among them, R cust This represents the reward value for customer satisfaction upon receiving the goods. This represents the feedback rating from the receiving user;

[0116] The formula for calculating the return trip time bonus is:

[0117]

[0118] Among them, R back Let ρ and τ represent the return time bonus, where ρ and τ are both positive numbers and ρ > τ.

[0119] The reward function of this invention is designed to balance the conflict between delivery efficiency, driver satisfaction, recipient satisfaction and return time, in order to maximize the number of orders completed per unit time and encourage drivers to return to familiar areas, thereby reducing the average delivery time of future orders.

[0120] Training the order allocation model includes the following steps:

[0121] Step 1: Randomly initialize the parameters of the policy network and the value network;

[0122] Step two, reset the environment to obtain the first state;

[0123] Step 3: Based on the interaction between the action and the environment, and the output of the current policy network, obtain the feedback r. t Next state s t+1 And whether the termination condition "done" has been triggered. The termination condition is that all drivers have returned to their respective familiar areas.

[0124] Experiences t ,a t ,r t ,s t+1 The done value is stored in the experience pool. One experience is a quaternion s. t ,a t ,r t ,s t+1 or s t ,a t ,r t ,done;

[0125] Step 4: Update the policy network and value network;

[0126] The method for updating the policy network includes the following steps:

[0127] 1. Randomly select a certain number of experience samples from the experience replay pool to form a batch dataset, and calculate the dominance function of the experience sample;

[0128] δ t =r t +γV(s t+1 )-V(s t )

[0129]

[0130] Where, r t V(s) represents the instantaneous reward obtained at time step t. t ) represents state s t Value function estimation, δ t Indicates TD error, Describes the Monte Carlo reward over n steps. Let represent the advantage function estimate at time step t, γ represent the discount factor, and λ represent the cutoff coefficient.

[0131] 2. Calculate the action probability π(a) using the current policy network. t |s t ;θ), calculate the action probability π(a) using the policy network updated last time. t |s t ;θ oid ), calculate ratio

[0132] 3. Use the clip objective function of PPO to limit the update step size;

[0133] The formula for calculating the policy loss function is:

[0134] L CLIP (θ)=E t [min(r t (θ)A t ,clip(r t (θ), 1-∈, 1+∈)A t )]

[0135] Among them, A t This represents the estimate of the advantage function, where ∈ denotes the clip parameter;

[0136] 4. Calculate the gradient of the policy loss function with respect to the policy network parameters θ, and update the parameters using the Adam optimizer;

[0137] The method for updating the value network includes the following steps:

[0138] 1. The policy loss function of the value network is:

[0139] Lv(φ)=E t [(V(s t |φ)-G t ) 2

[0140] wherein G t represents an actual return, V(s t |φ) represents a prediction of a value network;

[0141] 2. calculating a gradient of the loss function of the value network with respect to the parameter φ, and updating the parameter by an Adam optimizer;

[0142] step five: updating the policy network and the value network every fixed number of network updates, so that parameters thereof are the same as parameters of the current order allocation model;

[0143] step six: terminating the step until the policy network and the value network converge or the number of updates reaches a set value.

[0144] step 103: pushing the next order for each driver according to the order allocation strategy output by the order allocation model.

[0145] as shown in Figure 2 , a service intelligent matching system for multi-type downstream users comprises the following modules:

[0146] a data acquisition module 100, configured to acquire historical order data and driver data;

[0147] a data preprocessing module 200, configured to preprocess the acquired data;

[0148] a feature generation module 300, configured to generate order features and driver features, and combine the features into comprehensive state features;

[0149] an order allocation model module 400, configured to input the comprehensive state features and output an order allocation strategy;

[0150] an action and feedback module 500, configured to push the next order for each driver according to the output order allocation strategy.

[0151] A storage medium storing non-transitory computer-readable instructions, when the non-transitory computer-readable instructions are executed by a computer, the steps in the above service intelligent matching and pushing method for multi-type downstream users can be performed.

[0152] ​The embodiments of this example have been described above. However, this example is not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms based on the guidance of this example, and all of them are within the protection scope of this example.

Claims

1. A business intelligent matching and push method for multiple types of downstream users, characterized in that, Includes the following steps: Step 101: Collect historical order data and driver data. Order data includes order time, pickup location, delivery location, and order completion time. Driver data includes driver location, driver's familiar area, and driver's current status. Driver's current status includes delivery status and non-delivery status. Among them, the driver's familiar area is determined by using a clustering algorithm based on the driver's historical delivery records to identify the driver's main activity areas and generate the driver's familiar area; Data preprocessing is performed on the collected historical order data and driver data. The preprocessing methods include: outlier detection and handling, removing or correcting obviously erroneous data points; and data standardization, which normalizes or standardizes numerical features to reduce the impact of scale differences. Step 102: Generate order features through historical order data, generate driver features through driver data, combine order features and driver features to generate comprehensive status features, input them into the order allocation model, and the order allocation model outputs an order allocation strategy, which includes the next order received by each driver. The formula for calculating the combination of order characteristics and driver characteristics is as follows: ； in, Indicates the overall state characteristics. Indicates order characteristics, Indicates driver characteristics, This indicates vector concatenation; The order allocation model includes an input layer, a policy network, and a value network; The policy network includes a first hidden layer, a second hidden layer, and an output layer. The input layer inputs the comprehensive state features into the first hidden layer. The first hidden layer generates a first hidden feature, which is input into the second hidden layer. The second hidden layer generates a second hidden feature, which is input into the output layer. The output layer outputs the probability value of each possible order allocation strategy and selects the order from the order allocation strategy with the highest probability as the next order received by the driver. The formula for calculating the first hidden layer is: ； in, This represents the first hidden feature. Represents the ReLU activation function. This represents the first hidden weight matrix. Indicates the overall state characteristics. This represents the bias vector of the first hidden layer; The formula for calculating the second hidden layer is: ； in, This represents the second hidden feature. Represents the ReLU activation function. This represents the second hidden weight matrix. This represents the first hidden feature. This represents the bias vector of the second hidden layer; The formula for calculating the output layer is: ； in, This represents the probability of an order in the order allocation strategy. express function, This represents the policy network weight matrix of the output layer. This represents the second hidden feature. This represents the bias vector of the output layer; The value network consists of a value processing layer and a value generation layer. The input layer inputs the comprehensive state features into the value processing layer, the value processing layer generates a value vector, the value vector is input into the value generation layer, and the value generation layer outputs the estimated value of the current state. The calculation formula for the value processing layer is: ； in, Represents a value vector. Represents the ReLU activation function. This represents the weight matrix of the value processing layer. Indicates the overall state characteristics. Indicates the bias parameters of the value processing layer; The formula for calculating the value generation layer is: ； in, Indicates the state The estimated value under the given conditions, This represents the weight matrix of the value generation layer. Represents a value vector. Indicates the bias parameters of the value generation layer; The state space includes order data and driver data; Define the action space: Combine the next orders that each driver can accept to generate the action space; Orders include return tickets and non-return tickets; Each order is viewed as a vector, pointing from the merchant point to the user point; this vector represents the order vector. Definition of return trip: Define a return vector, pointing from the driver's current position to the center of the driver's familiar area, and use the upper and lower tangent vectors of the driver's current position and the boundary of the driver's familiar area as the upper and lower threshold ranges of the return vector. If the angle between the vector of any order and the return vector is within the upper and lower threshold ranges of the return vector, then the order is a return trip; otherwise, it is a non-return trip. An action includes each driver receiving the next order; The next order assigned to a driver is based on the principle of prioritizing return trips, unless a non-return trip can significantly improve overall efficiency; The reward is calculated after the driver completes the daily delivery tasks pushed by the platform. The reward calculation formula is as follows: ； in, This represents the total feedback reward value received. This represents the delivery efficiency bonus value. This represents the driver satisfaction reward value. This represents the reward value for customer satisfaction upon receiving the goods. This represents the return trip time bonus value. , , , These represent the first, second, third, and fourth reward parameters, respectively. The formula for calculating delivery efficiency bonus value is: ； in, This represents the delivery efficiency bonus value. This indicates the total time from order acceptance to delivery completion; The formula for calculating driver satisfaction bonus is: ； in, This represents the driver satisfaction reward value. This indicates the frequency at which drivers accept orders. Indicates the driver's working hours. This indicates the driver's income level. Indicates the regulating factor; The formula for calculating the customer satisfaction reward value is as follows: ； in, This represents the reward value for customer satisfaction upon receiving the goods. This represents the feedback rating from the recipient; The formula for calculating the return trip time bonus is: ； in, This represents the return trip time bonus value. , All are positive numbers, and ; Training the order allocation model includes the following steps: Step 1: Randomly initialize the parameters of the policy network and the value network; Step two, reset the environment to obtain the first state; Step 3: Obtain feedback based on the interaction between the action and the environment, and the output of the current policy network. Next state And whether the termination condition "done" has been triggered. The termination condition is that all drivers have returned to their respective familiar areas. Experience Storing experience into the experience pool, one experience is a quaternion. or ; Step 4: Update the policy network and value network; The method for updating the policy network includes the following steps: A certain number of experience samples are randomly selected from the experience replay pool to form a batch dataset, and the dominance function of the experience sample is calculated. ； ； ； in, This represents the immediate reward obtained at time step t. Representing state Value function estimation, Indicates TD error, Describes the Monte Carlo reward for n steps. This represents the estimated value of the dominance function at time step t. Indicates the discount factor. Indicates the cutoff coefficient; Calculate the action probability using the current policy network. Calculate the action probability using the policy network updated last time. Calculate the ratio ; Use the clip objective function of PPO to limit the update step size; The formula for calculating the policy loss function is: ； in, This represents the estimate of the advantage function. express parameter; Calculate the policy loss function with respect to the policy network parameters. The gradient is calculated, and the parameters are updated using the Adam optimizer; The method for updating the value network includes the following steps: The policy loss function of the value network is: ； in, Indicates actual return, Represents the prediction of the value network; Calculate the loss function of the value network with respect to the parameters. The gradient is calculated, and the parameters are updated using the Adam optimizer; Step 5: Every fixed number of network updates, update the policy network and value network so that their parameters are the same as those of the current order allocation model; Step six: Terminate the process once the policy network and value network converge or the number of updates reaches the set value. Step 103: Push the next order to each driver according to the order allocation strategy output by the order allocation model.

2. A business intelligent matching system for multiple types of downstream users, characterized in that, The method for performing the steps of the intelligent matching and push method for multi-type downstream users as described in claim 1 includes the following modules: The data acquisition module is used to collect historical order data and driver data; The data preprocessing module is used to preprocess the collected data; The feature generation module is used to generate order features and driver features, and combine them into comprehensive status features; The order allocation model module is used to take comprehensive status features as input and output order allocation strategies. The action and feedback module pushes the next order to each driver based on the output order allocation strategy.

3. A storage medium storing non-transitory computer-readable instructions, which, when executed by a computer, are capable of performing the steps in the business intelligent matching and push method for multiple types of downstream users as described in claim 1.

Images