Fourth-party logistics path planning method considering infection risk and tardiness risk

Abstract

The invention belongs to the technical field of logistics distribution path planning, and particularly discloses a fourth-party logistics path planning method considering an infection risk and a tardiness risk, and the method employs a CVaR model to carry out the expansion research of a fourth-party logistics optimization problem considering the tardiness risk, guarantees the minimum mean value of the tardiness risk, and improves the optimization efficiency of the fourth-party logistics path. The influence of the infection risk on a distribution scheme is fully considered, the infection risk is quantified by using the local accumulated diagnosed number of people in a transfer node city, a mathematical model taking the cost and the infection risk as constraint conditions and taking the minimization of CVaR as a target is established, and a reward function in an improved Q-learning algorithm is redesigned according to the characteristics of the model, so that the accuracy of the distribution scheme is improved. And solving and visualizing the distribution path of the minimum CVaR, which meets the requirements of customer infection risk and cost.

Description

Technical Field

[0001] The present invention relates to the technical field of logistics distribution route planning, and in particular to a fourth-party logistics route planning method that takes infection risk and delay risk into consideration. Background Art

[0002] Our previously applied invention patent (application number CN202211688071.6) discloses a fourth-party logistics path optimization method based on CVaR, including: determining the transportation relationship to be optimized including the supply city node, several transit city nodes and the destination city node, and constructing a fourth-party logistics optimization model based on CVaR according to the transportation relationship; solving the fourth-party logistics optimization model through the Q-learning algorithm to determine the optimal path for fourth-party logistics distribution. This application establishes a mathematical model with minimizing CVaR as the objective function and transportation cost as the constraint condition to evaluate the risk of delay, with the purpose of selecting the path that minimizes CVaR at a limited cost to provide customers with satisfactory delivery services. And combining the Q-learning algorithm with the fourth-party logistics path optimization problem, redesigning the state-action pair and defining the reward value, the constructed random model can effectively control the risk of delay, and can also quickly obtain a satisfactory solution according to the different confidence levels of customers.

[0003] During the epidemic, the rapid spread of the virus caused a serious shortage of medical and daily necessities. In the later period, the epidemic was effectively controlled and the supply of various materials was sufficient. The contribution of the logistics industry is indelible. At the same time, the epidemic has made 4PL companies deeply aware of the importance of improving emergency response capabilities, improving health protection measures, and optimizing logistics routes and models.

[0004] Since logistics warehousing centers and transfer nodes are densely populated and have large amounts of cargo, once an epidemic occurs, it may cause widespread transmission. While actively opening up logistics channels, it should be ensured that when providing distribution services, vehicles between city nodes transport goods, not viruses. Therefore, 4PL's supervision of infection risks during logistics transportation should become an important part of epidemic prevention and control. In this process, 4PL suppliers must formulate safe and efficient transportation routes with more scientific and accurate prevention and control measures to ensure safety while providing customers with more efficient services.

[0005] Therefore, it is necessary to study the optimization of fourth-party logistics routes taking into account infection risks and delay risks. Summary of the invention

[0006] In order to solve the above technical problems, the present invention provides a fourth-party logistics path planning method taking into account infection risk and delay risk.

[0007] To achieve the above object, the present invention is implemented according to the following technical solutions:

[0008] A fourth-party logistics path planning method considering infection risk and delay risk includes the following steps:

[0009] S1. Determine the transportation relationship to be optimized at the beginning of the epidemic, including the supply city node, several transit city nodes, and the destination city node, and build a CVaR-based fourth-party logistics path planning model based on the transportation relationship and with cost and infection risk as constraints:

[0010]

[0011] R = {v s ,...,v i ,k,v j ,...,v k}∈G (5)

[0012] x ijk (R),y j (R)∈{0,1}(6)

[0013] X j <50 (7);

[0014] Where: Formula (1) represents the objective function, i.e., minimizing CVaR, where β is the confidence level, indicating the customer's risk aversion; Formula (2) is the infection risk constraint, where F0 represents the maximum acceptable infection risk given by the customer, and X j represents the cumulative number of confirmed cases in the local area within 14 days in the node city j, and f represents the unit infection risk of the cumulative number of confirmed cases in the local area within 14 days; Formula (3) is the expression of the delay amount ΔT, which is a random variable; Formula (5) represents the constraint on the path, which must ensure that the path is a legal connected path from the initial city to the destination city; Formula (6) represents the constraint on the decision variable, and Formula (7) indicates that the transportation path cannot pass through high-risk areas;

[0015] S2. Based on the improved Q-learning algorithm, the CVaR-based fourth-party logistics path planning model is solved and the delivery path that minimizes CVaR that meets the customer's infection risk and cost requirements is visualized.

[0016] Furthermore, the classification criteria for infection risk in step S1 are:

[0017]

[0018] Furthermore, the step S2 specifically includes:

[0019] S21: Define the state space of the Q-learning algorithm: take the supply city node, the transit city node and the destination city node as different states, and the supply city node, several transit city nodes and the destination city node in the transportation relationship to be optimized as the state space composition; define the action space of the Q-learning algorithm: take the 3PL suppliers corresponding to the supply city node, the transit city node and the destination city node as different actions, and the 3PL suppliers corresponding to the supply city node, several transit city nodes and the destination city node in the transportation relationship to be optimized as the state space composition; assuming the current state s (the current node city), the 3PL supplier corresponding to the node city can be represented by the action space A={a1,a2,···,a k ,···,a K}, k = 1, 2, ···, K to represent; then import the known information into Matlab;

[0020] S22: Initialize parameters γ, α and Q table, set the initial state and final state, and generate the reward matrix R according to the existing data using the following formula;

[0021] r=ω1r(s,a)+ω2r(s,a)+ω3r(s,a)+ω4r0(s,a) (8)

[0022] in:

[0023]

[0024] ω1 is inversely related to the delivery cost corresponding to the selected 3PL supplier and transshipment node, that is, the smaller the delivery cost, the greater the reward value; ω2 is inversely related to the mean of the delivery time corresponding to the selected 3PL supplier and transshipment node, and the smaller the mean of the random time, the greater the reward value; ω3 is inversely related to the variance of the random time corresponding to the selected 3PL supplier and transshipment node, and the smaller the variance, the greater the corresponding reward value. k1 and k2 are the weighted coefficients of the reward function; ω4 is related to the number of infected people at the node;

[0025] S23: Initialization state is the initial node;

[0026] S24: using a random strategy to select an optional 3PL supplier corresponding to the action, i.e., the state;

[0027] S25: Execute action a, i.e. select a 3PL supplier of the current node, transfer to the new state s', i.e. the next node city, and update the Q table according to the reward matrix R and related parameter settings. The Q table formula is as follows:

[0028] Q *(s,a)←(s,a)+ρ[R(s,a,s′)+γmaxQ(s′,a′)-Q(s,a)];

[0029] Where R(s,a,s′) represents the reward value obtained by moving from state s to state s′ by executing action a;

[0030] γ represents the discount factor, γ∈(0,1); ρ represents the learning rate;

[0031] S26: Determine whether s' is in the termination state, if not, go to step S24, if yes, go to step S27;

[0032] S27: Determine whether the training times have been reached, if not, go to step S23, if so, go to step S28;

[0033] S28: Output Q table after training;

[0034] S29: Output the optimal delivery path according to the Q table.

[0035] Preferably, γ=0.8, α=0.9, and episode=100.

[0036] Preferably, k1 is 0.1; k2 is 0.2 or 0.8.

[0037] Compared with the prior art, the present invention adopts the CVaR model to expand the research on the fourth-party logistics optimization problem considering the delay risk. While ensuring the minimum mean delay risk, it fully considers the impact of infection risk on the distribution plan, and quantifies the infection risk by using the cumulative number of confirmed local cases in the transfer node city. A mathematical model with cost and infection risk as constraints and minimizing CVaR as the goal is established. According to the characteristics of the model, the reward function in the improved Q-learning algorithm is redesigned to solve and visualize the distribution path that minimizes CVaR and meets the customer's infection risk and cost requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 Description of the node problem.

[0039] Figure 2 Performance analysis for parameter α.

[0040] Figure 3 Performance analysis for parameter γ.

[0041] Figure 4 Performance analysis for the parameter episode.

[0042] Figure 5 Solve the path graph for seven nodes.

[0043] Figure 6 Solve the path graph for 15 nodes. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments. The specific embodiments described herein are only used to explain the present invention and are not used to limit the invention.

[0045] Taking into account the impact of infection risk during the epidemic, based on the previous article, the delivery route is set not to pass through high-risk node cities, and infection risk constraints are added. Therefore, this problem can be specifically described as: Figure 1 As shown in the figure, a batch of goods needs to be sent from the supply city s to the demand city t. There are multiple transfer nodes between the two cities. There are different 3PL suppliers to choose from between every two transfer cities. The cost and time spent by different 3PLs in providing services are different. Similarly, when transfer occurs at each city node, costs, time and infection risks caused by staying at the node will be incurred. Therefore, it is necessary to comprehensively consider the selection of transfer nodes and the relevant parameters of different 3PL suppliers to provide customers with a distribution plan that minimizes CVaR and meets the distribution cost and infection risk requirements.

[0046] In order to better study the above problems, the following assumptions are made:

[0047] Hypothesis 1: In view of the epidemic situation, in the actual transportation process, only when the node city changes the 3PL will operations such as handling and loading and unloading of goods be carried out, which will pose an infection risk. At the same time, the transportation process between the two cities will not cause a long stay in a certain place, so the infection risk is not considered.

[0048] Hypothesis 2: The size of the infection risk is related to the cumulative number of local confirmed cases in the transfer city in the past 14 days. At the same time, it is set that the transportation task does not pass through high-risk city nodes. For the infection risk, this embodiment studies the fourth-party logistics path optimization problem in the early stage of the epidemic. Therefore, the infection risk of the city node is quantified by referring to the division of low, medium and high risk areas in the early stage of the epidemic, as shown in Table 1.

[0049]

[0050] According to the demarcation standards, when there are no local confirmed cases in a certain area or no new local confirmed cases for fourteen consecutive days, it is identified as a low-risk area. When there are new local confirmed cases in a certain area within fourteen days and the cumulative number of local confirmed cases does not exceed 50, it is identified as a medium-risk area. When the cumulative number of local cases in a certain area within fourteen days exceeds 50, it is identified as a high-risk area. Because the incubation period of the virus is fourteen days, and the travel code also refers to the travel trajectory within 14 days, we use the cumulative number of local confirmed cases in a certain city node within 14 days as the basis, and the infection risk per confirmed person as the unit of infection risk f to quantify the size of the infection risk. If the cumulative number of confirmed cases in a certain city node within 14 days is 25, the infection risk is 25f, and the infection risk will only occur when transferring in the node city. Therefore, the constraint condition can be written as Formula 4.1, and F0 is the maximum infection risk acceptable to the customer.

[0051]

[0052] Based on the above problems, this embodiment provides a fourth-party logistics route planning method that takes into account infection risk and delay risk. The specific steps are as follows:

[0053] S1. First, determine the transportation relationships to be optimized at the beginning of the epidemic, including the supply city nodes, several transit city nodes, and the destination city nodes, and build a fourth-party logistics path planning model based on CVaR based on the transportation relationships and with cost and infection risk as constraints:

[0054]

[0055] R = {v s ,...,v i ,k,v j ,...,v k}∈G (5)

[0056] x ijk (R),y j (R)∈{0,1}(6)

[0057] X j <50 (7);

[0058] Where: Formula (1) represents the objective function, i.e., minimizing CVaR, where β is the confidence level, indicating the customer's risk aversion; Formula (2) is the infection risk constraint, where F0 represents the maximum acceptable infection risk given by the customer, and X jrepresents the cumulative number of confirmed cases in the local area within 14 days in the node city j, and f represents the unit infection risk of the cumulative number of confirmed cases in the local area within 14 days; Formula (3) is the expression of the delay amount ΔT, which is a random variable; Formula (5) represents the constraint on the path, which must ensure that the path is a legal connected path from the initial city to the destination city; Formula (6) represents the constraint on the decision variable, and Formula (7) indicates that the transportation path cannot pass through high-risk areas;

[0059] S2. Solve the CVaR-based fourth-party logistics path planning model based on the improved Q-learning algorithm and visualize the delivery path that minimizes CVaR and meets the customer's infection risk and cost requirements;

[0060] S21: Define the state space of the Q-learning algorithm: take the supply city node, the transit city node and the destination city node as different states, and the supply city node, several transit city nodes and the destination city node in the transportation relationship to be optimized as the state space composition; define the action space of the Q-learning algorithm: take the 3PL suppliers corresponding to the supply city node, the transit city node and the destination city node as different actions, and the 3PL suppliers corresponding to the supply city node, several transit city nodes and the destination city node in the transportation relationship to be optimized as the state space composition; assuming the current state s (the current node city), the 3PL supplier corresponding to the node city can be represented by the action space A={a1,a2,···,a k ,···,a K}, k = 1, 2, ···, K to represent; then import the known information into Matlab;

[0061] S22: Initialize parameters γ, α and Q table, set the initial state and final state, and generate the reward matrix R according to the existing data using the following formula;

[0062] r=ω1r(s,a)+ω2r(s,a)+ω3r(s,a)+ω4r0(s,a) (8)

[0063] in:

[0064]

[0065] This formula indicates that the transportation task cannot pass through high-risk areas, that is, the number of infected people at the transfer nodes passed by the path must not exceed 50. Therefore, when the number of infected people in a certain city j is less than or equal to 50, the reward is 1, and when the number of infected people in a certain city j is greater than 50, the reward is -100;

[0066]

[0067] This formula means that when there is a connection between cities i and j and j is not the end point, the reward is 1; when there is no connection between cities i and j, the reward is -1; when there is a connection between cities i and j and j is the end point, the reward is 100;

[0068] ω1 is inversely related to the delivery cost corresponding to the selected 3PL supplier and transshipment node, that is, the smaller the delivery cost, the greater the reward value; ω2 is inversely related to the mean of the delivery time corresponding to the selected 3PL supplier and transshipment node, and the smaller the mean of the random time, the greater the reward value; ω3 is inversely related to the variance of the random time corresponding to the selected 3PL supplier and transshipment node, and the smaller the variance, the greater the corresponding reward value. k1 and k2 are the weighted coefficients of the reward function; ω4 is related to the number of infected people at the node;

[0069] S23: Initialization state is the initial node;

[0070] S24: using a random strategy to select an optional 3PL supplier corresponding to the action, i.e., the state;

[0071] S25: Execute action a, i.e. select a 3PL supplier of the current node, transfer to the new state s', i.e. the next node city, and update the Q table according to the reward matrix R and related parameter settings. The Q table formula is as follows:

[0072] Q * (s,a)←(s,a)+ρ[R(s,a,s′)+γmaxQ(s′,a′)-Q(s,a)];

[0073] Where R(s,a,s′) represents the reward value obtained by moving from state s to state s′ by executing action a;

[0074] γ represents the discount factor, γ∈(0,1); ρ represents the learning rate;

[0075] S26: Determine whether s' is in the termination state, if not, go to step S24, if yes, go to step S27;

[0076] S27: Determine whether the number of training times has been reached. If not, go to step S23. If yes, go to step S28. In this embodiment, the best parameters of the Q-learning algorithm are γ =0.8, α=0.9, episode=100; k1 is 0.1; k2 is 0.2 or 0.8;

[0077] S28: Output Q table after training;

[0078] S29: Output the optimal delivery path according to the Q table.

[0079] In order to verify the feasibility of the method of the present invention, a simulation test is carried out, as follows:

[0080] First, taking seven nodes as an example, the influence of the number of training episodes, discount factor γ and learning rate α on the calculation results in the Q-learning algorithm is analyzed, and a set of optimal parameter combinations is obtained. Then, the mathematical model with the constraints of delivery cost and infection risk and the objective function of minimizing CVaR is solved, and finally the best delivery path obtained by the corresponding example is visualized. The software used in the algorithm is MATLAB2018a, and the operating environment is Intel(R)Core(TM)i7-2600@3.40GHz.

[0081] The parameters in the improved Q-learning algorithm are tested through a large number of experimental simulations. The method is to set other parameters unchanged and observe the impact of a certain parameter change on the solution result. It can be expressed by the best rate, that is, the probability of obtaining the best solution when the algorithm is solving the problem, out of the total number of times the algorithm is run.

[0082] Figure 2-Figure 4 As shown, when the confidence level is 0.9, the infection risk constraint is 8.5×10 -5 , the parameter testing process of the improved Q-learning algorithm for solving the seven-node case when the cost constraint is 80, where the "best rate" represents the probability of obtaining the best solution when the algorithm is used to solve the problem, and the total number of runs is 100. The test results show that the best parameters of the improved Q-learning algorithm are γ=0.8,α=0.9,episode=100.

[0083] By repeatedly experimenting with k1 and k2 in the reward function in examples of different scales, the algorithm achieves the best solution when k1 and k2 are set to the values ​​shown in Table 2.

[0084] Table 2

[0085]

[0086] Furthermore, in order to verify the effectiveness of the proposed model, the minimum CVaR value and the corresponding best path corresponding to two different scale examples of 7 and 15 nodes are solved. The relevant information of the 7 nodes is shown in Table 3. Since the amount of information of the 3PL supplier is too large, Table 4 only shows part of the information.

[0087] Table 3

[0088]

[0089] Table 4

[0090]

[0091] like Figure 5 This is the path diagram corresponding to the solution results of the 7 nodes, where the black five-pointed star s represents the initial node, the black five-pointed star t represents the destination node, and the node city 4 marked with a red five-pointed star represents the transfer city node that does not meet the infection number constraint, that is, transfer node 4 is in a high-risk area, so the path cannot pass through node 4. The remaining transfer nodes are cities that meet the infection number constraint. The path marked by the blue thick line is the distribution path corresponding to the minimum CVaR that meets the cost and infection risk constraints. The detailed data are shown in Table 5.

[0092] Table 5

[0093]

[0094] As shown in Table 5, under different confidence levels, the cost constraint C0=115, and the maximum infection risk that the customer can accept is 8.5×10 -5 The solution result of the CVaR model for the 7-node problem when . Among them, "β" represents the confidence level, that is, the degree of risk aversion of the customer, "CVaR" is the best solution obtained by the CVaR model (the evaluation standard is the objective function), "best path" is the distribution path corresponding to the best solution obtained, "F" is the infection risk of the distribution path corresponding to the best solution obtained, "best rate" represents the probability of the best solution obtained when the algorithm is used to solve the problem, and the total number of runs is 100 at this time, and "time" is the running time of the algorithm once, in seconds.

[0095] From the data in Table 5, we can see that when the confidence level is 0.9, the cost constraint C0 = 80, and the infection risk constraint F0 = 8.5 × 10 -5 When , the optimal CVaR value is 27.7892, which means that the corresponding delay risk of the delivery task is 27.7892, the corresponding delivery cost is 80, and the infection risk is 7.352×10 -5 , the corresponding best delivery path is R = {v s ,2,v1,1,v3,1,v t}, indicating that the transit cities selected from the source node city s to the destination node city t are 1 and 3 respectively, and the 3PL suppliers selected between every two transit cities are numbered 2, 1, 1; when the confidence level is 0.95, the cost constraint C0 = 80, and the infection risk constraint F0 = 8.5×10 -5 When , the minimum CVaR value is 35.1168, which means that the corresponding delay risk of the delivery task is 35.1168, the corresponding delivery cost is 80, and the infection risk is 7.352×10 -5 , the corresponding best delivery path is R = {v s,2,v1,1,v3,1,v t}, indicating that the transit cities selected from the source node city s to the destination node city t are 1 and 3 respectively, and the 3PL suppliers selected between every two cities are numbered 2, 1, 1; when the confidence level is 0.99, the cost constraint C0 = 80, and the infection risk constraint F0 = 8.5×10 -5 When , the optimal CVaR value is 49.4634, which means that the corresponding delay risk mean of the delivery task is 49.4634, the corresponding delivery cost is 80, and the infection risk is 7.352×10 -5 , the corresponding best delivery path is R = {v s ,2,v1,1,v3,1,v t}, indicating that the transit cities selected for transportation from source node city s to destination node city t are 1 and 3 respectively, and the 3PL suppliers selected between every two cities are numbered 2, 1, 1.

[0096] The relevant information of node 15 is shown in Table 6. Since there are 91 rows of information on the 3PL supplier corresponding to node 15, Table 7 only shows part of the information.

[0097] Table 6

[0098]

[0099]

[0100] Table 7

[0101]

[0102] like Figure 6 This is the path diagram corresponding to the solution results of 15 nodes, where the black five-pointed star s represents the initial node, the black five-pointed star t is the destination node, and the node cities 2, 4, and 7 marked with red five-pointed stars are the transfer city nodes that do not meet the infection number constraint, that is, transfer nodes 2, 4, and 7 are in high-risk areas, so the path cannot pass through nodes 2, 4, and 7. The remaining transfer nodes are cities that meet the infection number constraint. The path marked by the blue thick line is the distribution path corresponding to the minimum CVaR that meets the cost and infection risk constraints. The detailed data are shown in Table 8.

[0103] Table 8

[0104]

[0105]

[0106] As shown in Table 8, under different confidence levels, the cost constraint C0=115, and the maximum infection risk that the customer can accept is 1.6×10 -4The solution results of the CVaR model for the 15-node problem are shown in Figure 2.

[0107] From the data in Table 8, we can see that when the confidence level is 0.9, the cost constraint C0 = 115, and the infection risk constraint F0 = 1.6 × 10 -4 When , the optimal CVaR value is 12.1589, which means that the corresponding delay risk of the delivery task is 12.1589, the corresponding delivery cost is 115, and the infection risk is 1.28×10 -4 , the corresponding best delivery path is R = {v s ,3,v3,2,v6,3,v 13 ,2,v t}, indicating that the transit cities selected for transportation from source node city s to destination node city t are 3, 6, and 13, and the 3PL suppliers selected between every two cities are 3, 2, 3, 2; when the confidence level is 0.95, the cost constraint C0 = 115, and the infection risk constraint F0 = 1.6×10 -4 When , the optimal CVaR value is 14.2909, which means that the corresponding delay risk of the delivery task is 14.2909, the corresponding delivery cost is 115, and the infection risk is 1.28×10 -4 , the corresponding best delivery path is R = {v s ,3,v3,2,v6,3,v 13 ,2,v t}, indicating that the transit cities selected from the source node city s to the destination node city t are 3, 6, and 13, and the 3PL suppliers selected between every two cities are 3, 2, 3, 2; when the confidence level is 0.99, the cost constraint C0 = 115, and the infection risk constraint F0 = 1.6×10 -4 When , the optimal CVaR value is 18.4651, which means that the corresponding delay risk mean of the delivery task is 18.4651, the corresponding delivery cost is 115, and the infection risk is 1.28×10 -4 , the corresponding best delivery path is R = {v s ,3,v3,2,v6,3,v 13 ,2,v t}, indicating that the transit cities selected for transportation from source node city s to destination node city t are 3, 6, and 13, respectively, and the 3PL supplier numbers selected between every two cities are 3, 2, 3, 2.

[0108] The above data show that when the cost constraint and infection risk constraint remain unchanged, as the confidence level increases, the corresponding best delivery path will not change, so the infection risk will not change, but the average delay risk faced by the customer will be higher. Therefore, by using this model, 4PL suppliers can combine the customer's risk aversion and consider the impact of infection risk on the delivery plan, so that the delivery path does not pass through high-risk areas, and provide customers with a delivery path with the minimum average delay risk that meets the customer's infection risk and cost requirements at a given confidence level.

[0109] The technical solution of the present invention is not limited to the above-mentioned specific embodiments. All technical variations made according to the technical solution of the present invention fall within the protection scope of the present invention.

Claims

1. A fourth-party logistics path planning method considering infection risk and delay risk, characterized in that: The following steps are involved: S1. Determine the transportation relationship to be optimized at the beginning of the epidemic, including the supply city node, several transit city nodes, and the destination city node, and build a CVaR-based fourth-party logistics path planning model based on the transportation relationship and with cost and infection risk as constraints: R={v s ,...,v i ,k,v j ,...,v k }∈G (5) x ijk (R),y j (R)∈{0,1} (6) X j ＜50 (7)； Where: Formula (1) represents the objective function, i.e., minimizing CVaR, where β is the confidence level, indicating the customer's risk aversion; Formula (2) is the infection risk constraint, where F0 represents the maximum acceptable infection risk given by the customer, and X j represents the cumulative number of confirmed cases in the local area within 14 days in the node city j, and f represents the unit infection risk of the cumulative number of confirmed cases in the local area within 14 days; Formula (3) is the expression of the delay amount ΔT, which is a random variable; Formula (5) represents the constraint on the path, which must ensure that the path is a legal connection path from the initial city to the destination city; Formula (6) represents the constraint on the decision variable, and Formula (7) indicates that the transportation path cannot pass through high-risk areas; S2. Based on the improved Q-learning algorithm, the CVaR-based fourth-party logistics path planning model is solved and the delivery path that minimizes CVaR that meets the customer's infection risk and cost requirements is visualized.

2. The fourth-party logistics path planning method considering infection risk and delay risk according to claim 1 is characterized in that: The classification criteria for infection risk in step S1 are:

3. The fourth-party logistics path planning method considering infection risk and delay risk according to claim 1 is characterized in that: The step S2 specifically includes: S21: Define the state space of the Q-learning algorithm: take the supply city node, the transit city node and the destination city node as different states, and the supply city node, several transit city nodes and the destination city node in the transportation relationship to be optimized as the state space composition; define the action space of the Q-learning algorithm: take the 3PL suppliers corresponding to the supply city node, the transit city node and the destination city node as different actions, and the 3PL suppliers corresponding to the supply city node, several transit city nodes and the destination city node in the transportation relationship to be optimized as the state space composition; assuming the current state s (the current node city), the 3PL supplier corresponding to the node city can be represented by the action space A={a1,a2,···,a k ,···,a K }, k = 1, 2, ···, K to represent; then import the known information into Matlab; S22: Initialize parameters γ, α and Q table, set the initial state and final state, and generate the reward matrix R according to the existing data using the following formula; r=ω1r(s,a)+ω2r(s,a)+ω3r(s,a)+ω4r0(s,a) (8) in: ω1 is inversely related to the delivery cost corresponding to the selected 3PL supplier and transshipment node, that is, the smaller the delivery cost, the greater the reward value; ω2 is inversely related to the mean of the delivery time corresponding to the selected 3PL supplier and transshipment node, and the smaller the mean of the random time, the greater the reward value; ω3 is inversely related to the variance of the random time corresponding to the selected 3PL supplier and transshipment node, and the smaller the variance, the greater the corresponding reward value. k1 and k2 are the weighted coefficients of the reward function; ω4 is related to the number of infected people at the node; S23: Initialization state is the initial node; S24: using a random strategy to select an optional 3PL supplier corresponding to the action, i.e., the state; S25: Execute action a, i.e. select a 3PL supplier of the current node, transfer to the new state s', i.e. the next node city, and update the Q table according to the reward matrix R and related parameter settings. The Q table formula is as follows: Q * (s,a)←-Q(s,a)+ρ[R(s,a,s′)+γmaxQ(s′,a′)-Q(s,a)]; Where R(s,a,s′) represents the reward value obtained by moving from state s to state s′ by executing action a; γ represents the discount factor, γ∈(0,1); ρ represents the learning rate; S26: Determine whether s' is in the termination state, if not, go to step S24, if yes, go to step S27; S27: Determine whether the training times have been reached, if not, go to step S23, if so, go to step S28; S28: Output Q table after training; S29: Output the optimal delivery path according to the Q table.

4. The fourth-party logistics path planning method considering infection risk and delay risk according to claim 3 is characterized in that: The γ=0.8, α=0.9, and episode=100.

5. The fourth-party logistics path planning method considering infection risk and delay risk according to claim 3 is characterized in that: The k1 is 0.1; k2 is 0.2 or 0.8.

Images