Funeral service path optimization scheduling method based on deep reinforcement learning

Abstract

The invention discloses a funeral service path optimization scheduling method based on deep reinforcement learning, and the method comprises the steps: S1, collecting the data of a funeral vehicle, and constructing a data set; s2, initializing a particle swarm according to the data of the funeral vehicle, and defining a fitness function; s3, evaluating the fitness value of each particle in the particle swarm, and preliminarily obtaining the optimal position and the global optimal position of each particle; s4, temperature parameters are introduced through a simulated annealing algorithm, and whether the current solution is accepted as a historical optimal solution or not is judged according to a gradual temperature reduction mechanism of the simulated annealing algorithm; s5, combining a particle swarm and a simulated annealing algorithm and introducing a self-adaptive learning rate adjustment mechanism to update the speed and the position of the particle; and S6, iteratively executing the steps S3 to S5, and outputting an optimal funeral car path scheduling scheme. According to the method, an efficient and scientific optimization scheme can be provided for path planning when a funeral vehicle executes a task, and remarkable technical values and economic benefits are brought to practical application.

Description

Technical Field

[0001] The present invention relates to the field of path planning, and in particular, to a path optimization scheduling method for funeral services based on deep reinforcement learning. Background Art

[0002] In today's society, as a special field of public services, the scope and requirements of funeral services are gradually diversifying. When funeral vehicles provide the last farewell service for relatives, they need to complete tasks quickly and accurately in the complex urban traffic environment, which poses relatively high requirements for the optimization of the scheduling path of funeral vehicles. Most traditional funeral vehicle path scheduling methods rely on manual experience or rule-based algorithms, mainly considering factors such as the starting point, ending point, and service points on the way of the vehicle. However, these methods usually cannot adapt to the changes in traffic conditions in real time, nor can they fully consider the diversity of service requirements, resulting in low scheduling efficiency, slow response speed, and even problems such as unreasonable path planning and resource waste.

[0003] With the development of information technology, especially the application of deep reinforcement learning and optimization algorithms, new scheduling methods have begun to receive extensive attention. In recent years, the particle swarm optimization algorithm and the simulated annealing algorithm, as two common global optimization algorithms, have achieved remarkable results in path planning and scheduling optimization problems. The particle swarm optimization algorithm conducts global search by simulating the foraging behavior of bird flocks and can effectively avoid local optimal solutions. However, in complex path optimization problems, the particle swarm algorithm is easily restricted by the initial solution and local optimality, resulting in the inability to reach the global optimal solution. Although the simulated annealing algorithm can avoid falling into local optimal solutions by gradually reducing the temperature, its convergence speed is slow, and it may still converge to a suboptimal solution in the initial stage of high temperature, affecting the overall efficiency and accuracy.

[0004] Currently, scheduling methods based on the particle swarm optimization algorithm have been widely applied in many fields such as logistics, transportation, and aviation. However, in the field of funeral services, due to the particularity and complexity of the requirements, the existing scheduling methods based on particle swarm optimization still have many deficiencies. First, when facing diverse service requirements and complex environments, the traditional particle swarm optimization algorithm cannot flexibly adjust the path search strategy and is easily trapped in local optimality, resulting in inaccurate scheduling results and difficulty in meeting the needs of real-time dynamic changes. Second, the global search ability of the particle swarm optimization algorithm is weak, often relying on a large number of iterations to find a better solution, increasing the computational complexity and time cost. On the other hand, although the simulated annealing algorithm can avoid local optimality, in the scheduling of funeral vehicles, there may be problems such as too slow temperature adjustment and insufficient search ability, affecting the timeliness and accuracy of scheduling.

[0005] Therefore, the existing technical deficiencies are mainly manifested in the following aspects: on the one hand, the local optimum problem of the particle swarm optimization algorithm has not been effectively solved, and it is impossible to ensure the accuracy of the global optimum solution in complex path planning; on the other hand, the traditional simulated annealing algorithm lacks flexibility in setting temperature control and acceptance probability, resulting in poor adaptability in dynamic environments; in addition, there is still a large room for improvement in the combination of global search and local search, the convergence speed of the algorithm, and the real-time performance of path planning.

[0006] To solve these problems, the present invention proposes a path optimization scheduling method for funeral service based on deep reinforcement learning. This method combines the advantages of the particle swarm optimization algorithm and the simulated annealing algorithm. By introducing the simulated annealing mechanism, it enhances the global search ability and avoids the situation that the particle swarm optimization algorithm falls into the local optimum solution. By dynamically adjusting the temperature parameter and search strategy, this method can effectively increase the diversity of search during the global optimization process and achieve more efficient and accurate path optimization in real-time scheduling. Summary of the Invention

[0007] An object of the present invention is to propose a path optimization scheduling method for funeral service based on deep reinforcement learning. The present invention can provide an efficient and scientific optimization scheme in the path planning of funeral service vehicles, bringing significant technical value and economic benefits to practical applications.

[0008] The path optimization scheduling method for funeral service based on deep reinforcement learning according to an embodiment of the present invention includes the following steps:

[0009] S1. Collect funeral vehicle data and construct a data set;

[0010] S2. Initialize the particle swarm according to the funeral vehicle data and define a fitness function;

[0011] S3. Evaluate the fitness value of each particle in the particle swarm, initially obtain the best position and global optimal position of each particle, and update the velocity of each particle according to the historical optimal position and global optimal position of the particle;

[0012] S4. Introduce a temperature parameter through the simulated annealing algorithm, obtain the difference between the current fitness value and the fitness value of the historical optimal solution, and judge whether to accept the current solution as the historical optimal solution according to the mechanism of gradually decreasing temperature of the simulated annealing algorithm;

[0013] S5. Combine the particle swarm and the simulated annealing algorithm and introduce an adaptive learning rate adjustment mechanism to update the velocity and position of the particle, and output the historical optimal solution as the global optimal path;

[0014] S6. Iteratively execute steps S3 to S5 until the set stop criterion is met, stop the optimal path search, and output the optimal funeral vehicle path scheduling plan.

[0015] Optionally, S1 includes the following steps:

[0016] S11. Obtain the starting and ending positions of the funeral vehicle, and set multiple intermediate service points according to service requirements. Each service point includes service duration and vehicle accessibility information;

[0017] S12. Construct a data set based on the obtained funeral vehicle data.

[0018] Optionally, S2 includes the following steps:

[0019] S21. Initialize the positions and velocities of the particle swarm according to the spatial distribution and service requirements of the service points. The position of a particle is represented as a path sequence containing all service points, and the velocity is represented as the path adjustment velocity of each particle;

[0020] S22. The fitness value F of each particle i The calculation method is:

[0021]

[0022] Where, T l,l+1 is the driving time of the funeral vehicle from service point l to service point l + 1, S l is the service duration of service point l, C j is the constraint condition, n is the number of service points, and m is the number of constraint conditions.

[0023] Optionally, S3 includes the following steps:

[0024] S31. Obtain the position of each particle in the particle swarm and calculate the fitness value F i of each particle, and obtain the historical optimal position of the particle according to the fitness value and record the global optimal position

[0025] S32. Update the velocity v of each particle according to the historical optimal position of the particle and the global optimal position i :

[0026]

[0027] Where, v i (t + 1) is the velocity of particle i at time t + 1, v i(t) is the velocity of particle i at time t, w is the inertia weight, c1 and c2 are learning factors, r1 and r2 are random numbers, and x i (t) is the position of particle i at time t;

[0028] S33. Calculate the position x i (t + 1) of each particle after update according to the velocity of the particle swarm:

[0029] x i (t + 1) = x i (t) + v i (t + 1);

[0030] where x i (t) is the position of particle i at time t, and v i (t + 1) is the velocity of particle i at time t + 1;

[0031] S34. Re-calculate the fitness value F i of each particle according to the updated particle positions, and update the historical best position and the global best position

[0032] S35. Judge whether it is necessary to adjust the velocity and position of the particles according to the update situation of the fitness value, and judge whether to continue the iteration according to the algorithm stop criterion.

[0033] Optionally, the S4 includes the following steps:

[0034] S41. After each update of the particle position, introduce the temperature T k according to the simulated annealing algorithm, set the initial temperature T0 of the simulated annealing algorithm, and define the temperature drop rate α, then the temperature T k is:

[0035] T k = α · T k-1 ;

[0036] where T k-1 is the temperature of the (k - 1)-th iteration, α is the temperature drop rate, and T k is the temperature of the next iteration;

[0037] S42. Calculate the difference ΔF current between the fitness value F best of the current solution and the fitness value F

[0038] of the historical best solution: current ΔF = F best - F

[0039] S43. If ΔF < 0, accept the current solution as the new historical optimal solution. If ΔF ≥ 0, accept the current solution as the new historical optimal solution with probability P:

[0040]

[0041] where exp(.) is the natural exponential function, and T k is the current temperature.

[0042] Optionally, S5 includes the following steps:

[0043] S51. Introduce an adaptive learning rate adjustment mechanism, define the learning rate, and adjust the learning rate according to the change of temperature:

[0044]

[0045] where η k-1 is the learning rate of the current iteration step, η k is the learning rate of the next step, T k is the current temperature, and T max is the maximum value of the initial temperature;

[0046] S52. According to the velocity of the particle, combined with the adaptive learning rate, adjust the velocity of the particle. The new particle velocity v i (t + 1) is:

[0047]

[0048] where v i (t + 1) is the velocity of particle i at time t + 1, is the historical optimal position, is the global optimal position, v i (t) is the velocity of particle i at time t, and x i (t) is the position of particle i at time t. c1 and c2 are learning factors, P is the probability, r1 and r2 are random numbers, and η k is the learning rate of the current step, and ΔF is the fitness difference between the current solution and the historical optimal solution;

[0049] S53. Adopt an improved adaptive conjugate gradient method to dynamically adjust the step size of the particle:

[0050]

[0051] where S i,k is the step size of the k-th step, S i,k-1 is the step size of the (k - 1)-th step, and λ is the coefficient;

[0052] S54. Introduce a low-rank approximation of the Hessian matrix for second-order information adjustment and calculate the Hessian matrix H i of the low-rank approximation

[0053]

[0054] where x i is the position of particle i, and x j is the position of particle j, and N is the number of particles;

[0055] S55. Use the approximate information of the low-rank Hessian matrix to adjust the positions of the particles:

[0056]

[0057] where x i (t + 1) is the position of particle i at time t + 1, x i (t) is the position of particle i at time t, is the gradient of the fitness function of particle i, is the low-rank approximation of the Hessian matrix H i , and S i,k is the update step size;

[0058] S56. According to the updated particle velocities and positions, recalculate the fitness value F i of the particle, and further adjust the temperature decrease rate α according to the difference ΔF between the fitness value F current of the current solution and the fitness value F best of the historical optimal solution:

[0059]

[0060] where α0 is the initial temperature decrease rate, γ is the adjustment coefficient, and |F best | is the absolute value of the fitness value of the historical optimal solution;

[0061] S57. Gradually decrease the temperature T k according to the temperature decrease rate α until the temperature drops to the lowest value, then stop the update and output the historical optimal solution as the global optimal path.

[0062] The beneficial effects of the present invention are:

[0063] (1) The method for optimizing and scheduling the funeral service path based on deep reinforcement learning of the present invention combines the advantages of the particle swarm optimization algorithm and the simulated annealing algorithm, and effectively solves some key problems in the traditional path optimization algorithm on the basis of the existing technology. By introducing the simulated annealing mechanism, the global search ability of the particle swarm algorithm is significantly enhanced, and the situation that the particle swarm falls into the local optimal solution in the complex path planning is avoided, ensuring the global optimality of the scheduling result. This enables a more accurate path to be found in the complex funeral vehicle scheduling task, minimizing resource waste to the greatest extent and improving the timeliness of the service.

[0064] (2) The present invention effectively improves the problems of slow convergence speed and susceptibility to the initial solution in the early iteration process of the particle swarm algorithm. By dynamically adjusting the temperature parameter, the algorithm can flexibly adjust the search strategy during the search process, enhancing the adaptability of the algorithm to the dynamically changing environment. In funeral service scheduling, service demands are often uncertain and time-sensitive, and the algorithm can be adjusted in real time to ensure obtaining the optimal solution in the shortest time, improving the response speed of the entire scheduling system.

[0065] (3) The present invention not only improves the accuracy of path optimization but also greatly increases the convergence speed of the algorithm. In practical applications, the system can adjust the path planning strategy through a continuous feedback mechanism, thereby providing a fast and accurate funeral vehicle path scheduling scheme in the complex urban traffic environment. At the same time, by introducing an adaptive learning mechanism and a simulated annealing temperature adjustment mechanism, the method can effectively avoid imperfect path optimization caused by premature convergence, further enhancing the flexibility and robustness of the scheduling scheme. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The drawings are used to provide a further understanding of the present invention and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the present invention and do not constitute a limitation to the present invention. In the drawings:

[0067] Figure 1 is a flowchart of the method for optimizing and scheduling the funeral service path based on deep reinforcement learning proposed by the present invention;

[0068] Figure 2 is a flowchart of the simulated annealing algorithm in the method for optimizing and scheduling the funeral service path based on deep reinforcement learning proposed by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0069] Now, the present invention will be further described in detail with reference to the drawings. These drawings are all simplified schematic diagrams, only illustrating the basic structure of the present invention in a schematic manner, so they only show the components related to the present invention.

[0070] Reference Figure 1 - Figure 2, A method for optimizing the scheduling of funeral service paths based on deep reinforcement learning, including the following steps:

[0071] S1. Collect funeral vehicle data and construct a data set;

[0072] S2. Initialize the particle swarm according to the funeral vehicle data and define a fitness function;

[0073] S3. Evaluate the fitness value of each particle in the particle swarm, initially obtain the best position of each particle and the global optimal position, and update the velocity of each particle according to the historical optimal position and the global optimal position of the particle;

[0074] S4. Introduce a temperature parameter through the simulated annealing algorithm, obtain the difference between the current fitness value and the fitness value of the historical optimal solution, and judge whether to accept the current solution as the historical optimal solution according to the mechanism of gradually decreasing the temperature of the simulated annealing algorithm;

[0075] S5. Combine the particle swarm and the simulated annealing algorithm and introduce an adaptive learning rate adjustment mechanism to update the velocity and position of the particle, and output the historical optimal solution as the global optimal path;

[0076] S6. Iteratively execute steps S3 to S5 until the set stopping criterion is met, stop the optimization path search, and output the optimal funeral vehicle path scheduling plan.

[0077] In this embodiment, S1 includes the following steps:

[0078] S11. Obtain the starting and ending positions of the funeral vehicle, and set multiple intermediate service points according to the service requirements. Each service point includes the service duration and vehicle accessibility information;

[0079] S12. Construct a data set according to the obtained funeral vehicle data.

[0080] In this embodiment, S2 includes the following steps:

[0081] S21. Initialize the position and velocity of the particle swarm according to the spatial distribution and service requirements of the service points. The position of the particle is represented as a path sequence including all service points, and the velocity is represented as the path adjustment velocity of each particle;

[0082] S22. The fitness value F of each particle i The calculation method is:

[0083]

[0084] Among them, T l,l+1 is the driving time of the funeral vehicle from service point l to service point l + 1, S l is the service duration of service point l, C jTaking [constraint condition] as the constraint condition, n is the number of service points, and m is the number of constraint conditions.

[0085] In this embodiment, S3 includes the following steps:

[0086] S31. Obtain the position of each particle in the particle swarm and calculate the fitness value F of each particle i , and obtain the historical optimal position of the particle according to the fitness value and record the global optimal position

[0087] S32. Update the velocity v of each particle according to the historical optimal position of the particle and the global optimal position : i :

[0088]

[0089] where v i (t + 1) is the velocity of particle i at time t + 1, v i (t) is the velocity of particle i at time t, w is the inertia weight, c1 and c2 are learning factors, r1 and r2 are random numbers, and x i (t) is the position of particle i at time t;

[0090] S33. Calculate and update the position x of each particle according to the velocity of the particle swarm i (t + 1):

[0091] x i (t + 1) = x i (t) + v i (t + 1);

[0092] where x i (t) is the position of particle i at time t, and v i (t + 1) is the velocity of particle i at time t + 1;

[0093] S34. Recalculate the fitness value F of each particle according to the updated particle position i , and update the historical optimal position and the global optimal position

[0094] S35. Judge whether it is necessary to adjust the velocity and position of the particle according to the update situation of the fitness value, and judge whether to continue the iteration according to the algorithm stop criterion.

[0095] In this embodiment, S4 includes the following steps:

[0096] S41. After each update of the particle position, introduce the temperature T according to the simulated annealing algorithm. k , set the initial temperature T0 of the simulated annealing algorithm, and define the temperature decrease rate α. Then the temperature T k is:

[0097] T k = α·T k-1 ;

[0098] where T k-1 is the temperature at the (k - 1)-th iteration, α is the temperature decrease rate, and T k is the temperature at the next iteration;

[0099] S42. Calculate the difference ΔF between the fitness value F of the current solution and the fitness value F current of the historical optimal solution: best

[0100] ΔF = F current - F best ;

[0101] S43. If ΔF < 0, accept the current solution as the new historical optimal solution. If ΔF ≥ 0, accept the current solution as the new historical optimal solution with probability P:

[0102]

[0103] where exp(.) is the natural exponential function, and T k is the current temperature.

[0104] In this embodiment, S5 includes the following steps:

[0105] S51. Introduce an adaptive learning rate adjustment mechanism, define the learning rate, and adjust the learning rate according to the change of temperature:

[0106]

[0107] where η k-1 is the learning rate at the current iteration step, η k is the learning rate at the next step, T k is the current temperature, and T max is the maximum value of the initial temperature;

[0108] S52. According to the velocity of the particle, combine the adaptive learning rate to adjust the velocity of the particle. The new particle velocity v i (t + 1) is:

[0109]

[0110] where v i ​(t + 1) is the velocity of particle i at time t + 1, is the historical optimal position, is the global optimal position, v i (t) is the velocity of particle i at time t, x i (t) is the position of particle i at time t, c1 and c2 are learning factors, P is the probability, r1 and r2 are random numbers, η k is the learning rate of the current step, ΔF is the fitness difference between the current solution and the historical optimal solution;

[0111] S53. An improved adaptive conjugate gradient method is used to dynamically adjust the step size of the particle:

[0112]

[0113] where S i,k is the step size of the k-th step, S i,k-1 is the step size of the (k - 1)-th step, and λ is the coefficient;

[0114] S54. A low-rank approximation of the Hessian matrix is introduced for second-order information adjustment, and the low-rank approximation of the Hessian matrix H i is calculated

[0115]

[0116] where x i is the position of particle i, and x j is the position of particle j, and N is the number of particles;

[0117] S55. The position of the particle is adjusted using the approximate information of the low-rank Hessian matrix:

[0118]

[0119] where x i (t + 1) is the position of particle i at time t + 1, x i (t) is the position of particle i at time t, is the gradient of the fitness function of particle i, is the low-rank approximation of the Hessian matrix H i , and S i,k is the update step size;

[0120] S56. According to the updated particle velocity and position, the fitness value F i of the particle is recalculated, and the temperature decrease rate α is further adjusted according to the difference ΔF between the fitness value F current of the current solution and the fitness value F best of the historical optimal solution:

[0121]

[0122] Among them, α0 is the initial temperature drop rate, γ is the adjustment coefficient, and |F best | is the absolute value of the fitness value of the historical optimal solution;

[0123] S57. Gradually reduce the temperature T according to the temperature drop rate α k , and stop updating until the temperature drops to the lowest value, and output the historical optimal solution as the global optimal path.

[0124] Example:

[0125] An example selects a typical funeral service scenario in a large city for testing. The road network in this city is complex, and there is serious congestion during peak traffic periods. The main funeral service center in the city needs to provide services for multiple families within the urban area through funeral vehicles. The tasks of each funeral vehicle include picking up and delivering the remains and sending them to the funeral home. Considering the special nature of funeral vehicles (urgent service time, uncertain journey, frequent changes in traffic conditions), it is very difficult for traditional scheduling methods to effectively reduce the driving distance and time of vehicles while ensuring service quality.

[0126] In this scenario, the implementer applies the path optimization scheduling method for funeral vehicles based on deep reinforcement learning. Specifically, the implementer adopts the combination of the particle swarm optimization algorithm and the simulated annealing algorithm to solve the problems of local optimal solutions and slow path optimization in traditional methods. Through the temperature control mechanism of the simulated annealing algorithm and the global search ability of the particle swarm, it can avoid the problems of slow convergence and local optimality in traditional algorithms in complex traffic environments, thereby achieving global optimal path planning.

[0127] In practical applications, the system will first construct the objective function of the path optimization problem according to the starting and ending points of each funeral vehicle and the service demand points on the way. The objective function considers multiple factors, including the driving time of the vehicle, the time requirements of the service points, traffic conditions, and service time constraints, etc. The driving speed of the vehicle may be affected by factors such as traffic signals and road congestion. Therefore, the system will dynamically adjust the path selection according to real-time traffic data. At the same time, the time requirements of the service points include the start time of the funeral and the service duration after the vehicle arrives, etc., all of which need to be considered to ensure the smooth progress of the entire scheduling process. Next, the system initializes the particle swarm, sets the initial position and speed, and optimizes the path search process through the simulated annealing mechanism. The particle swarm continuously adjusts its position and speed according to the fitness value calculated each time (that is, the total cost of the path, including factors such as driving time and service duration), and finally realizes global optimal path planning.

[0128] During the process of path planning, the simulated annealing algorithm gradually reduces the temperature, introduces a certain degree of randomness, and avoids the particle swarm falling into the local optimal solution. In each iteration cycle, the algorithm calculates the update of the particle position based on the temperature and fitness value, and through repeated adjustment and optimization, ensures that the final scheduling plan of the path optimizes the overall service efficiency as much as possible.

[0129] To verify the effectiveness of the method of the present invention, the implementer conducted a one-month test in the actual urban environment. Two different funeral vehicles were selected for scheduling, and a comparative experiment of the traditional path scheduling method and the optimized path scheduling based on deep reinforcement learning was carried out respectively. During the test, the implementer simulated different traffic peak periods and emergencies (such as traffic accidents and road closures, etc.), and recorded the relevant data.

[0130] In the traditional path scheduling method, the path planning of vehicles mainly relies on manual experience or a rule-based scheduling system, while in the scheduling method based on deep reinforcement learning, the implementer adopted an algorithm combining particle swarm optimization and simulated annealing.

[0131] Table 1 shows the comparison data of the two scheduling methods under the same test scenario:

[0132] Table 1 Performance comparison between the present invention and the traditional method in the path planning of funeral vehicles

[0133]

[0134] Throughout the embodiment, the implementer not only solved the problems of path delay and resource waste caused by traffic peaks and emergencies in the traditional funeral vehicle path scheduling through the method of the present invention, but also significantly improved the optimization efficiency of path planning, achieving efficient scheduling and accurate path planning in the complex urban traffic environment.

[0135] The present invention combines the advantages of the particle swarm optimization algorithm and the simulated annealing algorithm, adopts a strategy of combining global search and local search, and avoids the limitation that the traditional particle swarm optimization algorithm is prone to fall into the local optimum. Under the guidance of the simulated annealing algorithm, the global search ability is optimized, and by dynamically adjusting the temperature parameter and acceptance probability, the diversity of the search is effectively increased, enabling the system to adjust the path planning in real time under complex traffic conditions and ensuring the realization of the global optimal path.

[0136] According to the actual application requirements, the present invention comprehensively considers multiple factors such as the driving time of the vehicle, the service duration, and the traffic conditions, and establishes a multi-objective optimization model. By dynamically adjusting the weights of the path objective function and combining with the gradually decreasing temperature process of the simulated annealing mechanism, the search for the globally optimal path is realized, while avoiding resource waste and scheduling delays. In the case of complex traffic and multiple service points, the optimization method can make adjustments according to real-time traffic information and quickly give the best path, thus ensuring the efficient scheduling of the funeral vehicle and the timeliness of the service.

[0137] Through the adaptive particle update mechanism and temperature adjustment strategy of the present invention, the path search is made more flexible, avoiding the local optimum problem caused by the randomness of the path search. Through the dynamic reinforcement mechanism, the system can adapt to the changes in different scenarios and service requirements, and optimize the scheduling scheme in real time, ensuring that the globally optimal performance of the optimization scheme is always maintained during various peak hours, emergencies, and complex environments.

[0138] The above is only the preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, making equivalent substitutions or changes, should be covered by the protection scope of the present invention.

Claims

1. A funeral service path optimization and scheduling method based on deep reinforcement learning, characterized in that It includes the following steps: S1. Collect the data of the hearse and construct a data set; S2. Initialize the particle swarm according to the hearse data and define the fitness function; S3. Evaluate the fitness value of each particle in the particle swarm, preliminarily obtain the best position of each particle and the global optimal position, and update the velocity of each particle according to the historical best position and the global optimal position of the particle; S4. Introduce the temperature parameter through the simulated annealing algorithm, obtain the difference between the current fitness value and the fitness value of the historical optimal solution, and judge whether to accept the current solution as the historical optimal solution according to the mechanism of gradually decreasing temperature in the simulated annealing algorithm; S5. Combine the particle swarm and the simulated annealing algorithm and introduce an adaptive learning rate adjustment mechanism to update the velocity and position of the particle, and output the historical optimal solution as the global optimal path; S6. Iteratively execute steps S3 to S5 until the set stop criterion is met, stop the optimization path search, and output the optimal hearse path scheduling scheme.

2. The optimized scheduling method for funeral service paths based on deep reinforcement learning according to claim 1, wherein The S1 includes the following steps: S11. Obtain the starting point and ending point positions of the hearse, and set multiple intermediate service points according to the service requirements. Each service point includes the service duration and vehicle accessibility information; S12. Construct a data set according to the obtained hearse data.

3. The optimized scheduling method for funeral service paths based on deep reinforcement learning according to claim 1, wherein, The S2 includes the following steps: S21. Initialize the position and velocity of the particle swarm according to the spatial distribution and service requirements of the service points. The position of the particle is represented as a path sequence containing all service points, and the velocity is represented as the path adjustment velocity of each particle; S22. Fitness value F of each particle i The calculation method is as follows: Among them, T l,l+1 is the driving time of the hearse from service point l to service point l + 1, S l is the service duration of service point l, C j is the constraint condition, n is the number of service points, and m is the number of constraint conditions.

4. The funeral service path optimization and scheduling method based on deep reinforcement learning according to claim 1, characterized in that The S3 includes the following steps: S31. Obtain the position of each particle in the particle swarm and calculate the fitness value F of each particle i , obtain the historical optimal position of the particle according to the fitness value and record the global optimal position S32. Update the velocity v of each particle according to the historical optimal position of the particle and the global optimal position as follows: i : Among them, v i (t + 1) is the velocity of particle i at time t + 1, v i (t) is the velocity of particle i at time t, w is the inertia weight, c1 and c2 are learning factors, r1 and r2 are random numbers, x i (t) is the position of particle i at time t; S33. Calculate the position x of each particle after update according to the velocity of the particle swarm i (t + 1): x i (t + 1)= x i (t)+ v i (t + 1); where x i (t) is the position of particle i at time t, and v i (t + 1) is the velocity of particle i at time t + 1; S34. Recalculate the fitness value F of each particle according to the updated particle positions i , and update the historical best position and the global best position S35. Judge whether it is necessary to adjust the velocity and position of the particle according to the update situation of the fitness value, and judge whether to continue the iteration according to the algorithm stop criterion.

5. The optimized scheduling method for the funeral service path based on deep reinforcement learning according to claim 1, characterized in that The S4 includes the following steps: S41. After each update of the particle position, introduce the temperature T according to the simulated annealing algorithm k , set the initial temperature T0 of the simulated annealing algorithm, and define the rate α of temperature decrease, then the temperature T k is as follows: T k = α · T k-1 ; where, T k-1 is the temperature of the (k - 1)-th iteration, α is the temperature decrease rate, and T k is the temperature of the next iteration; S42. Calculate the fitness value F of the current solution current and the fitness value F best of the historical optimal solution, and the difference ΔF therebetween: ΔF = F current -F best ; S43. If ΔF < 0, accept the current solution as the new historical optimal solution. If ΔF ≥ 0, accept the current solution as the new historical optimal solution with probability P: where exp(.) is the natural exponential function, and T k is the current temperature.

6. The funeral service path optimization scheduling method based on deep reinforcement learning according to claim 1, characterized in that The S5 includes the following steps: S51. Introduce an adaptive learning rate adjustment mechanism, define the learning rate, and adjust the learning rate according to the change of temperature: Among them, η k-1 is the learning rate of the current iteration step, η k is the learning rate of the next step, T k is the current temperature, T max is the maximum value of the initial temperature; S52. Adjust the velocity of the particle according to the velocity of the particle and in combination with the adaptive learning rate. The new particle velocity v i (t + 1) is as follows: Among them, v i (t + 1) is the velocity of particle i at time t + 1, is the historical best position, is the global best position, v i (t) is the velocity of particle i at time t, x i (t) is the position of particle i at time t, c1 and c2 are learning factors, P is the probability, r1 and r2 are random numbers, η k is the learning rate of the current step, and ΔF is the fitness difference between the current solution and the historical best solution; S53. Adopt an improved adaptive conjugate gradient method to dynamically adjust the step size of the particle: Among them, S i,k is the step size at the k-th step, and S i,k-1 is the step size at the (k - 1)-th step, and λ is a coefficient; S54. Introduce a low-rank approximation of the Hessian matrix for second-order information adjustment and calculate the low-rank approximation of the Hessian matrix H i of where x i is the position of particle i, and x j is the position of particle j, and N is the number of particles; S55. Use the approximate information of the low-rank Hessian matrix to adjust the position of the particle: where x i (t + 1) is the position of particle i at time t + 1, and x i (t) is the position of particle i at time t, is the gradient of the fitness function of particle i, is the low-rank approximation of the Hessian matrix H i , and S i,k is the update step size; S56. Recalculate the fitness value F of the particle according to the updated particle velocity and position i , and further adjust the temperature decrease rate α according to the difference ΔF between the fitness value F current of the current solution and the historical optimal solution F best : where α0 is the initial temperature decrease rate, γ is the adjustment coefficient, and |F best | is the absolute value of the fitness value of the historical optimal solution; S57. Gradually decrease the temperature T according to the temperature decrease rate α k until the temperature drops to the lowest value, then stop the update and output the historical optimal solution as the global optimal path.

Images