Large-range garbage collection path planning method and device

Abstract

The invention discloses a large-range garbage collection path planning method and device. The method comprises the following steps: S1, constructing a path planning data set; s2, a path planning model is constructed based on hierarchical reinforcement learning, the path planning model comprises an upper reinforcement learning model and a lower reinforcement learning model, and the upper reinforcement learning model performs global path planning in combination with the path planning data set; s3, dynamically adjusting the service priority of the garbage can through a multi-level priority scheduling mechanism in the lower-layer reinforcement learning model; s4, acquiring garbage can state data and road traffic condition data which are monitored in real time, and updating the priority scheduling mechanism; s5, in the driving process of the garbage truck, local path dynamic adjustment is carried out through a lower-layer reinforcement learning model; and S6, dynamically adjusting a garbage collection path and a priority scheduling mechanism. According to the method, efficient fusion of global path planning and local path dynamic adjustment is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of garbage collection, and in particular to a large-scale garbage collection path planning method and device. Background Art

[0002] In the prior art, urban garbage collection route planning usually relies on static algorithms or predefined scheduling schemes. Traditional static algorithms are generally based on fixed geographical locations and garbage bin distribution information, taking into account the capacity of garbage trucks, driving distance, and road traffic conditions to formulate collection routes.

[0003] Existing technologies face great limitations in dealing with dynamic urban environments. During the garbage collection process, the overflow of garbage bins, road traffic conditions, and emergencies in the city (traffic jams, road construction, etc.) are constantly changing, and existing technologies cannot dynamically respond to changes, resulting in garbage trucks often encountering problems such as failure to optimize driving routes, extended collection time, and increased energy consumption during actual execution. In other words, existing technologies lack real-time adjustment capabilities. When garbage trucks encounter traffic jams or the status of garbage bins changes, traditional algorithms cannot quickly adjust routes, resulting in garbage bins overflowing or garbage trucks taking long detours.

[0004] In addition, in large-scale urban garbage collection scenarios, how to simultaneously optimize both global and local paths is also a difficulty. Existing technologies are mostly based on global planning and lack a rapid response mechanism to local emergencies, which in turn reduces the efficiency of garbage collection.

[0005] Thirdly, the existing technology also has defects in priority scheduling. The collection order of garbage bins is usually set based on fixed rules, which fails to fully consider factors such as the risk of overflowing garbage bins, the importance of the area, and traffic congestion. As a result, garbage bins in high-priority areas may overflow due to failure to clean them in time, thereby affecting the city's environmental sanitation.

[0006] In general, the existing technologies have the following problems: First, the existing garbage collection path planning technology lacks dynamic adjustment capabilities and cannot respond to emergencies in the urban environment in real time; second, when dealing with large-scale urban environments, there is a lack of effective global and local path optimization mechanisms; finally, the existing technology is not flexible enough in garbage bin priority scheduling and fails to optimize and adjust according to the actual garbage bin status and regional needs. The existence of these problems not only increases the cost of garbage collection, but also reduces the efficiency of garbage collection and affects the cleanliness of the urban environment.

[0007] In view of this, this invention is proposed. Summary of the invention

[0008] The purpose of the present invention is to solve the shortcomings existing in the prior art, and to propose a large-scale garbage collection path planning method and device, which realizes the efficient integration of global path planning and local path dynamic adjustment, and can significantly improve the timeliness and overall efficiency of garbage collection, and by dynamically adjusting the parameters of the path planning model, make subsequent path planning more accurate.

[0009] In order to achieve the above object, the present invention also adopts the following technical solutions:

[0010] A large-scale garbage collection path planning method, comprising:

[0011] S1: Obtain the garbage bin data, road traffic data and garbage truck operation status data in the collection area, and construct a path planning dataset;

[0012] S2: Building a path planning model based on hierarchical reinforcement learning, including an upper-layer reinforcement learning model and a lower-layer reinforcement learning model. The upper-layer reinforcement learning model combines the path planning dataset to perform global path planning and generate a preliminary global optimal garbage collection path.

[0013] S3: In the lower reinforcement learning model, the service priority of the trash can is dynamically adjusted according to the overflow risk of the trash can, the regional population density and the importance of the region through a multi-level priority scheduling mechanism;

[0014] S4: Obtain real-time monitored trash bin status data and road traffic condition data to update the priority scheduling mechanism and reclassify the trash bins;

[0015] S5: During the driving process of the garbage truck, the local path is dynamically adjusted according to road congestion, changes in the status of garbage bins and emergencies through the lower-level reinforcement learning model;

[0016] S6: Feed the real-time path planning dataset and the historical path planning dataset back to the path planning model to dynamically adjust the garbage collection path and priority scheduling mechanism;

[0017] S7: Repeat the process of S1-S6 until the garbage collection work of all garbage bins in the urban area is completed.

[0018] Further, the S1 comprises the following steps:

[0019] S11: Obtain the geographic location L of each trash bin in the collection area i =(x i ,y i ), Capacity C i and historical garbage generation data G i (t), where i is the number of the trash can, x i and iis the latitude and longitude coordinates of the trash can, and t is the time;

[0020] S12: Obtain road traffic status data and construct a road network diagram:

[0021] G = (V, E);

[0022] Among them, V is the set of road network nodes, E is the set of road network edges, and each edge e jk ∈E has weight w jk , w jk represents the travel time or distance of the road segment from node j to node k;

[0023] S13: Get the current location L of the garbage truck c =(x c ,y c ), Load capacity C c and fuel consumption rate F c , where x c and c are the latitude and longitude coordinates of the garbage truck;

[0024] S14: Construct a path planning dataset based on the data obtained from S11-S13:

[0025] D={(L i ,C i ,G i (t)),(G,w jk ),(L c ,C c ,F c )}}.

[0026] Further, the S2 comprises the following steps:

[0027] S21: Construct path planning model M based on hierarchical reinforcement learning, including upper reinforcement learning model M upper and the underlying reinforcement learning model M lower ;

[0028] S22: Define the upper reinforcement learning model M upper The global state space S upper , action space A upper and the reward function R upper :

[0029] The global state space is defined as:

[0030]

[0031] in, is the average amount of garbage generated by trash bin i in time period T1;

[0032] Action Space A upper is the service order set of garbage bins in the global scope, which represents the optional route sequence of garbage trucks in global path planning;

[0033] Reward function R upper Defined as:

[0034]

[0035] Among them, α1, β1, γ1 are weight coefficients, s and a are states and actions, D(L c ,L i ) is the Euclidean distance between the current position of the garbage truck and garbage bin i, is the overflow risk factor of trash can i, E a is the set of edges in the path corresponding to action a;

[0036] S23. Use reinforcement learning algorithm to strengthen the upper reinforcement learning model M upper For training, use the strategy function π upper (s) Select the optimal action a to maximize the expected cumulative reward:

[0037]

[0038] Among them, γ upper is the discount factor, s t and a t are the state and action at time step t, respectively, and T2 is the time range of global planning;

[0039] S24. Optimal strategy based on training Generate a preliminary globally optimal garbage collection path:

[0040]

[0041] And the path P global Passed to the lower reinforcement learning model M lower Dynamically adjust and optimize local paths.

[0042] Further, the S3 comprises the following steps:

[0043] S31: Calculate the comprehensive service priority P of each trash bin i i :

[0044] P i =λ1R i +λ2D i +λ3I i ;

[0045] Among them, λ1, λ2, and λ3 are weight coefficients, satisfying λ1+λ2+λ3=1, is the overflow risk factor of trash can i, D i is the population density coefficient of the area where the trash can i is located, I i is the importance coefficient of the area where trash bin i is located;

[0046] S32: Based on the calculated comprehensive service priority P i For the preliminary global optimal garbage collection path P global The trash cans in the stub are reordered to form the adjusted path P adjusted ;

[0047] S33: Reinforce the learning model M at the lower level lower In the definition of local state space S lower , action space A lower and the reward function R lower :

[0048] The local state space is defined as:

[0049] S lower ={L c ,L i ,P i};

[0050] Action Space A lower The optional driving direction or next target trash can in the local area;

[0051] Reward function R lower Defined as:

[0052] R lower (s,a)=-(α2D(L c ,L i )-β2P i +γ2w jk );

[0053] Among them, α2, β2, and γ2 are weight coefficients;

[0054] S34: Use reinforcement learning algorithm to strengthen the lower layer reinforcement learning model M lower For training, use the strategy function π lower (s) Select the optimal action a to maximize the expected cumulative reward:

[0055]

[0056] Among them, γ lower is the discount factor, T3 is the time range of local planning;

[0057] S35: Optimal strategy based on training The adjusted path P adjusted Perform local optimization to generate the final global optimal garbage collection path P final .

[0058] Further, the S36 comprises the following steps:

[0059] S361: Using the best strategy For the adjusted path P adjusted Perform local optimization and select the current garbage truck position L c and the trash can L in the path i The optimal driving path between generates the local optimal path P local ;

[0060] S362: Calculate the optimal strategy for the local path through the reinforcement learning algorithm and define the local reward function R local (s,a) is to minimize the energy consumption and time of the garbage truck during driving, while maximizing the chance of processing high-priority garbage bins;

[0061] S363: Optimal strategy based on local paths Generate the final global optimal garbage collection path P final , where the garbage truck takes precedence over the adjusted path P adjusted Drive and use the strategy function π according to the real-time situation lower (s) Select the local optimal path P local :

[0062]

[0063] in, is the cumulative discounted sum at time step t, is the discount ratio of the short-term reward to the long-term reward in the future time step, is the travel time of the garbage truck on the local path, j is the jth garbage bin visited by the garbage truck on the local path, n is the total number of garbage bins visited by the garbage truck on the local path, D(L c,j ,L i,j ) is the current position of the garbage truck L c,j With trash can L i,j The Euclidean distance, v j (t) is the speed of the garbage truck at time t, is the impact of road conditions on the local path, w kl (t) represents the edge e at time t kl The weight on f(L k ,L l ) is a function that reflects the road traffic conditions.

[0064] Further, the S5 comprises the following steps:

[0065] S51: During the driving of the garbage truck, based on the lower layer reinforcement learning model M lower The dynamic adjustment mechanism obtains the road congestion situation, trash can status changes and emergency situations in real time, and constructs the real-time state space S real-time ;

[0066] S52: According to the real-time state space S real-time Define the local dynamically adjusted reward function R adjust (s,a) is to minimize the driving time and energy consumption caused by detours while ensuring that the garbage truck does not deviate from the global path planning goal:

[0067]

[0068] Among them, α3, α4, β2, β3 weight coefficients, T c,i is the time it takes for the garbage truck to reach garbage bin i, O jk is the road congestion coefficient from node j to node k, R i is the risk factor of the trash can overflowing;

[0069] S53: Use reinforcement learning algorithm to calculate the local dynamic adjustment strategy, combine the real-time state space and reward function, and generate the local optimal adjustment path P adjust-real-time , the local optimal adjustment path bypasses the congested area and optimizes the driving efficiency of the garbage truck.

[0070] Further, the S6 comprises the following steps:

[0071] S61: During the garbage collection task execution, the driving path of the garbage truck, the road traffic conditions, and the changes in the status of the garbage bins are collected in real time, and a real-time path planning dataset is constructed;

[0072] S62: Obtain a historical path planning dataset, including previous garbage truck driving paths, garbage bin collection conditions, and road traffic records;

[0073] S63: Compare and analyze the real-time path planning data set with the historical path planning data set, identify the deviation between the current path planning and the historical planning, and determine the main factors affecting the path deviation;

[0074] S64: Based on the analysis results of real-time and historical data, dynamically adjust the path planning model, update the parameters in the path planning model, and optimize the priority scheduling mechanism of the trash can;

[0075] S65: Self-optimize the adjusted path planning model and train the path planning model using real-time feedback data.

[0076] Further, the S7 comprises the following steps:

[0077] S71: After completing the collection task of the garbage bins in the current collection area, return to step S1, continue to obtain the garbage bin data, road traffic status data and garbage truck operation status data in the next collection area, and update the path planning data set D new ;

[0078] S72: Based on the updated path planning data set D new The hierarchical reinforcement learning model is used again to perform global path planning and local path dynamic adjustment, and steps S2 to S6 are repeated to generate a new global optimal garbage collection path P. new-final and the local optimal path P new-local ;

[0079] S73: Real-time monitoring of the status of trash cans and road traffic conditions in the urban area, and dynamic adjustment of the priority and path planning of trash cans;

[0080] S74: Continue to execute the process of S71-S73 until the garbage collection task of all garbage bins in the urban area is completed, and finally generate a garbage collection path P covering the entire area final-global .

[0081] In order to achieve the above object, the present invention also adopts the following technical solutions:

[0082] A large-scale garbage collection path planning device, comprising:

[0083] The data collection module is used to obtain the garbage bin data, road traffic condition data and garbage truck operation status data in the collection area, and construct a path planning data set;

[0084] The path planning module is used to build a path planning model based on hierarchical reinforcement learning. The path planning model includes an upper-layer reinforcement learning model and a lower-layer reinforcement learning model. The upper-layer reinforcement learning model combines the path planning data set to perform global path planning and generate a preliminary global optimal garbage collection path. The lower-layer reinforcement learning model dynamically adjusts the local path according to road congestion, changes in the status of garbage bins, and emergencies.

[0085] The priority scheduling module is used to dynamically adjust the service priority of the trash cans according to the overflow risk of the trash cans, the regional population density and the importance of the region through a multi-level priority scheduling mechanism in the lower-level reinforcement learning model;

[0086] The path optimization module is used to feed back the real-time path planning dataset and the historical path planning dataset to the path planning model to dynamically adjust the garbage collection path and priority scheduling mechanism;

[0087] The control module is used to transmit the final generated garbage collection path to the garbage truck scheduling system, control the garbage truck to collect garbage according to the optimized path, and dynamically adjust the path according to the real-time situation until the garbage collection task of all garbage bins in the urban area is completed.

[0088] In order to achieve the above object, the present invention also adopts the following technical solution:

[0089] A large-scale garbage collection path planning device is characterized by comprising: a memory and a processor, the memory is used to store instructions, and the instructions are used to control the processor to operate to execute any one of the methods provided by the present invention.

[0090] Compared with the prior art, the present invention has the following beneficial effects:

[0091] 1. A hierarchical reinforcement learning model is adopted. Through the division of labor and cooperation between the upper and lower models, an efficient integration of global path planning and local path dynamic adjustment is achieved. The upper reinforcement learning model is responsible for planning the global optimal path and optimizing the path based on historical data and real-time environmental data. The lower model makes real-time dynamic adjustments based on actual road congestion and sudden changes in the status of garbage bins. When dealing with complex and dynamic urban environments, the hierarchical structure can quickly adjust local paths to avoid delays caused by congestion or emergencies in garbage trucks, making garbage collection more flexible and efficient.

[0092] 2. The introduced multi-level priority scheduling mechanism can dynamically adjust the service priority of garbage bins according to multiple factors such as the overflow risk of garbage bins, regional population density, and regional importance. By comprehensively evaluating the service urgency of each garbage bin, it can give priority to garbage bins with higher overflow risks or in important areas, avoiding urban environmental problems caused by overflowing garbage bins. Compared with the traditional static priority scheduling method, it significantly improves the timeliness and overall efficiency of garbage collection.

[0093] 3. Obtain real-time monitoring data and dynamic adjustment mechanism. By real-time monitoring of road traffic, garbage bin status and emergencies, the garbage truck can flexibly adjust its path during driving. The lower-level reinforcement learning model dynamically adjusts the local path according to the real-time situation, giving priority to emergency garbage bins, while ensuring that the garbage truck does not deviate from the global optimal path. This mechanism ensures that the garbage truck can still operate efficiently in a complex and dynamic environment.

[0094] 4. By combining the real-time path planning data set with the historical path planning data set, the path planning model is dynamically adjusted and self-optimized through path planning deviation analysis. Each time the garbage truck completes the collection task, the system will compare the actual execution path with the planned path, analyze the reasons for the deviation, and then dynamically adjust the parameters of the path planning model to make subsequent path planning more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0095] Figure 1 A flowchart of a large-scale garbage collection path planning method;

[0096] Figure 2 It is a structural schematic diagram of a large-scale garbage collection path planning device;

[0097] Figure 3 The figure is a schematic diagram of the hardware structure of a large-scale garbage collection path planning device. DETAILED DESCRIPTION

[0098] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0099] Embodiment 1:

[0100] A large-scale garbage collection path planning method, such as Figure 1 As shown, including:

[0101] S1: Obtain the garbage bin data, road traffic condition data and garbage truck operation status data in the collection area, and build a path planning dataset.

[0102] S2: Construct a path planning model based on hierarchical reinforcement learning, including an upper-level reinforcement learning model and a lower-level reinforcement learning model. The upper-level reinforcement learning model combines the path planning dataset to perform global path planning and generate a preliminary global optimal garbage collection path.

[0103] S3: In the lower reinforcement learning model, the service priority of the trash can is dynamically adjusted according to the overflow risk of the trash can, the regional population density and the importance of the area through a multi-level priority scheduling mechanism.

[0104] S4: Obtain real-time monitored trash can status data and road traffic condition data to update the priority scheduling mechanism and reclassify the trash cans.

[0105] In this embodiment, real-time monitored garbage bin status data and road traffic condition data are obtained to update the priority scheduling mechanism and reclassify the garbage bins, so that the garbage truck can give priority to processing garbage bins in emergency overflow areas during driving without deviating from the global path planning goal.

[0106] S5: During the driving process of the garbage truck, the local path is dynamically adjusted according to road congestion, changes in the status of garbage bins and emergencies through the lower-level reinforcement learning model.

[0107] In this embodiment, during the driving process of the garbage truck, the lower-level reinforcement learning model dynamically adjusts the local path according to road congestion, changes in the status of garbage bins and emergencies, so that the garbage truck can bypass the congested area.

[0108] S6: Feed the real-time path planning dataset and the historical path planning dataset back to the path planning model to dynamically adjust the garbage collection path and priority scheduling mechanism.

[0109] In this embodiment, the real-time path planning data set and the historical path planning data set are fed back to the path planning model, and the garbage collection path and priority scheduling mechanism are dynamically adjusted, so that the path planning model can be self-optimized.

[0110] S7: Repeat the process of S1-S6 until the garbage collection work of all garbage bins in the urban area is completed.

[0111] The method of this embodiment adopts a hierarchical reinforcement learning model, and realizes the efficient integration of global path planning and local path dynamic adjustment through the division of labor and cooperation between the upper and lower models. The upper reinforcement learning model is responsible for the planning of the global optimal path and optimizes the path based on historical data and real-time environmental data. The lower model makes real-time dynamic adjustments according to actual road congestion and sudden changes in the status of garbage bins. When dealing with complex and dynamic urban environments, the hierarchical structure can quickly adjust local paths to avoid delays caused by congestion or emergencies in garbage trucks, making garbage collection more flexible and efficient.

[0112] The method of this embodiment introduces a multi-level priority scheduling mechanism, which can dynamically adjust the service priority of the trash cans according to multiple factors such as the overflow risk of the trash cans, the regional population density, and the importance of the region. By comprehensively evaluating the service urgency of each trash can, it can give priority to trash cans with a higher overflow risk or in important areas, thereby avoiding urban environmental problems caused by overflowing trash cans. Compared with the traditional static priority scheduling method, it significantly improves the timeliness and overall efficiency of garbage collection.

[0113] The method of this embodiment obtains real-time monitoring data and a dynamic adjustment mechanism. By real-time monitoring of road traffic, trash can status and emergencies, the garbage truck can flexibly adjust its path during driving. The lower-level reinforcement learning model dynamically adjusts the local path according to the real-time situation, giving priority to emergency trash cans, while ensuring that the garbage truck does not deviate from the global optimal path. This mechanism ensures that the garbage truck can still operate efficiently in a complex dynamic environment.

[0114] The method of this embodiment combines the real-time path planning data set with the historical path planning data set, and dynamically adjusts and self-optimizes the path planning model through path planning deviation analysis. Each time the garbage truck completes the collection task, the system will compare the actual execution path with the planned path, and analyze the reasons for the deviation, and then dynamically adjust the parameters of the path planning model to make subsequent path planning more accurate.

[0115] In an optional embodiment, the S1 comprises the following steps:

[0116] S11: Obtain the geographic location L of each trash bin in the collection area i =(x i ,y i ), Capacity C i and historical garbage generation data G i (t), where i is the number of the trash can, x i and i is the latitude and longitude coordinates of the trash can, and t is the time;

[0117] S12: Obtain road traffic condition data and construct a road network diagram:

[0118] G = (V, E);

[0119] Among them, V is the set of network nodes, E is the set of network edges, and a network edge is formed between two adjacent network nodes. jk ∈E has weight w jk , w jk Indicates the travel time or distance of the road section from node j to node k. In specific implementation, the road traffic status data can be obtained through software such as Amap;

[0120] S13: Get the current location L of the garbage truck c =(x c ,y c ), Load capacity C c and fuel consumption rate F c , where x c and c are the latitude and longitude coordinates of the garbage truck;

[0121] S14: Construct a path planning dataset based on the data obtained from S11-S13:

[0122] D={(L i ,C i ,G i (t)),(G,w jk ),(L c ,C c ,F c )}}.

[0123] In an optional embodiment, the S2 comprises the following steps:

[0124] S21: Construct path planning model M based on hierarchical reinforcement learning, including upper reinforcement learning model M upper and the underlying reinforcement learning model M lower ;

[0125] S22: Define the upper reinforcement learning model M upper The global state space S upper , action space A upper and the reward function R upper :

[0126] The global state space is defined as:

[0127]

[0128] in, is the average amount of garbage generated by trash bin i in time period T1;

[0129] Action Space A upper is the service order set of garbage bins in the global scope, which represents the optional route sequence of garbage trucks in global path planning;

[0130] Reward function R upper Defined as:

[0131]

[0132] Among them, s is the current state in the reward function, a is the action in the reward function, α1, β1, γ1 are weight coefficients, is the Euclidean distance between the current position of the garbage truck and garbage bin i, is the overflow risk factor of trash can i, E a is the set of edges in the path corresponding to action a;

[0133] S23. Use reinforcement learning algorithm to strengthen the upper reinforcement learning model M upper For training, use the strategy function π upper(s) Select the optimal action a to maximize the expected cumulative reward:

[0134]

[0135] Among them, γ upper is the discount factor, s t and a t are the state and action at time step t, respectively, and T2 is the time range of global planning;

[0136] S24. Optimal strategy based on training Generate a preliminary globally optimal garbage collection path:

[0137]

[0138] And the path P global Passed to the lower reinforcement learning model M lower Dynamically adjust and optimize local paths.

[0139] In an optional embodiment, S3 includes the following steps:

[0140] S31: Calculate the comprehensive service priority P of each trash bin i i :

[0141] P i =λ1R i +λ2D i +λ3I i ;

[0142] Among them, λ1, λ2, and λ3 are weight coefficients, satisfying λ1+λ2+λ3=1, is the overflow risk factor of trash can i, D i is the population density coefficient of the area where the trash can i is located, I i is the importance coefficient of the area where trash bin i is located;

[0143] S32: Based on the calculated comprehensive service priority P i For the preliminary global optimal garbage collection path P global The trash cans in the stub are reordered to form the adjusted path P adjusted ; Make the garbage truck give priority to serving the garbage bins with high comprehensive service priority in local path planning;

[0144] S33: Reinforce the learning model M at the lower level lower In the definition of local state space S lower , action space A lower and the reward function R lower :

[0145] The local state space is defined as:

[0146] S lower ={L c ,L i ,P i};

[0147] Action Space A lower The optional driving direction or next target trash can in the local area;

[0148] Reward function R lower Defined as:

[0149] R lower (s,a)=-(α2D(L c ,L i )-β2P i +γ2w jk );

[0150] Among them, α2, β2, γ2 are weight coefficients; D(L c ,L i ) is the Euclidean distance between the current position of the garbage truck and the target garbage bin i;

[0151] S34: Use reinforcement learning algorithm to strengthen the lower layer reinforcement learning model M lower For training, use the strategy function π lower (s) Select the optimal action a to maximize the expected cumulative reward:

[0152]

[0153] Among them, γ lower is the discount factor, T3 is the time range of local planning;

[0154] S35: Optimal strategy based on training For the adjusted path P adjusted Perform local optimization to generate the final global optimal garbage collection path P final , so that the garbage truck gives priority to garbage bins with high comprehensive service priority during driving, while not deviating from the global path planning goal.

[0155] In an optional embodiment, the S36 includes the following steps:

[0156] S361: Using the best strategy For the adjusted path P adjusted Perform local optimization and select the current garbage truck position L c and the trash can L in the path i The optimal driving path between generates the local optimal path P local ;

[0157] S362: Calculate the optimal strategy for the local path through the reinforcement learning algorithm and define the local reward function R local (s,a) is to minimize the energy consumption and time of the garbage truck during driving, while maximizing the chance of processing high-priority garbage bins;

[0158] S363: Optimal strategy based on local paths Generate the final global optimal garbage collection path P final , where the garbage truck takes precedence over the adjusted path P adjusted Drive and use the strategy function π according to the real-time situation lower (s) Select the local optimal path P local :

[0159]

[0160] in, is the cumulative discounted sum at time step t, is the discount ratio of the short-term reward to the long-term reward in the future time step, is the travel time of the garbage truck on the local path, j is the jth garbage bin visited by the garbage truck on the local path, n is the total number of garbage bins visited by the garbage truck on the local path, that is, the garbage bins that the garbage truck needs to collect garbage on the local path, D(L c,j ,L i,j ) is the current position of the garbage truck L c,j With trash can L i,j The Euclidean distance, v j (t) is the speed of the garbage truck at time t, is the impact of road conditions on the local path, w kl (t) represents the edge e at time t kl The weight on f(L k ,L l ) is a function that reflects the road traffic conditions.

[0161] In an optional embodiment, the S5 comprises the following steps:

[0162] S51: During the driving of the garbage truck, based on the lower layer reinforcement learning model M lower The dynamic adjustment mechanism obtains the road congestion situation, trash can status changes and emergency situations in real time, and constructs the real-time state space S real-time ;

[0163] S52: According to the real-time state space S real-time Define the local dynamically adjusted reward function R adjust(s,a) is to minimize the driving time and energy consumption caused by detours while ensuring that the garbage truck does not deviate from the global path planning goal:

[0164]

[0165] Among them, α3, α4, β2, β3 weight coefficients, T c,i is the time it takes for the garbage truck to reach garbage bin i, O jk is the road congestion coefficient from node j to node k, R i is the risk factor of the trash can overflowing;

[0166] S53: Use reinforcement learning algorithm to calculate the local dynamic adjustment strategy, combine the real-time state space and reward function, and generate the local optimal adjustment path P adjust-real-time , the local optimal adjustment path bypasses the congested area and optimizes the driving efficiency of the garbage truck.

[0167] In an optional embodiment, the S6 comprises the following steps:

[0168] S61: During the garbage collection task execution, the driving path of the garbage truck, the road traffic conditions, and the changes in the status of the garbage bins are collected in real time, and a real-time path planning dataset is constructed;

[0169] S62: Obtain a historical path planning dataset, including previous garbage truck driving routes, garbage bin collection conditions, and road traffic records; and form a historical path planning dataset for comparative analysis;

[0170] S63: Compare and analyze the real-time path planning data set with the historical path planning data set, identify the deviation between the current path planning and the historical planning, and determine the main factors affecting the path deviation;

[0171] S64: Based on the analysis results of real-time and historical data, the path planning model is dynamically adjusted, the parameters in the path planning model are updated, and the priority scheduling mechanism of the garbage bin is optimized to adapt to the current road and garbage collection conditions;

[0172] S65: Self-optimize the adjusted path planning model and train the path planning model using real-time feedback data.

[0173] In an optional embodiment, the S7 comprises the following steps:

[0174] S71: After completing the collection task of the garbage bins in the current collection area, return to step S1, continue to obtain the garbage bin data, road traffic status data and garbage truck operation status data in the next collection area, and update the path planning data set D new ;

[0175] S72: Based on the updated path planning data set D new The hierarchical reinforcement learning model is used again to perform global path planning and local path dynamic adjustment, and steps S2 to S6 are repeated to generate a new global optimal garbage collection path P. new-final and the local optimal path P new-local ;

[0176] S73: Real-time monitoring of the status of trash cans and road traffic conditions in the urban area, and dynamic adjustment of the priority and path planning of trash cans;

[0177] S74: Continue to execute the process of S71-S73 until the garbage collection task of all garbage bins in the urban area is completed, and finally generate a garbage collection path P covering the entire area final-global .

[0178] Embodiment 2:

[0179] A large-scale garbage collection path planning device, such as Figure 2 As shown, including:

[0180] The data collection module is used to obtain the garbage bin data, road traffic condition data and garbage truck operation status data in the collection area, and construct a path planning data set;

[0181] The path planning module is used to build a path planning model based on hierarchical reinforcement learning. The path planning model includes an upper-layer reinforcement learning model and a lower-layer reinforcement learning model. The upper-layer reinforcement learning model combines the path planning data set to perform global path planning and generate a preliminary global optimal garbage collection path. The lower-layer reinforcement learning model dynamically adjusts the local path according to road congestion, changes in the status of garbage bins, and emergencies.

[0182] The priority scheduling module is used to dynamically adjust the service priority of the trash cans according to the overflow risk of the trash cans, the regional population density and the importance of the region through a multi-level priority scheduling mechanism in the lower-level reinforcement learning model;

[0183] The path optimization module is used to feed back the real-time path planning data set and the historical path planning data set to the path planning model, dynamically adjust the garbage collection path and priority scheduling mechanism, and then self-optimize the path planning model;

[0184] The control module is used to transmit the final generated garbage collection path to the garbage truck scheduling system, control the garbage truck to collect garbage according to the optimized path, and dynamically adjust the path according to the real-time situation until the garbage collection task of all garbage bins in the urban area is completed.

[0185] like Figure 3The figure shows a hardware structure diagram of a large-scale garbage collection path planning device, including: a memory and a processor, the memory is used to store instructions, and the instructions are used to control the processor to operate to execute any method according to the first embodiment. Those skilled in the art can design instructions according to the technical solution disclosed in the present invention. How instructions control the processor to operate is well known in the art, so the embodiments of the present invention will not be described in detail here.

[0186] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A large-scale garbage collection path planning method, characterized in that: include: S1: Obtain the garbage bin data, road traffic data and garbage truck operation status data in the collection area, and construct a path planning dataset; S2: Building a path planning model based on hierarchical reinforcement learning, including an upper-layer reinforcement learning model and a lower-layer reinforcement learning model. The upper-layer reinforcement learning model combines the path planning dataset to perform global path planning and generate a preliminary global optimal garbage collection path. S3: In the lower reinforcement learning model, the service priority of the trash can is dynamically adjusted according to the overflow risk of the trash can, the regional population density and the importance of the region through a multi-level priority scheduling mechanism; S4: Obtain real-time monitored trash bin status data and road traffic condition data to update the priority scheduling mechanism and reclassify the trash bins; S5: During the driving process of the garbage truck, the local path is dynamically adjusted according to road congestion, changes in the status of garbage bins and emergencies through the lower-level reinforcement learning model; S6: Feed the real-time path planning dataset and the historical path planning dataset back to the path planning model to dynamically adjust the garbage collection path and priority scheduling mechanism; S7: Repeat the process of S1-S6 until the garbage collection work of all garbage bins in the urban area is completed.

2. A large-scale garbage collection path planning method according to claim 1, characterized in that: The S1 comprises the following steps: S11: Obtain the geographic location L of each trash bin in the collection area i =(x i ,y i ), Capacity C i and historical garbage generation data G i (t), where i is the number of the trash can, x i and i is the latitude and longitude coordinates of the trash can, and t is the time; S12: Obtain road traffic status data and construct a road network diagram: G = (V, E); Among them, V is the set of road network nodes, E is the set of road network edges, and each edge e jk ∈E has weight w jk , w jk represents the travel time or distance of the road segment from node j to node k; S13: Get the current location L of the garbage truck c =(x c ,y c ), Load capacity C c and fuel consumption rate F c , where x c and c are the latitude and longitude coordinates of the garbage truck; S14: Construct a path planning dataset based on the data obtained from S11-S13: D={(L i ,C i ,G i (t)),(G,w jk ),(L c ,C c ,F c )}}。 3. A large-scale garbage collection path planning method according to claim 2, characterized in that: The S2 comprises the following steps: S21: Construct path planning model M based on hierarchical reinforcement learning, including upper reinforcement learning model M upper and the underlying reinforcement learning model M lower ; S22: Define the upper reinforcement learning model M upper The global state space S upper , action space A upper and the reward function R upper : The global state space is defined as: in, is the average amount of garbage generated by trash bin i in time period T1; Action Space A upper is the service order set of garbage bins in the global scope, which represents the optional route sequence of garbage trucks in global path planning; Reward function R upper Defined as: Among them, α1, β1, γ1 are weight coefficients, s and a are states and actions, D(L c ,L i ) is the Euclidean distance between the current position of the garbage truck and garbage bin i, is the overflow risk factor of trash can i, E a is the set of edges in the path corresponding to action a; S23. Use reinforcement learning algorithm to strengthen the upper reinforcement learning model M upper For training, use the strategy function π upper (s) Select the optimal action a to maximize the expected cumulative reward: Among them, γ upper is the discount factor, s t and a t are the state and action at time step t, respectively, and T2 is the time range of global planning; S24. Optimal strategy based on training Generate a preliminary globally optimal garbage collection path: And the path P global Passed to the lower reinforcement learning model M lower Dynamically adjust and optimize local paths.

4. A large-scale garbage collection path planning method according to claim 3, characterized in that: The S3 comprises the following steps: S31: Calculate the comprehensive service priority P of each trash bin i i : P i =λ1R i +λ2D i +λ3I i ; Among them, λ1, λ2, and λ3 are weight coefficients, satisfying λ1+λ2+λ3=1, is the overflow risk factor of trash can i, D i is the population density coefficient of the area where the trash can i is located, I i is the importance coefficient of the area where trash bin i is located; S32: Based on the calculated comprehensive service priority P i For the preliminary global optimal garbage collection path P global The trash cans in the stub are reordered to form the adjusted path P adjusted ; S33: Reinforce the learning model M at the lower level lower In the definition of local state space S lower , action space A lower and the reward function R lower : The local state space is defined as: S lower ={L c ,L i ,P i }; Action Space A lower The optional driving direction or next target trash can in the local area; Reward function R lower Defined as: R lower (s,a)=-(α2D(L c ,L i )-β2P i +γ2w jk ); Among them, α2, β2, and γ2 are weight coefficients; S34: Use reinforcement learning algorithm to strengthen the lower layer reinforcement learning model M lower For training, use the strategy function π lower (s) Select the optimal action a to maximize the expected cumulative reward: Among them, γ lower is the discount factor, T3 is the time range of local planning; S35: Optimal strategy based on training The adjusted path P adjusted Perform local optimization to generate the final global optimal garbage collection path P final .

5. A large-scale garbage collection path planning method according to claim 4, characterized in that: The S36 comprises the following steps: S361: Using the best strategy For the adjusted path P adjusted Perform local optimization and select the location L of the current garbage truck. c and the trash can L in the path i The optimal driving path between generates the local optimal path P local ; S362: Calculate the optimal strategy for the local path through the reinforcement learning algorithm and define the local reward function R local (s,a) is to minimize the energy consumption and time of the garbage truck during driving, while maximizing the chance of processing high-priority garbage bins; S363: Optimal strategy based on local paths Generate the final global optimal garbage collection path P final , where the garbage truck takes precedence over the adjusted path P adjusted Drive and use the strategy function π according to the real-time situation lower (s) Select the local optimal path P local : in, is the cumulative discounted sum at time step t, is the discount ratio of the short-term reward to the long-term reward in the future time step, is the travel time of the garbage truck on the local path, j is the jth garbage bin visited by the garbage truck on the local path, n is the total number of garbage bins visited by the garbage truck on the local path, D(L c,j ,L i,j ) is the current position of the garbage truck L c,j With trash can L i,j The Euclidean distance, v j (t) is the speed of the garbage truck at time t, is the impact of road conditions on the local path, w kl (t) represents the edge e at time t kl The weight on f(L k ,L l ) is a function that reflects the road traffic conditions.

6. A large-scale garbage collection path planning method according to claim 4, characterized in that: The S5 comprises the following steps: S51: During the driving of the garbage truck, based on the lower layer reinforcement learning model M lower The dynamic adjustment mechanism obtains the road congestion situation, trash can status changes and emergency situations in real time, and constructs the real-time state space S real-time ; S52: According to the real-time state space S real-time Define the local dynamically adjusted reward function R adjust (s,a) is to minimize the driving time and energy consumption caused by detours while ensuring that the garbage truck does not deviate from the global path planning goal: Among them, α3, α4, β2, β3 weight coefficients, T c,i is the time it takes for the garbage truck to reach garbage bin i, O jk is the road congestion coefficient from node j to node k, R i is the risk factor of the trash can overflowing; S53: Use reinforcement learning algorithm to calculate the local dynamic adjustment strategy, combine the real-time state space and reward function, and generate the local optimal adjustment path P adjust-real-time , the local optimal adjustment path bypasses the congested area and optimizes the driving efficiency of the garbage truck.

7. A large-scale garbage collection path planning method according to claim 1, characterized in that: The S6 comprises the following steps: S61: During the garbage collection task execution, the driving path of the garbage truck, the road traffic conditions, and the changes in the status of the garbage bins are collected in real time, and a real-time path planning dataset is constructed; S62: Obtain a historical path planning dataset, including previous garbage truck driving paths, garbage bin collection conditions, and road traffic records; S63: Compare and analyze the real-time path planning data set with the historical path planning data set, identify the deviation between the current path planning and the historical planning, and determine the main factors affecting the path deviation; S64: Based on the analysis results of real-time and historical data, dynamically adjust the path planning model, update the parameters in the path planning model, and optimize the priority scheduling mechanism of the trash can; S65: Self-optimize the adjusted path planning model and train the path planning model using real-time feedback data.

8. A large-scale garbage collection path planning method according to claim 1, characterized in that: The S7 comprises the following steps: S71: After completing the collection task of the garbage bins in the current collection area, return to step S1, continue to obtain the garbage bin data, road traffic status data and garbage truck operation status data in the next collection area, and update the path planning data set D new ; S72: Based on the updated path planning data set D new The hierarchical reinforcement learning model is used again to perform global path planning and local path dynamic adjustment, and steps S2 to S6 are repeated to generate a new global optimal garbage collection path P. new-final and the local optimal path P new-local ; S73: Real-time monitoring of the status of trash cans and road traffic conditions in the urban area, and dynamic adjustment of the priority and path planning of trash cans; S74: Continue to execute the process of S71-S73 until the garbage collection task of all garbage bins in the urban area is completed, and finally generate a garbage collection path P covering the entire area final-global .

9. A large-scale garbage collection path planning device, characterized in that: include: The data collection module is used to obtain the garbage bin data, road traffic condition data and garbage truck operation status data in the collection area, and construct a path planning data set; The path planning module is used to build a path planning model based on hierarchical reinforcement learning. The path planning model includes an upper-layer reinforcement learning model and a lower-layer reinforcement learning model. The upper-layer reinforcement learning model combines the path planning data set to perform global path planning and generate a preliminary global optimal garbage collection path. The lower-layer reinforcement learning model dynamically adjusts the local path according to road congestion, changes in the status of garbage bins, and emergencies. The priority scheduling module is used to dynamically adjust the service priority of the trash cans according to the overflow risk of the trash cans, the regional population density and the importance of the region through a multi-level priority scheduling mechanism in the lower-level reinforcement learning model; The path optimization module is used to feed back the real-time path planning dataset and the historical path planning dataset to the path planning model to dynamically adjust the garbage collection path and priority scheduling mechanism; The control module is used to transmit the final generated garbage collection path to the garbage truck scheduling system, control the garbage truck to collect garbage according to the optimized path, and dynamically adjust the path according to the real-time situation until the garbage collection task of all garbage bins in the urban area is completed.

10. A large-scale garbage collection path planning device, characterized in that: include: The method comprises a memory and a processor, wherein the memory is used to store instructions, and the instructions are used to control the processor to operate so as to execute the method according to any one of claims 1 to 8.

Images