A dynamic task scheduling method and system for unmanned forklifts based on deep reinforcement learning in cold chain warehouses

Abstract

This invention discloses a method and system for dynamic task scheduling of unmanned forklifts based on deep reinforcement learning in cold chain warehouses. This method belongs to the field of unmanned forklift scheduling and includes: placing sensors in a target cold chain warehouse to collect environmental data of the target cold chain warehouse, and constructing a digital twin model of the target cold chain warehouse in the cloud based on digital twin technology; constructing the dynamic environmental state of the target cold chain warehouse based on the data fed back by the sensors, and generating an initial unmanned forklift trajectory based on a path generation algorithm; optimizing the initial unmanned forklift trajectory using a multi-feature coupled spatiotemporal joint path planning algorithm deployed in the cloud to generate an optimal path and scheduling plan; and finally, the cloud sends instructions to an edge gateway, which decomposes the instructions and sends them to each unmanned forklift. This invention improves the optimization effect of the scheduling scheme.

Description

Technical Field

[0001] The present invention relates to the field of unmanned forklift scheduling, and in particular to a method and system for dynamic task scheduling of unmanned forklifts based on deep reinforcement learning in cold chain warehouses. Background Art

[0002] With the rapid development of the cold chain logistics industry, traditional unmanned forklift scheduling methods are no longer able to meet the growing demand for efficiency, precision, and intelligence. Existing technologies for dynamic task scheduling for unmanned forklifts have numerous shortcomings, including difficulty responding to dynamic environmental changes in real time and failing to fully consider multiple optimization objectives. Summary of the Invention

[0003] One of the purposes of the present invention is to provide a method for dynamic task scheduling of unmanned forklifts based on deep reinforcement learning in cold chain warehouses, so as to solve the problem in the prior art that it is impossible to respond to dynamic environmental changes in real time. In response to the above problems, the present invention proposes a method for dynamic task scheduling of unmanned forklifts in cold chain warehouses based on deep reinforcement learning. This method arranges various sensors in the target cold chain warehouse, collects environmental data and builds a digital twin model, constructs a dynamic environmental state based on the data fed back by the sensors, and generates an initial unmanned forklift trajectory. Then, a path planning algorithm deployed in the cloud is used to optimize the initial trajectory, generate the optimal path and scheduling plan, and optimize the dynamic task scheduling of the unmanned forklift.

[0004] The present invention is implemented through the following technical solution, a dynamic task scheduling method for unmanned forklifts based on deep reinforcement learning in cold chain warehouses, including the following steps: arranging sensors in the target cold chain warehouse, and sending the collected data to the cloud to build a sensor database, collecting environmental data of the target cold chain warehouse, and building a digital twin model of the target cold chain warehouse in the cloud based on the collected environmental data based on digital twin technology; constructing the dynamic environmental state of the target cold chain warehouse according to the data feedback from the sensors, and generating the initial unmanned forklift trajectory based on the path generation algorithm, using the multi-feature coupled spatiotemporal joint path planning algorithm deployed on the cloud to optimize the initial unmanned forklift trajectory and generate the optimal path and scheduling plan; the cloud sends instructions to the edge gateway, the edge gateway decomposes the instructions and sends them to each unmanned forklift to realize the specific operations of dynamic task scheduling.

[0005] Furthermore, the arrangement of sensors in the target cold chain warehouse includes: deployment of environmental detection sensors, deployment of UWB anchor points and deployment of unmanned forklift sensors; the deployment of environmental detection sensors includes: installation of detection sensors in the target cold chain warehouse for real-time detection of the location of obstacles; the deployment of UWB anchor points includes: determining the number of UWB anchor points based on the area and structure of the target cold chain warehouse and installing multiple UWB anchor points to ensure signal coverage of the entire warehouse; the deployment of unmanned forklift sensors includes: installing anti-fogging cameras at the front and rear of the unmanned forklift, installing a low-temperature resistant lidar on the top of the unmanned forklift, and installing an IMU at the center of the unmanned forklift.

[0006] Furthermore, the data collected by the deployed sensors can be received through the edge gateway, and the collected data can be time synchronized and spatially aligned in the edge gateway to ensure the consistency of the data from each sensor.

[0007] Furthermore, collecting environmental data of the target cold chain warehouse includes: placing high-precision LiDAR and RGB-D camera equipment on a mobile platform, planning a scanning path covering the entire target cold chain warehouse, including all channels, shelves and important areas, moving along the predetermined path, ensuring that the LiDAR and RGB-D cameras fully cover every corner of the target cold chain warehouse, and collecting point cloud data of the entire warehouse; using the SLAM algorithm to convert the LiDAR data into a high-precision three-dimensional point cloud, and combining the RGB-D data to give the point cloud color information.

[0008] Furthermore, constructing a digital twin model of the target cold chain warehouse may also include marking key points of the target cold chain warehouse in the digital twin model. The key points may include: shelves, cargo locations, temperature zones, safety areas, and auxiliary facilities.

[0009] Furthermore, by numbering the shelves in the digital twin model, the specific location coordinates of each shelf are recorded to achieve shelf labeling; the coordinates and cargo type of each cargo location are marked to ensure clear classification of items and achieve labeling of cargo location information; the boundaries of different temperature zones in the cold storage are marked, and the temperature requirements of each temperature zone are recorded to achieve temperature zone division; the safety fence area is marked in the digital twin model, and the prohibited areas for unmanned forklifts are defined to achieve safety area labeling; the locations of charging piles and entrances and exits of unmanned forklifts are marked in the digital twin model to ensure that these auxiliary facilities are accurately reflected in the digital twin model and achieve auxiliary facility labeling.

[0010] Furthermore, the initial unmanned forklift trajectory is used to represent the state of the unmanned forklift at time t, which includes position, heading, and speed, and is expressed by the following formula:

[0011] ,in, is the horizontal coordinate position of the unmanned forklift at time t; is the ordinate position of the unmanned forklift at time t; is the heading angle of the unmanned forklift at time t; is the speed of the unmanned forklift at time t.

[0012] Furthermore, the dynamic environmental state includes a dynamic obstacle position set and a cargo location congestion index; the dynamic environmental state collects the location data of personnel and unmanned forklifts in the target cold chain warehouse in real time through sensors installed in the warehouse or factory, and obtains the task queue and space occupancy data of each cargo location through the cargo management system or task scheduling system, processes the sensor data into a dynamic obstacle position set, and calculates the congestion index of each cargo location based on the task queuing time and space occupancy rate.

[0013] Furthermore, the status information of obstacle location sets and congestion indexes is updated regularly to reflect the latest environmental changes, thereby acquiring and updating the dynamic environmental status in real time, and realizing effective path planning and scheduling management in complex dynamic environments.

[0014] Furthermore, the specific expression of the position set O(t) of the dynamic obstacles is:

[0015] ,in, is the position of the nth dynamic obstacle at time t. These positions are usually described by coordinates, such as in two-dimensional space Or in three-dimensional space .

[0016] Furthermore, the congestion index of cargo space The expression is:

[0017] ,in, and is the weight coefficient, which is used to balance the impact of task queuing time and space occupancy on the congestion index; is the task queue time of location k at time t, indicating how many tasks are waiting to be processed; is the space occupancy rate of cargo location k at time t, indicating the degree to which the cargo location is occupied;

[0018] Furthermore, the task queuing time can be calculated by the following formula:

[0019] ,in, is the number of tasks queued at cargo location k at time t; is the estimated processing time of the jth task at location k.

[0020] Furthermore, the space occupancy rate can be calculated by the following formula:

[0021] ,in, is the occupied space of cargo location k at time t, is the total space of cargo location k.

[0022] Furthermore, the spatiotemporal joint path planning algorithm is constructed by the following steps: constructing a main optimization target sub-model according to the main optimization objectives or constraints of the dynamic task scheduling of the unmanned forklift, wherein the main optimization target sub-model includes: a path time consumption sub-model that helps optimize the path of the unmanned forklift in the cold chain warehouse to minimize the total time consumption; a path energy consumption sub-model that helps optimize the path of the unmanned forklift to minimize energy consumption, extend battery life, and reduce operating costs; a collision risk sub-model that evaluates the collision risk on the path in real time and helps the forklift select the path with the lowest risk; a cargo congestion penalty sub-model that helps the forklift select the path with the lowest congestion level to avoid delays caused by congestion; based on the constructed sub-models, the total objective function of the spatiotemporal joint path planning model is formed by weighted aggregation. The total objective function creates a comprehensive path cost minimization target by combining the outputs of each sub-model. The total objective function is expressed by the following formula:

[0023] ,in, is the path time-consuming sub-model, is the weight coefficient of the path time sub-model; is the path energy consumption sub-model, is the weight coefficient of the path energy consumption sub-model; is the collision risk submodel, is the weight coefficient of the collision risk sub-model; is the cargo space congestion penalty sub-model, is the weight coefficient of the cargo congestion penalty sub-model.

[0024] Furthermore, the path time sub-model can be expressed as follows:

[0025] , whose constraints are: ,in, The path time sub-model is used to calculate the time required for the unmanned forklift to travel on the path; is the start time of the unmanned forklift driving on the path, is the end time of the unmanned forklift driving on the path, is the speed of the unmanned forklift at time t, is the acceleration of the unmanned forklift at time t, For the cold storage environment temperature The maximum acceleration under .

[0026] Furthermore, the path energy consumption sub-model can be expressed as follows:

[0027] ,in, is the path energy consumption sub-model, is the sports power, is the battery efficiency coefficient, At ambient temperature The battery efficiency coefficient under .

[0028] Furthermore, the exercise power can be calculated by the following formula:

[0029] , where the motion power is composed of the square term of velocity and the square term of acceleration, multiplied by the coefficient and , these two coefficients are constants related to motion resistance and acceleration loss respectively.

[0030] Furthermore, the collision risk sub-model can be expressed as follows:

[0031] ,in, is the collision risk submodel, is an exponential function, is the position of the obstacle at time t, is the safety radius, which represents the impact range of obstacles; when the channel is narrow, the safety radius will be smaller.

[0032] Furthermore, the cargo congestion penalty sub-model can be expressed as follows:

[0033] ,in, is the cargo space congestion penalty sub-model, is the collection of cargo spaces that the unmanned forklift passes through. is the logarithmic sign, is the partial differential symbol, and Represent the weights of static congestion and dynamic congestion respectively; is the congestion change rate of cargo location k at time t, that is, the congestion trend.

[0034] Furthermore, the spatiotemporal joint path planning algorithm also includes constraints, which are used to ensure that path planning not only minimizes the overall objective function but also ensures safety, feasibility and task requirements. The constraints include: obstacle avoidance hard constraints that require the distance between the unmanned forklift and all dynamic obstacles to be greater than or equal to the safety distance at any time; shelf aisle geometric constraints that limit the position of the unmanned forklift and must be within the boundary range of the shelf aisle; and dynamic task scheduling coupling constraints that are used to ensure the timeliness of task scheduling.

[0035] Furthermore, the obstacle avoidance hard constraint can be expressed as follows:

[0036] ,in, For safe distance; is the starting time, is the end time.

[0037] Furthermore, the safety distance can be calculated by the following formula:

[0038] ,in, is the ground friction coefficient, which is set according to the change of the ground friction coefficient under actual temperature; is the acceleration due to gravity.

[0039] Furthermore, the shelf aisle geometric constraints can be expressed as follows:

[0040] ,in, is the maximum position of the abscissa of the shelf aisle, is the minimum horizontal coordinate position of the shelf channel, is the maximum vertical coordinate position of the shelf channel, is the minimum vertical coordinate position of the shelf aisle. These positions together describe the boundary of the aisle and define the area where the forklift can move.

[0041] Furthermore, the dynamic task scheduling coupling constraint can be expressed as follows:

[0042] ,in, is the time it takes for unmanned forklift i to arrive at the pickup location for task j, is the specified deadline, which means that the time when unmanned forklift i arrives at the pickup location of task j must be less than or equal to the specified deadline; is the time it takes for unmanned forklift i to arrive at the delivery location of task j, is the maximum tolerable delay time, which means that the time when unmanned forklift i arrives at the delivery location of task j must be before the maximum tolerable delay time.

[0043] Furthermore, the edge gateway decomposes and distributes the instructions, including the following steps: first, the edge gateway verifies the instruction data to ensure the integrity and correctness of the data; then it converts it into control instructions and task instruction sets that each unmanned forklift can understand, and distributes the path and task instructions of each unmanned forklift to the corresponding forklift control system; the unmanned forklift drives along the designated path through its own navigation system according to the path instructions received from the edge gateway; the edge gateway monitors the status and location of each unmanned forklift in real time, and feeds the monitoring data back to the digital twin model in the cloud.

[0044] On the other hand, the present invention provides a dynamic task scheduling system for unmanned forklifts based on deep reinforcement learning in cold chain warehouses, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, it implements the dynamic task scheduling method for unmanned forklifts based on deep reinforcement learning in cold chain warehouses as described above.

[0045] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0046] 1. The present invention can respond to the dynamic environmental changes of cold chain warehouses in real time. By constructing a multi-feature coupled spatiotemporal joint path planning algorithm, it simultaneously optimizes multiple objectives such as the total time consumption, energy consumption, collision risk, and congestion level of the unmanned forklift path, and timely adjusts the scheduling plan of the unmanned forklift to improve scheduling efficiency.

[0047] 2. The present invention uses digital twin technology to accurately model the cold chain warehouse environment based on the collected environmental data, accurately describe the actual status of the warehouse, provide a reliable basis for scheduling decisions, and autonomously optimize the scheduling plan according to the dynamic environmental status, thereby improving the intelligence level of scheduling.

[0048] 3. The present invention leverages the powerful computing power of the cloud to efficiently process large amounts of sensor data, implement complex path planning calculations, and improve the optimization effect of the scheduling solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0050] Figure 1 This is a flow chart of the method provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0052] Example 1

[0053] In existing technology, cold chain warehouses feature densely packed shelves, narrow aisles, and are often kept at low temperatures year-round. Due to the unique characteristics of cold chain warehouses, the potential presence of obstacles such as personnel and equipment within the warehouse necessitates robust safety measures for unmanned forklifts (UAVs) to avoid collisions and ensure the safety of personnel and equipment. Furthermore, the dispatching system must optimize the dispatch of UAVs based on the specific pickup and drop-off points, the location and load information of each UAV, and the congestion level of each cargo bay. Furthermore, achieving precise positioning and navigation of UAVs within cold chain warehouses is a common challenge within the industry. It should be noted that the term "low temperature" in this application refers to the low temperatures found in cold chain warehouses. Those skilled in the art can refer to the relevant provisions of the national standard GB / T 28577-2021, "Classification and Basic Requirements for Cold Chain Logistics." It should be understood that "low temperature" in this application refers to frozen foods stored at temperatures below or equal to -18°C, while refrigerated foods must be stored between 0°C and 8°C. That is to say, those skilled in the art should understand that the low temperature range in this application should be at least: 8°C ~ -18°C.

[0054] This embodiment discloses a method for dynamic task scheduling of unmanned forklifts in the special environment of cold chain warehouses. Based on digital twin technology, a detailed model of the cold chain warehouse is constructed in the cloud. At the same time, multiple coupling features that take into account the dynamic task scheduling of unmanned forklifts in the cold chain warehouse are introduced into the constructed detailed model, thereby constructing a multi-feature coupled spatiotemporal joint path planning algorithm, which realizes efficient dynamic task scheduling of unmanned forklifts in the special environment of cold chain warehouses.

[0055] This embodiment includes two stages, namely the digital twin model construction stage, which is used for data collection and environmental modeling. By deploying sensors on-site in the cold chain warehouse and scanning and building a cloud-based digital twin model, a basic data perception layer is constructed that can reflect the on-site situation of the cold chain warehouse in real time on the cloud, providing basic data support for subsequent decision-making and scheduling.

[0056] The dynamic path planning and safety protection stage is used to perform path planning and obstacle avoidance based on the data collected in the first stage and the constructed spatiotemporal joint path planning algorithm. Finally, a decision layer that can output dynamic scheduling instructions is constructed. The decision layer generates decisions and sends instructions to the terminal. The terminal executes the dynamic scheduling instructions output by the decision layer, actually controls the actions of the unmanned forklift, and realizes dynamic task scheduling of unmanned forklifts in cold chain warehouses.

[0057] Figure 1 The flowchart of the method in this embodiment is shown. It can be seen from the figure that this embodiment includes the following steps:

[0058] The digital twin model construction phase includes two steps. The task of this phase is to deploy sensors in the target cold chain warehouse and collect environmental data of the target cold chain warehouse, and then build a refined cold chain warehouse digital twin model of the target cold chain warehouse based on digital twin technology in the cloud.

[0059] Step 1: First, deploy sensors in the target cold chain warehouse.

[0060] Sensor deployment specifically includes: deployment of environmental detection sensors, UWB anchor point deployment, and unmanned forklift sensor deployment.

[0061] The deployment of environmental detection sensors includes:

[0062] Detection sensors are installed in cold chain warehouses to detect the location of obstacles in real time, which will help analyze and check the location of obstacles in the warehouse later.

[0063] UWB anchor deployment includes:

[0064] Determine the number of UWB anchor points based on the cold storage area and structure to ensure signal coverage throughout the warehouse. Install UWB anchor points on the ceiling or walls along the cold storage aisles to form a complete coverage network. The spacing between each UWB anchor point should be less than 20 meters to ensure continuity and stability of signal coverage. During installation, be careful to avoid deploying UWB anchor points near metal shelves to reduce signal interference. Consider installing anchor points on top of shelves or away from metal structures. Each anchor point needs to wirelessly communicate with a central control system (such as a UWB positioning server) to ensure real-time data transmission. Ensure that each UWB anchor point has an independent power supply to avoid positioning system failure due to power supply issues.

[0065] Unmanned forklift sensor deployment includes:

[0066] Anti-fog cameras are installed on the front and rear of the unmanned forklift, preferably ones that can operate normally in low-temperature environments. A low-temperature-resistant lidar is installed on the top of the unmanned forklift to ensure 360-degree scanning without blind spots. An IMU (inertial measurement unit) is also installed in the center of the unmanned forklift to accurately measure the forklift's posture and motion.

[0067] Data collected by deployed sensors is received through edge gateways, where it is time-synchronized and spatially aligned to ensure consistency across all sensor data. This time-synchronized and spatially aligned data is then sent to the cloud to be built into a sensor database for subsequent data calculations.

[0068] Step 2: Collect environmental information of the cold chain warehouse, and build a refined digital twin model of the target cold chain warehouse in the cloud based on the collected environmental information. After the construction is completed, mark the locations of key points in the digital twin model.

[0069] Specifically, collecting environmental information about cold chain warehouses involves installing high-precision LiDAR and RGB-D cameras on a mobile platform, either an unmanned forklift or a handheld device. Planning a scanning path that covers the entire target cold chain warehouse, including all aisles, shelves, and important areas. Moving along the predetermined path, ensuring that the LiDAR and RGB-D cameras can fully cover every corner of the target cold chain warehouse. Obtaining point cloud data for the entire warehouse, using the SLAM (Simultaneous Localization and Mapping) algorithm to convert the LiDAR data into a high-precision three-dimensional point cloud, and combining it with the RGB-D data to assign color information to the point cloud, enhancing map visualization. Building a refined digital twin model of the target cold chain warehouse in the cloud based on the collected data and combined with digital twin technology. Furthermore, information about key points is annotated in the digital twin model.

[0070] Specifically, key points can include: shelves, cargo locations, temperature zone divisions, safety areas, and auxiliary facilities. Number the shelves in the digital twin model and record the specific location coordinates of each shelf to achieve shelf labeling. Mark the coordinates of each cargo location and the type of goods (frozen / refrigerated) to ensure that the items are clearly classified and to achieve the labeling of cargo location information. Mark the boundaries of different temperature zones in the cold storage and record the temperature requirements of each temperature zone to achieve temperature zone division. Mark the safety fence area in the digital twin model, define the prohibited areas for unmanned forklifts, and achieve safety area marking. Mark the locations of the charging piles and entrances and exits of the unmanned forklifts in the digital twin model to ensure that these auxiliary facilities are accurately reflected in the digital twin model to achieve auxiliary facility labeling.

[0071] Step 3: After constructing the digital twin model of the target cold chain warehouse, the dynamic environmental state of the target cold chain warehouse at the current moment is constructed based on the data fed back by the on-site sensors. An initial unmanned forklift trajectory is generated for the unmanned forklift using the existing path generation algorithm. The initial unmanned forklift trajectory is optimized using a multi-feature coupled spatiotemporal joint path planning mathematical model deployed in the cloud, ultimately generating an optimal path and scheduling plan so that all unmanned forklifts can efficiently complete all assigned tasks while meeting safety, geometry, and task scheduling constraints.

[0072] Specifically, in this embodiment, the RRT* algorithm can be used to obtain an initial unmanned forklift trajectory P(t). This trajectory is used to represent the state of the unmanned forklift at time t, including position, heading, and speed. It can be expressed as follows:

[0073] ,

[0074] in, is the horizontal coordinate position of the unmanned forklift at time t; is the ordinate position of the unmanned forklift at time t; is the heading angle of the unmanned forklift at time t (the direction the forklift is moving); is the speed of the unmanned forklift at time t.

[0075] In this embodiment, the dynamic environment state can be divided into the location set of dynamic obstacles (people, other forklifts) at time t; and the congestion index (task queuing time + space occupancy rate) of cargo location k at time t.

[0076] Sensors installed throughout the warehouse or factory collect real-time location data on personnel and other forklifts. The cargo management system or task scheduling system then collects task queue and space occupancy data for each cargo location. This sensor data is processed into a set of obstacle locations. Furthermore, a congestion index is calculated for each cargo location based on task queue time and space occupancy.

[0077] The obstacle location set and congestion index status information are regularly updated to reflect the latest environmental changes. This allows real-time acquisition and updating of dynamic environmental status, enabling effective path planning and scheduling management in complex dynamic environments.

[0078] Specifically, the location set of dynamic obstacles and the congestion index of each cargo location can be expressed as:

[0079] The position set O(t) of dynamic obstacles can be obtained from sensor data. The specific expression is:

[0080] ,

[0081] in, is the position of the nth dynamic obstacle at time t. These positions are usually described by coordinates, such as in two-dimensional space Or in three-dimensional space These locations can be obtained through the following methods: 1. Sensor data: Utilize sensors installed in the environment (such as lidar, cameras, ultrasonic sensors, etc.) to detect the location of obstacles in real time. 2. Positioning system: Use a real-time positioning system (RTLS), such as one based on ultra-wideband (UWB) or RFID technology, to track the location of personnel and forklifts in real time. 3. Communication system: Obtain the location data of other forklifts through vehicle-to-vehicle (V2V) or vehicle-to-infrastructure (V2I) communication.

[0082] Cargo space congestion index It can be composed of task queue time and space occupancy rate. The specific expression is:

[0083] ,

[0084] in, and is the weight coefficient, which is used to balance the impact of task queuing time and space occupancy on the congestion index.

[0085] is the task queue time of cargo location k at time t, indicating how many tasks are waiting to be processed. The task queue time can be obtained by obtaining the task queue information of each cargo location through the cargo management system or task scheduling system, and calculating the total time of the current queued tasks. Specifically, it can be calculated using the following formula:

[0086] ,

[0087] in, is the number of tasks queued at cargo location k at time t; is the estimated processing time of the jth task at location k.

[0088] is the space occupancy rate of cargo location k at time t, indicating the degree of occupancy of the cargo location. The occupancy rate can be calculated by tracking the occupancy of the cargo location through sensor data or cargo management system. It can be calculated using the following formula:

[0089] ,

[0090] in, is the occupied space of cargo location k at time t, is the total space of cargo location k.

[0091] Specifically, in this embodiment, the spatiotemporal joint path planning mathematical model is constructed through the following steps:

[0092] 1) First, considering the main optimization objectives or constraints of the dynamic task scheduling of unmanned forklifts, a main optimization objective sub-model is constructed.

[0093] In this embodiment, sub-models are constructed by considering the path time, energy consumption, collision risk, and congestion penalty for unmanned forklifts. Together, these sub-models form a mathematical model for dynamic path planning and multi-objective scheduling coupling in cold chain warehouses. Each sub-model calculates different path attributes (time, energy consumption, risk, and congestion) through time integration. Parameters (such as velocity v(t), acceleration a(t), and trajectory P(t)) are shared between the sub-models, allowing data and results to be transferred between them.

[0094] Specifically, in this embodiment, the sub-models may include: a path time consumption sub-model, a path energy consumption sub-model, a collision risk sub-model, and a cargo space congestion penalty sub-model.

[0095] The path time sub-model calculates the total time by integrating the reciprocal of speed. This is because speed is the rate of change of distance over time, while the reciprocal of speed is the rate of change of time over distance. The total path time is obtained by summing up all the small time segments along the integral path. This sub-model helps optimize the path of unmanned forklifts in cold chain warehouses to minimize the total time. It can be expressed as follows:

[0096] ,

[0097] The constraints are: ,in, The path time sub-model is used to calculate the time required for the unmanned forklift to travel on the path, taking into account the dynamic characteristics of the unmanned forklift and the influence of the friction coefficient of the cold storage floor. is the start time of the unmanned forklift driving on the path, is the end time of the unmanned forklift driving on the path, is the speed of the unmanned forklift at time t, is the acceleration of the unmanned forklift at time t, For the cold storage environment temperature When the ambient temperature of the cold storage is low, the ground is frozen and the friction coefficient decreases, which can easily lead to a decrease in the maximum acceleration. It is an acceleration constraint, which is used to ensure the safe acceleration limit of the forklift in the low temperature environment of the cold storage.

[0098] The path energy consumption sub-model considers the characteristics of battery efficiency and motion power. Battery efficiency is affected by ambient temperature, and low temperatures reduce battery efficiency. Motion power includes the energy consumption components of speed and acceleration. The greater the speed and acceleration, the greater the energy consumption. By dividing the motion power by the battery efficiency, the actual energy consumption of the unmanned forklift is reflected. This model helps optimize the forklift's path to minimize energy consumption, extend battery life, and reduce operating costs. It can be expressed as follows:

[0099] ,

[0100] in, is the path energy consumption sub-model, is the sports power, is the battery efficiency coefficient, At ambient temperature The battery efficiency coefficient under .

[0101] Specifically, in this embodiment, the exercise power can be calculated by the following formula:

[0102] ,

[0103] Among them, the motion power is composed of the square term of velocity and the square term of acceleration, which are multiplied by the coefficients and , these two coefficients are constants related to motion resistance and acceleration loss respectively.

[0104] The collision risk sub-model considers dynamic obstacles present at all time intervals along the path and maps the distance between the forklift's position and the obstacle to collision risk using a Gaussian function (exponential function). The closer the distance, the higher the risk index. This sub-model assesses collision risk along the path in real time, helping the forklift select the path with the lowest risk. It can be expressed as follows:

[0105]

[0106] in, is the collision risk submodel, is an exponential function, is the position of the obstacle at time t, is the safety radius, which represents the impact range of obstacles; when the channel is narrow, the safety radius will be smaller.

[0107] It should be noted that in the collision risk sub-model formula shown, the sum is calculated. arrive The cumulative negative exponential of the distance between all obstacles o and the unmanned forklift state P(t) within the time interval. The closer the obstacle is to the forklift, the larger the negative exponential value, and the higher the collision risk.

[0108] The cargo congestion penalty sub-model optimizes route selection by considering the degree of congestion and its dynamic changes, helping forklifts choose the path with the least congestion and avoid delays caused by congestion. It can be expressed as follows:

[0109] ,

[0110] in, is the cargo space congestion penalty sub-model, is the collection of cargo spaces that the unmanned forklift passes through. is the logarithmic sign, is the partial differential symbol, and Represent the weights of static congestion and dynamic congestion respectively; is the congestion change rate of cargo location k at time t, that is, the congestion trend.

[0111] It should be noted that this formula calculates the cumulative congestion costs of all cargo locations k that the path passes through by summing up. represents the static congestion cost; Indicates the dynamic congestion cost; the higher the congestion level or the greater the upward trend of congestion, the greater the penalty value.

[0112] 2) Based on the constructed sub-models, each sub-model is weighted and aggregated to form the overall objective function of the spatiotemporal joint path planning model. This overall objective function combines the outputs of each sub-model to create a comprehensive path cost minimization goal. Each sub-model optimizes path time, energy consumption, collision risk, and congestion penalty, which are ultimately weighted and combined to form the overall objective function. The weight coefficients can be adjusted to flexibly adapt to different operational strategies and needs.

[0113] The overall objective function can be expressed as follows:

[0114] ,

[0115] in, is the weight coefficient of the path time sub-model, is the weight coefficient of the path energy consumption sub-model, is the weight coefficient of the collision risk sub-model, is the weight coefficient of the cargo congestion penalty sub-model.

[0116] It's important to note that the weighting coefficients are used to adjust the importance of each sub-model in the overall objective. These coefficients can be adjusted based on the cold storage's operational strategy, specific needs, and actual circumstances. For example, if energy costs are a key concern for cold storage operations, the value of the path energy consumption sub-model can be increased to give energy consumption a more prominent position in the overall objective function.

[0117] 3) To ensure that the optimal path and scheduling plan ultimately outputted by the overall objective function minimizes the overall objective function while avoiding collisions, exceeding channel limits, and completing the task on time, constraints can also be set for the overall objective function to ensure safety, feasibility, and compliance with mission requirements. This ensures that the optimal path and scheduling plan ultimately outputted enables the efficient and safe operation of unmanned forklifts in the complex and dynamic environment of cold chain warehouses.

[0118] Constraints are restrictions or requirements placed on the variables in an optimization problem that must be satisfied during the optimization process. For the automated forklift path optimization problem, constraints ensure that the path planning not only optimizes the overall objective function but also ensures safety, feasibility, and mission requirements.

[0119] Specifically, in this embodiment, obstacle avoidance hard constraints, shelf channel geometry constraints, and dynamic task scheduling coupling constraints may be included.

[0120] The obstacle avoidance hard constraint requires that the distance between the unmanned forklift and all dynamic obstacles at any time must be greater than or equal to the safe distance. This can be expressed in the following formula:

[0121] ,

[0122] in, For a safe distance, is the starting time, is the end time. In this embodiment, the safety distance can be calculated by the following formula:

[0123] ,in, is the ground friction coefficient, which is set according to the change of the ground friction coefficient under actual temperature; is the acceleration due to gravity.

[0124] The rack aisle geometric constraint is used to limit the position of the unmanned forklift and must be within the boundary of the rack aisle. It can be expressed as follows:

[0125] ,

[0126] in, is the maximum position of the abscissa of the shelf aisle, is the minimum horizontal coordinate position of the shelf channel, is the maximum vertical coordinate position of the shelf channel, is the minimum vertical coordinate position of the shelf aisle. These positions together describe the boundary of the aisle and define the area where the forklift can move.

[0127] Dynamic task scheduling coupling constraints are used to ensure the timeliness of task scheduling. Specifically, it can be expressed as follows:

[0128] ,

[0129] in, is the time it takes for unmanned forklift i to arrive at the pickup location for task j, is the specified deadline, which means that the time when unmanned forklift i arrives at the pickup location of task j must be less than or equal to the specified deadline.

[0130] is the time it takes for unmanned forklift i to arrive at the delivery location of task j, is the maximum tolerable delay time, which means that the time when unmanned forklift i arrives at the delivery location of task j must be before the maximum tolerable delay time.

[0131] It's important to note that constraints ensure the feasibility of the path planning solution, ensuring that the unmanned forklift avoids collisions, stays within the corridor, and completes the mission on time. Under these constraints, the path planning model balances multiple objectives (such as route time, energy consumption, collision risk, and congestion penalties) to ultimately find the optimal overall path. This ensures that the unmanned forklift avoids collisions, stays within the corridor, and completes the mission on time while minimizing the overall objective function.

[0132] Step 4: The cloud sends the generated optimal path and scheduling plan to the edge gateway, which decomposes the instructions and sends them to each unmanned forklift to implement the specific operations of dynamic task scheduling.

[0133] Specifically, after receiving the optimal path and scheduling plan, the edge gateway will first verify the data to ensure its integrity and correctness.

[0134] This data is then converted into control instructions and task instruction sets that each unmanned forklift can understand. Based on the parsed data, the path and task instructions for each unmanned forklift are distributed to the corresponding forklift control system.

[0135] Based on the route instructions received from the edge gateway, the unmanned forklift uses its navigation system to drive along the designated route. The navigation system utilizes sensors on the forklift (such as lidar, cameras, and ultrasonic sensors) for real-time environmental perception and route tracking. Based on the dispatch plan, the unmanned forklift executes specific mission instructions during its journey, such as loading cargo, transporting cargo to a designated location, and unloading cargo.

[0136] The edge gateway monitors the status and location of each unmanned forklift in real time and feeds this data into the cloud-based digital twin model. Using feedback from on-site sensors and the forklifts, it ensures safe and efficient task execution. If unexpected situations arise during a task (such as new obstacles or changes in task priority), the edge gateway adjusts the unmanned forklift's path and task instructions in real time and sends these adjusted instructions to the corresponding forklifts to ensure successful task completion.

[0137] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A dynamic task scheduling method for unmanned forklifts based on deep reinforcement learning in cold chain warehouses, characterized by: The dynamic task scheduling method includes: Sensors are placed in the target cold chain warehouse and the collected data is sent to the cloud to build a sensor database. Collect environmental data of the target cold chain warehouse and build a digital twin model of the target cold chain warehouse in the cloud based on the collected environmental data and digital twin technology; The dynamic environment state of the target cold chain warehouse is constructed based on the data fed back by the sensors, and the initial unmanned forklift trajectory is generated based on the path generation algorithm. Using a multi-feature coupled spatiotemporal path planning algorithm deployed in the cloud, the initial unmanned forklift trajectory is optimized to generate the optimal path and scheduling plan. The cloud sends the instructions to the edge gateway, which decomposes the instructions and sends them to each unmanned forklift to implement the specific operations of dynamic task scheduling; The spatiotemporal joint path planning algorithm is constructed by the following steps: According to the main optimization objectives or constraints of the dynamic task scheduling of unmanned forklifts, a main optimization objective sub-model is constructed. The main optimization objective sub-model includes: Help optimize the path of unmanned forklifts in cold chain warehouses, making the path time sub-model with the shortest total time; A path energy consumption sub-model that helps optimize the routes of unmanned forklifts to minimize energy consumption, extend battery life, and reduce operating costs; A collision risk sub-model that assesses collision risk along a path in real time and helps the forklift select the path with the lowest risk; A cargo congestion penalty sub-model that helps forklifts choose the least congested route and avoid delays caused by congestion; Based on the constructed sub-models, the total objective function of the spatiotemporal joint path planning model is formed by weighted aggregation. The total objective function creates a comprehensive path cost minimization goal by combining the outputs of each sub-model. The total objective function is expressed as follows: ， in, is the path time-consuming sub-model, is the weight coefficient of the path time sub-model; is the path energy consumption sub-model, is the weight coefficient of the path energy consumption sub-model; is the collision risk submodel, is the weight coefficient of the collision risk sub-model; is the cargo space congestion penalty sub-model, is the weight coefficient of the cargo congestion penalty sub-model.

2. The method for dynamic task scheduling of unmanned forklifts based on deep reinforcement learning in cold chain warehouses according to claim 1 is characterized in that: The deployment of sensors in the target cold chain warehouse includes: deployment of environmental detection sensors, deployment of UWB anchor points, and deployment of unmanned forklift sensors; The deployment of the environmental detection sensor includes: installing a detection sensor in the target cold chain warehouse for detecting the location of obstacles in real time; The UWB anchor point deployment includes: determining the number of UWB anchor points according to the area and structure of the target cold chain warehouse and installing multiple UWB anchor points to ensure signal coverage throughout the warehouse; The unmanned forklift sensor deployment includes: installing anti-fogging cameras at the front and rear of the unmanned forklift, installing a laser radar on the top of the unmanned forklift, and installing an IMU at the center of the unmanned forklift.

3. The method for dynamic task scheduling of unmanned forklifts based on deep reinforcement learning in cold chain warehouses according to claim 1 is characterized in that: The environmental data of the target cold chain warehouse to be collected includes: High-precision LiDAR and RGB-D cameras are installed on a mobile platform to plan a scanning path covering the entire target cold chain warehouse, including all aisles, shelves and important areas. Move along the predetermined path to ensure that the LiDAR and RGB-D cameras fully cover every corner of the target cold chain warehouse and collect point cloud data for the entire warehouse; The SLAM algorithm is used to convert LiDAR data into a high-precision three-dimensional point cloud, and the RGB-D data is combined to give the point cloud color information.

4. The method for dynamic task scheduling of unmanned forklifts based on deep reinforcement learning in cold chain warehouses according to claim 1 is characterized in that: The dynamic environment state includes a dynamic obstacle position set and a cargo congestion index; The dynamic environment status collects the location data of personnel and unmanned forklifts in the target cold chain warehouse in real time through sensors installed in the warehouse or factory, and obtains the task queue and space occupancy data of each cargo location through the cargo management system or task scheduling system. The sensor data is processed into a set of dynamic obstacle locations, and the congestion index of each cargo location is calculated based on the task queue time and space occupancy rate.

5. The method for dynamic task scheduling of unmanned forklifts based on deep reinforcement learning in cold chain warehouses according to claim 1 is characterized in that: The spatiotemporal joint path planning algorithm also includes constraints, which are used to ensure that path planning not only minimizes the overall objective function but also ensures safety, feasibility and mission requirements. The constraints include: Obstacle avoidance hard constraints require that the distance between the unmanned forklift and all dynamic obstacles at any time must be greater than or equal to the safe distance; Geometric constraints for rack aisles that limit the position of unmanned forklifts and must be within the boundaries of the rack aisles; And dynamic task scheduling coupling constraints used to ensure the timeliness of task scheduling.

6. The method for dynamic task scheduling of unmanned forklifts based on deep reinforcement learning in cold chain warehouses according to claim 5 is characterized in that: The obstacle avoidance hard constraint is expressed by the following formula: ， Among them, O(t) is the location set of dynamic obstacles; P(t) is the initial unmanned forklift trajectory; For a safe distance, For the moment The position of the obstacle, is the starting time, is the end time.

7. The method for dynamic task scheduling of unmanned forklifts based on deep reinforcement learning in cold chain warehouses according to claim 6 is characterized in that: The safety distance is calculated by the following formula: ， in, is the ground friction coefficient, is the acceleration due to gravity, is the speed of the unmanned forklift at time t.

8. The method for dynamic task scheduling of unmanned forklifts based on deep reinforcement learning in cold chain warehouses according to claim 1 is characterized in that: The edge gateway decomposes the instruction and sends it down, including the following steps: First, the edge gateway verifies the command data to ensure its integrity and correctness; It is then converted into control instructions and task instruction sets that each unmanned forklift can understand, and the path and task instructions of each unmanned forklift are distributed to the corresponding forklift control system; The unmanned forklift drives along the designated path using its own navigation system based on the path instructions received from the edge gateway; The edge gateway monitors the status and location of each unmanned forklift in real time and feeds the monitoring data into the digital twin model in the cloud.

9. A dynamic task scheduling system for unmanned forklifts based on deep reinforcement learning in cold chain warehouses, characterized by: The dynamic task scheduling system includes: processor; A memory storing a computer program, which, when executed by a processor, implements the dynamic task scheduling method for unmanned forklifts based on deep reinforcement learning in cold chain warehouses as described in any one of claims 1 to 8.

Images