Multi-equipment cooperative scheduling method, device, equipment, medium and program product

Abstract

The invention relates to a multi-device cooperative scheduling method and device, equipment, a medium and a program product. The method comprises the following steps: acquiring multi-source fusion data for a plurality of logistics devices in a warehousing system; the multi-source fusion data is at least obtained by performing data fusion on to-be-executed tasks, equipment sensor data and storage space topological data of a plurality of logistics equipment in the storage system; performing task allocation and path collaborative prediction by the policy network based on the multi-source fusion data to obtain an initial action instruction set for the plurality of logistics devices; the initial action instruction set comprises initial action instructions corresponding to a plurality of logistics devices; performing action conflict detection based on the initial action instruction set to obtain a conflict detection result; performing instruction optimization on the initial action instruction set based on the conflict detection result to obtain an execution action instruction set; controlling the plurality of logistics devices to act based on the action execution instruction set; by adopting the method, equipment conflicts can be effectively avoided, and the equipment coordination capability is improved.

Description

Technical Field

[0001] This application relates to the field of warehousing and logistics technology, and in particular to a method, apparatus, equipment, medium, and program product for multi-device collaborative scheduling. Background Technology

[0002] With the development of automation technology, modern intelligent warehousing systems have emerged. These systems can achieve logistics automation and improve operational efficiency through multi-device collaborative scheduling technology. Multi-device collaborative scheduling technology mainly solves the problems of task allocation, path planning, and dynamic coordination of handling equipment in the warehouse, such as AGVs (Automated Guided Vehicles) and shuttles, and optimizes material flow efficiency through dynamic decision-making mechanisms.

[0003] Related collaborative scheduling technologies suffer from a high frequency of conflicts between multiple devices. Therefore, there is an urgent need for a multi-device collaborative scheduling method to improve the scheduling effect of multi-device scheduling. Summary of the Invention

[0004] Therefore, it is necessary to provide a multi-device collaborative scheduling method, apparatus, equipment, medium, and program product that can reduce planning conflicts among multiple devices in order to address the above-mentioned technical problems.

[0005] Firstly, this application provides a multi-device collaborative scheduling method, including:

[0006] A multi-source fusion data set is obtained for multiple logistics devices in a warehousing system. This multi-source fusion data is obtained by fusing data from the tasks to be executed by the multiple logistics devices in the warehousing system, device sensor data, and warehousing space topology data. A policy network performs task allocation and path coordination prediction based on the multi-source fusion data to obtain an initial action instruction set for the multiple logistics devices. The initial action instruction set includes initial action instructions that at least indicate the movement path and operating parameters of the logistics devices. Action conflict detection is performed based on the initial action instruction set to obtain conflict detection results. Based on the conflict detection results, the initial action instruction set is optimized to obtain an execution action instruction set. The execution action instruction set is used to control the multiple logistics devices to perform coordinated actions.

[0007] In one embodiment, after controlling multiple logistics devices to perform coordinated actions based on the execution action instruction set, the method further includes: acquiring device fault information and execution feedback data of the coordinated actions of the multiple logistics devices; performing multi-objective reward calculation based on the execution feedback data to determine the policy gradient update amount; determining the environmental state update amount based on the device fault information; and updating the parameters of the policy network based on the policy gradient update amount and the environmental state update amount.

[0008] In one embodiment, the execution feedback data includes the actual completion time and effective working time of each of the logistics devices, as well as the number of conflicts. The step of calculating multi-objective rewards based on the execution feedback data to determine the strategy gradient update amount includes: determining a task completion efficiency factor based on the deviation rate between the actual completion time and the planned task time of each of the logistics devices; determining an equipment utilization coefficient based on the ratio of the effective working time of each of the logistics devices to the total working time; and calculating multi-objective rewards based on the task completion efficiency factor, the equipment utilization coefficient, and the number of conflicts to determine the strategy gradient update amount.

[0009] In one embodiment, acquiring multi-source fusion data for multiple logistics devices in a warehousing system includes: acquiring tasks to be executed, device sensor data, and warehouse space topology data corresponding to each of the multiple logistics devices in the warehousing system; sorting the tasks based on their task types and priorities to obtain a hierarchical task sequence; and spatiotemporally associating the hierarchical task sequence, the device sensor data, and the warehouse space topology data to obtain multi-source fusion data for multiple logistics devices in the warehousing system.

[0010] In one embodiment, the policy network includes a graph convolutional layer, a long short-term memory layer, and a fully connected layer. The step of obtaining an initial action instruction set for multiple logistics devices by the policy network performing task allocation and path collaborative prediction based on the multi-source fused data includes: the graph convolutional layer in the policy network extracting topological features from the multi-source fused data to generate a topological neighborhood feature map; the long short-term memory layer in the policy network predicting device trajectories from the topological neighborhood feature map to generate a dynamic trajectory prediction vector; and the fully connected layer in the policy network performing task allocation and path collaborative prediction on the dynamic trajectory prediction vector to generate the initial action instruction set for multiple logistics devices.

[0011] In one embodiment, the step of performing action conflict detection based on the initial action instruction set to obtain conflict detection results includes: performing spatiotemporal trajectory prediction on each of the logistics devices based on the initial action instruction set to obtain the predicted spatiotemporal trajectory corresponding to each of the logistics devices; performing spatiotemporal overlap detection based on the predicted spatiotemporal trajectory corresponding to each of the logistics devices to determine the detection result of each of the predicted spatiotemporal trajectories; the detection result includes at least one of conflict time, conflict spatial coordinates, conflict type, and conflict severity; and determining the conflict detection result based on the detection result of each of the predicted spatiotemporal trajectories.

[0012] In one embodiment, the operating parameters include at least movement speed; the step of optimizing the initial action instruction set based on the conflict detection results to obtain an execution action instruction set includes: selecting instructions to be optimized from the initial action instruction set based on the conflict detection results; determining a speed attenuation coefficient based on the detection results of the predicted spatiotemporal trajectory corresponding to the instruction to be optimized, the task priority of the corresponding task, and spatial traffic conditions; re-determining alternative paths based on the start and end points contained in the instruction to be optimized; and optimizing the instruction to be optimized based on the speed attenuation coefficient and the alternative paths to obtain an execution action instruction set.

[0013] Secondly, this application also provides a multi-device collaborative scheduling device, comprising:

[0014] The acquisition module is used to acquire multi-source fusion data for multiple logistics devices in the warehousing system; the multi-source fusion data is obtained by fusing data from the tasks to be executed by the multiple logistics devices in the warehousing system, device sensor data, and warehousing space topology data.

[0015] The decision module is used by the policy network to perform task allocation and path collaborative prediction based on the multi-source fusion data to obtain an initial action instruction set for multiple logistics devices; wherein the initial action instruction set includes initial action instructions that at least indicate the movement path and operating parameters of the logistics devices;

[0016] The conflict detection module is used to perform action conflict detection based on the initial action instruction set and obtain the conflict detection result;

[0017] The optimization module is used to optimize the initial action instruction set based on the conflict detection results to obtain an execution action instruction set;

[0018] The control module is used to control multiple logistics devices to perform coordinated actions based on the set of execution action instructions.

[0019] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method.

[0020] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0021] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0022] The aforementioned multi-device collaborative scheduling methods, devices, equipment, media, and program products achieve a unified representation of various data, including tasks to be executed, equipment sensor data, and warehouse space topology data, through multi-source fusion data. This avoids data fragmentation caused by processing each data dimension separately, which leads to incomplete perception and provides comprehensive, accurate, and three-dimensional data support for equipment scheduling decisions. Subsequently, task allocation and path collaborative prediction are performed based on multi-source fusion data. By capturing equipment spatial constraints and task time constraints through multi-source fusion data, an initial action instruction set that takes into account both time and space is generated. Then, a rigorous action conflict detection is performed on the initial action instruction set to accurately locate potential issues such as overlapping equipment paths and conflicting operation sequences. Finally, the initial action instruction set is meticulously optimized based on the conflict detection results to form a feasible set of execution action instructions and control the collaborative actions of the equipment. In the above process, multi-device scheduling through multiple stages such as data fusion, instruction generation, conflict resolution, and instruction optimization can minimize action conflicts among multiple logistics equipment during warehousing operations, effectively reduce the probability of multi-device conflicts, and thus improve the overall collaborative operation efficiency and operational stability of warehousing equipment. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is an application environment diagram of a multi-device collaborative scheduling method in one embodiment;

[0025] Figure 2 This is a flowchart illustrating a multi-device collaborative scheduling method in one embodiment;

[0026] Figure 3 This is a schematic diagram of the policy network update process in one embodiment;

[0027] Figure 4 This is a structural block diagram of a multi-device collaborative scheduling device in one embodiment;

[0028] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0030] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0031] The multi-device collaborative scheduling method provided in this application embodiment can be applied to, for example... Figure 1 The application environment shown is illustrated. Terminal 101 communicates with server 102 via a network. A data storage system can store the data that server 102 needs to process. The data storage system can be integrated onto server 102, or it can be located in the cloud or on another network server.

[0032] Users can create logistics transportation tasks through terminal 101. Terminal 101 can then initiate a device scheduling request to server 102 based on the logistics transportation task. Upon receiving the device scheduling request, server 102 can determine the tasks to be performed by the logistics equipment in the warehousing system, and obtain equipment sensor data and warehousing space topology data from the involved logistics equipment. It then fuses the tasks to be performed by multiple logistics equipment in the warehousing system, the equipment sensor data, and the warehousing space topology data to obtain multi-source fused data for multiple logistics equipment in the warehousing system. A policy network then performs task allocation and path coordination prediction based on the multi-source fused data to obtain an initial action instruction set for multiple logistics equipment. The initial action instruction set contains initial action instructions that at least indicate the movement path and operating parameters of the logistics equipment. Action conflict detection is performed based on the initial action instruction set to obtain conflict detection results. Based on the conflict detection results, the initial action instruction set is optimized to obtain an execution action instruction set. Based on the execution action instruction set, multiple logistics equipment are controlled to perform coordinated actions, and the execution results of each logistics equipment are collected. Finally, server 102 can send the execution results of multiple logistics equipment to terminal 101.

[0033] Terminal 101 can be, but is not limited to, various logistics equipment, personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, projection equipment, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. Head-mounted devices can include virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc.

[0034] Server 102 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides cloud computing services.

[0035] In one exemplary embodiment, such as Figure 2 As shown, a multi-device collaborative scheduling method is provided, which can be applied to... Figure 1 Taking the server in the example, the explanation includes the following steps 201 to 205. Wherein:

[0036] Step 201: Obtain multi-source fusion data for multiple logistics devices in the warehousing system; the multi-source fusion data is obtained by fusing data from the tasks to be executed by multiple logistics devices in the warehousing system, device sensor data, and warehousing space topology data.

[0037] Among them, the pending tasks of multiple logistics equipment refer to the logistics transportation tasks assigned to multiple logistics equipment; further, the pending tasks can be a task queue formed by multiple logistics tasks, that is, each of the multiple logistics equipment completes its corresponding logistics task.

[0038] In some embodiments, the content of the task to be performed may include information such as task identifier, type of transported material, target location, task priority, and task time limit.

[0039] Equipment sensor data of logistics equipment refers to the data collected by sensors installed on the logistics equipment. There can be various types of sensors installed on logistics equipment, such as speed sensors, infrared sensors, and position sensors. Furthermore, equipment sensor data can include data collected by sensors over a period of time from the current moment.

[0040] In some embodiments, the device sensor data may include information such as the real-time location, operating speed, remaining power, load status, and actuator status of the logistics equipment.

[0041] Warehouse space topology data refers to a structured data set of nodes, paths, areas, and interrelationships of the warehouse space in which logistics equipment operates; for example, storage space topology data may include information such as shelf distribution, aisle access constraints, platform locations, platform functions, and obstacle area locations.

[0042] Understandably, the expected operational results of logistics equipment can be determined through tasks to be executed, such as the expected target location and expected arrival time. The current operating status of the logistics equipment can be determined through equipment sensor data, and the storage space topology data can provide data support for the path planning of the logistics equipment. In related technologies, each type of data is usually processed independently. This processing method ignores the inherent relationship between various types of data, resulting in data fragmentation and affecting the comprehensiveness of status perception. However, by fusing multiple types of data to obtain multi-source fused data, a unified representation of multiple types of data is achieved, strengthening the correlation between data in different dimensions, effectively reducing the risk of data omission, and thus improving the comprehensiveness and accuracy of the perception of the operating status of logistics equipment.

[0043] In some embodiments, multiple types of data can be fused based on spatiotemporal characteristics. For example, in the time dimension, the task time of the task to be executed (such as task start time, task end time, etc.) can be used as a time reference point to establish a time axis, and the collection timestamps of the equipment sensor data and the time limit requirements for task execution can be mapped to this time axis. At the same time, the equipment sensor data can also be processed by time-series slicing to associate the state changes of the same logistics equipment in continuous time-series slices, thereby achieving alignment between the task to be executed and the equipment sensor data in the time dimension. In the spatial dimension, the association between logistics equipment and spatial elements can be established based on the warehouse space topology data, the equipment locations included in the equipment sensors, and the target location of the task to be executed, and the spatial accessibility of the logistics equipment to the target shelf and the distance between the logistics equipment and the aisle entrance can be determined to achieve alignment in the spatial dimension. Finally, the time dimension and the spatial dimension are fused to obtain multi-source fused data.

[0044] In other embodiments, multiple data can be associated and fused based on the task characteristics of the task to be executed. For example, the target location of the task to be executed can be used as a benchmark to align the target nodes in the warehouse space topology data, and then a set of candidate paths from each logistics device to the target node can be generated based on the real-time location in the device sensor data to realize the association benchmark of the target location, target node and real-time location. Subsequently, other information, such as running speed, remaining power, load status and other parameters, are used as path filtering constraints to obtain multi-source fused data.

[0045] Furthermore, in some embodiments, different weights can be assigned to different data during the data fusion process, strengthening the weight of data with high attention, thereby determining the accuracy and reliability of multi-source fused data; for example, the weights of various types of data can be determined based on task priority, equipment failure risk, spatial congestion, etc., thereby performing weighted fusion of the data; then, the weighted data is normalized, for example, a normalization method is used to adjust the data range for continuous data, encoding is used for discrete data, and standardization is used for structured data, etc. Finally, all the processed data are fused to generate multi-source fused data.

[0046] Step 202: The strategy network performs task allocation and path coordination prediction based on multi-source fusion data to obtain the initial action instruction set for multiple logistics devices.

[0047] The initial action instruction set contains initial action instructions corresponding to multiple logistics devices. Each initial action instruction corresponding to a logistics device includes at least the task information, target location, movement path, and operating parameters of the task to be executed assigned to that logistics device. Furthermore, the task information of the task to be executed may include task identifier, task priority, task type, etc. The operating parameters may include the movement speed and start time of the logistics device. The movement path refers to the path from the current location of the logistics device to the target location.

[0048] Among them, the policy network is a pre-trained neural network model that has the ability to dynamically adapt to complex logistics scenarios.

[0049] Task allocation refers to assigning tasks to be performed to multiple logistics devices, such as logistics device 1 performing task a and logistics device 2 performing task b; path collaborative prediction refers to calculating and outputting the travel path of each logistics device from its current location to its target location within the same workspace based on multi-source fusion data.

[0050] Step 203: Perform action conflict detection based on the initial action instruction set to obtain the conflict detection results.

[0051] Among them, motion conflict detection refers to detecting whether there is a spatial position conflict between the mobile cabinet paths of any two logistics devices at the same point in time.

[0052] In some embodiments, the conflict detection results are used to describe the conflict information of the logistics equipment that are in conflict; for example, the conflict detection results may include the equipment identification, conflict time, conflict area, etc. of each logistics equipment that is in conflict.

[0053] In some embodiments, the movement path of the logistics equipment from its current location to its target location, contained in the initial action instruction set, can be combined with the path and associated with the time dimension to construct the spatial location information of the logistics equipment at different points in time, forming a spatiotemporal trajectory model of the equipment. Then, the spatiotemporal trajectory models of multiple logistics equipment are combined with the respective time trajectory models and compared and analyzed by a preset conflict detection algorithm to determine whether there is an overlapping area in the spatiotemporal trajectories of different equipment. If there is a situation where the spatial locations of two or more equipment are in the same area or have an intersection within a certain time period, it is determined that there is an action conflict. The system records the conflict information such as the equipment identifier involved in the conflict, the time interval of the conflict, and the spatial area corresponding to the conflict, and obtains the conflict detection result.

[0054] In some embodiments, the initial action instructions in the initial action instruction set can be paired up to obtain multiple instruction combinations, and then conflict detection can be performed on all instruction combinations to obtain conflict detection results; the conflict detection results include the detection results of each instruction combination.

[0055] Step 204: Optimize the initial action instruction set based on the conflict detection results to obtain the execution action instruction set.

[0056] Instruction optimization can be based on pre-defined optimization rules. For example, for at least two conflicting logistics devices, a priority score is calculated based on factors such as their respective task type, task priority, current operating status, and the proportion of completed action paths. The initial action instructions corresponding to the logistics device with the lower priority score are optimized, for example, by delaying the task start time or replanning the route, while retaining the initial action instructions corresponding to the logistics device with the higher priority score. If the two logistics devices have the same priority score, the adjustment target can be determined based on the distance from the current location of the logistics device to the conflict point; that is, the initial action instructions of the logistics device that is farther away are adjusted. Alternatively, for at least two conflicting logistics devices, the initial action instructions corresponding to each of the at least two logistics devices can be optimized to different degrees simultaneously.

[0057] In other embodiments, instruction optimization can also be implemented by an instruction optimization model; for example, conflicting initial action instructions are input into the instruction optimization model, and the instruction optimization model outputs optimized instructions.

[0058] The execution action instruction set is the instruction set without action conflicts. It can be understood that instruction optimization can be a process of multiple optimizations. For example, after the first instruction optimization, conflict detection is performed on the optimized instruction set again. If the conflict detection result indicates that there is no conflict, the optimized instruction set is used as the execution action instruction set. If the conflict detection result indicates that there is still a conflict, instruction optimization can be performed again until the conflict detection result indicates that there is no conflict.

[0059] Step 205: Control multiple logistics devices to perform coordinated actions based on the execution action instruction set.

[0060] In some embodiments, the various execution action instructions included in the execution action instruction set may be sent to the corresponding logistics equipment, and the logistics equipment may control the equipment according to the received execution action instructions, thereby realizing the coordinated operation of multiple logistics equipment.

[0061] It is worth mentioning that a preset communication protocol can be used during the issuance of action instructions to ensure the stability and integrity of instruction transmission. After receiving the instructions, the controller of the logistics equipment drives the equipment to perform the corresponding actions according to the instructions.

[0062] In the aforementioned multi-device collaborative scheduling method, multi-source fusion data achieves a unified representation of various data, including tasks to be executed, equipment sensor data, and warehouse space topology data. This avoids data fragmentation caused by processing each data dimension separately, which leads to incomplete perception and provides comprehensive, accurate, and three-dimensional data support for equipment scheduling decisions. Subsequently, task allocation and path collaborative prediction are performed based on multi-source fusion data. Multi-source fusion data can capture equipment spatial constraints and task time constraints, thereby generating an initial action instruction set that takes into account both time and space. Then, a rigorous action conflict detection is performed on the initial action instruction set to accurately locate potential issues such as overlapping equipment paths and conflicting operation sequences. Finally, the initial action instruction set is meticulously optimized based on the conflict detection results to form a feasible set of execution action instructions and control the collaborative actions of the equipment. In the above process, multi-device scheduling through multiple stages such as data fusion, instruction generation, conflict investigation, and instruction optimization can minimize action conflicts among multiple logistics equipment during warehousing operations, effectively reduce the probability of multi-device conflicts, and thus improve the overall collaborative operation efficiency and operational stability of warehousing equipment.

[0063] In one exemplary embodiment, the decision network can perform reverse optimization based on the execution status of the task to be performed, such as... Figure 3 As shown, Figure 3 The flowchart illustrating the optimization of the decision network is shown. After controlling multiple logistics devices to perform coordinated actions based on the execution action instruction set, the multi-device coordinated scheduling method further includes steps 301 to 304. Wherein:

[0064] Step 301: Obtain equipment fault information and execution feedback data of coordinated actions of multiple logistics devices.

[0065] Among them, equipment fault information refers to the equipment fault information of each logistics equipment collected in real time when decision network optimization is required; furthermore, equipment fault information is used to indicate whether there is a fault in the logistics equipment, and can also be used to indicate other fault information of the logistics equipment, such as fault type and fault severity.

[0066] In some embodiments, equipment fault information can be obtained through multiple means. For example, it can be monitored by local sensors installed on the logistics equipment, such as current sensors and vibration sensors mounted on the logistics equipment, which collect operating data in real time. If the collected data exceeds a preset normal range, a fault signal is generated. Alternatively, the server can monitor the communication status of the logistics equipment. Specifically, the server can periodically send heartbeat packets to the logistics equipment to detect the communication status between the equipment and the control node. If no heartbeat packet response is received for several consecutive times, a communication fault signal is generated. Another example is monitoring the deviation of the logistics equipment's actions by monitoring devices (such as cameras). The server compares the deviation between the actual actions of the equipment and the command actions. If the deviation continues to exceed the allowable range, an execution fault signal is generated. Of course, multiple means can be combined to obtain equipment fault information simultaneously.

[0067] It is understandable that if a logistics device malfunctions, the decision network should not assign a task to that device in the next decision-making process, or it should assign a specific task, such as a low-load task, to that device. In addition, if the malfunction of a logistics device prevents it from moving, the location of the device may also affect the spatial structure in the actual space, such as the device causing road obstruction.

[0068] Among them, the execution feedback data of logistics equipment refers to the data collected by logistics equipment during the execution of tasks; execution feedback data includes, for example, the actual completion time of the task, the actual running time of the equipment, the total running time of the equipment, and the delay time.

[0069] Step 302: Calculate multi-objective rewards based on execution feedback data to determine the policy gradient update amount.

[0070] Among them, the policy gradient update amount refers to the parameter variable used to characterize the direction and magnitude of the adjustment of the policy network parameters; the policy gradient update amount is used to update the parameters of the policy network.

[0071] In some embodiments, the server can first extract multi-dimensional indicators such as task completion rate, resource consumption rate, and execution efficiency based on execution feedback data, and calculate the comprehensive multi-objective reward value according to the preset multi-objective weight allocation rules. Then, the multi-objective reward value is substituted into the preset advantage function to calculate the advantage value. The advantage function is used to measure the degree of advantage of the current action relative to the average action. After that, the server can use a near-end policy optimization algorithm to construct a clipping objective function that includes probability ratio clipping operations based on the advantage value and the action probability distribution output by the policy network. By performing gradient solving and optimization calculation on the clipping objective function, the policy gradient update amount is finally obtained.

[0072] Step 303: Determine the environmental status update amount based on the equipment fault information.

[0073] Among them, the environmental state update quantity refers to the quantitative change value of the state space of the storage environment caused by the occurrence of fault events; it is used to realize the adaptive adjustment of the policy network under fault scenarios.

[0074] In some embodiments, after receiving device fault information, the server reconstructs the state space based on the device fault information. For example, it removes the faulty device identifier from the set of available devices, updates the adjacency matrix of the undirected graph of the storage space, and deletes the node connections related to the faulty device. Then, based on the reconstructed state space and the original state space before the fault occurred, the server calculates the difference in feature dimensions and extracts the change values ​​of key state parameters, including device availability weight, storage node connectivity coefficient, path planning cost factor, etc. The server then normalizes the above-mentioned difference parameters through a preset state quantification model to obtain the environmental state update.

[0075] For example, the server can parse the equipment fault information to extract the faulty equipment identifier, fault occurrence time, and fault impact range (such as whether it affects the equipment's movement function or whether task execution needs to be stopped). If the signal format is abnormal during the parsing process, a retransmission mechanism is triggered to ensure the integrity of the fault information. Based on the faulty equipment identifier, the system masks the faulty equipment node in the current state space, that is, removes all feature data such as the three-dimensional coordinates and operating status of the corresponding node, and no longer includes it in the entity set for subsequent network calculations. At the same time, the server updates the connectivity of the warehouse topology by adjusting the topology adjacency matrix: all connectivity identifiers (used to indicate whether nodes are passable) related to the faulty equipment node in the original adjacency matrix are set to invalid, and the shortest path and connectivity weight between the remaining valid nodes are recalculated to ensure that the topology can reflect the spatial passability status of the current fault-free equipment. Finally, the server integrates the faulty equipment node masking information and the topology connectivity update data to construct an incremental environmental state update quantity ΔS. ΔS includes a faulty node removal list and a topology adjustment matrix, both of which use standardized data formats to ensure that they can be directly read by the adaptation module of the primary optimization strategy network.

[0076] Step 304: Update the parameters of the policy network based on the policy gradient update amount and the environment state update amount.

[0077] In some embodiments, the policy gradient update amount and the environment state update amount can be combined. First, the policy gradient update amount is used to update the parameters of the policy network to obtain a primary optimized policy network. Then, the environment state update amount is used to update the network parameters of the primary optimized policy network.

[0078] For example, the weight matrix of the fully connected layer of the policy network can be updated first by updating the policy gradient amount according to the backpropagation algorithm to obtain the primary optimized policy network; then the graph convolution kernel parameters of the primary optimized policy network can be updated by updating the environment state amount.

[0079] For example, the server can determine the association mapping between the environment state update and the parameters in the parameter file of the policy network, and clarify which parameters of the graph convolution kernel corresponding to the topology adjustment matrix in the environment state update need to be fine-tuned (such as the neighborhood aggregation weight and spatial feature extraction coefficient of the convolution kernel). Then, it can adjust them using the convolution kernel parameter fine-tuning formula shown in the figure below:

[0080] .

[0081] in, These are the fine-tuned graph convolution kernel parameters. The parameters of the original convolutional kernel in the primary optimization policy network are obtained by updating the parameters using the policy gradient update. This is a fine-tuning coefficient used to control the magnitude of parameter adjustments and prevent network performance fluctuations caused by excessive adjustments. This represents the amount of environmental status updates. The gradient for adjusting the graph convolution kernel parameters is calculated based on the impact of topological changes on spatial feature extraction.

[0082] The server calculates the parameter adjustment values ​​for each dimension of the graph convolution kernel according to this formula. After the graph convolution kernel parameters are fine-tuned, the server synchronously updates the weights of the fully connected layers associated with the graph convolution layers in the policy network to ensure that the parameters of each layer of the network are coordinated. Furthermore, after the update is completed, the server can input the normalized state representation tensor corresponding to the environmental state update to verify whether the conflict-free action instruction set output by the policy network conforms to the current environmental state. After the verification is passed, the server confirms that the parameter weight update of the policy network is complete.

[0083] In the above embodiments, the environmental state update amount is determined by the equipment fault information; the policy gradient update amount is determined by the execution feedback data; and the parameters of the policy network are optimized by the environmental state update amount and the policy gradient update amount, which can realize the continuous optimization of the policy network and make the parameters of the policy network adapt to the real-time environmental state.

[0084] In some embodiments, the execution feedback data includes the actual completion time and effective working time of each logistics device, as well as the number of conflicts; based on the execution feedback data, multi-objective reward calculation is performed to determine the policy gradient update amount, including:

[0085] Based on the deviation rate between the actual completion time and the planned task time of each logistics device, the task completion efficiency factor is determined.

[0086] The deviation rate between actual completion time and task completion time refers to the ratio of the difference between the actual completion time and the planned task time to the planned task time. For example, the deviation rate DR can be calculated using the following formula:

[0087] .

[0088] in, , This is the actual completion time. This refers to the task completion time.

[0089] Understandably, the deviation rate reflects the degree of deviation between the actual completion time and the planned task time.

[0090] In some embodiments, the deviation rate can be directly determined as the task completion efficiency factor.

[0091] In other embodiments, the task completion efficiency factor can also be obtained by mapping based on the deviation rate, i.e., the task completion efficiency factor. When DR=0, EF=1, indicating that the task is completed as planned with the highest efficiency. When DR exceeds the preset reasonable range, the smaller the EF, the more abnormal the task efficiency. The server can generate corresponding EF for the tasks performed by each logistics device, organize them into a set of efficiency factors according to the task identifier, and store them in a specified directory in the data cache area for direct use in subsequent multi-objective fusion processing.

[0092] In some embodiments, before determining the task completion efficiency factor, the server can first use a preset verification algorithm to perform integrity verification on the obtained feedback data. By traversing the fields corresponding to each task identifier, it can determine whether there are null values ​​or abnormal formats in the actual completion time and the task plan time. If so, a data completion mechanism is triggered to retrieve the timestamp record in the task execution log from the device controller or query historical time data from the task progress module of the warehouse management system to supplement the missing information.

[0093] The equipment utilization coefficient is determined based on the ratio of the effective working time of each logistics equipment to the total working time.

[0094] Among them, the effective working time of logistics equipment refers to the cumulative time for logistics equipment to perform tasks such as handling and positioning; the total working time of logistics equipment refers to the cumulative time including effective working time and idle time, and the idle time refers to the time when logistics equipment is idle, malfunctioning or on standby.

[0095] Understandably, equipment utilization rate is used to describe the load level of logistics equipment during task execution.

[0096] In some embodiments, the server can collect operational status data of logistics equipment through device sensors and device controllers, classify and process the collected operational status data, establish data archives according to device identifiers, and calculate the effective working time for each device. and free time Then, according to the formula The total working time is calculated, and based on this, the equipment utilization rate is determined. It reflects the percentage of work done by the equipment during the statistical period; the server will Mapped to equipment utilization coefficient Using formula A mapping is implemented to ensure that the range of equipment utilization coefficients is consistent with the task completion efficiency factor, facilitating subsequent multi-objective fusion calculations. The system generates a corresponding equipment utilization coefficient for each logistics device, organizes them into a set of equipment utilization coefficients according to equipment identifiers, and stores them in association with the task completion efficiency factor set through equipment identifiers.

[0097] Based on the task completion efficiency factor, equipment utilization rate, and the number of conflicts, multi-objective reward calculation is performed to determine the policy gradient update amount.

[0098] Among them, the number of conflicts refers to the cumulative number of conflicts that occur during the execution of actions by all logistics equipment within a unit time window; furthermore, the server can determine the statistical period through time window division logic (e.g., determining the length of the time window based on the average execution time of the task) to ensure that the statistical results match the task execution period.

[0099] In some embodiments, a multi-objective reward value can be calculated using a pre-built dynamic weighted reward function, and then the policy gradient update amount can be determined based on the multi-objective reward value.

[0100] For example, this can be achieved through the following dynamically weighted reward function. To calculate multi-objective reward values:

[0101] .

[0102] in, The deviation between the actual completion time and the planned time of a task is also known as the task completion efficiency factor. The number of conflicts occurring within a unit time window. This is the equipment utilization rate coefficient. , , These are their respective weighting coefficients.

[0103] Then, the policy gradient update can be generated using a pre-defined gradient backpropagation algorithm. ,Right now:

[0104] .

[0105] in, This is the policy gradient update amount. For the parameters of the policy network, For the policy network in state Down Output Action The probability, This represents the mathematical expectation. The server uses this formula to propagate backwards along the network layers. The corresponding error signal is used to adjust the gradient direction and magnitude of the parameters of each layer of the network, ultimately generating the policy gradient update. This is used for subsequent parameter optimization of the policy network.

[0106] In some embodiments, after determining the policy gradient update amount, the server performs a validity check on the gradient data. Abnormal gradient values ​​exceeding a reasonable range are excluded using a preset gradient range judgment logic. If the proportion of abnormal gradients exceeds a threshold, the gradient is recalculated to ensure the reliability of the input policy gradient update amount. Based on the reliability of the policy gradient update amount, the server determines the execution path of the backpropagation algorithm, starting from the output layer of the policy network and traversing backwards layer by layer towards the fully connected layers. Each layer is associated with the corresponding policy gradient update amount data. For example, the update formula for the weight matrix of the fully connected layer can be as follows:

[0107] .

[0108] in, This is the updated weight matrix of the fully connected layer. This is the weight matrix of the fully connected layer before the update. The learning rate controls the magnitude of weight adjustments and dynamically adapts based on the network's training convergence state. This represents the policy gradient update amount, reflecting the direction and need for weight adjustments. The server calculates the weight adjustment value of each neuron in the fully connected layer row by row and column by column according to the above formula, and then superimposes the adjustment value with the original weight matrix to obtain a new weight matrix.

[0109] Furthermore, the server can write the updated fully connected layer weight matrix into the parameter storage module of the policy network, while retaining a backup of the parameters before the update. After the update is completed, the server performs a functional test on the network, inputs the historical normalized state representation tensor, and verifies whether the logic of the action instructions output by the network remains consistent. After the test is passed, a primary optimized policy network is generated and enters the state of waiting for dynamic adaptation.

[0110] In this embodiment, multi-objective reward calculation is performed based on execution feedback data to determine the policy gradient update amount, which can achieve precise optimization of the policy gradient, continuous optimization of the policy network, and thus effectively improve the decision-making accuracy and adaptability of the policy network.

[0111] In some embodiments, acquiring multi-source fusion data for multiple logistics devices in a warehousing system includes:

[0112] Acquire the tasks to be executed, equipment sensor data, and warehouse space topology data for each of the multiple logistics devices in the warehousing system.

[0113] In some embodiments, the task to be performed can be determined by parsing the task request; device sensor data can be synchronously acquired from logistics equipment; and warehouse space topology data can be pre-stored.

[0114] The tasks are sorted according to their type and priority to obtain a hierarchical task sequence.

[0115] In some embodiments, the server can extract complete task queue data of tasks to be executed from the warehouse management system. This data may include the task type identifier, task origin, task-related material information, task completion deadline, and task-related target operation area for each task. The server performs structured processing on the task queue data through preset data parsing logic, transforming unstructured task description information into quantifiable features, such as mapping the task type identifier to the corresponding type code and transforming the task completion deadline into a relative time difference.

[0116] Furthermore, the server can pre-build a task priority determination system, for example, using task type and urgency as the core determination dimensions. For instance, in the task type dimension, the server can set basic weights for different types of tasks based on the core objectives of warehouse operations. For example, task types can include replenishment tasks that directly affect the production line supply, outbound tasks involving order delivery, and regular inbound tasks, and different basic weights can be set for different types of tasks. In the urgency dimension, the server can determine the urgency level of a task by using information such as the difference between the task's required completion time and the current system time, and the urgency identifier of the task's origin. The higher the urgency level, the higher the corresponding weight coefficient. Finally, the server can add up the basic weight corresponding to the task type and the weight coefficient corresponding to the urgency level to calculate the comprehensive priority weight of each task.

[0117] In some implementations, the server can use a preset weight fusion formula to ensure the synergistic effect between the basic weight corresponding to the task type and the weight coefficient corresponding to the urgency, avoiding the excessive influence of a single dimension on priority determination. Finally, after obtaining the comprehensive priority weight of each task, the task queue is sorted in descending order of comprehensive priority weight to form an ordered task sequence.

[0118] Furthermore, the server can construct a hierarchical task feature set based on the sorted task sequence to facilitate subsequent association with other data. This feature set includes basic information features for each task, such as task identifier, type code, associated material code, target operation area coordinates, priority features, such as comprehensive weight coefficient and priority ranking, and time features, such as required completion deadline difference and task generation timestamp.

[0119] In some embodiments, the server can continuously receive real-time sensor data from each logistics device by establishing a real-time data transmission link with the sensor modules of the logistics devices. This sensor data may include device location sensing data, operating parameter data, power system status data, and load detection data. Since sensors may be affected by environmental interference, equipment vibration, and other factors during data acquisition, the data may contain instantaneous fluctuation noise, manifested as a significant deviation between the sensor value at a certain moment and the value at adjacent moments. Such outliers can affect the accuracy of subsequent device status judgments. Therefore, the server can perform outlier filtering on the sensor data to eliminate interference caused by outliers.

[0120] For example, the instantaneous fluctuation noise of the sensor-acquired signal can be eliminated by the method of calculating the mean of a sliding window, resulting in processed device sensor data, which can also be described as a smoothed device state set.

[0121] Furthermore, the server employs a sliding window averaging algorithm for filtering, constructing a sliding window model. Based on the sensor data acquisition frequency and the stability of device state changes, the length of the sliding window is logically determined to ensure that the window length covers a sufficient number of data points to smooth noise while also reflecting the true changes in device state in a timely manner. The sliding window starts with the currently received sensor data point and selects consecutive historical data points to form the data window. As new data points are continuously received, the window slides sequentially along the time axis, enabling dynamic processing of real-time data.

[0122] For each sliding window of data, the server first calculates the mean of all data points within the window, then compares the deviation of each data point within the window to the mean. If the deviation of a data point from the mean exceeds a preset range, the data point is determined to be an outlier. In this case, the server can replace the outlier with the window mean. If the deviation of the data point is within the preset range, the original data point is retained. In this way, the server can eliminate instantaneous fluctuation noise in the sensor's acquired signal, ensuring the stability of the device's sensor data.

[0123] Furthermore, to facilitate subsequent data fusion, the server can categorize and organize the device sensor data, after sliding window mean filtering, according to device identifiers to form a smoothed device state set. This state set includes smoothed position data, smoothed operating parameter data, smoothed power status data, and smoothed load data for each device at different timestamps. The device sensor data for each device is arranged in time-stamp order to ensure the temporal continuity of the data and provide accurate device state dimension input for subsequent spatiotemporal correlation of data.

[0124] By spatiotemporally correlating hierarchical task sequences, equipment sensor data, and warehouse space topology data, multi-source fusion data for multiple logistics devices in the warehouse system is obtained.

[0125] Spatiotemporal correlation refers to data correlation based on the time dimension as a unified axis and the spatial dimension as the correlation benchmark.

[0126] In some embodiments, the server can first obtain a hierarchical task feature set based on the hierarchical task sequence, and a smoothed device state set based on device sensor data. Then, the server performs data alignment processing on the hierarchical task feature set, the smoothed device state set, and the warehouse space topology data. In the time dimension, the server can construct a unified time axis according to a preset time interval, and map the time features of tasks in the hierarchical task feature set and the timestamps of device data in the smoothed device state set onto this unified time axis to ensure the synchronization of the three types of data in the time dimension and avoid correlation deviations caused by inconsistent time bases. In the spatial dimension, the server can regard the coordinate system in the warehouse space topology data as a reference, and convert the target operation area coordinates associated with tasks in the hierarchical task feature set and the location data of devices in the smoothed device state set into three-dimensional coordinate values ​​under this coordinate system. Furthermore, based on the overall coordinate range of the warehouse area, the coordinate boundaries of each functional area (shelf area, aisle area, operation platform area), and the three-dimensional coordinate information of fixed facilities (shelves, platforms) included in the warehouse space topology data, the server uses coordinate transformation logic to ensure the consistency of the task target location and the real-time location of the equipment in the spatial dimension.

[0127] Furthermore, the server can construct a spatiotemporal correlation matrix based on the three types of data after time alignment and spatial coordinate unification. The row dimension of the matrix corresponds to each time node on a unified time axis, and the column dimension corresponds to key entities in the warehousing system (such as tasks, equipment, fixed facilities, etc.). Each element in the matrix integrates the three-dimensional coordinate information and state variable information of the corresponding entity at the corresponding time node. For example, an element might contain the target coordinates and priority features of a task at a specific time point, the real-time coordinates and motion status of a device, and the coordinate information of surrounding fixed facilities, achieving deep fusion of three-dimensional coordinates and state variables. Finally, the server normalizes the constructed spatiotemporal correlation matrix using a preset normalization algorithm, mapping different types of feature values ​​(such as coordinate values, priority coefficients, and device operating parameters) to a unified numerical range. During the normalization process, appropriate normalization logic can be selected based on the value range characteristics of various features, ensuring that the normalized data retains the relative relationships of the original data while eliminating interference between features of different magnitudes. Ultimately, the normalized spatiotemporal correlation matrix is ​​reshaped into a three-dimensional tensor structure, generating a normalized state representation tensor, resulting in multi-source fusion data for multiple logistics devices in the warehousing system. This provides complete and standardized multi-source fusion data input for subsequent collaborative decision-making in the strategy network.

[0128] In the above implementation, by spatiotemporally correlating hierarchical task sequences, equipment sensor data, and warehouse space topology data, multi-source fusion data is obtained, which can achieve efficient and unified representation of logistics tasks, equipment status, and spatial topology features, effectively avoiding the problem of incomplete state perception caused by data fragmentation.

[0129] In one embodiment, the policy network includes graph convolutional layers, long short-term memory layers, and fully connected layers; the policy network performs task allocation and path coordination prediction based on multi-source fused data to obtain an initial action instruction set for multiple logistics devices, including:

[0130] The graph convolutional layer in the policy network performs topological structure feature extraction on the multi-source fused data to generate a topological neighborhood feature map.

[0131] Among them, the topological neighborhood feature map refers to the feature map obtained by aggregating the features of each node and its corresponding neighborhood nodes.

[0132] In some embodiments, an initial undirected graph can be constructed based on the entities and spatial constraints between entities contained in the multi-source fused data. Then, topological structure features are extracted from the initial undirected graph to obtain a topological neighborhood feature graph.

[0133] In some embodiments, after receiving multi-source fused data, the server can perform data structure parsing on the multi-source fused data through preset entity recognition logic. This is because the multi-source fused data contains the three-dimensional coordinates and state variables of core entities such as tasks, equipment, and fixed facilities. The server can extract the warehousing system entity information and spatial association data contained in the data, determine the feature dimensions and data ranges corresponding to various entities, and provide basic data support for subsequent graph structure construction. Then, the server constructs a graph structure input based on the classified entity data. The nodes of the graph correspond to various entities in the warehousing system. The feature vector of each node consists of the three-dimensional coordinate data and state variable data (such as equipment operating status, task priority, etc.) of the entity in the multi-source fused data. The server can construct the edges of the graph according to the spatial constraint relationship between entities. The spatial constraint relationship includes the distance relationship between entities (such as the spatial distance between equipment and adjacent equipment, the spatial distance between equipment and the target task area, etc.) and the passage constraint relationship (such as the positional relationship between equipment and the aisle boundary, the avoidance relationship between equipment and the shelf, etc.). If two entities have the above spatial constraint relationship, the system establishes an edge connection between the corresponding nodes to form an initial undirected graph.

[0134] Furthermore, the server can invoke the graph convolutional layer in the decision network to extract topological features from the initial undirected graph. The graph convolutional layer first defines the neighborhood of each node, determined based on the actual layout logic of the storage space, ensuring that only nodes with direct spatial influence are included in the neighborhood set. Subsequently, the graph convolutional layer aggregates the features of each node's neighboring nodes through multiple convolutional operations. During aggregation, a pre-defined weight allocation logic is used, assigning different weights based on the spatial correlation between neighboring nodes and the central node, so that neighboring nodes with closer spatial correlation contribute more to the feature updates of the central node.

[0135] After multi-layer graph convolution operations, feature vectors for each node are obtained, which incorporate the spatial constraints of their neighborhood. These feature vectors contain the node's own state, the states of neighboring entities, and the spatial constraints between them. Finally, the updated feature vectors of all nodes are integrated with the edge connections of the graph to generate a topological neighborhood feature graph. This feature graph visually presents the spatial distribution and constraint relationships of various entities in the warehousing system in a graphical structure. Simultaneously, each node carries feature data incorporating neighborhood information, providing spatial dimension feature support for subsequent time-series modeling of equipment trajectories.

[0136] The long short-term memory layer in the policy network performs device trajectory prediction on the topological neighborhood feature map, generating a dynamic trajectory prediction vector.

[0137] The long short-term memory layer can be constructed using a long short-term memory network.

[0138] In some embodiments, the policy network can extract device-related node data from the topological neighborhood feature map, classify and organize it according to device identifiers, and form a single-device time-series dataset. The single-device time-series dataset contains the device's feature data at consecutive time points, specifically including the three-dimensional coordinates and motion state variables (such as velocity direction, motion mode identifier, etc.) at different time points, as well as the state data of the device's neighboring entities at the corresponding time points (such as neighboring device locations, neighboring channel occupancy, etc.). The single-device time-series dataset is sorted according to timestamp order to ensure the temporal continuity of the data. Then, for each device's single-device time-series dataset, an input sequence for a Long Short-Term Memory (LSTM) network is constructed. The input sequence is divided into fixed time steps, and the input vector corresponding to each time step contains the device's feature data and neighboring entity feature data within that time step. A continuous input sequence is generated using a sliding time window, enabling the LSM network to process continuous time-series data and capture the dynamic changes in device trajectories.

[0139] Furthermore, the Long Short-Term Memory (LSTM) layer performs temporal modeling of the device trajectory. The LSM network processes the temporal data through a gating mechanism of forget gate, input gate, and output gate: the forget gate filters and retains historical information valuable for current trajectory modeling based on the importance of historical data, while eliminating redundant information; the input gate receives the input data of the current time step, merges it with the filtered historical information, and updates the cell state of the network; the output gate generates the hidden state of the current time step based on the updated cell state, and the hidden state contains the key features and historical dependencies of the device trajectory at that time step. As the input sequence is gradually input, the LSM network generates a sequence of hidden states unfolded over time steps, which reflects the dynamic changes and dependencies of the device trajectory in consecutive time steps; then, based on the hidden state sequence, it performs prediction of future trajectory coordinate changes, and through a preset prediction output layer, it maps the hidden state sequence to coordinate changes for multiple future time steps. The coordinate changes include the predicted displacement values ​​of the device in each direction in three-dimensional space. Finally, the coordinate changes of each future time step are integrated in chronological order to generate a dynamic trajectory prediction vector, which reflects the trend of the device's future motion trajectory.

[0140] The fully connected layer in the policy network performs task allocation and path collaborative prediction on the dynamic trajectory prediction vector, generating an initial action instruction set for multiple logistics devices.

[0141] In some embodiments, the dynamic trajectory prediction vectors of each logistics device are aggregated and combined with the spatial constraint information in the topological neighborhood feature map to construct the input data set of the fully connected layer. The input data set includes the future trajectory prediction information of each device, the current spatial location information, the trajectory prediction information of neighboring devices, and the spatial constraint information of the warehousing system. At the same time, the data set can be normalized to ensure that the data format is consistent with the input requirements of the fully connected layer and to eliminate the impact of data dimensionality differences on the decision-making process. Finally, the fully connected layer performs feature mapping operations to gradually convert the high-dimensional features in the input data set into low-dimensional action features. During the conversion process, the weighted calculation of multiple layers of neurons is used to fuse multi-dimensional information such as device trajectory prediction and spatial constraints, and finally outputs the action feature vector corresponding to each device. The action feature vector covers the potential features of various actions that the device may perform.

[0142] After the fully connected layer performs task allocation and path collaborative prediction, the output layer of the policy network maps the action feature vector to an action probability distribution. The action probability distribution contains the probability value corresponding to each action that the device can execute. The magnitude of the probability value reflects the suitability of the action in the current scenario. The action probability distribution is sampled through a preset probability sampling algorithm. During the sampling process, actions with higher probability values ​​are selected first. At the same time, a random mechanism is introduced to avoid the policy getting stuck in local optima and to ensure that better action selection can be explored in complex scenarios. After the target action is sampled, the sampled target action is transformed into an action space mapping, that is, the abstract action output by the policy network is converted into specific action parameters that the logistics device can execute. The specific action parameters include the target movement path of the device, the target movement speed, the start time and duration of the action execution, and obstacle avoidance parameters during the action execution. In addition, the specific action parameters of each logistics device can be classified and organized according to the device identifier to ensure that each action parameter is accurately associated with the corresponding device, and finally the initial action instruction set is generated.

[0143] In the above embodiments, the collaborative decision-making mechanism based on graph convolutional networks and long short-term memory networks can simultaneously capture the temporal dynamic characteristics of device spatial constraints and running trajectories, generate initial action instructions that take into account both path efficiency and continuity, and effectively avoid the risk of action conflicts caused by the separation of spatial and temporal features.

[0144] In some embodiments, action conflict detection is performed based on an initial action instruction set to obtain conflict detection results, including:

[0145] Based on the initial action instruction set, the spatiotemporal trajectory of each device is predicted to obtain the predicted spatiotemporal trajectory of each logistics device.

[0146] In some embodiments, the motion path of the logistics equipment from its current location to its target location contained in the initial motion instruction set can be combined with the time dimension to construct the spatial location information of the logistics equipment at different points in time, thus forming the predicted spatiotemporal trajectory of the logistics equipment.

[0147] Path overlap detection is performed based on the predicted spatiotemporal trajectories corresponding to each logistics device to determine the detection results of each predicted spatiotemporal trajectory; the detection results include at least one of the following: conflict time, conflict spatial coordinates, conflict type, and conflict severity.

[0148] In some embodiments, the minimum distance threshold of the device's movement direction in Euclidean space is calculated based on a set of single device action parameters. Before the calculation, three types of basic data are integrated: the device's physical dimensions, the storage space safety redundancy requirements, and the device's movement position deviation range. The relationship between the three types of data and the distance threshold is established through preset logic. The device's physical dimensions are converted into space occupancy range data, the safety redundancy requirements are converted into space buffer range data, and the position deviation range is converted into space tolerance range data. Then, the minimum distance threshold is generated through data fusion.

[0149] Preferably, the system can employ a two-dimensional time-space comparison logic to perform path space conflict detection. First, the time segment interval is determined based on the device's movement speed and the preset path length. The preset path coordinate sequence of each device is then divided into multiple time segments according to these intervals, with each time segment corresponding to a set of spatial coordinate ranges. The system iterates through the time-space coordinate ranges of all devices, performs Euclidean distance calculations on the same time segment for any two devices, compares the calculation results with a minimum distance threshold, and simultaneously determines whether the spatial coordinate ranges intersect. If any condition is met, a path overlap risk is identified. The system generates a path overlap risk identifier for the identified risks. The identifier includes a list of conflicting device identifiers, conflict time segment information, conflict spatial coordinate range, conflict type, and conflict severity assessment, as well as a timestamp indicating the risk identifier's generation, used for subsequent tracing of the risk detection time point.

[0150] In some embodiments, before performing path overlap detection, the integrity of the initial action instruction set can be verified first. Specifically, the action parameter data of each device is extracted from the initial action instruction set. During the extraction process, the data format is first verified to confirm the integrity of the device identifier, target movement direction, preset path coordinate sequence, action execution time interval, and movement speed-related parameters. If there is missing data, a data completion mechanism is triggered to obtain the corresponding information from the warehouse management system or device controller.

[0151] Based on the detection results of each predicted spatiotemporal trajectory, the conflict detection results are determined.

[0152] In some embodiments, the detection results of the predicted spatiotemporal trajectory can be associated with the device identifier of the corresponding logistics equipment to obtain the conflict detection results, thereby facilitating the subsequent retrieval of the corresponding detection results based on the device identifier.

[0153] In the above embodiments, the spatiotemporal trajectory of each logistics device is predicted according to the initial action instruction set, and conflict detection is performed based on the predicted spatiotemporal trajectory, taking into account both time constraints and spatial constraints, making the conflict detection results more comprehensive.

[0154] In some embodiments, the operating parameters include at least movement speed; the initial action instruction set is optimized based on the collision detection results to obtain an execution action instruction set, including:

[0155] Based on the conflict detection results, the instructions to be optimized are selected from the initial action instruction set.

[0156] In some embodiments, based on the conflict detection results, at least two conflicting logistics devices and their corresponding initial action instructions can be identified. In this case, the instructions to be optimized can be determined according to preset rules, such as low-priority avoidance or long-distance avoidance. That is, the instructions to be optimized are determined according to the priority of the tasks corresponding to the logistics devices. If the priority of the task corresponding to the logistics device is high, the corresponding initial action instruction will not be modified. If the priority of the task corresponding to the logistics device is low, the corresponding initial action instruction will be determined as an instruction to be optimized. If the task priorities are the same, and the current position of the logistics device is far from the conflict point, the corresponding initial action instruction will be determined as an instruction to be optimized.

[0157] Based on the detection results of the predicted spatiotemporal trajectory corresponding to the instruction to be optimized, the task priority of the corresponding task, and the spatial travel conditions, the velocity attenuation coefficient is determined.

[0158] In some embodiments, a mapping rule can be pre-established between the severity of the conflict, the priority of the equipment task, the access conditions of the storage space and the attenuation coefficient. The severity of the conflict is converted into the basic attenuation coefficient, the priority of the equipment task is converted into the attenuation correction coefficient, and the access conditions of the storage space are converted into the attenuation compensation coefficient. The three types of coefficients are fused and calculated by a preset algorithm to obtain the final speed attenuation coefficient, ensuring that the coefficient can adapt to the avoidance requirements of the current conflict scenario.

[0159] In some embodiments, considering that there may be multiple conflicts, the server can first deduplicate the conflict data, delete duplicate conflict location coordinates and equipment motion status data, sort the parsing results according to the severity of the conflict, and prioritize the processing of conflict information with high severity. Finally, the deduplicated and sorted information is integrated with the surrounding warehouse space environment data of the conflict area to form a conflict adjustment decision dataset. During the integration process, it is ensured that the timestamps of the core conflict information and the environmental data are consistent to avoid adjustment deviations due to data time differences.

[0160] Based on the start and end points contained in the instruction to be optimized, alternative paths are redefined.

[0161] In some embodiments, the server can first construct a path search range based on the warehouse space topology data, exclude conflict areas and the space occupied by obstacles, and then combine the conflict location with the movement trajectory of surrounding equipment to determine the path search range. The server can then call the path search algorithm to find feasible paths within the search range and obtain alternative paths.

[0162] In some embodiments, after obtaining alternative paths, the length difference between the alternative paths and the original paths can be compared to determine whether the difference is within the allowable range. If it exceeds the range, the search parameters are readjusted and path search is performed until an alternative path that meets the length requirement is generated.

[0163] Based on the speed attenuation coefficient and alternative paths, the instruction to be optimized is optimized to obtain the set of execution action instructions.

[0164] In some embodiments, the original speed can be corrected first based on the speed attenuation coefficient and the operating parameters contained in the initial action command. Then, the path coordinates of the alternative path are used to replace the movement path contained in the initial action command. Finally, the action execution time interval is recalculated based on the length of the alternative path and the corrected speed to ensure that the time interval matches the path length and speed parameters.

[0165] In some embodiments, the server may also generate conflict resolution action instructions for each device involved in the conflict, thereby achieving instruction optimization through conflict resolution instructions; the conflict resolution instructions include adjusted speed, path, time and conflict avoidance core parameters, and are accompanied by an instruction version number to distinguish instructions at different adjustment stages.

[0166] In the above embodiments, by simultaneously performing collaborative optimization operations on the instructions to be optimized from both the aspects of path and speed, the probability of internal conflicts still existing between optimized instructions can be effectively reduced, thereby improving the operational stability of the entire system.

[0167] In some embodiments, before sending the execution action instruction set to the logistics equipment, the communication status of each logistics equipment can be detected first. Test signals are sent to confirm whether the equipment has the ability to receive and send data. For equipment with abnormal communication status, a fault diagnosis mechanism is triggered. Link construction is performed after the communication of the logistics equipment is restored. Then, a data interaction channel between the equipment is established through a preset communication protocol. The communication rate, data transmission format and verification rules of the protocol are configured to ensure the stability and real-time performance of data transmission. In addition, the integrity of the execution action instructions can be verified. A data verification code is generated through a verification algorithm and sent to the logistics equipment along with the instruction data. After receiving the instruction data, the logistics equipment first verifies the verification code to confirm the integrity of the data, and then extracts the path coordinate sequence, motion speed parameters and action execution time interval from the instruction, and inputs them into the locally deployed instruction verification module for verification.

[0168] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0169] Based on the same inventive concept, this application also provides a multi-device collaborative scheduling apparatus for implementing the multi-device collaborative scheduling method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more embodiments of the multi-device collaborative scheduling apparatus provided below can be found in the limitations of the multi-device collaborative scheduling method described above, and will not be repeated here.

[0170] In one exemplary embodiment, such as Figure 4 As shown, Figure 4 A schematic diagram of a multi-device collaborative scheduling device is provided. The multi-device collaborative scheduling device 400 includes:

[0171] The acquisition module 401 is used to acquire multi-source fusion data for multiple logistics devices in the warehousing system; the multi-source fusion data is obtained by fusing data from the tasks to be executed by multiple logistics devices in the warehousing system, device sensor data and warehousing space topology data.

[0172] The decision module 402 is used to perform task allocation and path collaborative prediction based on multi-source fusion data by the policy network to obtain an initial action instruction set for multiple logistics devices; wherein the initial action instruction set contains initial action instructions that at least indicate the movement path and operating parameters of the logistics devices.

[0173] The conflict detection module 403 is used to perform action conflict detection based on the initial action instruction set and obtain the conflict detection result.

[0174] The optimization module 404 is used to optimize the initial action instruction set based on the conflict detection results to obtain the execution action instruction set.

[0175] The control module 405 is used to control multiple logistics devices to perform coordinated actions based on the execution action instruction set.

[0176] In one embodiment, the multi-device collaborative scheduling device 400 further includes an update module, which is used to acquire equipment fault information and execution feedback data of multiple logistics devices performing collaborative actions; calculate multi-objective rewards based on the execution feedback data to determine the policy gradient update amount; determine the environmental state update amount based on the equipment fault information; and update the parameters of the policy network based on the policy gradient update amount and the environmental state update amount.

[0177] In one embodiment, the execution feedback data includes the actual completion time and effective working time of each logistics device, as well as the number of conflicts. The update module is specifically used to: determine the task completion efficiency factor based on the deviation rate between the actual completion time of each logistics device and the task plan time; determine the equipment utilization coefficient based on the ratio of the effective working time of each logistics device to the total working time; and perform multi-objective reward calculation based on the task completion efficiency factor, the equipment utilization coefficient, and the number of conflicts to determine the strategy gradient update amount.

[0178] In one embodiment, the acquisition module 401 is specifically used to acquire the tasks to be executed, the equipment sensor data, and the warehouse space topology data corresponding to each of the multiple logistics devices in the warehousing system; sort the tasks based on the task type and task priority of the tasks to be executed to obtain a hierarchical task sequence; and perform spatiotemporal correlation on the hierarchical task sequence, the equipment sensor data, and the warehouse space topology data to obtain multi-source fusion data for the multiple logistics devices in the warehousing system.

[0179] In one embodiment, the policy network includes a graph convolutional layer, a long short-term memory layer, and a fully connected layer; the decision module 402 is specifically used to: extract topological features from the multi-source fused data using the graph convolutional layer in the policy network to generate a topological neighborhood feature map; predict device trajectories from the topological neighborhood feature map using the long short-term memory layer in the policy network to generate a dynamic trajectory prediction vector; and perform task allocation and path coordination prediction from the dynamic trajectory prediction vector using the fully connected layer in the policy network to generate an initial action instruction set for multiple logistics devices.

[0180] In one embodiment, the conflict detection module 403 is specifically used to: predict the spatiotemporal trajectory of each logistics device based on the initial action instruction set to obtain the predicted spatiotemporal trajectory corresponding to each logistics device; perform spatiotemporal overlap detection based on the predicted spatiotemporal trajectory corresponding to each logistics device to determine the detection result of each predicted spatiotemporal trajectory; the detection result includes at least one of conflict time, conflict spatial coordinates, conflict type and conflict severity; and determine the conflict detection result based on the detection result of each predicted spatiotemporal trajectory.

[0181] In one embodiment, the operating parameters include at least the movement speed; the optimization module 404 is specifically used to: based on the conflict detection results, select the instructions to be optimized from the initial action instruction set; determine the speed attenuation coefficient based on the detection results of the predicted spatiotemporal trajectory corresponding to the instruction to be optimized, the task priority of the corresponding task, and the spatial passage conditions; redetermine alternative paths based on the start point and end point contained in the instruction to be optimized; and optimize the instructions to be optimized based on the speed attenuation coefficient and the alternative paths to obtain the execution action instruction set.

[0182] Each module in the aforementioned multi-device collaborative scheduling device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0183] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data related to device scheduling. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a multi-device collaborative scheduling method.

[0184] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0185] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described above.

[0186] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0187] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method described above.

[0188] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0189] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0190] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0191] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A multi-device collaborative scheduling method, characterized in that, The method includes: Acquire multi-source fusion data for multiple logistics devices in a warehousing system; the multi-source fusion data is obtained by fusing data from the tasks to be executed by the multiple logistics devices in the warehousing system, device sensor data, and warehousing space topology data. The policy network performs task allocation and path collaborative prediction based on the multi-source fusion data to obtain an initial action instruction set for multiple logistics devices; wherein the initial action instruction set includes initial action instructions that at least indicate the movement path and operating parameters of the logistics devices; Action conflict detection is performed based on the initial action instruction set to obtain the conflict detection result; Based on the conflict detection results, the initial action instruction set is optimized to obtain the execution action instruction set; Based on the execution action instruction set, multiple logistics devices are controlled to perform coordinated actions.

2. The method according to claim 1, characterized in that, After controlling multiple logistics devices to perform coordinated actions based on the execution action instruction set, the method further includes: Acquire equipment fault information and execution feedback data of coordinated actions of multiple logistics devices; Based on the execution feedback data, multi-objective reward calculation is performed to determine the policy gradient update amount; Based on the equipment fault information, determine the environmental status update amount; The parameters of the policy network are updated based on the policy gradient update amount and the environment state update amount.

3. The method according to claim 2, characterized in that, The execution feedback data includes the actual completion time and effective working time of each of the logistics devices, as well as the number of conflicts. The step of calculating multi-objective rewards based on the execution feedback data and determining the policy gradient update amount includes: Based on the deviation rate between the actual completion time and the planned task time of each of the aforementioned logistics devices, the task completion efficiency factor is determined; The equipment utilization coefficient is determined based on the ratio of the effective working time of each of the aforementioned logistics equipment to the total working time. Based on the task completion efficiency factor, the equipment utilization rate, and the number of conflicts, multi-objective reward calculation is performed to determine the policy gradient update amount.

4. The method according to claim 1, characterized in that, The acquisition of multi-source fusion data for multiple logistics devices in the warehousing system includes: Acquire the tasks to be executed, equipment sensor data, and warehouse space topology data for each of the multiple logistics devices in the warehousing system. Based on the task type and task priority of the tasks to be executed, the tasks are sorted to obtain a hierarchical task sequence; By spatiotemporally correlating the hierarchical task sequence, the device sensor data, and the warehouse space topology data, multi-source fusion data for multiple logistics devices in the warehouse system is obtained.

5. The method according to claim 1, characterized in that, The policy network includes graph convolutional layers, long short-term memory layers, and fully connected layers; The strategy network performs task allocation and path coordination prediction based on the multi-source fusion data to obtain an initial action instruction set for multiple logistics devices, including: The graph convolutional layer in the policy network performs topological structure feature extraction on the multi-source fused data to generate a topological neighborhood feature map. The long short-term memory layer in the policy network performs device trajectory prediction on the topological neighborhood feature map to generate a dynamic trajectory prediction vector. The fully connected layer in the policy network performs task allocation and path collaborative prediction on the dynamic trajectory prediction vector to generate an initial action instruction set for multiple logistics devices.

6. The method according to claim 1, characterized in that, The step of performing action conflict detection based on the initial action instruction set to obtain conflict detection results includes: Based on the initial action instruction set, the spatiotemporal trajectory of each logistics device is predicted to obtain the predicted spatiotemporal trajectory of each logistics device. Spatiotemporal overlap detection is performed based on the predicted spatiotemporal trajectories corresponding to each of the aforementioned logistics equipment to determine the detection results of each of the predicted spatiotemporal trajectories; the detection results include at least one of the following: conflict time, conflict spatial coordinates, conflict type, and conflict severity. Based on the detection results of each predicted spatiotemporal trajectory, the conflict detection results are determined.

7. The method according to claim 6, characterized in that, The operating parameters include at least the movement speed; The step of optimizing the initial action instruction set based on the conflict detection results to obtain an execution action instruction set includes: Based on the conflict detection results, the instructions to be optimized are selected from the initial action instruction set; Based on the detection results of the predicted spatiotemporal trajectory corresponding to the instruction to be optimized, the task priority of the corresponding task, and the spatial travel conditions, the velocity attenuation coefficient is determined. Based on the start and end points contained in the instruction to be optimized, alternative paths are redefined; Based on the speed attenuation coefficient and the alternative paths, the instruction to be optimized is optimized to obtain an execution action instruction set.

8. A multi-device collaborative scheduling device, characterized in that, The device includes: The acquisition module is used to acquire multi-source fusion data for multiple logistics devices in the warehousing system; the multi-source fusion data is obtained by fusing data from the tasks to be executed by the multiple logistics devices in the warehousing system, device sensor data, and warehousing space topology data. The decision module is used by the policy network to perform task allocation and path collaborative prediction based on the multi-source fusion data to obtain an initial action instruction set for multiple logistics devices; wherein the initial action instruction set includes initial action instructions that at least indicate the movement path and operating parameters of the logistics devices; The conflict detection module is used to perform action conflict detection based on the initial action instruction set and obtain the conflict detection result; The optimization module is used to optimize the initial action instruction set based on the conflict detection results to obtain an execution action instruction set; The control module is used to control multiple logistics devices to perform coordinated actions based on the set of execution action instructions.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images