Method and device for scheduling automatic guided vehicle, electronic equipment and storage medium

Abstract

The application provides a scheduling method and device of an automatic guided vehicle, electronic equipment and a storage medium, and belongs to the technical field of data processing. The method comprises the following steps: performing aggregation coding processing on task characteristic data of each to-be-allocated task and vehicle characteristic data of each automatic guided vehicle respectively to obtain a target state vector; inputting the target state vector into a scheduling decision model to obtain a target scheduling rule output by the scheduling decision model; sorting all to-be-allocated tasks to obtain a sorting result; and determining a target automatic guided vehicle corresponding to each to-be-allocated task based on the sorting result and the vehicle characteristic data. The application comprehensively perceives the system state by performing aggregation coding on the characteristic data of the tasks and the vehicles, and dynamically selects a suitable scheduling rule according to the current state by using the scheduling decision model, so that the scheduling system can adapt to complex dynamic working conditions and ensure stable scheduling quality by avoiding the dependence on a few fixed indexes and static rules.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a scheduling method, apparatus, electronic device, and storage medium for automated guided vehicles. Background Technology

[0002] Automated Guided Vehicles (AGVs) are widely used in modern factories and warehousing logistics for material handling. The rational allocation of transportation tasks to be performed to each AGV is a crucial part of task scheduling in an AGV system.

[0003] Currently, static rules (such as first-come-first-served scheduling, shortest distance priority, etc.) are usually used for allocation to directly determine the matching relationship between tasks and automated guided vehicles.

[0004] However, this approach often only considers a few fixed indicators such as distance or waiting time, and when faced with different actual operating scenarios, this static rule cannot cope with complex dynamic conditions, resulting in unstable scheduling quality under different conditions. Summary of the Invention

[0005] This invention provides a scheduling method, device, electronic device, and storage medium for automated guided vehicles (AGVs), which addresses the shortcomings of existing technologies that often only consider a few fixed indicators such as distance or waiting time, and whose static rules cannot cope with complex dynamic working conditions, leading to unstable scheduling quality. The invention achieves comprehensive perception of the global state of the system and dynamic adjustment of scheduling rules, ensuring stable scheduling quality under different operating scenarios.

[0006] This invention provides a method for scheduling automated guided vehicles (AGVs), comprising the following steps: The task feature data of each task to be assigned and the vehicle feature data of each automated guided vehicle are aggregated and encoded to obtain the target state vector. The target state vector is input into the scheduling decision model to obtain the target scheduling rule output by the scheduling decision model; Based on the target scheduling rules, all the tasks to be assigned are sorted to obtain the sorting result; Based on the sorting results and the vehicle characteristic data of each of the automated guided vehicles, the target automated guided vehicle corresponding to each of the tasks to be assigned is determined.

[0007] According to a scheduling method for automated guided vehicles (AGVs) provided by the present invention, the step of aggregating and encoding the task feature data of each task to be assigned and the vehicle feature data of each AGV to obtain a target state vector includes: Multi-dimensional feature extraction is performed on the task feature data of each of the tasks to be assigned and the vehicle feature data of each of the automated guided vehicles to obtain task aggregate features and guided vehicle aggregate features. Global characteristics are determined based on the number of tasks to be assigned, the number of available automated guided vehicles, and the system load status at the current moment. Based on the resource occupancy status of the transit node, the number of stationed automated guided vehicles, and task prediction information, the pre-scheduling extension characteristics are determined. The target state vector is obtained by concatenating the task aggregation feature, the guided vehicle aggregation feature, the global feature, and the pre-scheduling extended feature. The target state vector is a feature vector of fixed dimensions.

[0008] According to a scheduling method for automated guided vehicles (AGVs) provided by the present invention, the step of performing multi-dimensional feature extraction on the task feature data of each of the tasks to be assigned and the vehicle feature data of each of the AGVs to obtain task aggregation features and AGV aggregation features includes: The task feature data of each of the tasks to be assigned are combined into a task feature matrix, and the vehicle feature data of each of the automated guided vehicles are combined into a vehicle feature matrix. Statistical calculations are performed on the task feature matrix and the vehicle feature matrix column by column to obtain the task aggregate feature and the guide vehicle aggregate feature.

[0009] The method for scheduling automated guided vehicles provided by the present invention further includes: The target state vector is input into the pre-scheduling decision model to obtain the pre-scheduling decision result output by the pre-scheduling decision model; If the pre-scheduling decision result is to execute the deployment, then based on the pre-scheduling decision result, the target transfer node and the number of vehicles to be deployed are determined; Based on the number of vehicles to be dispatched, the idle automated guided vehicles will be scheduled to the target transfer node.

[0010] The method for scheduling automated guided vehicles provided by the present invention further includes: After the current scheduling cycle ends, the first reward data corresponding to the scheduling decision model and the second reward data corresponding to the pre-scheduling decision model are determined respectively. Add the first reward data to the first experience buffer, and add the second reward data to the second experience buffer; The network parameters of the scheduling decision model are updated based on the data in the first experience buffer, and the network parameters of the pre-scheduling decision model are updated based on the data in the second experience buffer.

[0011] According to a scheduling method for automated guided vehicles provided by the present invention, the step of determining the first reward data corresponding to the scheduling decision model and the second reward data corresponding to the pre-scheduling decision model includes: Obtain task execution data within the current scheduling period, wherein the task execution data includes at least the task completion rate, the average task waiting time, and the average task completion time. Based on the task execution data, the first reward data corresponding to the scheduling decision model is determined; Obtain resource layout data within the current scheduling period, wherein the resource layout data includes at least the pre-scheduling hit rate and resource occupancy balance. Based on the resource layout data, the second reward data corresponding to the pre-scheduling decision model is determined.

[0012] According to a scheduling method for automated guided vehicles (AGVs) provided by the present invention, the step of determining the target AGV corresponding to each of the tasks to be assigned based on the sorting result and the vehicle characteristic data of each AGV includes: Based on the sorting result, all the tasks to be assigned are traversed to determine the current task to be assigned. Based on the vehicle feature data, multiple candidate automated guided vehicles are identified, and the current working state of the candidate automated guided vehicles is either idle or pre-scheduled. Determine the distance between the currently assigned task and each of the candidate automated guided vehicles; The candidate automated guided vehicle (AGV) that is closest to the current task to be assigned is determined as the target AGV corresponding to the current task to be assigned.

[0013] According to a scheduling method for automated guided vehicles (AGVs) provided by the present invention, after determining the candidate AGV with the smallest distance to the current task to be assigned as the target AGV corresponding to the current task to be assigned, the method further includes: If the current working state of the target automated guided vehicle is the pre-scheduled state, then an interrupt scheduling instruction is generated for the target automated guided vehicle. Based on the interrupt scheduling instruction, the target automated guided vehicle is controlled to move from its current position to execute the currently assigned task.

[0014] According to the present invention, a method for scheduling automated guided vehicles (AGVs) includes inputting the target state vector into a scheduling decision model to obtain the target scheduling rule output by the scheduling decision model, comprising: Obtain a candidate scheduling rule set, which includes multiple candidate scheduling rules; The target state vector is input into the scheduling decision model, and the evaluation score corresponding to each candidate scheduling rule is determined according to the scheduling decision model. The candidate scheduling rule with the highest evaluation score is determined as the target scheduling rule.

[0015] The present invention also provides a scheduling device for automated guided vehicles, comprising: The aggregation encoding module is used to perform aggregation encoding processing on the task feature data of each task to be assigned and the vehicle feature data of each automated guided vehicle to obtain the target state vector. The rule decision module is used to input the target state vector into the scheduling decision model to obtain the target scheduling rule output by the scheduling decision model; The task sorting module is used to sort all the tasks to be assigned based on the target scheduling rules to obtain a sorting result; The task allocation module is used to determine the target automated guided vehicle corresponding to each of the tasks to be assigned based on the sorting results and the vehicle characteristic data of each of the automated guided vehicles.

[0016] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the scheduling method of the automated guided vehicle as described above.

[0017] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the scheduling method of the automated guided vehicle described in any of the above embodiments.

[0018] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the scheduling method of the automated guided vehicle described above.

[0019] The automated guided vehicle (AGV) scheduling method, apparatus, electronic device, and storage medium provided by this invention comprehensively perceive the system state by aggregating and encoding the characteristic data of tasks and vehicles, and dynamically selects applicable scheduling rules based on the current state using a scheduling decision model. This avoids reliance on a few fixed indicators and static rules, enabling the scheduling system to adapt to complex dynamic working conditions and ensure stable scheduling quality. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0021] Figure 1 This is one of the flowcharts illustrating the scheduling method for automated guided vehicles provided by the present invention.

[0022] Figure 2 This is a schematic diagram of the process for determining the target state vector provided by the present invention.

[0023] Figure 3 This is the second flowchart illustrating the scheduling method for automated guided vehicles provided by this invention.

[0024] Figure 4 This is a schematic diagram of the structure of the scheduling device for automated guided vehicles provided by the present invention.

[0025] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0027] It should be noted that, in the description of this invention, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0028] The terms "first," "second," etc., used in this invention are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, without limiting the number of objects; for example, a first object can be one or more. Those skilled in the art will understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0029] Figure 1 This is one of the flowcharts illustrating the scheduling method for automated guided vehicles provided by the present invention, such as... Figure 1 As shown, the execution subject of the automated guided vehicle scheduling method provided by the present invention can be a server, a cloud computing platform, or a computer capable of executing the method of the present invention, etc. Unless otherwise specified, the following embodiments will be described using a server as an example.

[0030] As an optional embodiment, the scheduling method for the automated guided vehicle mainly includes, but is not limited to, the following steps: Step 110: Perform aggregation encoding on the task feature data of each task to be assigned and the vehicle feature data of each automated guided vehicle to obtain the target state vector.

[0031] Tasks awaiting assignment refer to handling work that is currently generated in the manufacturing and logistics scenario but has not yet been assigned to a specific vehicle for execution. For example, tasks awaiting assignment could be production line transportation tasks within a factory.

[0032] Task feature data refers to information that describes the attributes and status of these tasks to be assigned. For example, task feature data may include the current waiting time of each task, the total distance of the task, the type of the task's starting point, or the task's priority.

[0033] Vehicle characteristic data refers to information that reflects the current status and basic attributes of an automated guided vehicle (AGV). For example, vehicle characteristic data may include whether each AGV is currently idle or performing a task, as well as the coordinates of its current location or its current remaining battery power.

[0034] Aggregate coding refers to the statistical calculation of data generated by different tasks and different vehicles, thereby converting variable inputs into data in a fixed-dimensional form. For example, it can be achieved by calculating the mean, maximum, minimum, standard deviation, and statistical quantities such as the 25th percentile, 50th percentile, and 75th percentile for each feature data.

[0035] The target state vector refers to the data set generated after the above processing that can comprehensively reflect the current task status and the overall status of all vehicles. For example, the target state vector can be a fixed-dimensional vector containing various statistical values, which is used to provide a basis for judgment for subsequent identification of the current working condition.

[0036] Step 120: Input the target state vector into the scheduling decision model to obtain the target scheduling rule output by the scheduling decision model.

[0037] A scheduling decision model refers to an algorithmic model used to determine which allocation strategy to adopt based on the current state of the system. For example, a scheduling decision model can be a master scheduling agent network built based on reinforcement learning, or a network model using a competing deep Q-network (Dueling DQN) structure.

[0038] The target scheduling rule refers to the currently suitable task processing criterion selected by the model after scoring and evaluating multiple pre-set candidate rules. For example, the target scheduling rule can be the longest waiting rule arranged in descending order of task waiting time, or the shortest distance rule arranged in ascending order of the shortest distance from the available automated guided vehicle to the task starting point, or the highest priority rule arranged in descending order of priority score.

[0039] Specifically, the scheduling decision model can adopt a Dueling DQN structure, which includes an input layer, multiple hidden layers, and an output layer connected in sequence. The input to the scheduling decision model is the target state vector obtained through the aforementioned aggregation encoding process, and the dimension of the input layer is consistent with the fixed dimension of the target state vector. The output of the scheduling decision model is an N-dimensional score vector, where N is the number of candidate scheduling rules in the candidate scheduling rule set. Each output node of the output layer corresponds to a candidate scheduling rule, and the nth element in the score vector is the evaluation score of the nth candidate scheduling rule in the current system state.

[0040] In the process of deriving the target scheduling rule from the target state vector, firstly, the target state vector enters the hidden layer through the input layer. The hidden layer can be set as two fully connected layers, and the activation function can be the ReLU function. The target state vector undergoes a layer-by-layer nonlinear transformation through each fully connected layer to extract deep state features representing the current system condition. Then, the deep state features output from the hidden layer are fed into the value stream branch and the dominance stream branch, respectively. The value stream branch outputs the state value, which represents the overall goodness or badness of the current system state. The dominance stream branch outputs the dominance value corresponding to each candidate scheduling rule, which represents the relative superiority or inferiority of selecting the corresponding candidate scheduling rule relative to the average level under the current system state. Finally, the state value and the mean-reduced dominance value are aggregated to obtain the action value corresponding to each candidate scheduling rule output by the output layer, i.e.: ; in, Represents the target state vector. This represents the candidate scheduling rules currently being evaluated; This represents any candidate scheduling rule in the candidate scheduling rule set, and is used to iterate and sum over all candidate scheduling rules in the candidate scheduling rule set; This indicates the number of candidate scheduling rules in the candidate scheduling rule set; Represents the target state vector s Next, select candidate scheduling rules a The corresponding action value is used to characterize the candidate scheduling rule to be adopted in the current system state. a The expected long-term benefits that can be obtained by sorting tasks; The value of the output of the value stream branch is used to characterize the target state vector. s The overall quality of the current system state itself; The target state vector represents the output of the dominant flow branch. s Next candidate scheduling rules a The corresponding advantage value is used to characterize the selection of candidate scheduling rules under the current system state. a The relative merits or demerits of the scheduling rules compared to the average level of all candidate scheduling rules; Represents the target state vector s Next candidate scheduling rules The corresponding advantage value.

[0041] For example, if the candidate scheduling rule set includes five rules: longest waiting rule, shortest distance rule, longest path rule, shortest path rule, and highest priority rule, the output layer dimension of the scheduling decision model is 5. If the score vector output by the scheduling decision model is [0.62, 0.35, 0.18, 0.24, 0.87], and the highest priority rule has the largest evaluation score of 0.87, then the highest priority rule is determined as the target scheduling rule for the current scheduling cycle.

[0042] During the training phase, the scheduling decision model can adopt... ε - A greedy (epsilon-greedy) strategy is used to explore actions, i.e., using the exploration rate... ε A candidate scheduling rule is randomly selected based on probability ( Select the candidate scheduling rule with the highest evaluation score, and gradually decrease the exploration rate as the training progresses. ε The value of is chosen to fully explore the effects of each candidate scheduling rule in the early stage of training, and to utilize the learned decision-making experience in the later stage of training. After each scheduling cycle, an experience tuple consisting of the target state vector of the current cycle, the selected candidate scheduling rule, the first reward data, and the target state vector of the next scheduling cycle is stored in the aforementioned first experience buffer. During training, experience tuples are sampled from the first experience buffer. Based on the first reward data, the discount factor, and the maximum action value in the next state output by the target network with the same structure, the target action value is calculated. A loss function is constructed using the temporal difference error between the action value output by the current network and the target action value. The network parameters of the scheduling decision model are updated through gradient descent. At the same time, the parameters of the current network are synchronized to the target network every preset number of cycles to ensure the stability of the training process.

[0043] During the deployment phase, the network weights of the trained scheduling decision model are loaded, and random exploration is no longer performed. The real-time target state vector is directly input into the scheduling decision model, and the candidate scheduling rule with the highest evaluation score is selected as the target scheduling rule.

[0044] By adopting the scheduling decision model with the above structure, on the one hand, the separation of the value stream branch and the dominant stream branch enables the model to obtain an accurate assessment of the system state through state value learning even when some candidate scheduling rules are selected less frequently, thus accelerating the training convergence speed of the model and improving the accuracy of the evaluation score; on the other hand, the model's input is a fixed-dimensional target state vector, and the output is a fixed... NThe multidimensional score vector has an action space that is independent of the specific number of tasks and vehicles, ensuring that the time taken for a single decision does not increase with the increase in the number of vehicles and tasks. Furthermore, it can dynamically switch between multiple candidate scheduling rules based on the current working conditions, avoiding the problem of unstable scheduling quality caused by relying on a single static rule.

[0045] Step 130: Sort all tasks to be assigned based on the target scheduling rules to obtain the sorting results.

[0046] Specifically, according to the specific rules selected by the scheduling decision model, all tasks currently waiting in the queue are arranged in order to form an ordered task processing queue. For example, in a peak backlog scenario with a large number of tasks, if the target scheduling rule given by the scheduling decision model is the highest priority rule, all tasks will be arranged in descending order of priority score, so that tasks with higher priority scores are placed at the front of the queue and processed first.

[0047] Step 140: Based on the sorting results and the vehicle characteristic data of each automated guided vehicle, determine the target automated guided vehicle corresponding to each task to be assigned.

[0048] Specifically, after obtaining the sorted task queue, each task is retrieved sequentially. Combined with the current working status and location coordinates of each vehicle recorded in the vehicle feature data, a suitable available vehicle is found to perform the transport work for the current task. For example, when assigning vehicles to each task according to the task sorting results, the vehicle coordinate data can be used to calculate and find the available vehicle closest to the current task's starting point from the currently idle automated guided vehicles (AGVs). This closest vehicle is then designated as the target AGV for the current task, thus completing the task assignment.

[0049] The automated guided vehicle (AGV) scheduling method provided by this invention comprehensively perceives the system state by aggregating and encoding the feature data of tasks and vehicles, and dynamically selects applicable scheduling rules based on the current state using a scheduling decision model. This avoids reliance on a few fixed indicators and static rules, enabling the scheduling system to adapt to complex dynamic working conditions and ensuring stable scheduling quality.

[0050] Figure 2 This is a flowchart illustrating the process of determining the target state vector provided by the present invention, as shown below. Figure 2 As shown, as another optional embodiment provided by the present invention, the task feature data of each task to be assigned and the vehicle feature data of each automated guided vehicle are aggregated and encoded to obtain the target state vector, including but not limited to the following steps: Step 210: Perform multi-dimensional feature extraction on the task feature data of each task to be assigned and the vehicle feature data of each automated guided vehicle to obtain task aggregate features and guided vehicle aggregate features.

[0051] Task aggregation features refer to data that reflects the overall distribution of all tasks by performing statistical calculations on the specific feature values ​​of multiple tasks. For example, task aggregation features could be the maximum, minimum, and average values ​​of the waiting time for all tasks to be assigned, as well as statistics such as the 25th percentile and 50th percentile.

[0052] Guided vehicle aggregation features refer to data that reflects the overall operational status of a fleet after performing statistical calculations on the feature values ​​of multiple vehicles. For example, guided vehicle aggregation features may be statistical measures such as the mean, standard deviation, and 75th percentile calculated for the remaining battery power or location coordinates of all vehicles.

[0053] Step 220: Determine global characteristics based on the number of tasks to be assigned, the number of available automated guided vehicles, and the system load status at the current moment.

[0054] The number of tasks to be assigned, the number of available automated guided vehicles (AGVs), and the system load status refer to the total number of unassigned handling jobs in the current system, the total number of vehicles in an idle state that can directly respond to dispatch, and indicators reflecting the overall busyness of the current dispatch system, respectively. For example, the number of tasks to be assigned could be 20 tasks currently waiting in the queue, the number of available AGVs could be 18 vehicles currently idle and waiting, and the system load status could be the overall load rate of the current dispatch system.

[0055] Global features refer to a set of data that can describe the overall operating environment and state of the entire scheduling system at the current moment from a macroscopic perspective. For example, global features can be data that combines the aforementioned number of tasks, number of available vehicles, and load status, and adds information such as the current time node, the frequency of task arrival, and the proportion of high-priority tasks in the total number of tasks.

[0056] Step 230: Based on the resource occupancy status of the transit node, the number of resident automated guided vehicles, and task prediction information, determine the pre-scheduling extension characteristics.

[0057] A transit node refers to a temporary parking area set up inside a factory or warehouse to arrange for idle vehicles to go there in advance to wait for upcoming tasks. For example, a transit node can be a temporary standby position set up next to a production line.

[0058] Resource occupancy status, number of stationed automated guided vehicles (AGVs), and task prediction information refer to the current proportion of vehicles occupying the transit area, the specific number of idle vehicles currently parked in the area ready to accept orders, and the estimated number and type of new tasks that may be generated in the future. For example, resource occupancy status can be the current occupancy rate of a transit node group, the number of stationed AGVs can be the 3 idle vehicles waiting at the node, and task prediction information can be the expected number of various types of tasks arriving in the future and related supply and demand ratio data.

[0059] Pre-scheduling extended features refer to a set of data specifically used to help the model determine whether vehicles need to be pre-positioned to wait at a designated location. For example, pre-scheduling extended features may include information such as the previous node group occupancy rate, the number of stationed vehicles, the number of predictions for each type of task, and the vehicle stationing hit rate.

[0060] Step 240: The task aggregation features, the guide vehicle aggregation features, the global features, and the pre-scheduled extended features are concatenated to obtain the target state vector; wherein, the target state vector is a feature vector with fixed dimensions.

[0061] Specifically, in a fixed order, the previously calculated task-related statistical data and vehicle-related statistical data, together with macro data reflecting the overall system situation and predictive data used for advance scheduling, are combined to form a data column of constant length. For example, the average task waiting time, the maximum value of vehicle position coordinates, the current overall load rate of the system, and the dwell hit rate of transfer stations are sequentially concatenated to form a fixed-length feature vector that can comprehensively represent the current operating status of the system.

[0062] It should be noted that, considering the dynamic fluctuations in the number of tasks waiting to be assigned and the total number of vehicles actually available in actual logistics operations, directly using these variable raw data would result in inconsistent feature data lengths input to the decision model. Therefore, this invention extracts and merges fixed statistical quantities to ensure that input features of different scales can ultimately be transformed into target state vectors of fixed dimensions. This allows the same model weights to be directly applied to new scenarios with different vehicle and task scales. Even if the number of automated guided vehicles is subsequently increased or decreased in the factory, there is no need to redesign the scheduling network or train it from scratch, thus improving the adaptability of the scheduling method to different scenarios.

[0063] The automated guided vehicle (AGV) scheduling method provided by this invention extracts multi-dimensional aggregated features of tasks and vehicles separately, and combines them with global features reflecting the current macroscopic situation of the system and pre-scheduling extended features predicting future states to construct a target state vector with fixed dimensions. This provides a more comprehensive and objective basis for state judgment for the scheduling model. Furthermore, leveraging this fixed-dimensional characteristic, the scheduling network can directly load and reuse existing model weights without redesigning or retraining when facing new scenarios involving increases or decreases in the number of vehicles or expansions in task scale, thus improving the method's adaptability to different operating scenarios.

[0064] In another embodiment of the present invention, multi-dimensional feature extraction is performed on the task feature data of each task to be assigned and the vehicle feature data of each automated guided vehicle to obtain task aggregate features and guided vehicle aggregate features, including: combining the task feature data of each task to be assigned into a task feature matrix and combining the vehicle feature data of each automated guided vehicle into a vehicle feature matrix; and performing statistical calculations on the task feature matrix and the vehicle feature matrix column by column to obtain task aggregate features and guided vehicle aggregate features.

[0065] Specifically, the characteristic values ​​of all tasks to be assigned are arranged and combined in rows to form a task characteristic matrix, and the characteristic values ​​of all automated guided vehicles are arranged and combined in rows to form a vehicle characteristic matrix. Then, for each column of these two matrices, various statistical values ​​of the data in that column are calculated.

[0066] For example, for the column of data that records the waiting time in the task feature matrix, we can calculate the average, maximum, minimum, and standard deviation of the waiting time, as well as the 25th, 50th, and 75th percentiles, which are seven statistical measures. By combining these statistical values ​​calculated from all columns, we obtain the task aggregate features. The same process is used to calculate the vehicle feature matrix column by column.

[0067] It should be noted that, considering that simply using single data such as average values ​​often fails to accurately reflect the true distribution of task backlog in the system, this invention incorporates multiple statistical measures such as quantiles at different proportions to perform column-wise aggregation operations. This allows the calculated aggregated features to more objectively reflect the distribution skewness of data such as task waiting time. For example, when some tasks are backlogged for a long time, the calculated 75th percentile will be higher than the 50th percentile. This provides a more realistic basis for the subsequent scheduling model to accurately determine the current working condition and switch scheduling rules.

[0068] As an optional embodiment, the various statistics obtained by column-wise calculation (such as maximum, minimum, average, and quantiles) can be replaced with other types of statistics such as skewness, kurtosis, exponentially weighted mean, and truncated mean. Alternatively, the specific number of statistics can be increased or decreased according to computational power requirements. Furthermore, besides using statistical calculation methods, the column-wise statistical aggregation operation in this embodiment can also be replaced entirely with global pooling using multi-head attention mechanisms, graph-level aggregation using graph neural networks (GNNs), or [CLS] vector encoding in the Transformer model. As long as the core operation of converting the variable-length input feature matrix into a fixed-dimensional feature vector output remains unchanged, the above-mentioned encoding method replacements are all equivalent implementations applicable to this invention, and the replaced encoding methods are more suitable for complex scenarios with higher requirements for modeling relationships between entities.

[0069] The automated guided vehicle (AGV) scheduling method provided by this invention combines the task feature data of each task to be assigned into a task feature matrix and the vehicle feature data of each AGV into a vehicle feature matrix. Then, statistical calculations are performed on the task feature matrix and the vehicle feature matrix column by column. This method can effectively extract comprehensive indicators that reflect the data distribution status, objectively present the distribution skewness of tasks and vehicles in the system, and provide a more accurate basis for subsequent scheduling decisions.

[0070] In another embodiment provided by the present invention, the target state vector is input into the pre-scheduling decision model to obtain the pre-scheduling decision result output by the pre-scheduling decision model; if the pre-scheduling decision result is to execute the deployment, the target transfer node and the number of vehicles to be deployed are determined based on the pre-scheduling decision result; according to the number of vehicles to be deployed, the automated guided vehicles in the idle state are scheduled to the target transfer node.

[0071] A pre-scheduling decision model refers to a network model established independently outside of the main scheduler to determine whether vehicles need to be deployed in advance. For example, a pre-scheduling decision model can be an independent pre-scheduling agent.

[0072] The pre-schedule decision result refers to the model's judgment instruction on whether it is necessary to pre-deploy idle vehicles to a designated area. For example, the pre-schedule decision result may be a specific instruction to execute the deployment or not to execute the deployment.

[0073] Specifically, the pre-scheduling decision model can be an independent pre-scheduling agent constructed using a competing deep Q-network (Dueling DQN) structure. This pre-scheduling agent is independent of the scheduling decision model used by the aforementioned main scheduler, and the two do not share network parameters.

[0074] The network structure of the pre-scheduled decision model consists of an input layer, multiple fully connected hidden layers, and an output layer connected in sequence. The input to the pre-scheduled decision model is the target state vector obtained through the aforementioned aggregation encoding process. The dimension of the input layer is consistent with the fixed dimension of the target state vector, and it is used to receive the target state vector. The hidden layers can be set to two fully connected layers, with the number of neurons in each layer set to 128 and 64 respectively. The activation function is the Rectified Linear Unit (ReLU).

[0075] After the hidden layer, the system is divided into a value stream branch and an advantage stream branch. The value stream branch outputs the state value, which represents the overall quality of the current system state. The advantage stream branch outputs the advantage value corresponding to each pre-scheduled action. This advantage value represents the relative quality of executing the corresponding pre-scheduled action relative to the average level in the current system state.

[0076] The output of the pre-scheduling decision model is the action value corresponding to each pre-scheduling action. This action value is obtained by adding and aggregating the state value output by the value flow branch and the advantage value of each pre-scheduling action output by the advantage flow branch after mean removal. The aggregation method is the same as the aggregation method of action value in the aforementioned scheduling decision model, and will not be repeated here. The action value is used to characterize the expected long-term benefit that can be obtained by executing the corresponding pre-scheduling action in the current system state.

[0077] The action space of the pre-scheduling decision model is a predefined set of discrete actions, including the action "do not perform dispatch" and "to the first...". i Deployment at each transit node j The combined actions of the "automated guided vehicle" include i The value range is the total number of relay nodes. M , j The value range is the maximum number of vehicles allowed to be deployed in a single instance. K That is, the dimension of the action space is M × K +1, meaning the dimension of the output layer is M × K +1, where "1" refers to the action of "not executing the deployment". Each output node in the output layer corresponds one-to-one with a pre-scheduled action. For example, in a scenario with 3 transit nodes and a maximum of 2 vehicles deployed at a time, the action space includes a total of 7 discrete actions.

[0078] In the process of obtaining the pre-scheduling decision result from the target state vector, after the target state vector is input into the pre-scheduling decision model, the target state vector enters the hidden layer through the input layer. It then undergoes layer-by-layer nonlinear transformation through each fully connected layer to extract deep state features representing the current system operating condition. These features are then processed and aggregated through value stream branches and dominant stream branches, respectively. The output layer outputs the action value corresponding to each discrete action in the action space, and the action with the highest action value is determined as the pre-scheduling decision result. If the action is "do not perform dispatch," then no vehicle dispatch will be performed in this scheduling cycle; if the action is "to the first...", then... i Deployment at each transit node j "Automated Guided Vehicle" will then be the first i One transit node was identified as the target transit node. j The number of vehicles to be dispatched has been determined.

[0079] During the training phase, the pre-scheduling decision model can also be used. ε - A greedy (epsilon-greedy) strategy is used to explore actions, i.e., using the exploration rate... ε A pre-scheduled action is randomly selected with probability ( The pre-scheduled action with the highest action value is selected, and the exploration rate ε is gradually decayed as the training progresses. At the end of each scheduling cycle, the experience tuple consisting of the target state vector of the current cycle, the selected pre-scheduled action, the aforementioned second reward data, and the target state vector of the next scheduling cycle is stored in the aforementioned second experience buffer. During training, experience tuples are sampled from the second experience buffer. Based on the second reward data, the discount factor, and the maximum action value in the next state output by the target network with the same structure, the target action value is calculated. A loss function is constructed using the temporal difference error between the action value output by the current network and the target action value. The network parameters of the pre-scheduled decision model are updated through gradient descent, and the parameters of the current network are synchronized to the target network every preset number of cycles.

[0080] Since the pre-scheduling decision model and the scheduling decision model use independent experience buffers and update parameters separately, their reward signals do not interfere with each other during training, thus enabling independent training and collaborative optimization of the two models.

[0081] During the deployment phase, the network weights of the pre-scheduled decision model that has been trained are loaded. Instead of random exploration, the real-time target state vector is directly input into the pre-scheduled decision model, and the pre-scheduled action with the highest action value is selected as the pre-scheduled decision result.

[0082] By adopting the pre-scheduling decision model with the above structure, on the one hand, its separation structure of value stream branch and advantage stream branch can accelerate the training and convergence speed of the model. Even when some pre-scheduling actions are selected less often, an accurate assessment of the system state can be obtained through learning the state value. On the other hand, the pre-scheduling decision model is independent of the main scheduling decision model and can focus on the advance deployment decision of vehicles, transforming the passive response scheduling into an active pre-scheduling mode. This allows idle vehicles to be pre-deployed before the task arrives, thereby shortening the travel time of vehicles to the task starting point and improving the overall utilization rate of vehicles.

[0083] Specifically, the target state vector, containing overall system state information, is fed into the pre-scheduling decision model. The pre-scheduling decision model assesses whether there are any spare vehicles in the current system and whether specific areas will generate corresponding task demands in the future. If the assessment result is that deployment is required, it further clarifies which specific transfer location the spare vehicles should be sent to and how many vehicles need to be mobilized.

[0084] For example, during a low-load operation phase with a large number of idle vehicles and few tasks to be assigned, if the model, based on predictive information, finds that the expected task demand of a certain production line's transfer node group is higher than the number of vehicles currently parked, it will output the decision result of execution deployment and instruct the corresponding number of idle vehicles to be deployed to that transfer node in advance to enter a standby state.

[0085] It should be noted that, considering that previous scheduling methods were passively triggered when a task arrived and a vehicle was idle, resulting in the vehicle being completely idle during the waiting period, this invention proactively dispatches idle vehicles to transfer nodes to wait before a task arrives. This transforms passive response scheduling into an active pre-scheduling mode, allowing nearby vehicles to respond directly to new tasks as soon as they arise, shortening the travel time for vehicles to reach the task starting point and improving the overall utilization rate of vehicles.

[0086] The automated guided vehicle (AGV) scheduling method provided by this invention obtains a pre-scheduling decision result by inputting the target state vector into a pre-scheduling decision model. When the decision result is to execute deployment, the AGV in the idle state is pre-scheduled to the target transfer node to stand by. This can transform passive response scheduling into an active pre-scheduling mode, enabling vehicles to be pre-deployed before the task arrives, so that they can directly respond to the nearest location when the task occurs, shortening the time it takes for vehicles to travel to the task starting point and improving the overall utilization rate of vehicles.

[0087] In another embodiment provided by the present invention, after the current scheduling cycle ends, the first reward data corresponding to the scheduling decision model and the second reward data corresponding to the pre-scheduling decision model are determined respectively; the first reward data is added to the first experience buffer and the second reward data is added to the second experience buffer; the network parameters of the scheduling decision model are updated based on the data in the first experience buffer and the network parameters of the pre-scheduling decision model are updated based on the data in the second experience buffer.

[0088] The current scheduling period refers to the time interval between the system performing a complete scheduling and allocation operation. For example, the current scheduling period can be set from 5 seconds to 30 seconds according to the needs of the scenario.

[0089] The first reward data refers to the score used to evaluate the effectiveness of the main scheduling network in allocating tasks within the current period. For example, the first reward data can be a score calculated based on various task execution data such as task completion rate, average task waiting time, and average task completion time. It can be determined by obtaining the task execution data within the current scheduling period.

[0090] The second reward data refers to the score used to evaluate the effectiveness of the pre-scheduled network in dispatching vehicles ahead of schedule in the current period. For example, the second reward data can be a score calculated based on various resource layout data such as pre-scheduled hit rate and resource utilization balance, which can be determined by obtaining the resource layout data in the current scheduling period.

[0091] The first experience buffer and the second experience buffer refer to the storage spaces in the main scheduling network and the pre-scheduling network, respectively, which are dedicated to storing historical scheduling experience independently. For example, these two buffers can be independent priority experience replay buffers.

[0092] Specifically, after each scheduling cycle, the reward scores for the corresponding main scheduler and pre-scheduler are calculated separately and written to their respective independent storage spaces. Then, historical experience data is extracted from these two buffers. The data extracted from the first experience buffer is used to update the network parameters of the scheduling decision model, while the data extracted from the second experience buffer is used to update the network parameters of the pre-scheduling decision model.

[0093] For example, after every 10 scheduling cycles, samples are taken independently from the first and second experience buffers according to the priority experience replay mechanism to update the network parameters of the scheduling decision model and the pre-scheduling decision model respectively. By keeping the update process isolated from each other, both scheduling decision models can continuously improve the decision quality by accumulating historical experience.

[0094] It should be noted that the network model structure used for the main scheduler in this invention is not limited to the Dueling DQN in the embodiments. In actual deployment, it can also be replaced by Double Deep Q-Network (DoubleDQN), a variant of Priority Experience Replay DQN, or policy gradient algorithms such as Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC). As long as the two-level decision structure of "fixed-dimensional state input and rule number output" is maintained, any change in the network structure is an equivalent substitution of this invention.

[0095] Furthermore, for decoupled training of dual agents, the independent setting of the first experience buffer and the second experience buffer in the embodiment can also be replaced by a mechanism that uses a single shared experience buffer but samples independently according to the agents; the execution order of the main scheduler and the pre-scheduler can also be interchanged or changed to parallel decision-making; the reward functions used for the corresponding first reward data and second reward data can also be replaced equivalently according to the focus of the actual business.

[0096] The automated guided vehicle (AGV) scheduling method provided by this invention determines the first reward data and the second reward data corresponding to the scheduling decision model and the pre-scheduling decision model respectively after the end of the current scheduling cycle, and adds them to independent first experience buffers and second experience buffers for separate network parameter updates. This enables gradient isolation and independent training of the two decision models, avoids mutual interference between the reward signals of the two models during updates, and allows the scheduling system to continuously accumulate experience from historical scheduling processes and independently iterate and optimize the model network, thereby improving the accuracy of collaborative decision-making between the two models.

[0097] In another embodiment provided by the present invention, determining the first reward data corresponding to the scheduling decision model and the second reward data corresponding to the pre-scheduling decision model includes: obtaining task execution data within the current scheduling period, wherein the task execution data includes at least the task completion rate, the average waiting time of the task, and the average completion time of the task; determining the first reward data corresponding to the scheduling decision model based on the task execution data; obtaining resource layout data within the current scheduling period, wherein the resource layout data includes at least the pre-scheduling hit rate and the resource occupancy balance; and determining the second reward data corresponding to the pre-scheduling decision model based on the resource layout data.

[0098] Task completion rate, average task waiting time, and average task completion time refer to the proportion of tasks successfully assigned to the total number of tasks in the current time period, the average time from when all tasks start queuing to when they are accepted by a vehicle, and the average time from when a task is generated to when it is finally transported.

[0099] For example, the task completion rate could be that 90% of the tasks in the current period have been processed, the average task waiting time could be the average time that all tasks have been waiting in the queue, and the average task completion time could be the average time that the test set reaches the starting point, which is 1.75 minutes.

[0100] The primary reward data can be determined by comprehensively considering these task execution data, assigning a positive score to the task completion rate, a penalty score to the average task waiting time and average task completion time, and adding a component such as the penalty for high-priority tasks. The final main scheduler reward score is obtained by calculating all the scores.

[0101] Pre-scheduled hit rate and resource utilization balance refer to the proportion of vehicles pre-arranged to transfer nodes that subsequently successfully receive surrounding tasks, and the evenness of the distribution of the number of vehicles parked in each transfer area of ​​the system, respectively.

[0102] For example, the pre-scheduling hit rate can be defined as the success rate of vehicles dispatched in advance responding to new tasks directly and nearby, reaching 80%. Resource utilization balance can be defined as data reflecting whether there is excessive vehicle clustering at each node.

[0103] The second reward data can be determined by comprehensively evaluating factors such as deployment timing, pre-schedule hit rate, and resource utilization balance. A higher hit rate is given a positive score, while points are deducted for situations that result in idle resources or uneven distribution. This is how the reward score corresponding to the pre-schedule decision model is calculated.

[0104] The automated guided vehicle scheduling method provided by this invention determines the first reward data by acquiring task execution data including task completion rate, average task waiting time, and average task completion time, and determines the second reward data by acquiring resource layout data including pre-scheduling hit rate and resource occupancy balance. This method can provide specific quantitative evaluation indicators that match the optimization objectives of the scheduling decision model and the pre-scheduling decision model, making the reward signals of the two models clearly targeted.

[0105] In another embodiment of the present invention, the target automated guided vehicle (AGV) corresponding to each task to be assigned is determined based on the sorting results and the vehicle feature data of each AGV. This includes: traversing all tasks to be assigned according to the sorting results to determine the current task to be assigned; determining multiple candidate AGVs based on the vehicle feature data, wherein the current working state of the candidate AGVs is idle or pre-scheduled; determining the distance between the current task to be assigned and each candidate AGV; and determining the candidate AGV with the smallest distance to the current task to be assigned as the target AGV corresponding to the current task to be assigned.

[0106] Specifically, each task that needs to be processed is viewed in the order of the sorting results. Then, the vehicles that can currently accept orders are selected as candidates based on their status information. These candidate vehicles include not only idle vehicles with no tasks, but also pre-scheduled vehicles that have been pre-assigned to transfer nodes and are in the standby stage.

[0107] Next, calculate the distance between the starting point of the current task and the current location of all candidate vehicles. Finally, use a greedy matching method to select the available vehicle closest to the task starting point and assign it directly to the current task to perform the transport work.

[0108] For example, if the current task requires moving materials from a production line, there are two idle vehicles and one pre-scheduled vehicle parked at a nearby transfer node in the scheduling system. If the distance calculation shows that the pre-scheduled vehicle parked at the transfer node is closest to the task start point of the production line, then this pre-scheduled vehicle will be directly used as the target vehicle for executing the task.

[0109] It should be noted that, considering that traditional methods often only focus on completely idle vehicles when allocating tasks, while vehicles that are heading to transfer nodes or are already waiting at transfer stations are not included in the scope of real-time allocation, which can easily lead to the idleness of nearby resources, this invention includes vehicles in the pre-scheduled state when determining candidate vehicles for unified distance calculation and matching. This expands the screening range of available vehicles, allowing the pre-scheduled resources closest to the task to participate in and respond to the allocation needs of the main scheduler at any time, further shortening the time it takes for vehicles to travel to the task starting point.

[0110] The automated guided vehicle (AGV) scheduling method provided by this invention expands the selection range of available vehicles by using vehicles in an idle or pre-scheduled state as candidate AGVs and matching the candidate AGV with the shortest distance to the currently assigned task. This allows pre-scheduled vehicles that have been sent to transfer nodes in advance to participate in and respond to the allocation requirements of the main scheduler in a timely manner, avoiding the idleness of nearby resources and further shortening the travel time of vehicles to the task starting point.

[0111] In another embodiment of the present invention, after determining the candidate automated guided vehicle (AGV) with the smallest distance to the current task to be assigned as the target AGV corresponding to the current task to be assigned, the method further includes: if the current working state of the target AGV is a pre-scheduled state, generating an interrupt scheduling instruction for the target AGV; and based on the interrupt scheduling instruction, controlling the target AGV to move from its current position to execute the current task to be assigned.

[0112] Specifically, after confirming the target automated guided vehicle (AGV), if the selected vehicle is currently in a pre-scheduled state, either waiting to proceed to a transfer node or already waiting at a transfer node, an interrupt operation will be triggered. This will release the pre-scheduled resources originally bound to the vehicle and send an interrupt scheduling command to the vehicle. Upon receiving the command, the vehicle will abandon its original waiting arrangement and depart directly from its current location to the starting point of the new task to perform the transport work.

[0113] For example, an automated guided vehicle (AGV) was originally assigned to stand by transfer node 1. When it was halfway there, the main scheduling process found that it was closest to the starting point of a newly generated task and selected it as the target vehicle. At this time, an interrupt scheduling command was sent to it, and the vehicle would immediately stop its action of standing by transfer node and directly change its route from the current road segment to the starting point of the new task.

[0114] It should be noted that, considering that if a vehicle is forcibly bound to a standby task during the pre-scheduled execution period, it will be unable to respond flexibly when a new task suddenly appears nearby, which may easily lead to a waste of vehicle resources, this invention introduces an interrupt release mechanism to allow vehicles in the pre-scheduled state to be called by the main scheduling process at any time and depart directly from the current location. This ensures the flexible allocation of vehicle resources, avoids vehicles rigidly executing pre-assigned tasks, and improves the overall utilization rate of vehicles.

[0115] The automated guided vehicle (AGV) scheduling method provided by this invention generates an interrupt scheduling command when the target AGV is in a pre-scheduled state and controls the target AGV to go directly from its current location to execute the currently assigned task. This enables the immediate release of vehicle resources that are on standby or en route to a transfer point, breaks the mandatory binding of pre-assigned tasks, and avoids vehicles rigidly executing pre-scheduled tasks and missing the opportunity to accept orders nearby. This ensures the flexibility of vehicle resource allocation and further improves the overall utilization rate of vehicles.

[0116] In another embodiment provided by the present invention, the target state vector is input into the scheduling decision model to obtain the target scheduling rule output by the scheduling decision model, including: obtaining a candidate scheduling rule set, the candidate scheduling rule set including multiple candidate scheduling rules; inputting the target state vector into the scheduling decision model, determining the evaluation score corresponding to each candidate scheduling rule according to the scheduling decision model; and determining the candidate scheduling rule with the largest evaluation score as the target scheduling rule.

[0117] Candidate scheduling rules refer to a variety of pre-defined strategies and criteria for prioritizing tasks. For example, candidate scheduling rules may include the longest waiting rule arranged in descending order of task waiting time, the nearest distance rule arranged in ascending order of distance, or the deadline priority rule.

[0118] Specifically, first, a set of rules containing multiple strategies is obtained. Then, the target state vector representing the current overall operating condition of the system is input into the scheduling decision model. The scheduling decision model will calculate each rule in the set based on the current state and give a score for each rule under the current operating condition. Finally, the server selects the rule with the highest score as the target scheduling rule to be executed.

[0119] For example, in a peak backlog scenario, after the scheduling decision model scores the five rules in the set, if the highest priority rule is given the highest evaluation score, it will be determined as the target scheduling rule for the current period, so as to prioritize the processing of backlogged high-priority tasks.

[0120] It should be noted that, considering that if the scheduling decision model were to directly determine the specific pairing relationship between each task and each vehicle, the number of matching combinations that the scheduling decision model needs to calculate would become very large as the number of vehicles and tasks increases, making training difficult, this invention defines the output of the scheduling decision model as selecting a rule from a preset set. This makes the action space of the scheduling decision model fixed and independent of the specific number of tasks and vehicles, thereby accelerating the training and convergence speed of the scheduling decision model, ensuring that the time taken for a single decision does not increase with the increase in the number of vehicles, and automatically switching to appropriate scheduling rules according to different system operating conditions.

[0121] The predefined candidate scheduling rule set of this invention is not limited to the five rules exemplified in the foregoing embodiments, such as longest waiting time and closest distance. In practical applications, the candidate scheduling rule set can be expanded to N types according to the specific factory environment and logistics scenario requirements. For example, it can also include comprehensive cost priority rules, deadline priority rules, regional aggregation rules, etc. Correspondingly, the action space of the scheduling decision model will also be adjusted to N dimensions.

[0122] Figure 3 This is the second flowchart illustrating the scheduling method for automated guided vehicles provided by this invention, as shown below. Figure 3 As shown, after the scheduling process begins, the first step is to perceive the system status. Specifically, this involves collecting data from the current task queue and the automated guided vehicle queue, and then performing aggregation and encoding to extract various features. Next, DQN is used to score the rules. Figure 3 The document shows five specific candidate rules involved in the scoring evaluation: longest wait, shortest distance, longest journey, shortest journey, and highest priority.

[0123] After scoring and evaluating each rule, the operation of selecting the rule with the highest score is executed. Tasks are then assigned to AGVs according to the rule; that is, the tasks to be assigned are sorted according to the specific requirements of the selected rule, and suitable AGVs are matched to each task for execution. Next, a determination is made as to whether to deploy the AGV to a transfer station. If the determination is yes, an idle AGV is controlled to go to the transfer station in advance to wait and complete the active deployment of the vehicle; if the determination is no, the process proceeds directly to the reward calculation step.

[0124] In the reward calculation step, the reward data corresponding to the main scheduler and pre-scheduler in the current period are calculated separately. Then, the DQN update operation is performed, updating the network parameters using saved experience data. After the network parameter update is complete, the next scheduling cycle begins, and the process loops back to the step of sensing the system state to continue execution, thus enabling the entire scheduling process to run cyclically according to a fixed scheduling cycle and continuously optimize the decision network.

[0125] Figure 4 This is a schematic diagram of the structure of the automated guided vehicle (AGV) scheduling device provided by the present invention, as shown below. Figure 4 As shown, the scheduling device for automated guided vehicles mainly includes, but is not limited to: The aggregation encoding module 410 is used to perform aggregation encoding processing on the task feature data of each task to be assigned and the vehicle feature data of each automated guided vehicle to obtain the target state vector.

[0126] The rule decision module 420 is used to input the target state vector into the scheduling decision model and obtain the target scheduling rule output by the scheduling decision model.

[0127] The task sorting module 430 is used to sort all tasks to be assigned based on the target scheduling rules and obtain the sorting result; The task allocation module 440 is used to determine the target automated guided vehicle corresponding to each task to be assigned based on the sorting results and the vehicle characteristic data of each automated guided vehicle.

[0128] It should be noted that the automated guided vehicle scheduling device provided by the present invention can execute the automated guided vehicle scheduling method described in any of the above embodiments during actual operation, which will not be elaborated in this embodiment.

[0129] The automated guided vehicle (AGV) scheduling device provided by this invention comprehensively perceives the system state by aggregating and encoding the feature data of tasks and vehicles, and dynamically selects applicable scheduling rules based on the current state using a scheduling decision model. This avoids reliance on a few fixed indicators and static rules, enabling the scheduling system to adapt to complex dynamic working conditions and ensuring stable scheduling quality.

[0130] Figure 5This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 5 As shown, the electronic device may include a processor 510, a communications interface 520, a memory 530, and a communication bus 540. The processor 510, communications interface 520, and memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a scheduling method for automated guided vehicles (AGVs). This method includes: performing aggregation encoding processing on the task feature data of each task to be assigned and the vehicle feature data of each AAV to obtain a target state vector; inputting the target state vector into a scheduling decision model to obtain the target scheduling rule output by the scheduling decision model; sorting all tasks to be assigned based on the target scheduling rule to obtain a sorting result; and determining the target AAV corresponding to each task to be assigned based on the sorting result and the vehicle feature data of each AAV.

[0131] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0132] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the automated guided vehicle (AGV) scheduling method provided by the above methods. The method includes: performing aggregation encoding processing on the task feature data of each task to be assigned and the vehicle feature data of each AGV to obtain a target state vector; inputting the target state vector into a scheduling decision model to obtain a target scheduling rule output by the scheduling decision model; sorting all tasks to be assigned based on the target scheduling rule to obtain a sorting result; and determining the target AGV corresponding to each task to be assigned based on the sorting result and the vehicle feature data of each AGV.

[0133] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the scheduling method for automated guided vehicles (AGVs) provided by the above methods. The method includes: performing aggregation encoding processing on the task feature data of each task to be assigned and the vehicle feature data of each AAV to obtain a target state vector; inputting the target state vector into a scheduling decision model to obtain a target scheduling rule output by the scheduling decision model; sorting all tasks to be assigned based on the target scheduling rule to obtain a sorting result; and determining the target AAV corresponding to each task to be assigned based on the sorting result and the vehicle feature data of each AAV.

[0134] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0135] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for scheduling automated guided vehicles, characterized in that, include: The task feature data of each task to be assigned and the vehicle feature data of each automated guided vehicle are aggregated and encoded to obtain the target state vector. The target state vector is input into the scheduling decision model to obtain the target scheduling rule output by the scheduling decision model; Based on the target scheduling rules, all the tasks to be assigned are sorted to obtain the sorting result; Based on the sorting results and the vehicle characteristic data of each of the automated guided vehicles, the target automated guided vehicle corresponding to each of the tasks to be assigned is determined.

2. The scheduling method for automated guided vehicles according to claim 1, characterized in that, The process of aggregating and encoding the task feature data of each task to be assigned and the vehicle feature data of each automated guided vehicle to obtain the target state vector includes: Multi-dimensional feature extraction is performed on the task feature data of each of the tasks to be assigned and the vehicle feature data of each of the automated guided vehicles to obtain task aggregate features and guided vehicle aggregate features. Global characteristics are determined based on the number of tasks to be assigned, the number of available automated guided vehicles, and the system load status at the current moment. Based on the resource occupancy status of the transit node, the number of stationed automated guided vehicles, and task prediction information, the pre-scheduling extension characteristics are determined. The target state vector is obtained by concatenating the task aggregation feature, the guided vehicle aggregation feature, the global feature, and the pre-scheduling extended feature. The target state vector is a feature vector of fixed dimensions.

3. The scheduling method for automated guided vehicles according to claim 2, characterized in that, The step involves performing multi-dimensional feature extraction on the task feature data of each of the tasks to be assigned and the vehicle feature data of each of the automated guided vehicles to obtain task aggregate features and guided vehicle aggregate features, including: The task feature data of each of the tasks to be assigned are combined into a task feature matrix, and the vehicle feature data of each of the automated guided vehicles are combined into a vehicle feature matrix. Statistical calculations are performed on the task feature matrix and the vehicle feature matrix column by column to obtain the task aggregate feature and the guide vehicle aggregate feature.

4. The scheduling method for automated guided vehicles according to claim 1, characterized in that, The method further includes: The target state vector is input into the pre-scheduling decision model to obtain the pre-scheduling decision result output by the pre-scheduling decision model; If the pre-scheduling decision result is to execute the deployment, then based on the pre-scheduling decision result, the target transfer node and the number of vehicles to be deployed are determined; Based on the number of vehicles to be dispatched, the idle automated guided vehicles will be scheduled to the target transfer node.

5. The scheduling method for automated guided vehicles according to claim 4, characterized in that, The method further includes: After the current scheduling cycle ends, the first reward data corresponding to the scheduling decision model and the second reward data corresponding to the pre-scheduling decision model are determined respectively. Add the first reward data to the first experience buffer, and add the second reward data to the second experience buffer; The network parameters of the scheduling decision model are updated based on the data in the first experience buffer, and the network parameters of the pre-scheduling decision model are updated based on the data in the second experience buffer.

6. The scheduling method for automated guided vehicles according to claim 5, characterized in that, The step of determining the first reward data corresponding to the scheduling decision model and the second reward data corresponding to the pre-scheduling decision model includes: Obtain task execution data within the current scheduling period, wherein the task execution data includes at least the task completion rate, the average task waiting time, and the average task completion time. Based on the task execution data, the first reward data corresponding to the scheduling decision model is determined; Obtain resource layout data within the current scheduling period, wherein the resource layout data includes at least the pre-scheduling hit rate and resource occupancy balance. Based on the resource layout data, the second reward data corresponding to the pre-scheduling decision model is determined.

7. The scheduling method for automated guided vehicles according to claim 1, characterized in that, The step of determining the target automated guided vehicle (AGV) corresponding to each of the tasks to be assigned based on the sorting results and the vehicle characteristic data of each AGV includes: Based on the sorting result, all the tasks to be assigned are traversed to determine the current task to be assigned. Based on the vehicle feature data, multiple candidate automated guided vehicles are identified, and the current working state of the candidate automated guided vehicles is either idle or pre-scheduled. Determine the distance between the currently assigned task and each of the candidate automated guided vehicles; The candidate automated guided vehicle (AGV) that is closest to the current task to be assigned is determined as the target AGV corresponding to the current task to be assigned.

8. The scheduling method for automated guided vehicles according to claim 7, characterized in that, After determining the candidate automated guided vehicle (AGV) with the smallest distance to the current task to be assigned as the target AGV corresponding to the current task to be assigned, the method further includes: If the current working state of the target automated guided vehicle is the pre-scheduled state, then an interrupt scheduling instruction is generated for the target automated guided vehicle. Based on the interrupt scheduling instruction, the target automated guided vehicle is controlled to move from its current position to execute the currently assigned task.

9. The scheduling method for automated guided vehicles according to claim 1, characterized in that, The step of inputting the target state vector into the scheduling decision model to obtain the target scheduling rule output by the scheduling decision model includes: Obtain a candidate scheduling rule set, which includes multiple candidate scheduling rules; The target state vector is input into the scheduling decision model, and the evaluation score corresponding to each candidate scheduling rule is determined according to the scheduling decision model. The candidate scheduling rule with the highest evaluation score is determined as the target scheduling rule.

10. A scheduling device for automated guided vehicles, characterized in that, include: The aggregation encoding module is used to perform aggregation encoding processing on the task feature data of each task to be assigned and the vehicle feature data of each automated guided vehicle to obtain the target state vector. The rule decision module is used to input the target state vector into the scheduling decision model to obtain the target scheduling rule output by the scheduling decision model; The task sorting module is used to sort all the tasks to be assigned based on the target scheduling rules to obtain a sorting result; The task allocation module is used to determine the target automated guided vehicle corresponding to each of the tasks to be assigned based on the sorting results and the vehicle characteristic data of each of the automated guided vehicles.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the scheduling method for the automated guided vehicle as described in any one of claims 1 to 9.

12. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the scheduling method of the automated guided vehicle as described in any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the scheduling method of the automated guided vehicle as described in any one of claims 1 to 9.

Images