A method for collaborative optimization of time-of-use electricity price and AGV transportation production scheduling

Abstract

This application proposes a production scheduling collaborative optimization method considering time-of-use electricity pricing and AGV transportation, relating to the field of workshop scheduling technology. The method includes: reverse-order traversal of the process sequence corresponding to the target scheme individual; determining the optimal displacement of the target process within the maximum feasible right shift range corresponding to the currently traversed target process; decoding the target scheme individual at the optimal displacement and initial position based on a non-dominated sorting genetic algorithm to obtain a first bi-objective value and a second bi-objective value; determining the total electricity cost savings and maximum delay increment based on the difference between the first and second bi-objective values; and using an investment return strategy to determine whether the target process should be right-shifted according to the optimal displacement; updating the bi-objective values ​​of the target scheme individual based on the determination result, thereby completing the update of the entire external archive set. This application can provide a series of optimal production plans that achieve a balance between electricity cost reduction and on-time delivery.

Description

Technical Field

[0001] This application relates to the field of workshop scheduling technology, and in particular to a production scheduling collaborative optimization method that considers time-of-use electricity pricing and AGV transportation. Background Technology

[0002] In the development of modern intelligent manufacturing, the integration of workshop production with AGVs (Automated Guided Vehicles) has become standard, especially in power-intensive manufacturing industries such as automotive parts and heavy machinery processing. Products typically need to be processed sequentially on multiple machines with different functions according to a predetermined process route. Unlike traditional replacement-based assembly lines, non-replacement-based assembly lines allow for changes in the processing sequence of tasks on different machines. They also introduce a limited number of AGVs to handle the transportation of tasks between loading / unloading areas and various machines, and utilize time-of-use electricity pricing policies to adjust production plans and save on electricity costs.

[0003] In non-displacement assembly line workshops, changes in the processing sequence of tasks affect the allocation of AGV transport tasks and electricity costs. Related technologies often treat these factors separately, ignoring the limitations of the number of AGVs or the impact of electricity prices. Moreover, when faced with conflicting optimization objectives, these technologies struggle to strike a balance between ensuring delivery and reducing costs. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to address the shortcomings of the prior art by providing a production scheduling collaborative optimization method that considers time-of-use electricity pricing and AGV transportation. This method can provide a series of optimal production plans that achieve a balance between reducing electricity costs and on-time delivery for power-intensive manufacturing enterprises that widely use non-displacement assembly lines and AGVs.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0006] Browse the scheme individuals in the external archive set, which includes multiple scheme individuals. The scheme individuals are used to represent the workshop scheduling scheme. The workshop scheduling scheme is a three-layer coding structure including a machine processing sequence matrix, a process sequence, and a transportation task allocation sequence. The machine processing sequence matrix is ​​used to record the order of each machine processing process. The process sequence is used to represent the priority of multiple processes. The transportation task allocation sequence is used to allocate transportation tasks according to the machine processing sequence matrix and the process sequence.

[0007] Traverse the process sequence corresponding to the individual target scheme in reverse order. Within the maximum feasible right shift range corresponding to the target process currently being traversed, determine the optimal displacement of the target process. The optimal displacement is used to indicate that the power cost corresponding to the target process is the lowest under the current displacement.

[0008] Based on the non-dominated sorting genetic algorithm, the individual target schemes of the target process under the optimal displacement and initial position are decoded respectively to obtain the first bi-objective value and the second bi-objective value, wherein the bi-objective value includes the total power cost and the maximum delay.

[0009] Based on the difference between the first dual target value and the second dual target value, the total electricity cost savings and the maximum delay increment are determined;

[0010] Using an investment return strategy, based on the total electricity cost savings and the maximum delay increment, it is determined whether the target process should be shifted to the right according to the optimal displacement.

[0011] Update the dual-objective values ​​of the individual target schemes based on the judgment results;

[0012] The optimized external archive set is obtained by updating all the individual schemes in the external archive set.

[0013] Furthermore, it also includes:

[0014] Using multiple processes in the process sequence as nodes and the process flow between the multiple processes as directed edges, a directed acyclic graph is constructed.

[0015] Based on the machine processing sequence matrix, the elements in the machine processing sequence matrix are traversed according to the processing order of each machine, and adjacent elements are searched. The elements are used to represent the process and the machine number corresponding to the process.

[0016] Add the pointers corresponding to the adjacent elements to the directed acyclic graph;

[0017] Calculate the in-degree of each node in the directed acyclic graph, and add all nodes with an in-degree of zero to the zero-degree node set;

[0018] Randomly select a target node from the set of zero-degree nodes and arrange the target node at the end of the process sequence;

[0019] Traverse the directed acyclic graph, decrement the in-degree of the successor nodes of the target node by 1, and add the successor nodes with an in-degree of zero to the set of zero-degree nodes.

[0020] The target node is selected from the set of zero-degree nodes in a loop until the set of zero-degree nodes is empty, thus obtaining the process sequence without self-locking.

[0021] Furthermore, the step of using an investment return strategy to determine whether the target process should be shifted to the right, based on the total electricity cost savings and the maximum delay increment, includes:

[0022] If the maximum delay increment is less than or equal to zero, or if the maximum delay increment is greater than zero and the ratio of the total power saving cost to the maximum delay increment is greater than the judgment threshold, the target process will be adjusted according to the optimal displacement.

[0023] If the maximum delay increment is greater than zero and the ratio of the total electricity cost savings to the maximum delay increment is less than or equal to a threshold, the target process will not be moved.

[0024] Furthermore, the determination threshold is determined according to the following formula: ,in, This indicates the determination threshold. This represents the average processing power of all said processes on all said machines. and These represent peak electricity price and off-peak electricity price, respectively. This represents the sensitivity coefficient.

[0025] Furthermore, it also includes:

[0026] The initial population size is divided into multiple initial population sizes according to different generation rules. The generation rules include mixed heuristic rules and completely random rules. The initial population size consists of multiple processes, multiple machines for processing the processes, and multiple guided vehicles.

[0027] The multiple steps in the initial multi-part population are arranged according to the hybrid heuristic rule and the completely random rule, respectively, to obtain an ordered step sequence and an unordered step sequence.

[0028] The ordered process sequence is segmented according to priority, and the processes in each segment of the ordered process sequence are randomly arranged to generate multiple reference sequences;

[0029] The reference sequence and the unordered process sequence are mapped into a machine processing sequence matrix, which is represented as follows: , This represents the machine processing sequence matrix. This refers to either the reference sequence or the unordered process sequence. Indicates the machine used in the process;

[0030] The machine sequence matrix is ​​used to generate the process sequence without locking, and the transportation task allocation sequence is generated based on the process sequence without locking to obtain the initial individual;

[0031] Decode the initial individual to obtain the bi-target value of the initial individual;

[0032] Based on the bi-objective values ​​of all the initial individuals, a screening strategy is executed to select non-dominated solutions. The screening strategy includes a method based on fast non-dominated sorting and crowding distance calculation to select individuals with high ranking.

[0033] The external archive set is generated based on the initial individuals corresponding to all the non-dominated solutions.

[0034] Furthermore, it also includes:

[0035] Based on a binary tournament selection strategy, the first and second parent individuals are selected cyclically from the parent population, wherein the parent population includes all the initial individuals.

[0036] Perform crossover and mutation operations on the first parent individual and the second parent individual to generate the first child individual and the second child individual;

[0037] When the number of offspring individuals reaches a preset value, the parent population and the offspring population are merged, and the offspring population includes all the offspring individuals.

[0038] Based on the dual objective values ​​of individuals in the merged population, the screening strategy is executed to select multiple individuals to form the first mixed population.

[0039] The first mixed population is added to the external archive set, the screening strategy is executed, and the external archive set is updated.

[0040] Furthermore, it also includes:

[0041] A Markov decision model is constructed, which includes multiple state feature vectors and multiple action operators. The state feature vectors are used to reflect the characteristics of the workshop scheduling scheme related to the scheduling target and resource status. The action operators are used to perturb the individuals in the first mixed population according to the state feature vectors.

[0042] The first target state feature vector of the first target individual in the first mixed population and the multiple action operators are input into the Dueling Double DQN neural network to calculate the first evaluation Q value of the action operators;

[0043] In the first evaluation Q value, select the first target action operator corresponding to the maximum value;

[0044] The first target individual is perturbed using the first target action operator to obtain the second target individual;

[0045] When the second target individual dominates the first target individual, the second target individual is added to the real-time archive;

[0046] The real-time archive is added to the first mixed population, and the screening strategy is executed to obtain the second mixed population;

[0047] The second mixed population is added to the external archive set, the screening strategy is executed, and the external archive set is updated.

[0048] Furthermore, it also includes:

[0049] Based on the first target action operator and the first target state feature vector of the first target individual, the second target state feature vector of the second target individual is obtained;

[0050] The second target state feature vector and the multiple action operators are input into the Dueling DoubleDQN neural network to calculate the second evaluation Q value of the action operators;

[0051] In the second evaluation Q value, the second target action operator corresponding to the maximum value is selected;

[0052] The target Q value corresponding to the second target action operator is calculated as follows:

[0053] ;

[0054] in, This represents the target Q value. Indicates the reward factor. Indicates the target network. This represents the feature vector of the second target state. This represents the action operator. This represents the action operator corresponding to the largest Q-value, and the reward factor is used to reflect the dominance relationship between the first target individual and the second target individual;

[0055] The evaluation network parameters of the Dueling Double DQN neural network are optimized based on the target Q value.

[0056] Furthermore, it also includes:

[0057] When the dual-target value of the first target individual dominates the second target individual, the reward factor takes a first positive number;

[0058] When the dual-target value of the first target individual and the second target individual are not mutually exclusive, the reward factor takes a second positive number, and the first positive number is greater than the second positive number;

[0059] The reward factor is set to zero if the dual objective value of the first target individual is dominated by the second target individual or if the two are the same.

[0060] In summary, the production scheduling collaborative optimization method considering time-of-use electricity pricing and AGV transportation provided by this invention traverses the process sequence of the workshop scheduling scheme from back to front, delaying the operation time of each process until the electricity cost is lowest. The dual objective values ​​at this point are calculated: total electricity cost and maximum task delay. These are compared with the dual objective values ​​at the initial position of the process. An investment return strategy is used to find a balance between electricity cost and task duration, achieving cost reduction and efficiency improvement while ensuring on-time delivery. This determines whether the process should be shifted to the right. After performing the same judgment on all processes in the entire process sequence, the dual objective values ​​of individual schemes are decoded based on the judgment results, and these solutions are used to update individual schemes. Similarly, this process is applied to all individuals in the external archive set, completing the optimization of all schemes in the external archive set. Attached Figure Description

[0061] Figure 1 A flowchart illustrating a production scheduling collaborative optimization method considering time-of-use electricity pricing and AGV transportation, provided as an embodiment of this application;

[0062] Figure 2 A three-layer coding structure diagram of a workshop scheduling scheme provided in this application embodiment;

[0063] Figure 3 This is a schematic diagram of the algorithm flow for a production scheduling collaborative optimization method that considers time-of-use electricity pricing and AGV transportation, provided as an embodiment of this application. Detailed Implementation

[0064] In non-displacement assembly line scenarios, existing algorithms for related technologies lack post-processing mechanisms for time-of-use pricing, making it impossible to scientifically weigh the cost-effectiveness of increased delays against electricity cost savings. This results in missing out on significant opportunities to reduce electricity costs and causing serious energy waste.

[0065] To address the above-mentioned technical deficiencies, this application provides a production scheduling optimization method that considers time-of-use electricity pricing and AGV transportation. Figure 1 A flowchart illustrating a production scheduling collaborative optimization method considering time-of-use electricity pricing and AGV transportation, as provided in this application embodiment, is shown below. Figure 1 As shown, the above method includes:

[0066] Step S1: Browse the individual schemes in the external archive set.

[0067] The external archive set includes multiple scheme individuals, each of which is used to characterize a workshop scheduling scheme. The workshop scheduling scheme is a three-layer coding structure including a machine processing sequence matrix, a process sequence, and a transportation task allocation sequence. The machine processing sequence matrix is ​​used to record the order of each machine processing process, the process sequence is used to represent the priority of multiple processes, and the transportation task allocation sequence is used to allocate transportation tasks according to the machine processing sequence matrix and the process sequence.

[0068] It should be noted that in a non-displacement assembly line workshop, there are multiple machines and equipment, multiple tasks, and correspondingly multiple different process sequences, as well as multiple automated guided vehicles. Therefore, in practice, there are multiple workshop scheduling schemes, which are stored in an external archive.

[0069] This application focuses on the non-displacement assembly line, which, compared to the traditional displacement assembly line, allows for changes in the processing sequence of tasks on different machines. It also introduces a limited number of AGVs to handle the transportation of tasks between the loading / unloading area and various machines. Therefore, in complex physical scenarios, the production scheduling of manufacturing enterprises is no longer a simple processing sequencing problem; the following decisions must be considered simultaneously:

[0070] 1. Machine processing sequence decision: Determine the specific processing order of each task on each machine in order to balance the workload of each machine and reduce machine idle waiting time.

[0071] 2. AGV Transportation Allocation Decision: For each movement of a task between machines, a specific AGV must be assigned to be responsible for it, and the order in which the same AGV performs multiple transportation tasks must be determined to avoid logistics conflicts and deadlocks.

[0072] 3. Processing time planning decision: Calculate the start and end times of each process and each transport, and arrange processing and transport times.

[0073] In view of this, embodiments of this application propose a three-layer encoding structure for a workshop scheduling scheme, and design a corresponding deadlock-free decoding mechanism based on graph theory topological sorting, such as... Figure 2 As shown, Figure 2 This is a three-layer coding structure diagram of a workshop scheduling scheme provided in an embodiment of this application. Figure 2 In this matrix, SE represents the machine processing sequence matrix, OS represents the process sequence, and AS represents the transportation task allocation sequence. The method also includes:

[0074] Using multiple processes in the process sequence as nodes and the process flow between the multiple processes as directed edges, a directed acyclic graph is constructed.

[0075] For example, Figure 2The OS is The sequence is a one-dimensional sequence, where J1, J2, and J3 represent different processes. From left to right, the sequence represents the completion priority of each process. Figure 2 SE in the middle represents The machine processing sequence matrix, where each element in SE is... It indicates the first Machine number Each processing step ID is represented by a layer in SE, which indicates the processing sequence of the corresponding machine. For example, in SE, M1, M2, and M3 are vertically arranged to represent each machine, and the horizontal arrangement represents the processing sequence of M1, M2, and M3 by J1, J2, and J3. Figure 2 AS is A one-dimensional sequence, where A1, A2, and A3 represent different AGVs. It indicates the first Which AGV is responsible for performing each transportation process? For example, the dashed box indicates that process J1 is transported to M2 and is handled by AGV A1.

[0076] It should be noted that the constructed directed acyclic graph Nodes in The above-mentioned process is, i.e. Figure 2 J1, J2 and J3, The directed edges represent the sequence of the technological processes corresponding to J1, J2, and J3, i.e., the process priority.

[0077] Based on the machine processing sequence matrix, the elements in the machine processing sequence matrix are traversed according to the processing order of each machine, and adjacent elements are searched. The elements are used to represent the process and the machine number corresponding to the process.

[0078] Add the pointers corresponding to the adjacent elements to the directed acyclic graph.

[0079] It should be noted that searching for adjacent elements can be done by iterating through the machine-processed sequence matrix SE and finding adjacent elements, such as... Figure 2 In row M1, J1 and J2, as well as J2 and J3, are adjacent elements, and J1 and J2, as well as J2 and J3, represent the processing order. This processing order is then added as a directed edge to the directed acyclic graph described above.

[0080] It should be noted that the above machine processing sequence matrix is ​​given in advance, and the horizontal and vertical directions in the matrix can be transposed.

[0081] Calculate the in-degree of each node in the directed acyclic graph, and add all nodes with an in-degree of zero to the zero-degree node set.

[0082] For example, nodes with an in-degree of zero in a directed acyclic graph are added to the set of zero-degree nodes. A node with an in-degree of zero can be the origin node in the directed acyclic graph.

[0083] A target node is randomly selected from the set of zero-degree nodes and arranged at the end of the process sequence.

[0084] Traverse the directed acyclic graph, decrement the in-degree of the successor nodes of the target node by 1, and add the successor nodes with an in-degree of zero to the set of zero-degree nodes.

[0085] The target node is selected from the set of zero-degree nodes in a loop until the set of zero-degree nodes is empty, thus obtaining the process sequence without self-locking.

[0086] For example, a target node is randomly selected from the aforementioned set of zero-degree nodes, and its corresponding ID is added to the end of the above process sequence. The IDs of this node are J1, J2, and J3. Then, the in-degree of all successor nodes of this target node is decremented by 1; the successor node is the next node adjacent to the target node. The successor nodes whose in-degree becomes zero after decrementing are added to the aforementioned set of zero-degree nodes. Another target node is then found from the set of zero-degree nodes and added after the previous target node. This process is repeated until the set of zero-degree nodes is empty, resulting in a process sequence without self-locking.

[0087] Step S2: Traverse the process sequence corresponding to the individual target scheme in reverse order, and determine the optimal displacement of the target process within the maximum feasible right shift range corresponding to the currently traversed target process.

[0088] It should be noted that shifting the process to the right means postponing the process in terms of time.

[0089] For example, the algorithm iterates through each process sequence in the scheduling scheme from back to front. For the target process, it first calculates the maximum feasible right shift that satisfies the machine processing sequence, process sequence, and AGV transportation constraints. This algorithm can be implemented using the NSGA-II (Non-dominated Sorting Genetic Algorithm) algorithm in ranges from 0 to... Within the specified range, the algorithm searches for the optimal displacement that minimizes the power cost of the target process. It can be the time period with the lowest electricity price in the time-of-use pricing system.

[0090] Step S3: Based on the non-dominated sorting genetic algorithm, decode the individual target schemes of the target process under the optimal displacement and initial position respectively to obtain the first bi-objective value and the second bi-objective value. The bi-objective value includes the total power cost and the maximum delay.

[0091] Step S4: Based on the difference between the first dual target value and the second dual target value, determine the total electricity cost savings and the maximum delay increment.

[0092] For example, after obtaining the optimal displacement of the target process, the dual objective values ​​of the overall process sequence are calculated under both the displacement and the initial position: TEC (Total Electricity Cost) and the maximum delay Z. The difference between these two dual objective values ​​is then calculated to obtain the total electricity cost savings. and maximum delay increment .

[0093] Step S5: Using the return on investment strategy, based on the total electricity cost savings and the maximum delay increment, determine whether the target process should be shifted to the right according to the optimal displacement.

[0094] It should be noted that in actual production, minimizing both TEC and Z simultaneously presents a fundamental conflict in terms of production cycle time and resource utilization. Therefore, TEC and Z need to be addressed in a coordinated manner to achieve a balance between cost reduction and efficiency improvement and on-time delivery.

[0095] For example, when high-energy-consuming processes are moved to periods with low electricity prices by utilizing time windows between processes, and ROI (Return On Investment) is used to measure the savings in electricity costs by delaying a single process, a trade-off between ensuring delivery and reducing costs is achieved.

[0096] In some instances, step S5 above may include:

[0097] If the maximum delay increment is less than or equal to zero, or if the maximum delay increment is greater than zero and the ratio of the total power saving cost to the maximum delay increment is greater than the judgment threshold, the target process will be adjusted according to the optimal displacement.

[0098] If the maximum delay increment is greater than zero and the ratio of the total electricity cost savings to the maximum delay increment is less than or equal to a threshold, the target process will not be moved.

[0099] For example, because the target process was selected to operate at the lowest electricity cost, the total electricity cost savings amounted to [amount missing]. Since it is greater than or equal to zero, the maximum delay increment is judged first. Please refer to the following principles:

[0100] Scenario 1 (Lossless Optimization): If This indicates that the move reduced electricity costs without worsening the delay, and the algorithm unconditionally accepts the move.

[0101] Scenario II (Paid Optimization): If This indicates a target conflict. At this point, calculation... .like This indicates that the move is cost-effective, and the algorithm accepts it. Among other things, This indicates the threshold for judgment.

[0102] Case III (No optimization): If ,and The move is not accepted.

[0103] In some examples, the above-mentioned threshold is used. The following method can be used for calculation: ;in, This indicates the aforementioned threshold value. This represents the average processing power of all said processes on all said machines. and These represent peak electricity price and off-peak electricity price, respectively. This represents the sensitivity coefficient.

[0104] It should be noted that step S5 determines whether each process in the process sequence needs to be shifted to the right, and the determination of the next process is based on the determination result of the previous process.

[0105] Step S6: Update the dual target values ​​of the target scheme individual based on the judgment result.

[0106] For example, each process in the process sequence is processed by step S4 to determine whether it should be shifted to the right. After all processes are completed, the solution of the target scheme individual is decoded, including the bi-objective values, which represent the total power cost TEC and the maximum delay Z of the target scheme individual.

[0107] Step S7: Update all individual schemes in the external archive set to obtain the optimized external archive set.

[0108] For example, by following steps S5 and S6, all individual schemes in the external archive set are updated, thus completing the optimization of the entire external archive set.

[0109] Figure 3 A schematic diagram of the algorithm flow for a production scheduling collaborative optimization method considering time-of-use electricity pricing and AGV transportation, provided in an embodiment of this application, is shown below. Figure 3 As shown, Figure 3Steps A1 and A2 are about generating the initial population and the initial external archive set. Steps A3 to A4 use a binary tournament strategy to select the environment for the external archive set in the first round. Steps A5 to A6 use a neural network to select the environment in the second round. Step A7 is based on the ROI investment strategy to select the environment in the third round. Step A8 updates the parameters of the neural network. Step A9 is iterative loop judgment. Step A10 outputs the final external archive set.

[0110] In order to generate high-quality initialized individuals, the method proposed in this application embodiment further includes:

[0111] The initial population size is divided into multiple initial population sizes according to different generation rules, including mixed heuristic rules and completely random rules. The initial population size consists of multiple processes, multiple machines for processing the processes, and multiple guided vehicles.

[0112] For example, the hybrid heuristic rules may include sorting by delay urgency in descending order, by overdue urgency in descending order, by task release time in ascending order, and by task delivery date in ascending order. These four heuristic rules and one completely randomly generated rule can each account for 20% of the total initial population size.

[0113] The multiple processes in the initial multi-part population are arranged according to the hybrid heuristic rule and the completely random rule, respectively, to obtain ordered process sequences and unordered process sequences.

[0114] For example, processes arranged according to a hybrid heuristic rule generate an ordered sequence of processes, while processes arranged according to a completely random rule generate an unordered sequence of processes.

[0115] The ordered process sequence is segmented according to priority, and the processes in each segment of the ordered process sequence are randomly arranged to generate multiple reference sequences.

[0116] For example, the above ordered process sequence is evenly divided into 10 segments, denoted as R1, R2, R3...R9, R 10 R1 includes the top 10% of priority processes, R2 includes the next 10% of priority processes, and so on. Maintaining the relative order between sets, the processes in each segment are randomly arranged to generate multiple baseline sequences, which can be processed using sampling without replacement.

[0117] It should be noted that the number of segments and whether they are evenly divided can be set according to actual needs, and there are no specific limitations.

[0118] The reference sequence and the unordered process sequence are mapped into a machine processing sequence matrix, which is represented as follows: , This represents the machine processing sequence matrix. This refers to either the reference sequence or the unordered process sequence. This refers to the machine used in the process described.

[0119] For example, the above-mentioned baseline sequence and unordered process sequence are mapped into a machine processing sequence matrix. Each row in this matrix is ​​identical to both the baseline sequence and the unordered process sequence.

[0120] The process sequence without self-locking is generated based on the machine sequence matrix, and the transportation task allocation sequence is generated based on the process sequence without self-locking to obtain the initial individual.

[0121] For example, based on the above machine processing sequence matrix Using the aforementioned method for generating a non-locking process sequence, a non-locking process sequence and a corresponding transportation task allocation sequence are generated to obtain the initial individual.

[0122] Decode the initial individual to obtain the dual objective value of the initial individual.

[0123] For example, a non-dominated sorting genetic algorithm can be used to decode the initial individuals sequentially to obtain their respective bi-objective values.

[0124] Based on the bi-objective values ​​of all the initial individuals, a screening strategy is executed to select non-dominated solutions. The screening strategy includes a method based on fast non-dominated sorting and crowding distance calculation to select individuals with high ranking.

[0125] The external archive set is generated based on the initial individuals corresponding to all the non-dominated solutions.

[0126] For example, the aforementioned fast non-dominated sorting determines the corresponding individual rank based on the bi-objective values ​​of the initial individuals, sorts them according to the rank, selects solutions with high value, and selects solutions with diversity based on crowding distance. Based on these two indicators, a certain number of top-ranked individuals are selected. These individuals constitute the aforementioned external archive set. Since the external archive set initialized in this embodiment is generated after sorting, the generated solutions are of relatively high quality, thus improving the speed of subsequent processing.

[0127] After initializing the external archive set, this application also introduces some new solutions to selectively filter the solutions in the external archive set. In some examples, the method proposed in the embodiments of this application further includes:

[0128] Based on a binary tournament selection strategy, the first parent individual and the second parent individual are selected cyclically from the parent population, wherein the parent population includes all the initial individuals.

[0129] For example, when selecting two parent individuals each time, the individual with the lower non-dominance ranking is preferred. If the rankings are the same, the individual with the larger crowding distance value is selected as the parent.

[0130] Perform crossover and mutation operations on the first parent individual and the second parent individual to generate the first offspring individual and the second offspring individual.

[0131] When the number of offspring individuals reaches a preset value, the parent population and the offspring population are merged, and the offspring population includes all the offspring individuals.

[0132] For example, crossover and mutation operations are performed on the first parent individual P1 and the second parent individual P2 to generate the first offspring individual Q1 and the second offspring individual Q2. This operation is repeated until the number of offspring individuals reaches a preset number.

[0133] For example, the above crossover and mutation operations are performed as follows:

[0134] (1) SE sequence crossover: In each row, a gene segment is randomly extracted from a certain row of the SE matrix of the parent generation P1 and placed in the same position of Q1. The remaining empty slots in the row of Q1 are filled in the order of the process in P2. At the same time, the same operation is performed on Q2. P2 is extracted and P1 is filled.

[0135] (2) OS sequence crossover: OS is generated by topological sorting based on the SE matrix of the offspring.

[0136] (3) AS Sequence Crossover: Assigning sequences to generate the AGV for Q1. The steps are as follows: Divide the AS sequences of P1 and P2 into several segments, where the length of each segment is equal to the total number of tasks. Based on the process sequence of Q1, calculate the corresponding idle transport time of P1 and P2 in each segment. For each segment, if P1 exhibits less idle transport time, the offspring inherits the AS sequence of that segment of P1; otherwise, it inherits the corresponding segment of P2. Apply the same process to Q2 to generate its AS sequence, at which point the OS of Q2 is used to determine the selection of AS segments.

[0137] (4) SE sequence mutation: Traverse the machine processing sequence matrix SE row by row. For each row, a mutation is triggered with a 10% probability. Two different positions are randomly selected in the current machine's processing sequence, and the task numbers at these two positions are swapped, thereby changing the local processing order of the task on the machine.

[0138] (5) OS sequence variation: randomly generated based on the topological sorting of the SE matrix.

[0139] (6) AS sequence variation: Calculate how many tasks each AGV is currently responsible for transporting, find the AGV1 with the most tasks and the AGV2 with the fewest tasks. If the difference between the two tasks is not less than 2, then randomly select one task of AGV1 and assign it to AGV2.

[0140] Based on the dual objective values ​​of individuals in the merged population, the screening strategy is executed to select multiple individuals to form the first mixed population.

[0141] The first mixed population is added to the external archive set, the screening strategy is executed, and the external archive set is updated.

[0142] For example, after merging the parent and offspring populations, multiple individuals are selected based on a selection strategy to form a first mixed population. The first mixed population is then combined with an external archive set, and the selection strategy is continued to be applied to complete the update of the external archive set and the first round of environmental selection for the external archive set is completed.

[0143] To overcome the problems of high blindness and low efficiency of traditional random mutation in the complex solution space of non-permutation flow shop scheduling, the method proposed in this application further includes:

[0144] A Markov decision model is constructed, which includes multiple state feature vectors and multiple action operators. The state feature vectors are used to reflect the characteristics of the workshop scheduling scheme related to the scheduling target and resource status. The action operators are used to perturb individuals in the first mixed population according to the state feature vectors.

[0145] For example, unlike directly using high-dimensional chromosome encoding as the state, in order to reduce training difficulty and improve the model's generalization ability to problems of different scales, the Markov decision model constructed in this application uses features related to scheduling objectives and resource states to form a state feature vector, for example:

[0146] (1) Normalized total TEC: The ratio of the current solution's TEC value to the estimated maximum TEC. The estimated maximum TEC is calculated as the product of the total processing time of all processes, the maximum operating power of the machine, and the highest unit price during the time-of-use electricity pricing cycle.

[0147] (2) Normalized maximum delay: The ratio of the maximum delay of the current solution to the estimated maximum delay. The upper bound of the maximum delay is calculated as follows: the end time of the entire time-of-use electricity pricing cycle minus the earliest delivery date of all tasks.

[0148] (3) AGV load balance: The ratio of the standard deviation to the mean of the workload of all AGVs. The workload of a single AGV is defined as the sum of its total loaded travel time and total unloaded travel time during the entire scheduling cycle.

[0149] (4) AGV empty running rate: the proportion of the total empty running time of all AGVs to their total workload.

[0150] (5) AGV idle rate: reflects the average idleness of AGV resources. It is calculated as the difference between 1 and the product of the total workload of all AGVs and the maximum completion time and the total number of AGVs.

[0151] (6) Proportion of delayed tasks: The proportion of delayed tasks to the total number of tasks.

[0152] (7) Proportion of tasks completed ahead of schedule: The proportion of tasks whose completion time is earlier than the delivery date.

[0153] (8) Delay distribution dispersion: the normalized standard deviation of the delay time of all tasks. It is calculated as follows: the standard deviation of the difference between the completion time and the delivery date of each task, divided by the estimated maximum delay.

[0154] (9) Machine load balance: the ratio of the standard deviation of the completion time of each machine to its average value.

[0155] (10) Average machine utilization: The ratio of the total effective processing time of the system to the total machine operating capacity. The total machine operating capacity is defined as the sum of the differences between the final completion time and the initial start time of each machine.

[0156] For example, the above multiple action operators are divided into operators of machine sequence SE, such as:

[0157] (1) Random machine sequence swap: Randomly swap two tasks in a machine queue and reconstruct the OS based on topological sorting to ensure no deadlock.

[0158] (2) Maximum Delay Forward Insertion: Identify the task with the most severe delay and move its position forward on a random machine to compress the completion time.

[0159] (3) Minimum delay backward insertion: The earliest completed task is moved back to the later stage, using the relaxation time to exchange for low electricity price periods or free up resources.

[0160] (4) Critical path forward linkage insertion: The position of the longest-delayed task on the current machine and all subsequent machines is moved forward synchronously to simulate critical path compression.

[0161] (5) Critical path backward linkage insertion: The position of the least delayed task on the current machine and the preceding machine is synchronously moved backward.

[0162] (6) Simulation feedback sequence synchronization: The S matrix is ​​reconstructed in reverse using the actual arrival time after decoding to repair the inconsistency between genotype and phenotype.

[0163] And operators based on the transportation task allocation sequence AS, such as:

[0164] (7) AGV single-point mutation: randomly change the AGV assignment for a certain task.

[0165] (8) AGV load balancing: Transfer the transportation task of the AGV with the largest load to the AGV with the smallest load to reduce the maximum completion time.

[0166] (9) Minimize idle time: Identify the task with the longest idle time and reassign it to the AGV that can shorten the connection distance.

[0167] Operators based on process sequence OS:

[0168] (10) Topological order resampling: Keep S unchanged, use the topological sorting algorithm to resample OS to obtain different deadlock-free transport timing sequences.

[0169] To effectively integrate the aforementioned local search operators and adaptively determine the optimal search strategy based on the current scheduling solution state, this application employs a Dueling Double DQN neural network. To better understand the embodiments of this application, the Dueling Double DQN neural network will first be introduced.

[0170] Dueling Double DQN is a high-efficiency deep reinforcement learning algorithm that learns from historical experience and makes optimal decisions. Dueling Double DQN combines two improvement strategies to the original DQN algorithm:

[0171] Double DQN addresses the Q-value overestimation problem in DQN by introducing two Q-value functions: an online Q-network and a target Q-network. When calculating the target Q-value, the action selection and action evaluation processes are separated. The online Q-network selects the action corresponding to the maximum Q-value in the next state, and then the target Q-network evaluates the Q-value of the selected action. This effectively reduces the degree of overestimation.

[0172] The Dueling Double DQN network is divided into two parts: one part is the value function part that is only related to the state. The other part is the advantage function, which is related to both state and action. Dueling Double DQN can better distinguish the value of different states and the advantage of actions relative to states, thus improving the ability to identify the importance of states.

[0173] The first target state feature vector of the first target individual in the first mixed population and the multiple action operators are input into the Dueling Double DQN neural network to calculate the first evaluation Q value of the action operators.

[0174] It should be noted that each individual in the first mixed population mentioned above is used as the initial state. The input is fed into a neural network, and a fixed number of consecutive searches are performed. The following explanation uses an individual operation as an example.

[0175] The Dueling Double DQN neural network has two networks: an evaluation network and an evaluation network. and target network The formula for calculating the Q-value of a Dueling Double DQN neural network is shown below:

[0176] ;

[0177] in, Represents the state value function. Denotes the dominance function, | | represents the size of the action space, i.e., the multiple action operators mentioned above.

[0178] The first target state feature vector of the first target individual in the first mixed population mentioned above By substituting multiple action operators into the above formula, we can obtain the first target state feature vector of the first target individual. The first evaluation Q value for all action operators.

[0179] In some examples, the step includes:

[0180] use The strategy is to select the action operator.

[0181] For example, that is, in terms of probability Randomly select actions to maintain exploratory nature, using probability. Choose the action with the highest Q-value output from the online network to leverage existing experience.

[0182] The first target state feature vector of the first target individual in the first mixed population and the selected action operator are input into the Dueling Double DQN neural network to calculate the first evaluation Q value of the action operator.

[0183] In the first evaluation Q value, the first target action operator corresponding to the maximum value is selected.

[0184] For example, among the first evaluation Q values ​​corresponding to all the above working operators, the largest value is selected, and the corresponding first target action operator is determined based on this value. .

[0185] The first target individual is perturbed using the first target action operator to obtain the second target individual.

[0186] For example, using the first target action operator described above The first target individual is perturbed to obtain the second target individual.

[0187] When the second target individual dominates the first target individual, the second target individual is added to the real-time archive.

[0188] For example, when the binocular value of the second target individual dominates the binocular value of the second target individual, the second target individual is added to the real-time archive.

[0189] For example, in each interaction of the Dueling Double DQN continuous search, if a new solution is generated that is non-dominant relative to the current search path, it is directly added to the real-time archive.

[0190] The real-time archive is added to the first mixed population, and the screening strategy is executed to obtain the second mixed population.

[0191] For example, after processing all individuals in the first mixed population, all non-dominated solutions generated constitute the real-time archive. The real-time archive is then mixed with the first mixed population, and the screening strategy described above is executed to obtain the second mixed population.

[0192] The second mixed population is added to the external archive set, the screening strategy is executed, and the external archive set is updated.

[0193] In this embodiment, after a fixed number of DQN consecutive searches are completed for the current target individual, all improved solutions accumulated in the real-time archive are merged with the original population to form a mixed population. Subsequently, fast non-dominated sorting and crowding distance calculation are performed again on the mixed population to select the top-ranked individuals of equal size to form a second mixed population, completing the second round of environmental selection.

[0194] In some examples, embodiments of this application can also train the neural network to optimize its network structure parameters, and the above method further includes:

[0195] Based on the first target action operator and the first target state feature vector of the first target individual, the second target state feature vector of the second target individual is obtained.

[0196] For example, the aforementioned first target action operator is used as a perturbation and applied to the first target state feature vector of the aforementioned first target individual. The second target individual is obtained above. The second target individual is then decoded to obtain the second target state feature vector. .

[0197] The second target state feature vector and the multiple action operators are input into the Dueling DoubleDQN neural network to calculate the second evaluation Q value of the action operators.

[0198] In the second evaluation Q value, the second target action operator corresponding to the maximum value is selected.

[0199] For example, the second target state feature vector is calculated according to the aforementioned Q-value calculation formula. For different action operators, calculate the maximum value and determine the corresponding second target action operator.

[0200] The target Q value corresponding to the second target action operator is calculated as follows.

[0201] ;

[0202] in, This represents the target Q value. Indicates the reward factor. Indicates the target network. This represents the feature vector of the second target state. This represents the action operator. This represents the action operator corresponding to the largest Q value, and the reward factor is used to reflect the dominance relationship between the first target individual and the second target individual.

[0203] For example, the target network Q-value corresponding to the second target action operator is calculated.

[0204] In some examples, the above reward factor can be determined in the following way:

[0205] When the dual-target value of the first target individual dominates the second target individual, the reward factor takes a first positive number.

[0206] When the dual-target value of the first target individual and the second target individual are not mutually exclusive, the reward factor takes a second positive number, and the first positive number is greater than the second positive number.

[0207] The reward factor is set to zero if the dual objective value of the first target individual is dominated by the second target individual or if the two are the same.

[0208] For example, a reward function can be constructed to compare the solutions before and after executing the action operator with the solution of the first target individual. Solution for the second target individual The reward factor can be calculated in the following way:

[0209] (1) If Dominate If the quality of the solution is significantly improved, a reward of +10 will be given.

[0210] (2) If and If they do not dominate each other, it is considered a discovery of a new trade-off solution that helps to expand the Pareto frontier, and is awarded the highest reward of +5.

[0211] (3) If quilt If the target values ​​of the two targets are exactly the same, the search is considered a failure and the reward is 0.

[0212] The evaluation network parameters of the Dueling Double DQN neural network are optimized based on the target Q value.

[0213] For example, in each step of a DQN successive search, the generated non-dominated solution and the old solution form a quadruple, i.e. ( , , ),in This represents the state feature vector of the first target individual. This represents the state feature vector of the second target individual. The action operator represents the action to be performed. This represents the reward factor.

[0214] All quadruplets generated during the search are added to an experience pool. A batch of interaction samples is randomly sampled from this pool, and the network parameters are evaluated based on the mean squared error loss function. Update.

[0215] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A production scheduling collaborative optimization method considering time-of-use electricity pricing and AGV transportation, characterized in that, include: Browse the scheme individuals in the external archive set, which includes multiple scheme individuals. The scheme individuals are used to represent the workshop scheduling scheme. The workshop scheduling scheme is a three-layer coding structure including a machine processing sequence matrix, a process sequence, and a transportation task allocation sequence. The machine processing sequence matrix is ​​used to record the order of each machine processing process. The process sequence is used to represent the priority of multiple processes. The transportation task allocation sequence is used to allocate transportation tasks according to the machine processing sequence matrix and the process sequence. Traverse the process sequence corresponding to the individual target scheme in reverse order. Within the maximum feasible right shift range corresponding to the target process currently being traversed, determine the optimal displacement of the target process. The optimal displacement is used to indicate that the power cost corresponding to the target process is the lowest under the current displacement. Based on the non-dominated sorting genetic algorithm, the individual target schemes of the target process under the optimal displacement and initial position are decoded respectively to obtain the first bi-objective value and the second bi-objective value, wherein the bi-objective value includes the total power cost and the maximum delay. Based on the difference between the first dual target value and the second dual target value, the total electricity cost savings and the maximum delay increment are determined; Using an investment return strategy, based on the total electricity cost savings and the maximum delay increment, it is determined whether the target process should be shifted to the right according to the optimal displacement. Update the dual-objective values ​​of the individual target schemes based on the judgment results; The optimized external archive set is obtained by updating all the individual schemes in the external archive set.

2. The production scheduling collaborative optimization method considering time-of-use electricity pricing and AGV transportation as described in claim 1, characterized in that, Also includes: Using multiple processes in the process sequence as nodes and the process flow between the multiple processes as directed edges, a directed acyclic graph is constructed. Based on the machine processing sequence matrix, the elements in the machine processing sequence matrix are traversed according to the processing order of each machine, and adjacent elements are searched. The elements are used to represent the process and the machine number corresponding to the process. Add the pointers corresponding to the adjacent elements to the directed acyclic graph; Calculate the in-degree of each node in the directed acyclic graph, and add all nodes with an in-degree of zero to the zero-degree node set; Randomly select a target node from the set of zero-degree nodes and arrange the target node at the end of the process sequence; Traverse the directed acyclic graph, decrement the in-degree of the successor nodes of the target node by 1, and add the successor nodes with an in-degree of zero to the set of zero-degree nodes. The target node is selected from the set of zero-degree nodes in a loop until the set of zero-degree nodes is empty, thus obtaining a process sequence without self-locking.

3. The production scheduling collaborative optimization method considering time-of-use electricity pricing and AGV transportation as described in claim 1, characterized in that, The method of using a return on investment strategy to determine whether the target process should be shifted to the right, based on the total electricity cost savings and the maximum delay increment, includes: If the maximum delay increment is less than or equal to zero, or if the maximum delay increment is greater than zero and the ratio of the total power saving cost to the maximum delay increment is greater than the judgment threshold, the target process will be adjusted according to the optimal displacement. If the maximum delay increment is greater than zero and the ratio of the total electricity cost savings to the maximum delay increment is less than or equal to a threshold, the target process will not be moved.

4. The production scheduling collaborative optimization method considering time-of-use electricity pricing and AGV transportation as described in claim 3, characterized in that, The determination threshold is determined according to the following formula: ,in, This indicates the determination threshold. This represents the average processing power of all said processes on all said machines. and These represent peak electricity price and off-peak electricity price, respectively. This represents the sensitivity coefficient.

5. The production scheduling collaborative optimization method considering time-of-use electricity pricing and AGV transportation as described in claim 2, characterized in that, Also includes: The initial population size is divided into multiple initial population sizes according to different generation rules. The generation rules include mixed heuristic rules and completely random rules. The initial population size consists of multiple processes, multiple machines for processing the processes, and multiple guided vehicles. The multiple steps in the initial multi-part population are arranged according to the hybrid heuristic rule and the completely random rule, respectively, to obtain an ordered step sequence and an unordered step sequence. The ordered process sequence is segmented according to priority, and the processes in each segment of the ordered process sequence are randomly arranged to generate multiple reference sequences; The reference sequence and the unordered process sequence are mapped into a machine processing sequence matrix, which is represented as follows: , This represents the machine processing sequence matrix. This refers to either the reference sequence or the unordered process sequence. Indicates the machine used in the process; The machine sequence matrix is ​​used to generate the process sequence without locking, and the transportation task allocation sequence is generated based on the process sequence without locking to obtain the initial individual; Decode the initial individual to obtain the bi-target value of the initial individual; Based on the bi-objective values ​​of all the initial individuals, a screening strategy is executed to select non-dominated solutions. The screening strategy includes a method based on fast non-dominated sorting and crowding distance calculation to select individuals with high ranking. The external archive set is generated based on the initial individuals corresponding to all the non-dominated solutions.

6. The production scheduling collaborative optimization method considering time-of-use electricity pricing and AGV transportation as described in claim 5, characterized in that, Also includes: Based on a binary tournament selection strategy, the first and second parent individuals are selected cyclically from the parent population, wherein the parent population includes all the initial individuals. Perform crossover and mutation operations on the first parent individual and the second parent individual to generate the first child individual and the second child individual; When the number of offspring individuals reaches a preset value, the parent population and the offspring population are merged, and the offspring population includes all the offspring individuals. Based on the dual objective values ​​of individuals in the merged population, the screening strategy is executed to select multiple individuals to form the first mixed population. The first mixed population is added to the external archive set, the screening strategy is executed, and the external archive set is updated.

7. The production scheduling collaborative optimization method considering time-of-use electricity pricing and AGV transportation as described in claim 6, characterized in that, Also includes: A Markov decision model is constructed, which includes multiple state feature vectors and multiple action operators. The state feature vectors are used to reflect the characteristics of the workshop scheduling scheme related to the scheduling target and resource status. The action operators are used to perturb the individuals in the first mixed population according to the state feature vectors. The first target state feature vector of the first target individual in the first mixed population and the multiple action operators are input into the Dueling Double DQN neural network to calculate the first evaluation Q value of the action operators; In the first evaluation Q value, select the first target action operator corresponding to the maximum value; The first target individual is perturbed using the first target action operator to obtain the second target individual; When the second target individual dominates the first target individual, the second target individual is added to the real-time archive; The real-time archive is added to the first mixed population, and the screening strategy is executed to obtain the second mixed population; The second mixed population is added to the external archive set, the screening strategy is executed, and the external archive set is updated.

8. The production scheduling collaborative optimization method considering time-of-use electricity pricing and AGV transportation as described in claim 7, characterized in that, The step of inputting the first target state feature vector of the first target individual in the first mixed population and the multiple action operators into the Dueling Double DQN neural network, and calculating the first evaluation Q value of the action operators, includes: The action operator is selected using the ε-greedy strategy; The first target state feature vector of the first target individual in the first mixed population and the selected action operator are input into the Dueling Double DQN neural network to calculate the first evaluation Q value of the action operator.

9. The production scheduling collaborative optimization method considering time-of-use electricity pricing and AGV transportation as described in claim 8, characterized in that, Also includes: Based on the first target action operator and the first target state feature vector of the first target individual, the second target state feature vector of the second target individual is obtained; The second target state feature vector and the multiple action operators are input into the Dueling Double DQN neural network to calculate the second evaluation Q value of the action operators; In the second evaluation Q value, the second target action operator corresponding to the maximum value is selected; The target Q value corresponding to the second target action operator is calculated as follows: ； in, This represents the target Q value. Indicates the reward factor. Indicates the target network. This represents the feature vector of the second target state. This represents the action operator. This represents the action operator corresponding to the largest Q-value, and the reward factor is used to reflect the dominance relationship between the first target individual and the second target individual; The evaluation network parameters of the Dueling Double DQN neural network are optimized based on the target Q value.

10. The production scheduling collaborative optimization method considering time-of-use electricity pricing and AGV transportation according to claim 9, characterized in that, Also includes: When the dual-target value of the first target individual dominates the second target individual, the reward factor takes a first positive number; When the dual-target value of the first target individual and the second target individual are not mutually exclusive, the reward factor takes a second positive number, and the first positive number is greater than the second positive number; The reward factor is set to zero if the dual objective value of the first target individual is dominated by the second target individual or if the two are the same.

Images