Production and logistics cooperative scheduling method for dynamic flexible job shop

Abstract

The invention relates to a production and logistics cooperative scheduling method for a dynamic flexible job shop, and the method comprises the following steps: obtaining the real-time information of the dynamic flexible job shop, inputting a production and logistics cooperative scheduling problem model for the dynamic flexible job shop, carrying out the solving through a deep reinforcement learning method, and obtaining a shop scheduling scheme; wherein the production and logistics collaborative scheduling problem model oriented to the dynamic flexible job shop sets respective optimization targets for production activities and logistics activities, fault scenes and corresponding processing strategies of logistics equipment are considered, and the solving process is as follows: S301, designing an adaptive multi-agent nested hierarchical framework; s302, designing Markov decision process key elements of each agent; and S303, performing cooperative training on each agent by using a multi-agent near-end strategy optimization algorithm. Compared with the prior art, a more reasonable and effective flexible job scheduling scheme can be obtained.

Description

Technical Field

[0001] The invention belongs to the technical field of flexible job shop scheduling, and in particular relates to a production and logistics collaborative scheduling method for a dynamic flexible job shop. Background Art

[0002] At present, the increasingly diversified and customized market demands force the manufacturing industry to adopt a more flexible production model. In this context, flexible job shops with flexible processing advantages have become the first choice for the development of various manufacturing industries. Unlike traditional job shops, flexible job shops are equipped with a series of multifunctional machines to manufacture products. The use of multifunctional machines makes the processing paths of various products in the workshop extremely flexible and diverse, which directly leads to the urgent need for frequent and flexible transfer of semi-finished products between machines. Especially in some flexible job shops with highly versatile machines and high processing efficiency, such as aviation parts production workshops and automobile parts production workshops, this transfer demand is more urgent and complex. It can be said that in flexible job shops, the completion of manufacturing tasks is closely related to production activities and logistics activities.

[0003] More specifically, for production activities and logistics activities, the former mainly involves allocating multifunctional machines to form diversified processing paths to manufacture products, while the latter requires arranging logistics resources according to the diverse processing paths to provide the necessary semi-finished product transfer. Obviously, there is an intricate coupling relationship between production and logistics, that is, flexible production arrangements lead to complex transfer requirements, which restricts and challenges the effective organization of logistics; and logistics arrangements determine the production rhythm, thus affecting the flexibility and efficiency of production. In view of the necessity of production activities and logistics activities in flexible workshops and the intricate coupling relationship between the two, the two must be coordinated to ensure effective production.

[0004] At present, many studies have focused on the problem of coordinated scheduling of production and logistics under flexible job shops. For example, in the Chinese invention patent "Intelligent Scheduling Decision Method for Flexible Job Shops Combined with Transportation Equipment Constraints" (publication number CN112949077A), for the intelligent scheduling problem of flexible job shops considering transportation equipment constraints, it is proposed to use genetic algorithms to schedule the corresponding processing equipment and transportation equipment to optimize the completion time. In the Chinese invention patent "Distributed Flexible Job Shop Scheduling Method Considering Limited Transportation Resources and Multi-Objectives" (publication number CN116187093A), it is proposed to combine the multi-objective comprehensive decision-making method with the intelligent optimization algorithm to solve the multi-objective distributed flexible job shop scheduling problem considering limited transportation resources and low carbon, and to achieve simultaneous optimization of the maximum completion time, total processing energy consumption and total processing quality. In the Chinese invention patent "A flexible job shop scheduling method considering transportation time and adjustment time" (publication number CN116109084A), in order to solve the flexible job shop scheduling problem that simultaneously considers transportation time and adjustment time, an ant colony algorithm integrating reinforcement learning is provided to optimize the completion time, total machine load, critical machine load and delivery time penalty value of the workpiece.

[0005] However, existing research mainly focuses on static environments and does not consider disturbance factors. In actual production, disturbances are inevitable, and ignoring disturbances will greatly limit the application value of the research. In addition, existing research mainly focuses on optimizing goals related to production activities (such as completion time, machine load, etc.), while ignoring goals related to logistics. In fact, production and logistics are often delivered to different managers in the workshop for separate management, and they each pursue specific goals related to their own tasks or resources. Therefore, to achieve synergy between these two separately managed activities, it is essentially necessary to optimize the goals of both activities at the same time, not just one of them. In summary, there is still a need to design a new production and logistics collaborative scheduling method to effectively deal with disturbances in actual production and simultaneously optimize the goals pursued by both production and logistics activities. Summary of the invention

[0006] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a production and logistics coordinated scheduling method for a dynamic flexible job shop, so as to effectively deal with disturbances in actual production and simultaneously optimize the goals pursued by both production and logistics activities.

[0007] The purpose of the present invention can be achieved by the following technical solutions:

[0008] The present invention provides a production and logistics collaborative scheduling method for a dynamic flexible job shop, comprising the following steps:

[0009] Obtain real-time information of dynamic flexible workshops, input the production and logistics collaborative scheduling problem model for dynamic flexible workshops, and solve to obtain workshop scheduling solutions;

[0010] The construction process of the production and logistics collaborative scheduling problem model for dynamic flexible job shops is as follows:

[0011] S1. Set the optimization goals of production activities and logistics activities, set task processing and resource allocation constraints, and establish a production and logistics collaborative scheduling problem model for static flexible workshops. The optimization goal of production activities is to minimize the maximum completion time, and the optimization goal of logistics activities is to minimize the total logistics cost.

[0012] S2. According to the failure scenarios of logistics equipment and the corresponding processing strategies, the optimization objectives of logistics activities are updated, and the task processing and resource allocation constraints under disturbance are further set. The problems are added to the production and logistics collaborative scheduling problem model for static flexible workshops, and the production and logistics collaborative scheduling problem model for dynamic flexible workshops is established.

[0013] The production and logistics collaborative scheduling problem model for dynamic flexible job shops is solved using a deep reinforcement learning method. The specific process is as follows:

[0014] S301, designing a multi-agent nested hierarchical framework adapted to the production and logistics collaborative scheduling problem model for the dynamic flexible job shop, wherein the multi-agent nested hierarchical framework includes a target agent, a production agent, and a logistics agent, wherein the production agent and the logistics agent are independent of each other and are used to make decisions on production scheduling and logistics scheduling, respectively, and the target agent is used to guide the decision-making direction of the production agent and the logistics agent;

[0015] S302, designing a reward function, state characteristics and action space according to the functions and management authority of each agent in the multi-agent nested hierarchical framework;

[0016] S303. Use the multi-agent proximal strategy optimization algorithm to collaboratively train each agent, input the real-time information of the dynamic flexible job shop into the trained agent, and obtain the workshop scheduling plan.

[0017] Furthermore, in step S1, the expression of the optimization target of the production activity is specifically as follows:

[0018]

[0019] Among them, MS represents the maximum completion time, ct ij represents the completion time of operation j of manufacturing task i, I represents the total number of manufacturing tasks, and J irepresents the total number of operations required to complete manufacturing task i;

[0020] The expression of the optimization objective of logistics activities is as follows:

[0021]

[0022] Where TC represents the total logistics cost, c represents the debugging cost required to activate an AGV, uc represents the logistics cost per unit transportation time of the AGV, and y v is a binary variable, which is 1 if and only if the AGV v is enabled in the scheduling scheme. V represents the total number of AGVs, nst ij and nct ij denote the start time and end time of the non-load phase of the transport task derived from operation j of manufacturing task i, respectively. ij and lct ij They represent the start time and end time of the load phase in the transportation task derived from operation j of manufacturing task i, respectively.

[0023] Furthermore, the task processing and resource allocation constraints include:

[0024] The completion time of each operation of each manufacturing task is the sum of its start time and the machine processing time. The specific expression is as follows:

[0025]

[0026] Among them, K represents the total number of machines, st ij represents the start time of operation j of manufacturing task i, pt ijk represents the processing time of operation j of manufacturing task i on machine k, λ ijk is a binary variable, which is 1 if and only if operation j of manufacturing task i is assigned to machine k for processing;

[0027] Each operation of each manufacturing task can only be processed after the task is transported to the corresponding machine. The specific expression is as follows:

[0028]

[0029] Each machine device can only process one operation at a time, the expression is as follows:

[0030] st ij +M(1-σ iji′j′k )≥ct i′j′

[0031]

[0032] Among them, σ iji′j′kis a binary variable that is 1 if and only if operation j of manufacturing task i is processed on machine k immediately after operation j′ of manufacturing task i′. i′j′ represents the completion time of operation j′ of manufacturing task i′, δ ijk is a binary variable that is 1 if and only if operation j of manufacturing task i is processed first on machine k;

[0033] Each automated guided vehicle can only transport one manufacturing task at a time, and the expression is as follows:

[0034] nst ij +M(1-ω iji′j′v )≥lct i′j′

[0035]

[0036] Among them, ω iji′j′v is a binary variable, which is 1 if and only if the transport task corresponding to operation j of manufacturing task i is processed on the AGV v immediately after the transport task corresponding to operation j′ of manufacturing task i′ is processed. ij is a binary variable, which is 1 if and only if the transportation task corresponding to operation j of manufacturing task i is processed first on the assigned AGV;

[0037] Each AGV has sufficient time to complete the non-load phase of the transportation task without being affected by disturbances. The specific expression is as follows:

[0038]

[0039]

[0040]

[0041] Among them, tt ll′ represents the transportation time from location l to l′, x ijv is a binary variable, which is 1 if and only if the transportation task corresponding to operation j of manufacturing task i is completed by the automatic guided vehicle v. kl is a binary variable that is 1 if and only if it indicates that machine k is located at position l, i′j′k is a binary variable that is 1 if and only if operation j′ of manufacturing task i′ is assigned to machine k for processing. ij-1k is a binary variable, which is 1 if and only if operation j-1 of manufacturing task i is assigned to machine k for processing;

[0042] The loading phase of the transport task corresponding to operation j of manufacturing task i can only start after the previous operation of the operation is completed and the assigned automatic guided vehicle completes its non-loading phase. The specific expression is as follows:

[0043]

[0044]

[0045] Among them, lst ij represents the start time of the load phase in the transport task derived from operation j of manufacturing task i;

[0046] Each AGV has sufficient time to complete the loading phase of the transport task without being affected by disturbances. The specific expression is as follows:

[0047]

[0048]

[0049] Among them, lct ij represents the end time of the load phase in the transport task derived from operation j of manufacturing task i;

[0050] Each operation of each manufacturing task can only be assigned to one machine to complete, and the expression is as follows:

[0051]

[0052] The transportation tasks derived from each operation of each manufacturing task can only be assigned to one automated guided vehicle to complete. The specific expression is as follows:

[0053]

[0054] Any two operations can only have a close-before-close-after relationship on one processing machine. The specific expression is as follows:

[0055]

[0056] Any two operations can only have a close relationship on one automated guided vehicle. The specific expression is as follows:

[0057]

[0058] Furthermore, in step S2, the logistics equipment is an automatic guided vehicle, and the failure scenarios of the logistics equipment and the corresponding processing strategies are as follows:

[0059] If an AGV fails while idle, it is repaired at the current location to restore availability;

[0060] If the AGV fails during the non-load phase, it is moved to a location closer to the failure location between the starting location and the destination location for repair;

[0061] If the AGV fails during the loading phase, it will be moved to a location closer to the failure location between the starting and destination locations for repair, and its loading task will be moved synchronously and rescheduled.

[0062] Furthermore, the expression of the optimization objective of the updated logistics activities is as follows:

[0063]

[0064] Where TC represents the total logistics cost, c represents the debugging cost required to activate an AGV, uc represents the logistics cost per unit transportation time of the AGV, and y v is a binary variable, which is 1 if and only if the AGV v is enabled in the scheduling scheme. V represents the total number of AGVs, nst ij and nct ij denote the start time and end time of the non-load phase of the transport task derived from operation j of manufacturing task i, respectively. ij and lct ij denote the start and end time of the load phase in the transport task derived from operation j of manufacturing task i, at n It represents the transportation time consumed by the logistics task being processed by the faulty vehicle when fault n occurs, and N represents the total number of faults.

[0065] Furthermore, the task processing and resource allocation constraints under disturbance include:

[0066] After a breakdown, the AGV can only start processing new logistics tasks after being repaired. The specific expression is as follows:

[0067]

[0068] Among them, tr n Indicates the time when fault n occurs, rt n represents the repair time required to repair fault n, is a binary variable that is 1 if and only if the AGV assigned to handle the logistics task derived from operation j of manufacturing task i has experienced a failure before handling the task, ξ ij is a binary variable, which is 1 if and only if the logistics task derived from operation j of manufacturing task i is the first task handled by the assigned AGV after repairing the fault, ξ ijn is a binary variable, which is 1 if and only if the logistics task derived from operation j of manufacturing task i is the first task processed by the assigned AGV after repairing fault n;

[0069] The transportation tasks that are interrupted by the failure can be restarted immediately after the interruption. The specific expression is as follows:

[0070]

[0071] Wherein, μijn is a binary variable, which is 1 if and only if the logistics task derived from operation j of manufacturing task i is interrupted due to fault n;

[0072] Each automated guided vehicle still satisfies the transportation-related constraints in the task processing and resource allocation constraints when not directly affected by the disturbance;

[0073] Each AGV starts from the repair site to handle the first logistics task after repair. The expression is as follows:

[0074]

[0075]

[0076]

[0077] Among them, z ij Indicates that the load phase of the logistics task derived from operation j of manufacturing task i is interrupted due to the failure, Indicates that from position rl n Transportation time to L+1, rl n represents the repair position of the faulty AGV after the nth fault occurs, λ ij-1k is a binary variable that is 1 if and only if operation j-1 of manufacturing task i is assigned to machine k for processing. kl is a binary variable, which is 1 if and only if it indicates that machine k is located at position l;

[0078] For logistics tasks with interrupted loading process, each AGV will go to the corresponding repair location to complete its non-loading phase task. The specific expression is as follows:

[0079]

[0080]

[0081]

[0082]

[0083] Among them, ω iji′j′v is a binary variable, which is 1 if and only if the transport task corresponding to operation j of manufacturing task i is processed on the AGV v immediately after the transport task corresponding to operation j′ of manufacturing task i′ is processed. ij is a binary variable, which is 1 if and only if the transportation task corresponding to operation j of manufacturing task i is processed first on the assigned AGV.ijv is a binary variable, which is 1 if and only if the corresponding transport task of operation j of manufacturing task i is completed by the automated guided vehicle v;

[0084] For logistics tasks that are interrupted due to failures during the load phase, each AGV starts from the corresponding repair position to complete the load phase task. The specific expression is as follows:

[0085]

[0086] Furthermore, in step S301, the design of the multi-agent nested hierarchical framework is as follows:

[0087] The target agent is deployed on the upper layer of the production agent and the logistics agent to control the production agent and the logistics agent;

[0088] The production agent is deployed on the upper layer of the logistics agent, and controls the logistics agent together with the target agent.

[0089] Further, in step S302, the state characteristics of the target agent include resource utilization information, task processing information and optimization preference, the resource utilization information includes machine utilization information and logistics equipment utilization information, the task processing information includes production task processing information and logistics task processing information, and the action space of the target agent includes selecting optimization to minimize the maximum completion time and selecting to minimize the total logistics cost;

[0090] The state characteristics of the production agent include machine utilization information, production task processing information, optimization preference and target agent decision information, and the action space of the production agent includes multiple combined scheduling rules for selecting the earliest pending operation of the unfinished manufacturing task and assigning it to the machine for processing;

[0091] The state characteristics of the logistics agent include logistics equipment utilization information, logistics task processing information, optimization preferences and target agent decision information. The action space of the logistics agent includes multiple scheduling rules for assigning derivative flow tasks of the selected operation to automatic guided vehicles for processing.

[0092] Furthermore, in step S302, the reward function includes a reward function related to minimizing the maximum completion time target and a reward function related to minimizing the total logistics cost, and the specific expressions are as follows:

[0093]

[0094]

[0095] Among them, RMS tTo minimize the maximum completion time target related reward function, w1 and w2 are weights, nms t Represents the normalized value of the minimum maximum completion time change before and after the state transfer, ms t represents the change in the minimum maximum completion time before and after the state transfer, pms and maxp represent the optimization preference for the goal of minimizing the maximum completion time and the maximum possible preference for a goal, respectively, ms max and ms min are the maximum and minimum values ​​of the change in the maximum completion time minimized before and after the state transfer, respectively;

[0096]

[0097]

[0098] Among them, RTC t To minimize the total logistics cost related reward function, ntc t Represents the normalized value of the logistics cost generated by this state transition, tc t represents the logistics cost generated by this state transition, ptc and maxp represent the optimization preference for minimizing the total logistics cost and the maximum possible preference for a goal, respectively, tc max and tc min They are respectively the maximum and minimum logistics costs generated by this state transfer.

[0099] Furthermore, in step S303, the specific process of using the multi-agent proximal strategy optimization algorithm to perform collaborative training on each agent is as follows:

[0100] Randomly initialize a policy network and a value network for each agent in the multi-agent nested hierarchical framework, and build an experience replay pool for each agent, wherein the policy network is used to select an action according to the current state output, and the value network is used to estimate the output value according to the state-action pair;

[0101] Initialize the random environment required for a single training. At each decision point in the environment, adopt the ε-greedy strategy combined with noise. According to the structure of the multi-agent nested hierarchical framework, the interaction and decision transmission between the environment and the agent, and between the agents are sequentially performed. The interaction trajectories are accumulated in the corresponding experience replay pool. If the number of interaction trajectories in the experience replay pool reaches a fixed batch size, the parameters of the policy network and the value network are updated using the Adam optimizer based on the update logic of the proximal policy optimization algorithm and combined with stochastic gradient descent until all manufacturing tasks in the environment are completed.

[0102] If the maximum number of training times is not reached after all manufacturing tasks in the environment are completed, the random environment required for a single training is reinitialized, the aforementioned training process is repeated, and the network parameters are updated until the maximum number of training times is reached.

[0103] Compared with the prior art, the present invention has the following beneficial effects:

[0104] 1. The present invention provides a production and logistics collaborative scheduling method for a dynamic flexible operation workshop, and obtains a workshop scheduling solution by solving the production and logistics collaborative scheduling problem model for the dynamic flexible operation workshop, wherein the construction process of the production and logistics collaborative scheduling problem model for the dynamic flexible operation workshop is as follows: first, taking minimizing the maximum completion time as the optimization goal of the production activity, taking minimizing the total logistics cost as the optimization goal of the logistics activity, setting task processing and resource allocation constraints, and establishing a production and logistics collaborative scheduling problem model for a static flexible operation workshop; secondly, according to the failure scenario of the logistics equipment and the corresponding processing strategy, updating the optimization goal of the logistics activity, and further setting the task processing and resource allocation constraints under disturbance, adding them to the production and logistics collaborative scheduling problem model for the static flexible operation workshop, and establishing a production and logistics collaborative scheduling problem model for the dynamic flexible operation workshop, the model can simultaneously optimize the goals pursued by the two activities of production and logistics, and at the same time, taking into account the possible failure scenarios and impacts of the logistics equipment, extracting new constraints, which are more in line with the actual manufacturing environment, and obtaining a more scientific, reasonable and effective flexible operation workshop scheduling solution.

[0105] 2. In the present invention, a deep reinforcement learning method is used to solve the production and logistics collaborative scheduling problem model for dynamic flexible job workshops. The specific process is: first, a multi-agent nested hierarchical framework adapted to the production and logistics collaborative scheduling problem model for dynamic flexible job workshops is designed. The multi-agent nested hierarchical framework includes a target agent, a production agent and a logistics agent. The production agent and the logistics agent are independent of each other and are used to make decisions on production scheduling and logistics scheduling respectively. The target agent is deployed on the upper layer of the production agent and the logistics agent to guide the decision-making direction of the production agent and the logistics agent. The production agent is deployed on the upper layer of the logistics agent and is connected with the target agent. The target agents jointly control the logistics agents. The framework fully considers the functions and management rights of each agent and is consistent with the actual manufacturing environment. Secondly, according to the functions and management rights of each agent in the multi-agent nested hierarchical framework, the reward function, state characteristics and action space are designed. The above process clarifies the Markov key elements that are compatible with the actual decision-making process, and provides a practical and specific solution to the problem. Finally, the multi-agent proximal strategy optimization algorithm is used to collaboratively train the agents. The algorithm can more effectively utilize the collected data to accelerate the learning process. It is suitable for processing complex multi-agent optimization problems and has good scalability and flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0106] Figure 1 is a flow chart of the method of the present invention;

[0107] Figure 2 It is a schematic diagram of the structure of the multi-agent nested hierarchical framework;

[0108] Figure 3 The workshop layout diagram of the embodiment is as follows:

[0109] Among them, L / U represents loading and unloading station, M m Represents machine m. DETAILED DESCRIPTION

[0110] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0111] Example:

[0112] This embodiment provides a production and logistics collaborative scheduling method for a dynamic flexible job shop. Figure 1 As shown, the following steps are included:

[0113] S1. Set the optimization goals of production activities and logistics activities respectively, set the task processing and resource allocation constraints, and establish a production and logistics collaborative scheduling problem model for static flexible workshops.

[0114] The optimization goal of production activities is to minimize the maximum completion time. The specific expression is as follows:

[0115]

[0116] Among them, MS represents the maximum completion time, ct ij represents the completion time of operation j of manufacturing task i, I represents the total number of manufacturing tasks, j i Represents the total number of operations required to complete manufacturing task i.

[0117] The optimization goal of logistics activities is to minimize the total logistics cost. The specific expression is as follows:

[0118]

[0119] Where TC represents the total logistics cost, c represents the debugging cost required to activate an AGV, uc represents the logistics cost per unit transportation time of the AGV, and y v is a binary variable, which is 1 if and only if the AGV v is enabled in the scheduling scheme. V represents the total number of AGVs, nst ij and nct ij denote the start time and end time of the non-load phase of the transport task derived from operation j of manufacturing task i, respectively. ij and lct ij They represent the start time and end time of the load phase in the transportation task derived from operation j of manufacturing task i, respectively.

[0120] Task processing and resource allocation constraints include:

[0121] (1) The completion time of each operation of each manufacturing task is the sum of its start time and the machine processing time. The specific expression is as follows:

[0122]

[0123] Among them, K represents the total number of machines, st ij represents the start time of operation j of manufacturing task i, pt ijk represents the processing time of operation j of manufacturing task i on machine k, λ ijk is a binary variable that is 1 if and only if operation j of manufacturing task i is assigned to machine k for processing.

[0124] (2) Each operation of each manufacturing task can only be processed after the task is transported to the corresponding machine. The specific expression is as follows:

[0125]

[0126] (3) Each machine device can only process one operation at a time. The specific expression is as follows:

[0127] st ij +M(1-σ iji′j′k )≥ct i′j′

[0128]

[0129] Among them, σ iji′j′k is a binary variable that is 1 if and only if operation j of manufacturing task i is processed on machine k immediately after operation j′ of manufacturing task i′. i′j′ represents the completion time of operation j′ of manufacturing task i′, δ ijk is a binary variable that is 1 if and only if operation j of manufacturing task i is the first to be processed on machine k.

[0130] (4) Each AGV can only transport one manufacturing task at a time. The specific expression is as follows:

[0131] nst ij +M(1-ω iji′j′v )≥lct i′j′

[0132]

[0133] Among them, ω iji′j′v is a binary variable, which is 1 if and only if the transport task corresponding to operation j of manufacturing task i is processed on the AGV v immediately after the transport task corresponding to operation j′ of manufacturing task i′ is processed. ij is a binary variable that is 1 if and only if the transportation task corresponding to operation j of manufacturing task i is first processed on the assigned AGV.

[0134] (5) Each AGV has sufficient time to complete the non-load phase of the transport task without being affected by disturbances. The specific expression is as follows:

[0135]

[0136]

[0137]

[0138] Among them, tt ll′represents the transportation time from location l to l′, x ijv is a binary variable, which is 1 if and only if the transportation task corresponding to operation j of manufacturing task i is completed by the automatic guided vehicle v. kl is a binary variable that is 1 if and only if it indicates that machine k is located at position l, i′j′k is a binary variable that is 1 if and only if operation j′ of manufacturing task i′ is assigned to machine k for processing. ij-1k is a binary variable that is 1 if and only if operation j-1 of manufacturing task i is assigned to machine k for processing.

[0139] (6) The loading phase of the transport task corresponding to operation j of manufacturing task i can only start after the previous operation of the operation is completed and the assigned AGV completes its non-loading phase. The specific expression is as follows:

[0140]

[0141]

[0142] Among them, lst ij Represents the start time of the load phase in the transport task derived from operation j of manufacturing task i.

[0143] (7) Each AGV has sufficient time to complete the loading phase of the transport task without being affected by disturbances. The specific expression is as follows:

[0144]

[0145]

[0146] Among them, lct ij Represents the end time of the load phase in the transport task derived from operation j of manufacturing task i.

[0147] (8) Each operation of each manufacturing task can only be assigned to one machine to complete. The specific expression is as follows:

[0148]

[0149] (9) The transportation tasks derived from each operation of each manufacturing task can only be assigned to one automated guided vehicle to complete. The specific expression is as follows:

[0150]

[0151] (10) Any two operations can only have a close-before-close-after relationship on one processing machine. The specific expression is as follows:

[0152]

[0153] (11) Any two operations can only have a close relationship on one automated guided vehicle. The specific expression is as follows:

[0154]

[0155] S2. According to the failure scenarios of logistics equipment and the corresponding processing strategies, the optimization objectives of logistics activities are updated, and the task processing and resource allocation constraints under disturbance are further set. The problems are added to the production and logistics collaborative scheduling problem model for static flexible workshops, and the production and logistics collaborative scheduling problem model for dynamic flexible workshops is established.

[0156] Taking the automated guided vehicle as the logistics equipment object, in the flexible operation workshop, each automated guided vehicle may fail at any time. The failure scenarios of logistics equipment and the corresponding processing strategies are as follows:

[0157] (1) If an AGV fails while idle, it is repaired at its current location to restore availability.

[0158] (2) If an AGV fails during the non-load phase, it is moved to a location closer to the failure location between the starting location and the destination location for maintenance, thereby avoiding interference with other activities caused by maintenance during the journey.

[0159] (3) If an AGV fails during the loading phase, it will be moved to a location closer to the failure location between the starting and destination locations for repair, and its loading task will be moved synchronously and rescheduled. The logistics activities affected by the failure will be rescheduled, and the failed vehicle will be restored to service after repair.

[0160] The expression of the optimization objective of the updated logistics activities is as follows:

[0161]

[0162] Where TC represents the total logistics cost, c represents the debugging cost required to activate an AGV, uc represents the logistics cost per unit transportation time of the AGV, and y v is a binary variable, which is 1 if and only if the AGV v is enabled in the scheduling scheme. V represents the total number of AGVs, nst ij and nct ij denote the start time and end time of the non-load phase of the transport task derived from operation j of manufacturing task i, respectively. ij and lct ij denote the start and end time of the load phase in the transport task derived from operation j of manufacturing task i, at nIt represents the transportation time consumed by the logistics task being processed by the faulty vehicle when fault n occurs, and N represents the total number of faults.

[0163] The task processing and resource allocation constraints under disturbance include:

[0164] (1) After a fault occurs, the AGV can only start processing new logistics tasks after being repaired. The specific expression is as follows:

[0165]

[0166] Among them, tr n Indicates the time when fault n occurs, rt n represents the repair time required to repair fault n, is a binary variable that is 1 if and only if the AGV assigned to handle the logistics task derived from operation j of manufacturing task i has experienced a failure before handling the task, ξ ij is a binary variable, which is 1 if and only if the logistics task derived from operation j of manufacturing task i is the first task handled by the assigned AGV after repairing the fault, ξ ijn is a binary variable that is 1 if and only if the logistics task derived from operation j of manufacturing task i is the first task processed by the assigned AGV after repairing fault n.

[0167] (2) The transportation task that is interrupted by the failure can be restarted immediately after the interruption. The specific expression is as follows:

[0168]

[0169] Wherein, μijn is a binary variable, which is 1 if and only if the logistics task derived from operation j of manufacturing task i is interrupted due to fault n.

[0170] (3) Each AGV still meets the transportation-related constraints in the task processing and resource allocation constraints when it is not directly affected by the disturbance (i.e., the load process of the logistics task it is responsible for is not affected or it is not the first task to be processed after the fault is repaired).

[0171] (4) Each AGV starts from the repair site to handle the first logistics task after the repair. The specific expression is as follows:

[0172]

[0173]

[0174]

[0175] Among them, z ijIndicates that the load phase of the logistics task derived from operation j of manufacturing task i is interrupted due to the failure, Indicates that from position rl n Transportation time to L+1, rl n represents the repair position of the faulty AGV after the nth fault occurs, λ ij-1k is a binary variable that is 1 if and only if operation j-1 of manufacturing task i is assigned to machine k for processing. kl is a binary variable, which is 1 if and only if it indicates that machine k is located at position l;

[0176] (5) For logistics tasks with interrupted loading process, each AGV will go to the corresponding repair location to complete its non-loading phase task. The specific expression is as follows:

[0177]

[0178]

[0179]

[0180]

[0181] Among them, ω iji′j′v is a binary variable, which is 1 if and only if the transport task corresponding to operation j of manufacturing task i is processed on the AGV v immediately after the transport task corresponding to operation j′ of manufacturing task i′ is processed. ij is a binary variable, which is 1 if and only if the transportation task corresponding to operation j of manufacturing task i is processed first on the assigned AGV. ijv is a binary variable, which is 1 if and only if the corresponding transportation task of operation j of manufacturing task i is completed by the automatic guided vehicle v.

[0182] (6) For logistics tasks that are interrupted due to failures during the load phase, each AGV starts from the corresponding repair position to complete the load phase task. The specific expression is as follows:

[0183]

[0184] S3. Use deep reinforcement learning methods to solve the production and logistics collaborative scheduling problem model for dynamic flexible workshops and obtain workshop scheduling solutions in real time. The specific process is as follows:

[0185] S301. Design a multi-agent nested hierarchical framework that is adapted to the production and logistics collaborative scheduling problem model for dynamic flexible job shops.

[0186] Specifically, the multi-agent nested hierarchical framework includes a target agent, a production agent, and a logistics agent. Considering that the above problem requires separate decisions on production scheduling and logistics scheduling, and the two are usually managed separately in actual manufacturing, a production agent and a logistics agent with independent state space and action space are designed to make decisions on production scheduling and logistics scheduling, respectively. Secondly, considering that the above problem requires the simultaneous optimization of two objectives, combined with the idea of ​​adaptive hierarchical reinforcement learning, a target agent is designed to guide the decision-making direction of the production agent and the logistics agent, guiding the two to achieve a good compromise between different objectives in long-term decision-making, and realize the required multi-objective optimization.

[0187] like Figure 2 As shown in the figure, the target agent is deployed on the upper layer of the production agent and the logistics agent, and makes decisions first to control the production agent and the logistics agent. For the production agent and the logistics agent at the lower layer, considering the sequential decision relationship between production scheduling and logistics scheduling in practice, they are also deployed in a hierarchical structure, that is, the production agent is deployed on the upper layer of the logistics agent, and controls the logistics agent together with the target agent.

[0188] S302. Design reward functions, state characteristics and action spaces according to the functions and management rights of each agent in the multi-agent nested hierarchical framework.

[0189] The state characteristics of the target agent include resource (including machines and logistics equipment) utilization information, task processing information (including production task processing information and logistics task processing information) and optimization preference; the state characteristics of the production agent include machine utilization information, production task processing information, optimization preference and target agent decision information; the state characteristics of the logistics agent include logistics equipment utilization information, logistics task processing information, optimization preference and target agent decision information. The state characteristics of each agent are shown in Table 1:

[0190] Table 1 Agent state characteristics

[0191]

[0192]

[0193] The action space of the target agent includes two possible target selections, namely, optimizing to minimize the maximum completion time and minimizing the total logistics cost; the action space of the production agent includes nine combined scheduling rules, which are used to select the earliest pending operation of the unfinished manufacturing task and assign it to the appropriate machine for processing; the action space of the logistics agent includes three scheduling rules, which are used to assign the derivative flow tasks of the selected operation to the automatic guided vehicle for processing. The action space of each agent is shown in Table 2:

[0194] Table 2 Action space of intelligent agents

[0195]

[0196] For the reward function, this embodiment designs two reward functions corresponding to two optimization objectives to evaluate the decision quality of the intelligent agent, including a reward function related to minimizing the maximum completion time objective and a reward function related to minimizing the total logistics cost. Both reward functions are composed of two parts: the degree of objective optimization and the degree of satisfaction of manager preferences, and are called accordingly according to the temporary objective decision of the target intelligent agent.

[0197] At decision point t, minimize the maximum completion time target related reward function RMS t The specific expression is as follows:

[0198]

[0199]

[0200] Among them, w1 and w2 are weights, nms t Represents the normalized value of the minimum maximum completion time change before and after the state transfer, ms t represents the change in the minimum maximum completion time before and after the state transfer, pms and maxp represent the optimization preference for the goal of minimizing the maximum completion time and the maximum possible preference for a goal, respectively, ms max and ms min are the maximum and minimum values ​​of the change in the maximum completion time before and after the state transfer.

[0201] At decision point t, minimize the total logistics cost related reward function RTC t The specific expression is as follows:

[0202]

[0203]

[0204] Among them, NTC t Represents the normalized value of the logistics cost generated by this state transition, tc t represents the logistics cost generated by this state transition, ptc and maxp represent the optimization preference for minimizing the total logistics cost and the maximum possible preference for a goal, respectively, tc max and tc min They are respectively the maximum and minimum logistics costs generated by this state transfer.

[0205] S303. Use the multi-agent proximal strategy optimization algorithm to collaboratively train each agent, and the trained agents will be directly used for subsequent decision making.

[0206] The specific process of model training is as follows:

[0207] A policy network and a value network are randomly initialized for each agent in the multi-agent nested hierarchical framework, and an experience replay pool with a specific capacity is built for each agent. The policy network is used to select actions according to the current state output, and the value network is used to estimate the output value based on the state-action pair. The structures of the two are shown in Table 3:

[0208] Table 3 Structure of strategy network and value network

[0209]

[0210]

[0211] Initialize the random environment required for a single training. At each decision point in the environment, the ε-greedy strategy combined with noise is used. According to the structure of the multi-agent nested hierarchical framework, the interaction and decision transmission between the environment and the agent, and between the agents are performed in sequence, and the interaction trajectories are accumulated in the corresponding experience replay pool. If the number of interaction trajectories in the experience replay pool reaches a fixed batch size, the update logic of the proximal policy optimization algorithm is combined with stochastic gradient descent to update the parameters of the policy network and the value network using the Adam optimizer until all manufacturing tasks in the environment are completed. The ε-greedy strategy combined with noise exists in the training process to balance exploration and development. It uses a certain probability ε to add random noise to the action probability distribution output by the policy network to affect the action selection of the agent. As the training progresses, ε will gradually decrease until it reaches a minimum value.

[0212] If the maximum number of training times L is not reached after all manufacturing tasks in the environment are completed, the random environment required for a single training is reinitialized, the above training process is repeated, and the network parameters are updated until the maximum number of training times L is reached.

[0213] The relevant parameters of the above training process are shown in Table 4:

[0214] Table 4 Training parameters

[0215] parameter value Number of training sessions 1000 Target agent / production agent / logistics agent playback pool size 32 / 32 / 500 Batch size 32 ε parameter 0.9 Lower bound of ε 0.1 Learning rate of target agent / production agent / logistics agent 0.0001 / 0.0001 / 0.001 Discount Factor 0.99 Pruning parameters 0.2

[0216] This example uses an aviation parts production workshop (a typical complex flexible operation workshop) as the object to verify the effectiveness of the above method. The workshop consists of 18 multifunctional machines and 6 automatic guided vehicles, which can process 7 types of workpieces with a certain process route, including compressor disks, turbine disks, fan disks, integral drums, compressor front journals, compressor rear journals and long shafts. The workshop layout is as follows Figure 3 As shown, L / U represents loading and unloading station, Mm represents machine m. The manufacturing tasks and fault-related information in the workshop are randomly adjusted to generate different instances for training and validation, including the number of jobs to be processed is randomly set to 10 to 40, and the type of each job is also randomly assigned. The processing time of each job on each available machine is randomly set to 2 to 10 time units. Faults are randomly assigned to automated guided vehicles. The interval time between two consecutive automated guided vehicle failures is exponentially distributed with an average value between 50 and 100. The repair time for each failure is randomly generated between 5 and 10. In addition, the manager's optimization preference for the two objectives is also randomly set within the corresponding range. Each instance is named "Exp(a / b)", indicating that it involves a jobs and b operations.

[0217] The method of the present invention is compared with a random scheduling method (i.e., a method that randomly selects a goal, a production scheduling rule, and a logistics scheduling rule at each decision point), a scheduling method based on scheduling rules (i.e., 27 composite scheduling rules composed of the actions of production agents and logistics agents), and two popular scheduling methods based on deep reinforcement learning, namely, a scheduling method based on a double deep Q network (DDQN) and a scheduling method based on a deep deterministic policy gradient algorithm (DDPG). Tables 5 and 6 list the results of the diversity index (Spacing, SP) and hypervolume (HV) of the Pareto scheme obtained after running each method 20 times on different instances. In order to effectively display the results, in each instance, only the results of the three composite scheduling rules with the best performance among all scheduling rules are listed, and an additional winning rate is added to represent the overall comparison between the method of the present invention and all rules. The winning rate is calculated by dividing the number of rules that the method of the present invention is better than or equal to by the total number of rules.

[0218] Table 5 Comparison results of SP indicators

[0219]

[0220]

[0221] Table 6 HV index comparison results

[0222]

[0223]

[0224] It can be seen from Table 5 (“-” indicates that only one solution is obtained by this method, resulting in the inability to measure SP or win rate) and Table 6 that compared with the random scheduling method, the method of the present invention can achieve better results in both indicators in almost all cases, which shows that the method of the present invention has learned an effective agent behavior strategy instead of using a random strategy to select feasible actions for each decision point. This comparison preliminarily verifies the learning effect of the method of the present invention and the effectiveness of subsequent experiments. Compared with the scheduling rules and other scheduling methods based on deep reinforcement learning, the effective results in Table 5 show that the method of the present invention obtains better or equivalent SP in most cases, indicating that the Pareto solution set obtained by the method of the present invention maintains better or equal distribution. Table 6 shows that in almost all cases, the HV value obtained by the method of the present invention is better than the HV value based on the scheduling rules or deep reinforcement learning method, which shows that the approximate Pareto solution set obtained by the method of the present invention has better convergence and diversity than that obtained by the benchmark method. These results further confirm that the method of the present invention is superior to other benchmark methods in solving the proposed multi-objective problem. In addition, it can be seen that the method of the present invention shows good performance when solving different instances, which also confirms its excellent versatility in dealing with different production environments. In summary, it can be concluded that the method of the present invention can effectively solve the production and logistics collaborative scheduling problems in different dynamic flexible job shop production environments.

[0225] If the above method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0226] The above description of the embodiments is to facilitate the understanding and use of the invention by those skilled in the art. It is obvious that those skilled in the art can easily make various modifications to these embodiments and apply the general principles described herein to other embodiments without creative work. Therefore, the present invention is not limited to the above embodiments, and improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope of the present invention should be within the scope of protection of the present invention.

Claims

1. A production and logistics collaborative scheduling method for a dynamic flexible job shop, characterized in that: The following steps are involved: Obtain real-time information of dynamic flexible workshops, input the production and logistics collaborative scheduling problem model for dynamic flexible workshops, and solve to obtain workshop scheduling solutions; The construction process of the production and logistics collaborative scheduling problem model for dynamic flexible job shops is as follows: S1. Set the optimization goals of production activities and logistics activities, set task processing and resource allocation constraints, and establish a production and logistics collaborative scheduling problem model for static flexible workshops. The optimization goal of production activities is to minimize the maximum completion time, and the optimization goal of logistics activities is to minimize the total logistics cost. S2. According to the failure scenarios of logistics equipment and the corresponding processing strategies, the optimization objectives of logistics activities are updated, and the task processing and resource allocation constraints under disturbance are further set. The problems are added to the production and logistics collaborative scheduling problem model for static flexible workshops, and the production and logistics collaborative scheduling problem model for dynamic flexible workshops is established. The production and logistics collaborative scheduling problem model for dynamic flexible job shops is solved using a deep reinforcement learning method. The specific process is as follows: S301, designing a multi-agent nested hierarchical framework adapted to the production and logistics collaborative scheduling problem model for the dynamic flexible job shop, wherein the multi-agent nested hierarchical framework includes a target agent, a production agent, and a logistics agent, wherein the production agent and the logistics agent are independent of each other and are used to make decisions on production scheduling and logistics scheduling, respectively, and the target agent is used to guide the decision-making direction of the production agent and the logistics agent; S302, designing a reward function, state characteristics and action space according to the functions and management authority of each agent in the multi-agent nested hierarchical framework; S303. Use the multi-agent proximal strategy optimization algorithm to collaboratively train each agent, input the real-time information of the dynamic flexible job shop into the trained agent, and obtain the workshop scheduling plan.

2. The method for coordinated production and logistics scheduling for a dynamic flexible job shop according to claim 1 is characterized in that: In step S1, the expression of the optimization target of the production activity is as follows: Among them, MS represents the maximum completion time, ct ij represents the completion time of operation j of manufacturing task i, I represents the total number of manufacturing tasks, and J i represents the total number of operations required to complete manufacturing task i; The expression of the optimization objective of logistics activities is as follows: Where TC represents the total logistics cost, c represents the debugging cost required to activate an AGV, uc represents the logistics cost per unit transportation time of the AGV, and y v is a binary variable, which is 1 if and only if the AGV v is enabled in the scheduling scheme. V represents the total number of AGVs, nst ij and nct ij denote the start time and end time of the non-load phase of the transport task derived from operation j of manufacturing task i, respectively. ij and lct ij They represent the start time and end time of the load phase in the transportation task derived from operation j of manufacturing task i, respectively.

3. The method for coordinated production and logistics scheduling for a dynamic flexible job shop according to claim 2 is characterized in that: The task processing and resource allocation constraints include: The completion time of each operation of each manufacturing task is the sum of its start time and the machine processing time. The specific expression is as follows: Among them, K represents the total number of machines, st ij represents the start time of operation j of manufacturing task i, pt ijk represents the processing time of operation j of manufacturing task i on machine k, λ ijk is a binary variable, which is 1 if and only if operation j of manufacturing task i is assigned to machine k for processing; Each operation of each manufacturing task can only be processed after the task is transported to the corresponding machine. The specific expression is as follows: Each machine device can only process one operation at a time, the expression is as follows: st ij +M(1-σ iji′j′k )≥ct i′j′ Among them, σ iji′j′k is a binary variable that is 1 if and only if operation j of manufacturing task i is processed on machine k immediately after operation j′ of manufacturing task i′. i′j′ represents the completion time of operation j′ of manufacturing task i′, δ ijk is a binary variable that is 1 if and only if operation j of manufacturing task i is processed first on machine k; Each automated guided vehicle can only transport one manufacturing task at a time, and the expression is as follows: next ij +M(1-ω iji′j′v )≥lct i′j′ Among them, ω iji′j′v is a binary variable, which is 1 if and only if the transport task corresponding to operation j of manufacturing task i is processed on the AGV v immediately after the transport task corresponding to operation j′ of manufacturing task i′ is processed. ij is a binary variable, which is 1 if and only if the transportation task corresponding to operation j of manufacturing task i is processed first on the assigned AGV; Each AGV has sufficient time to complete the non-load phase of the transportation task without being affected by disturbances. The specific expression is as follows: Among them, tt ll′ represents the transportation time from location l to l′, x ijv is a binary variable, which is 1 if and only if the transportation task corresponding to operation j of manufacturing task i is completed by the automatic guided vehicle v. kl is a binary variable that is 1 if and only if it indicates that machine k is located at position l, i′j′k is a binary variable that is 1 if and only if operation j′ of manufacturing task i′ is assigned to machine k for processing. ij-1k is a binary variable, which is 1 if and only if operation j-1 of manufacturing task i is assigned to machine k for processing; The loading phase of the transport task corresponding to operation j of manufacturing task i can only start after the previous operation of the operation is completed and the assigned automatic guided vehicle completes its non-loading phase. The specific expression is as follows: Among them, lst ij represents the start time of the load phase in the transport task derived from operation j of manufacturing task i; Each AGV has sufficient time to complete the loading phase of the transport task without being affected by disturbances. The specific expression is as follows: Among them, lct ij represents the end time of the load phase in the transport task derived from operation j of manufacturing task i; Each operation of each manufacturing task can only be assigned to one machine to complete, and the expression is as follows: The transportation tasks derived from each operation of each manufacturing task can only be assigned to one automated guided vehicle to complete. The specific expression is as follows: Any two operations can only have a close-before-close-after relationship on one processing machine. The specific expression is as follows: Any two operations can only have a close relationship on one automated guided vehicle. The specific expression is as follows:

4. The method for coordinated production and logistics scheduling for a dynamic flexible job shop according to claim 1 is characterized in that: In step S2, the logistics equipment is an automatic guided vehicle, and the failure scenarios of the logistics equipment and the corresponding processing strategies are as follows: If an AGV fails while idle, it is repaired at the current location to restore availability; If the AGV fails during the non-load phase, it is moved to a location closer to the failure location between the starting location and the destination location for repair; If the AGV fails during the loading phase, it will be moved to a location closer to the failure location between the starting and destination locations for repair, and its loading task will be moved synchronously and rescheduled.

5. The method for coordinated production and logistics scheduling for a dynamic flexible job shop according to claim 4 is characterized in that: The expression of the optimization objective of the updated logistics activities is as follows: Where TC represents the total logistics cost, c represents the debugging cost required to activate an AGV, uc represents the logistics cost per unit transportation time of the AGV, and y v is a binary variable, which is 1 if and only if the AGV v is enabled in the scheduling scheme. V represents the total number of AGVs, nst ij and nct ij denote the start time and end time of the non-load phase of the transport task derived from operation j of manufacturing task i, respectively. ij and lct ij denote the start and end time of the load phase in the transport task derived from operation j of manufacturing task i, at n It represents the transportation time consumed by the logistics task being processed by the faulty vehicle when fault n occurs, and N represents the total number of faults.

6. The method for coordinated production and logistics scheduling for a dynamic flexible job shop according to claim 5, characterized in that: The task processing and resource allocation constraints under the disturbance include: After a breakdown, the AGV can only start processing new logistics tasks after being repaired. The specific expression is as follows: Among them, tr n Indicates the time when fault n occurs, rt n represents the repair time required to repair fault n, is a binary variable that is 1 if and only if the AGV assigned to handle the logistics task derived from operation j of manufacturing task i has experienced a failure before handling the task, ξ ij is a binary variable, which is 1 if and only if the logistics task derived from operation j of manufacturing task i is the first task handled by the assigned AGV after repairing the fault, ξ ijn is a binary variable, which is 1 if and only if the logistics task derived from operation j of manufacturing task i is the first task processed by the assigned AGV after repairing fault n; The transportation tasks that are interrupted by the failure can be restarted immediately after the interruption. The specific expression is as follows: Among them, μ ijn is a binary variable, which is 1 if and only if the logistics task derived from operation j of manufacturing task i is interrupted by fault n; Each automated guided vehicle still satisfies the transportation-related constraints in the task processing and resource allocation constraints when not directly affected by the disturbance; Each AGV starts from the repair site to handle the first logistics task after repair. The expression is as follows: Among them, z ij Indicates that the load phase of the logistics task derived from operation j of manufacturing task i is interrupted due to the failure, Indicates that from position rl n Transportation time to L+1, rl n represents the repair position of the faulty AGV after the nth fault occurs, λ ij-1k is a binary variable that is 1 if and only if operation j-1 of manufacturing task i is assigned to machine k for processing. kl is a binary variable, which is 1 if and only if it indicates that machine k is located at position l; For logistics tasks with interrupted loading process, each AGV will go to the corresponding repair location to complete its non-loading phase task. The specific expression is as follows: Among them, ω iji′j′v is a binary variable, which is 1 if and only if the transport task corresponding to operation j of manufacturing task i is processed on the AGV v immediately after the transport task corresponding to operation j′ of manufacturing task i′ is processed. ij is a binary variable, which is 1 if and only if the transportation task corresponding to operation j of manufacturing task i is processed first on the assigned AGV. ijv is a binary variable, which is 1 if and only if the corresponding transport task of operation j of manufacturing task i is completed by the automated guided vehicle v; For logistics tasks that are interrupted due to failures during the load phase, each AGV starts from the corresponding repair position to complete the load phase task. The specific expression is as follows:

7. The method for coordinated production and logistics scheduling for a dynamic flexible job shop according to claim 1, characterized in that: In step S301, the design of the multi-agent nested hierarchical framework is as follows: The target agent is deployed on the upper layer of the production agent and the logistics agent to control the production agent and the logistics agent; The production agent is deployed on the upper layer of the logistics agent, and controls the logistics agent together with the target agent.

8. The method for coordinated production and logistics scheduling for a dynamic flexible job shop according to claim 1 is characterized in that: In step S302, the state characteristics of the target intelligent agent include resource utilization information, task processing information and optimization preference, the resource utilization information includes machine utilization information and logistics equipment utilization information, the task processing information includes production task processing information and logistics task processing information, and the action space of the target intelligent agent includes selecting to optimize minimizing the maximum completion time and selecting to minimize the total logistics cost; The state characteristics of the production agent include machine utilization information, production task processing information, optimization preference and target agent decision information, and the action space of the production agent includes multiple combined scheduling rules for selecting the earliest pending operation of the unfinished manufacturing task and assigning it to the machine for processing; The state characteristics of the logistics agent include logistics equipment utilization information, logistics task processing information, optimization preferences and target agent decision information. The action space of the logistics agent includes multiple scheduling rules for assigning derivative flow tasks of the selected operation to automatic guided vehicles for processing.

9. The method for coordinated production and logistics scheduling for a dynamic flexible job shop according to claim 1, characterized in that: In step S302, the reward function includes a reward function related to minimizing the maximum completion time target and a reward function related to minimizing the total logistics cost, and the specific expressions are as follows: Among them, RMS t To minimize the maximum completion time target related reward function, w1 and w2 are weights, nms t Represents the normalized value of the minimum maximum completion time change before and after the state transfer, ms t represents the change in the minimum maximum completion time before and after the state transfer, pms and maxp represent the optimization preference for the goal of minimizing the maximum completion time and the maximum possible preference for a goal, respectively, ms max and ms min are the maximum and minimum values ​​of the change in the maximum completion time minimized before and after the state transfer, respectively; Among them, RTC t To minimize the total logistics cost related reward function, ntc t Represents the normalized value of the logistics cost generated by this state transition, tc t represents the logistics cost generated by this state transition, ptc and maxp represent the optimization preference for minimizing the total logistics cost and the maximum possible preference for a goal, respectively, tc max and tc min They are respectively the maximum and minimum logistics costs generated by this state transfer.

10. The method for coordinated production and logistics scheduling for a dynamic flexible job shop according to claim 1, characterized in that: In step S303, the specific process of using the multi-agent proximal strategy optimization algorithm to perform collaborative training on each agent is as follows: Randomly initialize a policy network and a value network for each agent in the multi-agent nested hierarchical framework, and build an experience replay pool for each agent, wherein the policy network is used to select an action according to the current state output, and the value network is used to estimate the output value according to the state-action pair; Initialize the random environment required for a single training. At each decision point in the environment, adopt the ε-greedy strategy combined with noise. According to the structure of the multi-agent nested hierarchical framework, the interaction and decision transmission between the environment and the agent, and between the agents are sequentially performed. The interaction trajectories are accumulated in the corresponding experience replay pool. If the number of interaction trajectories in the experience replay pool reaches a fixed batch size, the parameters of the policy network and the value network are updated using the Adam optimizer based on the update logic of the proximal policy optimization algorithm and combined with stochastic gradient descent until all manufacturing tasks in the environment are completed. If the maximum number of training times is not reached after all manufacturing tasks in the environment are completed, the random environment required for a single training is reinitialized, the aforementioned training process is repeated, and the network parameters are updated until the maximum number of training times is reached.

Images