A scheduling method and system for multi-module configuration reconfigurable production line
By combining a 3D configuration-coupled heterogeneous disjunctive graph and a relation-aware heterogeneous graph fusion network with near-end strategy optimization and artificial bee colony algorithm, the problem of scheduling infeasibility and local optima in multi-module configuration reconfigurable production lines is solved, and high-quality scheduling decisions are achieved.
Patent Information
- Application Number
- CN202611465821.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-20
- Publication Date
- 2026-11-06
AI Technical Summary
Existing scheduling methods struggle to accurately characterize the complex coupling constraints between processes, configurations, and auxiliary modules in multi-module reconfigurable production lines. Furthermore, single reinforcement learning algorithms are prone to getting trapped in local optima, leading to scheduling infeasibility or excessive computational overhead.
A fusion network of three-dimensional configuration-coupled heterogeneous disjunctive graph and relation-aware heterogeneous graph is adopted, combined with near-end policy optimization algorithm and artificial bee colony algorithm. Through Markov decision process and reinforcement learning, the coupling relationship between order process, configuration and auxiliary module is uniformly described, thereby improving the feasibility and quality of scheduling scheme.
It enables more precise scheduling decisions in multi-module reconfigurable production lines, reduces scheduling infeasibility and computational overhead, improves the quality and stability of scheduling schemes, and avoids resource deadlock and local optima traps.
Smart Images

Figure CN123051027A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the technical field of intelligent scheduling in workshop manufacturing processes, and more specifically, relates to a scheduling method and system for reconfigurable production lines with multi-module configuration. Background Technology
[0002] Reconfigurable production lines can rapidly adjust the structure, functional modules, and processing capabilities of processing equipment according to changes in product type, processing tasks, and capacity requirements. This makes them an important production organization form for multi-variety, small-batch, and rapid-response manufacturing scenarios. In these production lines, reconfigurable processing equipment typically consists of basic modules and multiple auxiliary modules combined to form different configurations, each corresponding to different processing capabilities. Compared to traditional flexible manufacturing systems, scheduling reconfigurable production lines requires not only determining the processing sequence of order steps and equipment allocation, but also simultaneously determining equipment configuration selection, configuration switching timing, and the occupancy and release relationships of auxiliary modules.
[0003] In real-world reconfigurable production lines, an effective configuration is often not determined by a single auxiliary module, but rather by multiple auxiliary modules working together. Auxiliary modules typically come from a shared and limited pool of modules, and the same module copy can only serve one configuration at a time. When multiple order processes simultaneously compete for the same configuration, or multiple configurations simultaneously compete for the same auxiliary module, the feasibility of the scheduling scheme is severely impacted. Even slight coding perturbations or action selection deviations can lead to module conflicts, configuration unavailability, resource waiting, or even scheduling deadlocks. Therefore, the scheduling problem in multi-module reconfigurable production lines exhibits significant high-dimensionality, strong coupling, and feasibility sensitivity.
[0004] In existing scheduling methods, heuristic rules typically rely on manually set priorities, making it difficult to adaptively adjust based on order progress, configuration contention, and auxiliary module occupancy status. While traditional metaheuristic algorithms possess some global search capabilities, they require complex coding repair and feasibility assessment in multi-module configuration scenarios, easily generating numerous infeasible solutions and increasing computational overhead. Methods based solely on deep reinforcement learning can avoid explicit coding through sequential decision-making, but if the state representation cannot fully capture the heterogeneous relationships between order processes, configurations, and auxiliary modules, the policy network struggles to identify key coupling factors affecting feasibility and completion time.
[0005] Furthermore, the reconfigurable production line scheduling state with multi-module configurations naturally possesses a heterogeneous graph structure: there are process sequence constraints between order processes, processing compatibility relationships between order processes and configurations, competition between different configurations due to handling the same process, competition for module resources due to sharing auxiliary modules, and collaborative relationships between multiple auxiliary modules within the same configuration. If these relationships are compressed into ordinary vectors or a single isomorphic graph, the semantics of these relationships are easily lost, making it impossible for the policy network to accurately determine the feasibility of actions and the quality of scheduling.
[0006] Therefore, there is an urgent need for a reconfigurable production line scheduling method and system that can uniformly describe the coupling relationship between order processes, configurations, and auxiliary modules, maintain the feasibility of actions during the sequential decision-making process, and take into account both the stability of strategy learning and the ability to explore the whole world. Summary of the Invention
[0007] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention provides a scheduling method and system for multi-module configurable reconfigurable production lines. It aims to solve the problem of how to more accurately represent the complex coupling constraint relationship between processes, configurations and auxiliary modules in the scheduling of multi-module configurable reconfigurable production lines, and overcome the problem that a single reinforcement learning algorithm is prone to getting trapped in local optima, so as to obtain a higher quality scheduling scheme while ensuring scheduling feasibility.
[0008] To achieve the above objectives, according to one aspect of the present invention, a scheduling method for a multi-module configurable reconfigurable production line is provided, comprising the following steps: (1) The scheduling problem model of a multi-module configurable reconfigurable production line is mapped to a three-dimensional configuration coupling heterogeneous disjunction graph. The three-dimensional configuration coupling heterogeneous disjunction graph includes process nodes, configuration nodes, auxiliary module nodes, order process sequence arcs, allocation edges between processes and configurations, demand edges between configurations and auxiliary modules, and start and end boundaries of the scheduling process represented by start virtual nodes and end virtual nodes. (2) The multi-module configuration reconfigurable production line scheduling problem is characterized as a Markov decision process: When the system time reaches a decision point where there is a ready process and a feasible configuration, the current state of the decision point is obtained through the three-dimensional configuration coupling heterogeneous disjunction graph encoding. Based on the current state, an action consisting of order process and equipment configuration is selected from the set of actionable actions. After executing the action, the system time, configuration state, auxiliary module occupancy state, order process state and graph state are updated to obtain the state of the next decision point. (3) Based on the state of all decision points, a relation-aware heterogeneous graph fusion network is used to perform representation learning on the three-dimensional configuration coupled heterogeneous disjunctive graph: the three-dimensional configuration coupled heterogeneous disjunctive graph is decomposed into multiple meta-path subgraphs representing the sequential relationship of the process, the competitive relationship of the configuration to the same process, the competitive relationship of the configuration to the shared auxiliary module, and the collaborative relationship of the auxiliary module within the same configuration. The multiple meta-path subgraphs are processed by the edge-enhanced graph attention mechanism and the multi-path fusion mechanism to obtain the embedded representation of the process node, configuration node, auxiliary module node and global system state. (4) Construct a reinforcement learning policy network and a value network based on the embedding representation. After concatenating the process embedding, configuration embedding and auxiliary module context vector corresponding to the candidate action, input the network into the reinforcement learning decision network to obtain the priority score of the candidate action. Then, process the priority score of the candidate action through a normalization function to obtain the probability distribution of the actionable action. Based on the probability distribution of the actionable action, determine the scheduling action by greedy selection or sampling. At the same time, use the value network to output the state value of the scheduling action based on the embedding representation. (5) Based on the obtained scheduling actions and state values, the parameters of the relation-aware heterogeneous graph fusion network, the reinforcement learning policy network, and the value network are optimized by combining the near-end policy optimization algorithm and the artificial bee colony algorithm: After performing policy gradient updates based on environment interaction on the relation-aware heterogeneous graph fusion network, the policy network, and the value network by the near-end policy optimization algorithm, the population-based parameter search is performed on the relation-aware heterogeneous graph fusion network, the reinforcement learning policy network, and the value network by the artificial bee colony algorithm to obtain a policy network that satisfies the configuration, auxiliary module, and process constraints; and then the optimal scheduling scheme is obtained based on the policy network.
[0009] Furthermore, multiple meta-path subgraphs include OO , COC , CAC and ACA Four types of meta-path subgraphs, among which OO Subgraphs represent the sequential constraints between order processes. COC The subgraph represents the competitive relationship between multiple configurations for the same incomplete order process. CAC The subgraph represents the competitive relationship between multiple configurations for a shared auxiliary module. ACA Subgraphs represent the collaborative relationships between multiple auxiliary modules within the same configuration.
[0010] Furthermore, on COC The subgraph introduces process-level competition features based on a shared order process set, for... CAC The subgraph introduces module-level competition features based on a set of shared auxiliary modules.
[0011] Furthermore, during the construction of the action set, when there is a ready order process but no feasible configuration exists due to conflicts in auxiliary modules, a virtual release mechanism is triggered to temporarily treat the auxiliary modules currently occupied by the idle reconfigurable processing equipment as available, thereby re-evaluating the configuration feasibility.
[0012] Furthermore, bidirectional collaboration between the near-end policy optimization algorithm and the artificial bee colony algorithm is achieved through individual injection from the near-end policy optimization algorithm to the artificial bee colony algorithm and adaptive probabilistic soft update from the artificial bee colony algorithm to the near-end policy optimization algorithm.
[0013] Furthermore, during the parameter optimization process, the parameters of the relation-aware heterogeneous graph fusion network are initialized using the Xavier initialization method, and the parameters of the decision network and value network based on the multilayer perceptron are initialized using the Kaiming initialization method, to obtain the near-end policy optimization network parameter vector. This network parameter vector is then used as the first initialized individual in the artificial bee colony population, and the remaining individuals are generated by superimposing a perturbation that follows a zero-mean Gaussian distribution onto this network parameter vector.
[0014] Furthermore, the artificial bee colony algorithm searches the strategy parameter space according to the hired bee phase, the observation bee phase, and the scout bee phase, and uses the maximum completion time of the scheduling scheme as the fitness index to evaluate the quality of individuals.
[0015] Furthermore, the reinforcement learning policy network adopts the actor network in the actor-critic structure; the value network adopts a multilayer perceptron.
[0016] The present invention also provides a scheduling system for a multi-module configurable reconfigurable production line. The system includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the scheduling method for a multi-module configurable reconfigurable production line as described above.
[0017] The present invention also provides a computer-readable storage medium storing machine-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the scheduling method for a multi-module configurable reconfigurable production line as described above.
[0018] In summary, compared with the prior art, the scheduling method and system for reconfigurable production lines with multi-module configuration provided by the present invention have the following advantages: 1. This invention uses a three-dimensional configuration-coupled heterogeneous disjunctive graph to uniformly represent the process sequence, processing compatibility, module requirements, configuration competition, and module collaboration relationships among order processes, configurations, and auxiliary modules. This enables the scheduling scheme to fully reflect the structural characteristics of a multi-module configuration reconfigurable production line. The relation-aware heterogeneous graph fusion network learns neighborhood information for different relational semantics and fuses multi-source competition information of configuration nodes, thereby improving the policy network's ability to identify configuration competition and auxiliary module competition. Through bidirectional collaboration between the near-end policy optimization algorithm and the artificial bee colony algorithm, gradient-based policy updates and population-based global search complement each other, improving learning stability while reducing the probability of the policy getting trapped in local optima. This ensures high-quality scheduling schemes while maintaining scheduling feasibility.
[0019] 2. This invention constrains action selection through a set of feasible actions and a virtual release mechanism, which can avoid obviously infeasible order process-configuration matching during the sequential decision-making process and reduce subsequent repair costs.
[0020] 3. In relation-aware heterogeneous graph fusion networks, for COC The subgraph introduces process-level competition features based on a shared order process set, for... CAC The subgraph introduces module-level competition features based on a shared set of auxiliary modules, making the graph attention weights depend not only on node features but also on the actual competition intensity between configurations.
[0021] 4. During the construction of the action set, when there is a ready order process but no feasible configuration is available due to conflicts in auxiliary modules, the virtual release mechanism is triggered. The auxiliary modules currently occupied by the idle reconfigurable processing equipment are temporarily regarded as available, and the configuration feasibility is reassessed, thereby reducing the risk of deadlock caused by the lag in resource occupancy status. Attached Figure Description
[0022] Figure 1 This is a flowchart of a scheduling method for a multi-module configurable reconfigurable production line provided by an embodiment of the present invention; Figure 2 This is a typical structural schematic diagram of the reconfigurable production line and reconfigurable processing equipment involved in the present invention; Figure 3 This is a Gantt chart diagram of a feasible scheduling scheme of the present invention; Figure 4 This is a schematic diagram of the three-dimensional configuration coupling heterogeneous extraction diagram provided by the present invention; Figure 5 This is a flowchart of the relation-aware evolutionary reinforcement learning scheduling framework provided by the present invention; Figure 6 This is a schematic diagram of the structure of the relation-aware heterogeneous graph fusion network provided by the present invention; Figure 7 This is the evolutionary reinforcement learning training flowchart provided by the present invention. Detailed Implementation
[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.
[0024] Please see Figure 1 This invention provides a scheduling method for reconfigurable production lines with multi-module configuration. The method uses a three-dimensional configuration coupling heterogeneous extraction graph to uniformly represent the multiple constraint relationships between order processes, equipment configurations, and auxiliary modules. It extracts state representations under different relational semantics through a relation-aware heterogeneous graph fusion network, and improves the policy search capability through a bidirectional collaborative training mechanism of near-end policy optimization algorithm and artificial bee colony algorithm, thereby obtaining a high-quality scheduling scheme while ensuring scheduling feasibility.
[0025] Please see Figure 2 , Figure 3 , Figure 4 , Figure 5 , Figure 6 and Figure 7 The method includes the following steps: Step 1: After determining the set of reconfigurable processing equipment, the set of equipment convertible configurations, the set of auxiliary modules, the set of orders, and the set of order operations, the configuration of each reconfigurable processing equipment is limited to the processing capacity formed by the combination of one or more auxiliary modules. Based on the configuration switching time, the operation processing time, and the number of module replicas, a scheduling problem model is established with the optimization objective of minimizing the maximum completion time.
[0026] The set of reconfigurable processing equipment is denoted as Each reconfigurable processing device Supports a set of interchangeable configurations. Each configuration A specific processing capability is formed by at least one auxiliary module; the set of auxiliary modules is denoted as... Each type of auxiliary module has b a There are identical copies; the order set is denoted as Each order J j Includes a set of processes arranged in the order of their operation. Each process It must be compatible with at least one device configuration and have compatibility with the device. Processing time on configuration c The scheduling problem model assigns each order process to the corresponding reconfigurable processing equipment and its configuration, and minimizes the maximum completion time. C max To optimize the objective.
[0027] Step 2: Map the scheduling problem model into a three-dimensional configuration coupling heterogeneous disjunction graph. The three-dimensional configuration coupling heterogeneous disjunction graph includes process nodes, configuration nodes, auxiliary module nodes, order process sequence arcs, allocation edges between processes and configurations, demand edges between configurations and auxiliary modules, and start and end boundaries of the scheduling process represented by start virtual nodes and end virtual nodes.
[0028] The three-dimensional configuration coupling heterogeneous dissociation graph is defined as follows: ,in O Represents the set of order process nodes. C Represents the set of configuration nodes. A Represents the set of auxiliary module nodes. U This represents a directed arc used to describe the sequence constraints of processes within the same order. V This represents the connection edge used to describe the allocation of flexibility between order processes and manufacturable configurations. W This represents the connection edges used to describe the combination of auxiliary modules required for each configuration; among them, the order process node, configuration node, and auxiliary module node together represent the scheduling status of the multi-module configuration reconfigurable production line, starting with the virtual node. S and terminate virtual nodes E These represent the start and end points of the scheduling process, respectively.
[0029] The state space is obtained by encoding the three-dimensional configuration coupled heterogeneous disjunction graph, and at each decision point, only incomplete order processes, configurations capable of processing the remaining processes, auxiliary module occupancy and availability status, configuration reconfiguration information, and process-configuration processing time information are retained; the action space... A ( t This includes all feasible order process-configuration pairs, which are determined to be feasible when the selected order process is in a ready state, the selected configuration is available, and the required auxiliary module is available or can be obtained through reconfiguration; when there is a ready process but no feasible configuration exists due to auxiliary module conflicts, a virtual release mechanism is triggered, and the auxiliary module on the idle reconfigurable processing equipment is temporarily regarded as available and the configuration feasibility is reassessed.
[0030] Step 3: The multi-module configuration reconfigurable production line scheduling problem is characterized as a Markov decision process: When the system time reaches a decision point where there is a ready process and a feasible configuration, the current state of the decision point is obtained through the three-dimensional configuration coupling heterogeneous disjunctive graph encoding. Based on the current state, an action consisting of order process and equipment configuration is selected from the set of actionable actions. After executing the action, the system time, configuration state, auxiliary module occupancy state, order process state and graph state are updated to obtain the state of the next decision point.
[0031] At each decision point t, the observation is encoded by a three-dimensional configuration-coupled heterogeneous disjunctive graph to obtain the current state s of decision point t. t and from the set of actionable actions A ( t Select the order process - equipment configuration action. After executing the selected action, update the system time, order status, device configuration status, and auxiliary module occupancy status, and return to the next status. t+ 1 and rewards Repeat the above process until all order processes are completed.
[0032] In one implementation, after executing the selected action, it is determined whether the corresponding reconfigurable processing equipment needs to be reconfigured. If reconfiguration is required, the current auxiliary module is released and the auxiliary module required for the target configuration is acquired. If the auxiliary module is unavailable, module reallocation is performed based on a virtual release mechanism. After the selected order process is completed, the order status, the reconfigurable processing equipment status, and the set of actionable actions are updated. If no actionable actions exist, the system time is advanced to the next event occurrence time. The Markov decision process uses a reward function to encourage actions that can reduce the estimated maximum completion time. The reward function is defined as the difference between the estimated maximum completion times of two adjacent decision points. The completion time of an unscheduled process is estimated based on the completion time of its preceding process and the minimum processing time in a compatible configuration.
[0033] Step four: Based on the state of all decision points, a relation-aware heterogeneous graph fusion network is used to perform representation learning on the three-dimensional configuration coupled heterogeneous disjunctive graph. The three-dimensional configuration coupled heterogeneous disjunctive graph is decomposed into multiple meta-path subgraphs representing the sequential relationship of processes, the competitive relationship of configurations to the same process, the competitive relationship of configurations to shared auxiliary modules, and the collaborative relationship of auxiliary modules within the same configuration. The multiple meta-path subgraphs are then processed through an edge-enhanced graph attention mechanism and a multi-path fusion mechanism to obtain embedded representations of process nodes, configuration nodes, auxiliary module nodes, and the global system state.
[0034] The relation-aware heterogeneous graph fusion network decomposes the 3D configuration-coupled heterogeneous dissociative graph into... OO , COC , CAC and ACA Four types of meta-path subgraphs, among which OO Subgraphs represent the sequential constraints between order processes. COC The subgraph represents the competitive relationship between multiple configurations for the same incomplete order process. CAC The subgraph represents the competitive relationship between multiple configurations for a shared auxiliary module. ACA Subgraphs represent the collaborative relationships between multiple auxiliary modules within the same configuration; for COC The subgraph introduces process-level competition features based on a shared order process set, targeting... CAC The subgraph introduces module-level competitive features based on a shared set of auxiliary modules to aggregate neighborhood information according to relational semantics. An edge-enhanced graph attention mechanism is used to independently encode each metapath subgraph, and multi-path fusion is performed on configuration nodes participating in multiple competitive relationships to obtain embedded representations of node-level and global system states.
[0035] For those who participate at the same time COC Subgraph and CAC The configuration nodes of the subgraph are obtained respectively. COC Metapath and CAC The embedding representation under the metapath is obtained, and learnable scalar weights are assigned to different metapath subgraphs. The learnable scalar weights are normalized and then used to calculate the fusion embedding of the configuration node. After completing the multi-head attention aggregation, the outputs of each attention head are concatenated to form the final node representation. Average pooling is performed on the order process node, configuration node and auxiliary module node respectively. The type-level context vector after average pooling is concatenated with the corresponding node embedding to form the enhanced node representation and the global system representation used for downstream scheduling decisions.
[0036] Step 5: Construct a reinforcement learning policy network and a value network based on the embedded representation. Concatenate the process embedding, configuration embedding, and auxiliary module context vector corresponding to the candidate action and input them into the reinforcement learning decision network to obtain the priority score of the candidate action. Process the priority score of the candidate action through a normalization function to obtain the probability distribution of the actionable action. Based on the probability distribution of the actionable action, determine the scheduling action by greedy selection or sampling. At the same time, use the value network to output the state value of the scheduling action based on the embedded representation.
[0037] The reinforcement learning policy network employs an actor-critic structure, where, at each decision step, it considers candidate actions... Order process nodes Embedded and configuration nodes The embedded and pooled context vectors of the auxiliary modules are concatenated and then input into the reinforcement learning decision network. The network outputs priority scores of candidate actions and normalizes the priority scores of all possible actions to obtain policy probabilities. The value network employs a multilayer perceptron, using a global system representation as input and outputting state values. .
[0038] Step Six: Based on the obtained scheduling actions and state values, the parameters of the relation-aware heterogeneous graph fusion network, the reinforcement learning policy network, and the value network are optimized by combining the proximal policy optimization algorithm and the artificial bee colony algorithm. After performing policy gradient updates based on environment interaction on the relation-aware heterogeneous graph fusion network, the policy network, and the value network through the proximal policy optimization algorithm, a population-based parameter search is performed on the relation-aware heterogeneous graph fusion network, the reinforcement learning policy network, and the value network through the artificial bee colony algorithm to obtain a policy network that satisfies the constraints of configuration, auxiliary modules, and processes. Then, the optimal scheduling scheme is obtained based on the policy network.
[0039] An evolutionary reinforcement learning training mechanism is formed by combining the proximal policy optimization algorithm and the artificial bee colony algorithm. The evolutionary reinforcement learning training mechanism realizes the bidirectional collaboration between the proximal policy optimization algorithm and the artificial bee colony algorithm through individual injection from the proximal policy optimization algorithm to the artificial bee colony algorithm and adaptive probabilistic soft update from the artificial bee colony algorithm to the proximal policy optimization algorithm.
[0040] In the initialization phase, the parameters of the relation-aware heterogeneous graph fusion network are initialized using the Xavier initialization method, and the parameters of the decision network and value network based on the multilayer perceptron are initialized using the Kaiming initialization method, resulting in a near-end policy optimization network parameter vector. This network parameter vector is then used as the first initialized individual in the artificial bee colony population, and the remaining individuals are generated by superimposing a perturbation following a zero-mean Gaussian distribution onto this network parameter vector. During the parameter optimization process, the near-end policy optimization algorithm performs gradient-based policy updates, and the artificial bee colony algorithm searches the policy parameter space according to the mercenary bee stage, the observer bee stage, and the scout bee stage. The maximum completion time of the scheduling scheme is used as the fitness index to evaluate the quality of individuals.
[0041] In the individual injection path, the parameters corresponding to the current proximal strategy optimization are injected into the artificial bee colony and the individual with the worst fitness is replaced to guide the population to search for parameters closer to the gradient optimization region. In the soft update path, the acceptance probability is calculated based on the initial temperature, minimum temperature, current temperature, temperature decay rate, basic acceptance probability, minimum acceptance probability, maximum acceptance probability, fitness of the best individual in the artificial bee colony, and fitness corresponding to the current proximal strategy optimization. When the random number is less than the acceptance probability, the parameters of the best individual in the artificial bee colony are transferred to the proximal strategy optimization network parameters according to the soft update coefficient, and the current temperature is updated according to the temperature decay rule.
[0042] During the construction of the action set, when there is a ready order process but no feasible configuration is available due to conflicts in auxiliary modules, a virtual release mechanism is triggered. The auxiliary modules currently occupied by the idle reconfigurable processing equipment are temporarily regarded as available, and the configuration feasibility is reassessed, thereby reducing the risk of deadlock caused by the lag in resource occupancy status.
[0043] In relation-aware heterogeneous graph fusion networks, COC The subgraph introduces process-level competition features based on a shared order process set, for... CAC The subgraph introduces module-level competition features based on a shared set of auxiliary modules, making the graph attention weights depend not only on node features but also on the actual competition intensity between configurations.
[0044] This invention also provides a relation-aware evolutionary reinforcement learning scheduling system for multi-module configurable reconfigurable production lines, including a problem modeling module, a graph construction module, a Markov decision module, a relation-aware representation learning module, a policy decision module, an evolutionary reinforcement learning training module, and a scheduling output module. The problem modeling module performs scheduling problem modeling; the graph construction module performs 3D configuration-coupled heterogeneous disjunctive graph construction; the Markov decision module performs state, action, state transition, and reward calculations; the relation-aware representation learning module performs graph representation learning; the policy decision module performs action probability generation and state value estimation; the evolutionary reinforcement learning training module performs parameter training and bidirectional collaborative optimization; and the scheduling output module outputs a scheduling scheme composed of order processes, reconfigurable processing equipment, configuration, and auxiliary module occupancy relationships.
[0045] The present invention will be further described in detail below with reference to specific embodiments.
[0046] This invention provides a relation-aware evolutionary reinforcement learning scheduling method for multi-module configurable reconfigurable production lines. Please refer to [link to relevant documentation]. Figures 2 to 7This method first establishes a multi-module configuration reconfigurable production line scheduling problem model. Then, it constructs a three-dimensional configuration coupled heterogeneous disjunction graph based on order processes, equipment configurations, and auxiliary modules. The three-dimensional configuration coupled heterogeneous disjunction graph is then input into a relationship-aware heterogeneous graph fusion network to obtain a state representation. Subsequently, a policy network selects feasible order process-configuration actions. Finally, a scheduling policy is obtained through collaborative training of a near-end policy optimization algorithm and an artificial bee colony algorithm.
[0047] The method mainly includes the following steps: Step 1: Construct a multi-module configurable reconfigurable production line scheduling problem model.
[0048] In this embodiment, the reconfigurable production line includes m reconfigurable processing equipment, and the equipment set is represented as follows: Each reconfigurable processing device Supports a set of interchangeable device configurations Each configuration This represents a processing capability formed by a combination of auxiliary modules. This configuration can only be used for order processing when copies of all the required auxiliary modules are available. Switching between different configurations involves reconfiguration time and equipment... By configuration c 1. Switch to configuration c 2. The required reconstruction time is denoted as .
[0049] The set of auxiliary modules is represented as Each type of auxiliary module A a There are ba identical copies. Since the auxiliary modules come from a shared module pool, a single auxiliary module copy can only be occupied by one device configuration at a time. When a device releases its current configuration or switches to another configuration, the auxiliary modules occupied by the original configuration are released, and the auxiliary modules required by the new configuration are reacquired.
[0050] The order set is represented as Each order J j Includes several order processes arranged in the order of technology. Each order process At least one device configuration must be compatible, in the device Configuration c The processing time is recorded as p j,i,k,c If the order process With configuration If incompatible, the order process cannot be processed by that configuration. The scheduling objective is to determine the processing equipment, equipment configuration, start time, and completion time for each order process to maximize the completion time. Cmax Minimum.
[0051] Maximum completion time C max It can be represented as: , in , Indicates order The Middle i Order process The completion time, This represents the set of all order processes.
[0052] Step 2: Construct a 3D configuration coupling heterogeneous disjunction graph.
[0053] like Figure 4 As shown, the three-dimensional configuration coupled heterogeneous disjunction graph is defined as follows: .in, O Represents the set of order process nodes. C Represents the set of configuration nodes. A Represents the set of auxiliary module nodes. U An arc representing the process sequence between adjacent operations within the same order. V This represents the allocation edge between order processes and processable configurations. W This represents the requirement edge between the configuration and its necessary auxiliary modules. (Starting virtual node) S Used to indicate the start and end of scheduling for virtual nodes. E Used to indicate that all order processes have been completed.
[0054] In the 3D configuration-coupled heterogeneous disjunctive graph, the order process node records the order number, process number, processing status, completion status of the preceding process, scheduled flag, and estimated completion time; the configuration node records the associated reconfigurable processing equipment, current configuration status, reconfigurable status, set of processable order processes, and configuration switching information; the auxiliary module node records the auxiliary module type, number of copies, current occupied quantity, available quantity, and release time. The process-configuration edge records the processing time of the order process on the corresponding configuration, and the configuration-auxiliary module edge records the combination of auxiliary modules necessary for configuration formation.
[0055] Through a 3D configuration-coupled heterogeneous disjunctive graph, the process sequence of order operations, the processing compatibility of configurations, and the sharing and competition relationships of auxiliary modules are all incorporated into the same state representation space. Compared with representing system states only as vectors, the 3D configuration-coupled heterogeneous disjunctive graph can more accurately reflect the feasibility of scheduling actions in a multi-module configuration reconfigurable production line.
[0056] Step 3: Establish the Markov decision process.
[0057] like Figure 5 As shown, in system time T ( t Arrival at the decision point t If at least one ready order process and at least one feasible configuration exist, the policy network performs an action selection. A ready order process is one whose preceding processes have all been completed and have not yet been scheduled. A feasible configuration is one that can process the selected order process, and whose corresponding reconfigurable processing equipment and auxiliary modules meet the availability conditions or can be obtained through reconfiguration.
[0058] state It is obtained by encoding the coupled heterogeneous disjunctive graph of the current 3D configuration. To reduce the state size, only entities related to subsequent decisions, i.e., the set of incomplete order processes, are retained at decision point t. A set of configurations capable of processing remaining order steps. Auxiliary module collection A And related edge characteristics. As the scheduling process progresses, the set of unfinished order procedures and the set of available configurations gradually shrink, thereby reducing the complexity of action evaluation.
[0059] The action space A(t) consists of all feasible order operation-configuration pairs. , A ( t () indicates selecting an order process And by reconfigurable processing equipment M k The configuration c is used for processing. For an action to be deemed feasible, it must simultaneously satisfy: the order process... In ready state; configured The equipment is capable of processing the order's steps. M k Available at the appropriate time; configuration c All required auxiliary modules are available, or can be obtained by releasing the auxiliary modules currently occupied by idle device configurations.
[0060] When a ready order process exists but no feasible configuration is possible due to conflicts in auxiliary modules, a virtual release mechanism is triggered. This mechanism temporarily treats the auxiliary modules currently occupied by idle reconfigurable processing equipment as available and reassesses the feasibility of each configuration based on the set of temporarily available modules. If a feasible configuration is found after the reassessment, the corresponding module release and reallocation are allowed during state transition, thereby reducing resource deadlock caused by auxiliary modules being occupied by idle equipment for a long time.
[0061] The state transition process is as follows: during the action Once selected, determine the corresponding reconfigurable processing equipment. M kDo we need to switch from the current configuration to the target configuration? c If a switch is required, first release the auxiliary modules currently occupied by the configuration, then obtain the auxiliary modules required by the target configuration, and add the corresponding configuration switch time; then execute the order process. The system updates the start time, completion time, and order progress of each process, and also updates the equipment status, auxiliary module occupancy status, and diagram status. If no action is available at the current system time, the system time advances to the moment when the next process is completed, the equipment is released, or the module is released.
[0062] A reward function is used to incentivize actions that reduce the estimated maximum completion time. r t Defined as the difference between the estimated maximum completion time of two adjacent decision points: .
[0063] For scheduled order processes, their completion time is calculated based on the actual environment; for unscheduled order processes, their completion time is estimated based on the completion time of the preceding process and the minimum processing time in the compatible configuration, i.e.: ,in , .
[0064] Based on the above estimated completion time When an action reduces the estimated maximum completion time, the reward is positive; otherwise, the reward is negative or smaller, thus guiding the policy network to learn better scheduling actions.
[0065] Step 4: Construct a relation-aware heterogeneous graph fusion network.
[0066] While the 3D configuration-coupled heterogeneous disjunctive graph comprehensively describes the scheduling state, it simultaneously contains process sequence relationships, configuration competition relationships, module competition relationships, and module collaboration relationships. Directly aggregating these relationships on the complete heterogeneous graph can easily lead to semantic interference between different relationships. Therefore, this invention decomposes the 3D configuration-coupled heterogeneous disjunctive graph into multiple meta-path subgraphs based on meta-paths, and learns each subgraph separately.
[0067] Specifically, OO Subgraphs are used to represent the sequential constraints between order processes; COC The subgraph is used to represent the competitive relationship between multiple configurations for the same unfinished order process; that is, if two configuration nodes can both process at least one of the same unfinished order processes, then in... COC Connect in the subgraph; CAC The subgraph is used to represent the competition relationship between multiple configuration pairs sharing auxiliary modules; that is, if two configuration nodes need at least one of the same auxiliary modules, then in... CAC Connect in the subgraph; ACASubgraphs are used to represent the collaborative relationships between multiple auxiliary modules within the same configuration.
[0068] for COC Subgraph, Configuration C k,c and C m,n The set of shared order processes can be represented as:
[0069] in, express Capable of processing ; express Capable of processing Based on this set of shared order processes, the competitive characteristics at the process level can be composed of the number of shared order processes, the difference in processing time of shared processes, or the statistical measures of processing time of shared processes.
[0070] In one implementation, the following approach may be adopted:
[0071] This indicates the intensity of process-level competition between the two configurations.
[0072] For CAC subgraphs, configure and The set of shared auxiliary modules can be represented as:
[0073] Based on this set, module-level competition characteristics can be composed of the number of shared auxiliary modules, the number of remaining copies of shared auxiliary modules, or the occupancy status of shared modules.
[0074] In one implementation, the following approach may be adopted:
[0075] This indicates the intensity of module-level competition between the two configurations.
[0076] Each metapath subgraph is encoded using an edge-enhanced graph attention mechanism. For the central node in metapath subgraph p... u and its neighboring nodes , No. r The unnormalized attention score under each attention head can be expressed as:
[0077] in, h ( u )and h ( v ) represent nodes respectively u and nodesv The original characteristics, Represents the node feature projection matrix. Represents the edge feature projection matrix. Let p represent the edge features in the metapath subgraph. Let || denote the learnable attention vector, and || denote vector concatenation. For OO Subgraph and ACA If a subgraph does not have explicit competing edge features, then Set as the zero vector; for COC Subgraph and CAC Subgraph, It consists of process-level competitive features and module-level competitive features, respectively.
[0078] The normalized attention coefficient can be expressed as:
[0079] node u In the r The embedding under each attention head is represented as follows:
[0080] Here, σ represents a non-linear activation function. The multi-head attention outputs are concatenated to form the embedding representation of the node under the corresponding metapath subgraph.
[0081] Because the configuration node participates simultaneously COC Subgraph and CAC The subgraph is affected by both order process competition and auxiliary module competition. To integrate these two types of competitive information, this invention assigns learnable scalar weights to different metapath subgraphs. and for learnable scalar weights Normalization yields And then according to The configuration node embeddings under the COC subgraph and CAC subgraph are weighted and summed to obtain the configuration node fusion embedding.
[0082] After L After fusing the layer-relationship-aware heterogeneous graph, embeddings for order process nodes, configuration nodes, and auxiliary module nodes are obtained. Average pooling is performed on each type of node to obtain order process context vectors, configuration context vectors, and auxiliary module context vectors. These three context vectors are then embedded and concatenated with their respective node counterparts to obtain context-enhanced node representations. Finally, the three pooled vectors are concatenated to form the embedded representation, which is used for subsequent policy decisions and value estimation.
[0083] Step 5: Construct the policy network and value network.
[0084] The policy network employs an actor network within an actor-critic structure. For each candidate action... Order process nodes Embedded and configuration nodes After concatenating the embedded and pooled context vectors of the auxiliary modules, the results are input into the decision network to obtain candidate action priority scores. a t | s t ).
[0085] Normalize the priority scores of all candidate actions in the current set of possible actions to obtain the policy probabilities:
[0086] The final action can be sampled based on policy probabilities to improve exploration capabilities during the training phase; alternatively, the action with the highest probability can be selected during the inference phase to obtain a deterministic scheduling scheme. The value network employs a multilayer perceptron, using embedded representations as input and outputting state values. It is used for advantage estimation and policy update in near-end policy optimization algorithms.
[0087] Step six: Perform evolutionary reinforcement learning training.
[0088] like Figure 7 As shown, this invention employs a collaborative training mechanism combining the proximal policy optimization algorithm and the artificial bee colony algorithm. In the initialization phase, a proximal policy optimization agent is first established. The parameters of the relation-aware heterogeneous graph fusion network are initialized using the Xavier initialization method, and the parameters of the multilayer perceptron decision network and value network are initialized using the Kaiming initialization method, resulting in the proximal policy optimization agent parameter vector. w PPO .
[0089] Subsequently w PPO As the first individual in an artificial bee colony, The remaining individuals, through their understanding of... w PPO Perturbation generation:
[0090] in, Indicates the first i The first individual k One parameter, This is the perturbation term sampled from a zero-mean Gaussian distribution. This initialization method ensures the population is distributed around the current policy while preserving sufficient search diversity.
[0091] During the near-end policy optimization phase, the system interacts with the scheduling environment and collects samples of state, action, reward, and next state. The parameters of the policy network are updated based on the policy probability ratio, advantage estimation, and pruning objective function. The parameters of the value network are also updated based on the value loss. By pruning the policy update magnitude, excessive policy shifts caused by single updates are avoided.
[0092] During the artificial bee colony renewal phase, the population search process includes a hired bee phase, an observation bee phase, and a scout bee phase. In the hired bee phase, each individual randomly selects another individual, and candidate individuals are generated according to the following formula:
[0093] in i,k ∈[-1, 1]. This stage is used to explore the policy parameter space within a larger neighborhood. If the maximum completion time of the scheduling scheme corresponding to the candidate individual is better than that of the original individual, then the candidate individual is used to replace the original individual; otherwise, the unimproved count of the individual is increased.
[0094] During the observation phase, a roulette wheel method was used to select a guiding individual based on individual fitness, and candidate individuals were generated with a small perturbation range. i,k ∈[-0.5, 0.5]. This phase is used to perform a fine-grained search around the better individuals. In the scout bee phase, for several consecutive individuals that have not been improved, perturbation individuals are regenerated around the current best individual, thereby introducing new diversity and reducing the risk of premature convergence.
[0095] This invention uses the maximum completion time as an indicator of the fitness of individual bee colonies. The smaller the maximum completion time, the better the strategy parameters of the individual.
[0096] Step 7: Implement the two-way collaboration mechanism.
[0097] The bidirectional collaborative mechanism includes an individual injection path from the proximal policy optimization algorithm to the artificial bee colony algorithm, and a soft update path from the artificial bee colony algorithm to the proximal policy optimization algorithm. In the individual injection path, the current proximal policy optimization agent parameters are... w PPO Introduce artificial bee colonies and replace the least fit individuals. This allows population search to be continuously guided by gradient optimization strategies.
[0098] In the soft update path, the optimal individual obtained from the artificial bee colony search is... An adaptive probability is fed back to the proximal policy optimization agent. This adaptive probability is determined by the initial temperature. T init Minimum temperature T minCurrent temperature T cur Temperature decay rate decay Basic acceptance probability P basic Minimum acceptance probability P min Maximum acceptance probability P max Optimal individual fitness in artificial bee colonies f ABC And current near-field strategy optimization agent fitness f PPO The parameters are determined jointly. Higher temperatures in the early stages of training make it easier for the system to accept new parameters obtained from artificial colony searches, thus enhancing exploration. As the temperature gradually decreases in the later stages of training, parameter acceptance becomes more conservative to maintain policy stability. The adaptive probability is calculated as follows:
[0099] When random number rand (0,1) is less than the acceptance probability. P At that time, according to the soft update coefficient Execution parameter update:
[0100] Then update the current temperature. In this way, the high-quality policy knowledge discovered by the artificial bee colony algorithm in the global search can be smoothly injected into the near-end policy optimization agent, avoiding training oscillations caused by directly replacing parameters.
[0101] Step 8: Output the scheduling scheme.
[0102] After training, the obtained policy network is used to make sequential decisions for the instances to be scheduled. Each decision generates action probabilities based on the current state of the 3D configuration coupled heterogeneous disjunctive graph, selects the order process-configuration action, executes equipment reconfiguration, auxiliary module acquisition, and order process processing, and continuously updates the graph state until all order processes are completed. The final output scheduling scheme includes the processing equipment, equipment configuration, required auxiliary modules, start time, completion time, and configuration switching relationships for each order process.
[0103] Specific application examples Please see Figures 2 to 7 In a reconfigurable production line scheduling scenario, the production line includes 3 reconfigurable processing machines, 3 orders to be processed, and 3 types of auxiliary modules, each type of auxiliary module having 3 identical copies. Each reconfigurable processing machine supports several configurations, each configuration is formed by one or more auxiliary modules, and each order includes several order operations arranged in process sequence.
[0104] This embodiment includes the following steps: Step 1: Construct a multi-module reconfigurable production line scheduling problem model. Based on the available configurations of the three reconfigurable processing machines, the required auxiliary module combinations for each configuration, the order process processing time, and the configuration switchover time, establish a scheduling problem model with the objective of minimizing the maximum completion time.
[0105] Step two, construct a three-dimensional configuration-coupled heterogeneous disjunction graph. For example... Figure 4 As shown, order processes are mapped to process nodes, the configurations of each reconfigurable processing equipment are mapped to configuration nodes, and auxiliary module types are mapped to auxiliary module nodes. Order process sequence arcs, process-configuration allocation edges, and configuration-auxiliary module requirement edges are constructed respectively.
[0106] Step 3: Establish decision points and generate a set of actionable actions. If the current process of an order already satisfies the preceding constraints, the process is added to the set of ready order processes. For each ready order process, check whether the configuration of each reconfigurable processing equipment is compatible with the process, and further check whether the corresponding auxiliary module is available, thereby obtaining the set of actionable actions.
[0107] Step four involves state embedding using a relation-aware heterogeneous graph fusion network. For example... Figure 6 As shown, the current graph state is decomposed into OO , COC , CAC and ACA The four types of meta-path subgraphs are used to calculate the embedded representations of the sequence of processes, the competitive relationship of configurations for processes in the same order, the competitive relationship of configurations for shared auxiliary modules, and the collaborative relationship of auxiliary modules. The comprehensive representation of configuration nodes is obtained through multi-path fusion.
[0108] Step 5: The policy network selects scheduling actions. For each candidate order process-configuration action, the corresponding order process embedding, configuration embedding, and auxiliary module context vector are input into the decision network to obtain action priority scores and form a probability distribution. After selecting the action with higher probability, the environment executes the corresponding order process and performs equipment configuration switching and auxiliary module reallocation as needed.
[0109] Step Six: Repeat Status Updates and Action Selection. After each order process is completed, update the order progress, current equipment configuration, auxiliary module occupancy status, and 3D configuration coupling heterogeneous analysis diagram. If there are no actionable actions at the current system time, advance the system time to the next equipment release or process completion event. Repeat the above process until all order processes are completed, resulting in the following: Figure 3 The feasible scheduling schemes are shown.
[0110] Step 7, Training Phase Figure 7The evolutionary reinforcement learning process shown optimizes policy parameters. The proximal policy optimization algorithm updates the policy network and value network based on interactive samples from the scheduling environment, while the artificial bee colony algorithm periodically searches the policy parameter space. The two optimization processes are connected through individual injection and soft updates.
[0111] By executing the above steps in sequence, the scheduling strategy can simultaneously consider the order process progress, configuration processing capacity, auxiliary module availability, and configuration switching cost at each decision point, thereby generating a reconfigurable production line scheduling scheme that meets the constraints.
[0112] The present invention also provides a scheduling system for a multi-module configurable reconfigurable production line. The system includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the scheduling method for a multi-module configurable reconfigurable production line as described above.
[0113] The present invention also provides a computer-readable storage medium storing machine-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the scheduling method for a multi-module configurable reconfigurable production line as described above.
[0114] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A scheduling method for a multi-module configurable reconfigurable production line, characterized in that, Includes the following steps: (1) The scheduling problem model of a multi-module configurable reconfigurable production line is mapped to a three-dimensional configuration coupling heterogeneous disjunction graph. The three-dimensional configuration coupling heterogeneous disjunction graph includes process nodes, configuration nodes, auxiliary module nodes, order process sequence arcs, allocation edges between processes and configurations, demand edges between configurations and auxiliary modules, and start and end boundaries of the scheduling process represented by start virtual nodes and end virtual nodes. (2) The multi-module configuration reconfigurable production line scheduling problem is characterized as a Markov decision process: When the system time reaches a decision point where there is a ready process and a feasible configuration, the current state of the decision point is obtained through the three-dimensional configuration coupling heterogeneous disjunction graph encoding. Based on the current state, an action consisting of order process and equipment configuration is selected from the set of actionable actions. After executing the action, the system time, configuration state, auxiliary module occupancy state, order process state and graph state are updated to obtain the state of the next decision point. (3) Based on the state of all decision points, a relation-aware heterogeneous graph fusion network is used to perform representation learning on the three-dimensional configuration coupled heterogeneous disjunctive graph: the three-dimensional configuration coupled heterogeneous disjunctive graph is decomposed into multiple meta-path subgraphs representing the sequential relationship of the process, the competitive relationship of the configuration to the same process, the competitive relationship of the configuration to the shared auxiliary module, and the collaborative relationship of the auxiliary module within the same configuration. The multiple meta-path subgraphs are processed by the edge-enhanced graph attention mechanism and the multi-path fusion mechanism to obtain the embedded representation of the process node, configuration node, auxiliary module node and global system state. (4) Construct a reinforcement learning policy network and a value network based on the embedding representation. After concatenating the process embedding, configuration embedding and auxiliary module context vector corresponding to the candidate action, input the network into the reinforcement learning decision network to obtain the priority score of the candidate action. Then, process the priority score of the candidate action through a normalization function to obtain the probability distribution of the actionable action. Based on the probability distribution of the actionable action, determine the scheduling action by greedy selection or sampling. At the same time, use the value network to output the state value of the scheduling action based on the embedding representation. (5) Based on the obtained scheduling actions and state values, the parameters of the relation-aware heterogeneous graph fusion network, the reinforcement learning policy network, and the value network are optimized by combining the near-end policy optimization algorithm and the artificial bee colony algorithm: After performing policy gradient updates based on environment interaction on the relation-aware heterogeneous graph fusion network, the policy network, and the value network by the near-end policy optimization algorithm, the population-based parameter search is performed on the relation-aware heterogeneous graph fusion network, the reinforcement learning policy network, and the value network by the artificial bee colony algorithm to obtain a policy network that satisfies the configuration, auxiliary module, and process constraints; and then the optimal scheduling scheme is obtained based on the policy network.
2. The scheduling method for a multi-module configurable reconfigurable production line as described in claim 1, characterized in that: Multiple metapath subgraphs include OO , COC , CAC and ACA Four types of meta-path subgraphs, among which OO Subgraphs represent the sequential constraints between order processes. COC The subgraph represents the competitive relationship between multiple configurations for the same incomplete order process. CAC The subgraph represents the competitive relationship between multiple configurations for a shared auxiliary module. ACA Subgraphs represent the collaborative relationships between multiple auxiliary modules within the same configuration.
3. The scheduling method for a multi-module configurable reconfigurable production line as described in claim 2, characterized in that: right COC The subgraph introduces process-level competition features based on a shared order process set, for... CAC The subgraph introduces module-level competition features based on a set of shared auxiliary modules.
4. The scheduling method for a multi-module configurable reconfigurable production line as described in claim 1, characterized in that: During the construction of the action set, when there is a ready order process but no feasible configuration is available due to conflicts in auxiliary modules, a virtual release mechanism is triggered. The auxiliary modules currently occupied by the idle reconfigurable processing equipment are temporarily regarded as available, and then the configuration feasibility is reassessed.
5. The scheduling method for a multi-module configurable reconfigurable production line as described in any one of claims 1-4, characterized in that: Bidirectional collaboration between the near-end policy optimization algorithm and the artificial bee colony algorithm is achieved through individual injection from the near-end policy optimization algorithm to the artificial bee colony algorithm and adaptive probabilistic soft update from the artificial bee colony algorithm to the near-end policy optimization algorithm.
6. The scheduling method for a multi-module configurable reconfigurable production line as described in any one of claims 1-4, characterized in that: During parameter optimization, the parameters of the relation-aware heterogeneous graph fusion network are initialized using the Xavier initialization method, and the parameters of the decision network and value network based on the multilayer perceptron are initialized using the Kaiming initialization method, resulting in a near-end policy optimization network parameter vector. This network parameter vector is then used as the first initialized individual in the artificial bee colony population, and the remaining individuals are generated by superimposing a perturbation that follows a zero-mean Gaussian distribution onto this network parameter vector.
7. The scheduling method for a multi-module configurable reconfigurable production line as described in claim 6, characterized in that: The artificial bee colony algorithm searches the strategy parameter space according to the hired bee phase, the observation bee phase, and the scout bee phase, and uses the maximum completion time of the scheduling scheme as the fitness index to evaluate the quality of individuals.
8. The scheduling method for a multi-module configurable reconfigurable production line as described in any one of claims 1-4, characterized in that: The reinforcement learning strategy network employs an actor network in an actor-critic structure; the value network employs a multilayer perceptron.
9. A scheduling system for a multi-module configurable reconfigurable production line, characterized in that: The system includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it performs the scheduling method for a multi-module configurable reconfigurable production line as described in any one of claims 1-8.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores machine-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the scheduling method for a multi-module configurable reconfigurable production line as described in any one of claims 1-8.
