Production scheduling method and device, equipment, storage medium and program product

Abstract

The invention provides a production scheduling method, device and equipment, a storage medium and a program product, and relates to the technical field of intelligent manufacturing and workshop scheduling, and the method comprises the steps: obtaining the resource configuration data of a plurality of RMCs (Reconfigurable Manufacturing Units) and the operation process structure data of a to-be-scheduled operation; according to the resource configuration data and the operation process structure data, determining a state characteristic value input into the production scheduling agent, and obtaining a scheduling scheme of the to-be-scheduled operation output by the production scheduling agent according to the state characteristic value; the scheduling scheme comprises at least one of the following items: information of a selected process path, an execution sequence of process characteristics, an execution sequence of process operation and information of a selected RMC, the scheme of the invention can effectively cope with complex production and manufacturing scenes such as multi-operation, multi-process path, multi-RMC resource and processing capability switching, a global scheduling scheme is obtained, and the scheduling efficiency is improved. And the production scheduling efficiency and the system resource utilization rate are obviously improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent manufacturing and workshop scheduling technology, specifically to a production scheduling method, device, equipment, storage medium, and program product. Background Technology

[0002] In modern manufacturing systems, process planning and production scheduling are two core components of the manufacturing execution process. Traditional methods typically handle process planning and production scheduling separately, that is, determining the process path first and then scheduling the tasks. However, this separate approach cannot simultaneously consider equipment resource constraints and processing path flexibility, easily leading to problems such as uneven equipment load, task delays, or path conflicts, thereby reducing production efficiency and system responsiveness. Therefore, Integrated Process Planning and Scheduling (IPPS) has become a key research direction for improving manufacturing flexibility and system performance.

[0003] Meanwhile, as manufacturing models shift from mass production to small-batch, multi-variety customization, traditional manufacturing systems struggle to adapt to market changes and personalized demands. To enhance the flexibility and scalability of manufacturing systems, Reconfigurable Manufacturing Systems (RMS) have emerged. The Reconfigurable Manufacturing Cell (RMC), as the core component of an RMS, can flexibly switch between turning, milling, drilling, and inspection capabilities by changing tools, fixtures, or program modules, significantly improving the adaptability of the manufacturing system.

[0004] Introducing RMC into IPPS to form the IPPS-RMCs problem, while enhancing scheduling flexibility, also presents challenges such as the time constraint of RMC processing capacity reconstruction, and the technical issues of path selection, resource allocation, and processing capacity switching. However, existing production scheduling methods, such as exact algorithms, metaheuristic methods, or rule-based scheduling methods, fail to achieve a globally optimal scheduling solution when addressing these technical challenges. Summary of the Invention

[0005] At least one embodiment of the present invention provides a production scheduling method, apparatus, equipment, storage medium, and program product to solve the problem that a global scheduling scheme cannot be obtained in production scheduling tasks in the prior art.

[0006] To solve the above-mentioned technical problems, the present invention is implemented as follows:

[0007] Firstly, a production scheduling method is provided, including:

[0008] Acquire resource configuration data and job process structure data of multiple reconfigurable manufacturing cells (RMCs) and jobs to be scheduled;

[0009] Based on the resource configuration data and the work process structure data, determine the state characteristic value of the input production scheduling agent, and obtain the scheduling scheme of the work to be scheduled output by the production scheduling agent based on the state characteristic value;

[0010] The scheduling scheme includes at least one of the following: information on the selected process path, the execution order of process features, the execution order of process operations, and information on the selected RMC.

[0011] Optionally, in the production scheduling method, before acquiring resource configuration data of multiple RMCs and job process structure data of the jobs to be scheduled, the method further includes:

[0012] Obtain RMC resource configuration training data and operational process structure training data;

[0013] Based on the resource configuration training data and the work process structure training data, the first state feature value is obtained;

[0014] The initial reinforcement learning network is trained based on the first state feature value to obtain the second state feature value and the first reinforcement learning network.

[0015] The first reinforcement learning network is trained based on the second state feature value until a preset number of training iterations are reached, thereby obtaining the production scheduling agent.

[0016] Optionally, the production scheduling method, wherein training an initial reinforcement learning network based on first state feature values ​​to obtain second state feature values ​​and the first reinforcement learning network includes:

[0017] The initial reinforcement learning network is trained based on the first state feature value and the preset composite scheduling rule to obtain the first composite scheduling action selected by the initial reinforcement learning network.

[0018] The initial reinforcement learning network executes the first composite scheduling action to obtain the second state feature value;

[0019] The first reward value is obtained based on the first state feature value, the second state feature value, and the reward function;

[0020] The network parameters of the initial reinforcement learning network are updated based on the first state feature value, the first composite scheduling action, the first reward value, and the second state feature value to obtain the first reinforcement learning network.

[0021] Optionally, in the production scheduling method, the state feature values ​​include at least one of the following: scheduling progress related feature values, RMC related feature values, completion time related feature values, and delay rate related feature values;

[0022] The scheduling progress-related characteristic values ​​include at least one of the following: total number of jobs, total number of process features, average job completion rate, standard deviation of job completion rate, and average completion rate of process features;

[0023] The RMC-related characteristic values ​​include at least one of the following: total number of RMCs, average RMC utilization rate, standard deviation of average RMC utilization rate, average configuration time percentage, and standard deviation of configuration time percentage.

[0024] The completion time-related characteristic values ​​include: the standard deviation of the current completion time of RMC and / or the current average completion time of RMC;

[0025] The procrastination rate-related characteristic values ​​include: the actual average procrastination rate.

[0026] Optionally, in the production scheduling method, the preset composite scheduling rule includes at least one of the following: job selection rule, node sorting rule, or node selection rule and RMC allocation rule;

[0027] The job selection rule is used to indicate the jobs that are currently scheduled with priority.

[0028] The AND node sorting rule and the OR node selection rule are used to indicate at least one of the following: the information of the selected process path, the execution order of the process features, and the execution order of the process operations;

[0029] The RMC allocation rule is used to indicate the information of the selected RMC.

[0030] Optionally, in the production scheduling method, the reward function is used to indicate minimizing the difference between the average RMC utilization rate in the first state feature value and the average RMC utilization rate in the second state feature value.

[0031] Secondly, a production scheduling device is also provided, comprising:

[0032] The first acquisition module is used to acquire resource configuration data of multiple RMCs and job process structure data of jobs to be scheduled.

[0033] The determination module is used to determine the state feature value of the input production scheduling agent based on the resource configuration data and the operation process structure data, and to obtain the scheduling scheme of the operation to be scheduled output by the production scheduling agent based on the state feature value.

[0034] The scheduling scheme includes at least one of the following: information on the selected process path, the execution order of process features, the execution order of process operations, and information on the selected RMC.

[0035] Thirdly, a production scheduling device is also provided, comprising: a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the processor executes the program or instructions to implement the production scheduling method as described in the first aspect.

[0036] Fourthly, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the production scheduling method as described in the first aspect.

[0037] Fifthly, a computer program product is also provided, including computer instructions that, when executed by a processor, implement the production scheduling method as described in the first aspect.

[0038] Compared with existing technologies, embodiments of the present invention provide a production scheduling method, apparatus, equipment, storage medium, and program product, which acquires resource configuration data of multiple RMCs and job process structure data of jobs to be scheduled; determines the state feature value of the input production scheduling agent based on the resource configuration data and the job process structure data, and obtains the scheduling scheme of the job to be scheduled output by the production scheduling agent based on the state feature value; wherein, the scheduling scheme includes at least one of the following: information on the selected process path, the execution order of process features, the execution order of process operations, and information on the selected RMC, which can effectively cope with complex production and manufacturing scenarios such as multiple jobs, multiple process paths, multiple RMC resources, and switching of processing capabilities, and obtain a global scheduling scheme, thereby significantly improving production scheduling efficiency and system resource utilization. Attached Figure Description

[0039] Figure 1 This is a network example diagram of the IPPS-RMCs problem in an embodiment of the present invention;

[0040] Figure 2 This is an example diagram of the process feature structure data in an embodiment of the present invention;

[0041] Figure 3 This is a flowchart illustrating the production scheduling method described in an embodiment of the present invention;

[0042] Figure 4 This is a flowchart illustrating the training process of the production scheduling intelligent agent in an embodiment of the present invention;

[0043] Figure 5 This is a block diagram of the production scheduling device described in an embodiment of the present invention;

[0044] Figure 6 This is a hardware block diagram of the production scheduling method described in an embodiment of the present invention. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] The terms "first," "second," etc., used in the specification and claims of this invention are used to distinguish similar objects and are not used to describe a specified order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, not limited in number; for example, the first object can be one or more. Furthermore, in the specification and claims, "and" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0047] Before describing at least one embodiment of the present invention, the following will be explained first:

[0048] IPPS-RMCs Problem Description:

[0049] The IPPS-RMCs problem solved by this invention is based on a matrix-structured manufacturing system (MMS), which includes the following core features:

[0050] Reconfigurability feature: Multiple RMCs in the system can switch capabilities according to the requirements of the job to be scheduled;

[0051] Matrix layout features: workstations are distributed in a grid structure, allowing for spatial flexibility in resource utilization.

[0052] Understandably, under this system, the production scheduling process must simultaneously complete at least one of the following tasks:

[0053] Select the process path for each job, arrange the execution order of process features, arrange the execution order of process operations, and select the appropriate RMC and assign processing tasks.

[0054] While completing the above tasks, it is necessary to consider the time cost of RMC capability configuration and switching, with the ultimate goal of minimizing the maximum completion time of all jobs.

[0055] Task description:

[0056] Figure 1 This is a network example diagram illustrating the IPPS-RMCs problem in an embodiment of the present invention. For example... Figure 1 As shown, this embodiment of the invention constructs a production scheduling agent for MMS, using an And-Or graph to describe the process structure of a job. In this And-Or graph, OR nodes are OR nodes, representing optional process paths; JOIN nodes are join nodes, representing points where process operations are merged; and arrows between nodes represent the sequential constraints of process operations. Each job consists of multiple process features, and each process feature is composed of several process operations. There are sequential or parallel relationships between different process features.

[0057] As shown in Figure 1, S represents the start node, E represents the end node, the numbers in {} represent the processing capacity codes required for the process operation, and the numbers in [] represent the processing time. Some process features contain multiple optional process paths (existing or nodes), each process path consisting of a different number of process operations. Process features without or nodes default to a single process path.

[0058] Figure 2 This is an example diagram of the process feature structure data in an embodiment of the present invention. For example... Figure 2 As shown, process feature 2 includes two optional process paths, consisting of one and two process operations respectively, reflecting the flexibility of the process paths. An example of the job process structure information is shown in Table 1 below, including the available RMC processing capacity code, processing time, processing sequence constraints, and whether it is an optional process path for each process operation. Different jobs may correspond to different deadlines, and production scheduling needs to coordinate all jobs to be scheduled to optimize overall performance.

[0059] Table 1: Examples of Work Process Structure Information

[0060]

[0061] RMC Explanation:

[0062] The workshop is equipped with multiple Resource Management Controllers (RMCs), each with a set of initial processing capabilities and supporting switching between multiple processing capabilities. As shown in Table 2, the processing capability range and reconfiguration time vary for different RMCs. During scheduling, if an RMC consecutively executes two process operations with the same processing capability requirements, no reconfiguration is needed; however, if the processing capabilities are different, corresponding reconfiguration time is required before subsequent process operations can be executed. Therefore, the scheduling scheme needs to consider not only resource and process path matching but also optimize reconfiguration overhead to avoid time waste caused by frequent switching.

[0063] Table 2: Example Information for RMC

[0064]

[0065] Next, at least one embodiment of the present invention will be provided, including a production scheduling method, apparatus, device, storage medium, and program product, as described below:

[0066] like Figure 3 As shown, an embodiment of the present invention provides a production scheduling method, including:

[0067] Step 301: Obtain the resource configuration data of each of the multiple RMCs and the job process structure data of the job to be scheduled;

[0068] Optionally, the resource configuration data includes: processing capacity indication data and reconstruction time. Specifically, the processing capacity indication data may be the processing capacity code in Table 2 above, and the reconstruction time is the reconstruction time in Table 2 above.

[0069] Optionally, the operation process structure data includes: the total number of operations and multiple process feature structure data, wherein one of the process feature structure data corresponds to at least one process operation, and the process feature structure data includes: indication data of the optional process path of the process operation, indication data of the processing capacity required by the process operation, and processing time required by the process operation.

[0070] It is understood that, prior to step 301 above, the method further includes:

[0071] Collect the resource configuration data and the operational process structure data.

[0072] In one embodiment, optionally, after step 301 above, the method further includes:

[0073] Based on the aforementioned operational process structure data, a process network diagram is constructed to indicate the matching relationship between operations and resources;

[0074] The process network diagram is visualized.

[0075] In this embodiment of the invention, a system based on the operational process structure data can be constructed. Figure 1 or Figure 2 The process network diagram shown is an AND-OR graph structure. The process network diagram is used to indicate the matching relationship between tasks and resources, and to visualize the process network diagram so that relevant personnel can clearly understand the process structure of the tasks to be scheduled.

[0076] Step 302: Based on the resource configuration data and the work process structure data, determine the state characteristic value of the input production scheduling agent, and obtain the scheduling scheme of the work to be scheduled output by the production scheduling agent based on the state characteristic value;

[0077] The scheduling scheme includes at least one of the following: information on the selected process path, the execution order of process features, the execution order of process operations, and information on the selected RMC.

[0078] The production scheduling method described in this invention can effectively cope with complex production and manufacturing scenarios such as multiple jobs, multiple process paths, multiple RMC resources, and switching of processing capabilities, and obtain a global scheduling solution, thereby significantly improving production scheduling efficiency and system resource utilization.

[0079] Optionally, the status feature values ​​include at least one of the following: scheduling progress related feature values, RMC related feature values, completion time related feature values, and delay rate related feature values;

[0080] The scheduling progress-related characteristic values ​​include at least one of the following: total number of jobs, total number of process features, average job completion rate, standard deviation of job completion rate, and average completion rate of process features;

[0081] The RMC-related characteristic values ​​include at least one of the following: total number of RMCs, average RMC utilization rate, standard deviation of average RMC utilization rate, average configuration time percentage, and standard deviation of configuration time percentage.

[0082] The completion time-related characteristic values ​​include: the standard deviation of the current completion time of RMC and / or the current average completion time of RMC;

[0083] The procrastination rate-related characteristic values ​​include: the actual average procrastination rate.

[0084] In one embodiment, optionally, before step 301 above, the method further includes:

[0085] Obtain RMC resource configuration training data and operational process structure training data;

[0086] Based on the training data of the operation process structure and the training data of the resource allocation, the first state feature value is obtained;

[0087] Based on the resource configuration training data and the operation process structure training data, the second state feature value and the first reinforcement learning network are obtained.

[0088] The first reinforcement learning network is trained based on the second state feature value until a preset number of training iterations are reached, thereby obtaining the production scheduling agent.

[0089] It should be noted that the initial reinforcement learning network can be a Dueling Double Deep Q-Network (D3QN), which uses an algorithm with a Dueling DQN structure to decompose the Q-value function into a state value function and an advantage function, thus more accurately evaluating state value and action advantage. The Double DQN mechanism is used to decouple action selection and Q-value evaluation, thereby reducing the overestimation bias of Q-value.

[0090] It is understood that the production scheduling agent in the embodiments of the present invention is trained based on an initial reinforcement learning network, and therefore the production scheduling agent has the advantages of the initial reinforcement learning network mentioned above, which will not be repeated here.

[0091] In this embodiment of the invention, the training process of the production scheduling agent uses job process structure training data and RMC resource configuration training data to obtain a first state feature value, and uses the first state feature value as input to learn and output scheduling actions and reward values.

[0092] In one implementation, optionally, training an initial reinforcement learning network based on first state feature values ​​to obtain second state feature values ​​and the first reinforcement learning network includes:

[0093] The initial reinforcement learning network is trained based on the first state feature value and the preset composite scheduling rule to obtain the first composite scheduling action selected by the initial reinforcement learning network.

[0094] The initial reinforcement learning network executes the first composite scheduling action to obtain the second state feature value;

[0095] The first reward value is obtained based on the first state feature value, the second state feature value, and the reward function;

[0096] The network parameters of the initial reinforcement learning network are updated based on the first state feature value, the first composite scheduling action, the first reward value, and the second state feature value to obtain the first reinforcement learning network.

[0097] It should be noted that, based on training the initial reinforcement learning network, the production scheduling agent also proposes an improved exploration strategy, with an exploration rate... The decay rate decreases with increasing training iterations, and this decay rate can be dynamically adjusted according to the scale of the IPPS-RMCs problem, significantly improving learning efficiency. Furthermore, in the training process of the production scheduling agent, a soft update method is used to synchronize the network parameters of the initial reinforcement learning network, thereby improving the stability of the training process.

[0098] Figure 4 This is a flowchart illustrating the training process of the production scheduling agent in an embodiment of the present invention. Figure 4 As shown, the training process for the production scheduling agent includes the following steps:

[0099] Initialize the initial reinforcement learning network;

[0100] Determine whether the preset number of training iterations has been reached; if the preset number of training iterations has been reached, then end the training.

[0101] If the pre-training count is not reached, then the first state feature value is obtained based on the work process structure data and RMC resource configuration data. It also resets the state space and initializes the state of the scheduled tasks;

[0102] Determine whether the scheduling task has been completed. If the scheduling task has been completed, update the number of training iterations and return to the step of determining whether the preset number of training iterations has been reached.

[0103] If the scheduled task is not completed, then based on the first state feature value in the state space... Preset composite scheduling rules and exploration strategies to select the first composite scheduling action. ;

[0104] Execute the first composite scheduling action Obtain the second state feature value And update the state space;

[0105] Based on the first state feature value Second state eigenvalues And the reward function, to obtain the first reward value;

[0106] The first state feature value First composite scheduling action First reward value and the second state eigenvalue As an empirical tuple ( , , , ), and the empirical tuple ( , , , Stored in the experience replay buffer;

[0107] Learn by sampling from the experience replay buffer and update the network parameters of the initial reinforcement learning network.

[0108] Here, the network parameters of the initial reinforcement learning network can be updated based on the calculated temporal difference (TD) error and through gradient descent.

[0109] Therefore, the production scheduling agent learns from the experience replay buffer by interacting with the environment of IPPS-RMCs problems, including observing states, selecting actions, executing actions, obtaining rewards, and state transitions, continuously optimizing its strategy. After multiple rounds of training, the trained production scheduling agent can select the optimal scheduling action based on resource configuration data from multiple RMCs and the operational process structure data of the jobs to be scheduled, generating a global scheduling scheme for the jobs. This iterative learning process enables the production scheduling agent to continuously improve its decision-making ability, ultimately mastering the ability to select the optimal scheduling action under various shop floor conditions, contributing to more efficient production scheduling and significantly improving operational performance.

[0110] The production scheduling method provided in this invention addresses the IPPS-RMCs problem in MMS (Manufacturing Module Management), taking into account process path selection, resource allocation, and equipment reconfiguration constraints, and generates a globally optimized scheduling scheme in a dynamic environment. This method integrates the Dueling DQ3N structure and adaptive policy optimization mechanism from deep reinforcement learning, possessing efficient learning and generalization capabilities.

[0111] Here, we provide supplementary explanations regarding the scheduling tasks for IPPS-RMCs problems related to MMS:

[0112] This invention modeles the scheduling task of the IPPS-RMCs problem as a Markov Decision Process (MDP), including:

[0113] The workshop status is characterized from multiple perspectives, including scheduling progress, RMC utilization, completion time deviation, and delay rate, with 13 status characteristic values ​​designed.

[0114] The 36 compound scheduling rules (CDRs) are composed of job selection rules, node sorting rules, or node selection rules and RMC allocation rules.

[0115] Reward function.

[0116] The above 13 state feature values ​​constitute the state space, and the 36 composite scheduling rules constitute the action space.

[0117] Next, the state space, action space, and reward function will be explained in detail as follows:

[0118] I. State Space:

[0119] In this embodiment of the invention, the generation of scheduling actions depends on the perception of the scheduling environment. The construction of the state space aims to provide the production scheduling agent with quantifiable and identifiable state features to guide it in selecting the optimal scheduling action at each decision moment. The design of the state feature space needs to accurately reflect the job execution progress, resource utilization efficiency, and system responsiveness, while maintaining a balance between expressive power and dimensional control. To avoid excessively high dimensionality of the state space leading to decreased training efficiency, this embodiment of the invention extracts representative state information from four perspectives, constructing a total of 13 state feature values ​​to achieve a comprehensive characterization of the scheduling environment. Specifically, this includes:

[0120] (1) Relevant characteristics of scheduling progress: The progress of scheduling tasks is evaluated by the overall completion rate and its standard deviation, revealing the differences between the progress of each task;

[0121] (2) RMC related characteristic values: average utilization rate and its standard deviation and other indicators can provide insight into the resource efficiency in the workshop and highlight the differences in utilization rates between different RMCs;

[0122] (3) Completion time related characteristic values: From the perspective of completion time, the gap between the existing scheduling plan and the optimal completion time can be identified. By minimizing this gap, the maximum completion time can be shortened, thereby aligning with the optimization objective.

[0123] (4) Delay rate related characteristics: The delay rate reflects the difference between the actual task completion time and the planned task completion time. Adjusting the scheduling strategy according to the GAI delay rate helps to reduce delays and thus optimize the overall results.

[0124] The 13 state characteristic values ​​mentioned above, as shown in Table 3, collectively constitute the state space of the scheduling environment and serve as the input to the production scheduling agent. By rationally extracting key state information, the adaptability and stability of scheduling actions in complex workshop environments are improved.

[0125] Table 3: State Feature Values

[0126]

[0127] II. Movement Space:

[0128] In this embodiment of the invention, the scheduling action is controlled by the production scheduling agent, and its core lies in selecting a compound dispatching rule (CDR) at each decision moment. Each CDR is selected based on the following four scheduling rules:

[0129] (1) Job selection rules, used to determine the jobs that should be scheduled first;

[0130] (2) Node sorting rules, used to select the next executable process feature in a process path with sequential flexibility;

[0131] (3) Or node selection rules, used to select a specific process path in scenarios where there are multiple parallel optional paths;

[0132] (4) RMC allocation rules, used to select execution devices from among the RMCs that meet the capability requirements.

[0133] By arranging and combining the above four scheduling rules, a total of 36 composite scheduling rules are formed, namely the preset composite scheduling rules. Scheduling actions are selected based on these 36 composite scheduling rules to form the action space. The execution of each scheduling action represents the complete scheduling decision regarding job, path, and resource selection at the current moment. The action space is shown in Table 4 below.

[0134] It should be noted that in each round of scheduling decision, the production scheduling agent applies job selection rules to determine a target job from the currently available job set; if the job has sequential flexibility, it further determines the executable process features through node sorting rules; if the process feature has multiple optional paths, it selects the optimal path using node selection rules; finally, based on RMC status and configuration constraints, it selects a suitable RMC to complete the operation using RMC allocation rules.

[0135] The embodiments of the present invention employ the above-mentioned action space, which, compared with directly searching the scheduling solution space, reduces the action dimension, improves decision-making efficiency, and has good scalability and interpretability.

[0136] Table 4: Preset Composite Scheduling Rules

[0137]

[0138] III. Reward Function:

[0139] In this embodiment of the invention, to achieve reinforcement learning optimization of the scheduling objective, an immediate reward function directly related to system resource utilization is defined, aiming to guide the production scheduling agent to learn a strategy to minimize the maximum completion time. The growth direction of the cumulative reward value must be consistent with the optimization objective of production scheduling. Minimizing the completion time means that each RMC should be fully balanced and efficiently utilized, thereby reducing idle time and minimizing downtime.

[0140] The reward function for completion time is defined as the difference in average RMC utilization between two decision steps, expressed as: ,in, The average utilization rate of RMC is given by the following formula (1):

[0141] (1).

[0142] It should be noted that due to the initial utilization rate The cumulative reward value is ultimately equivalent to the final average utilization rate of the RMC after all jobs have been scheduled. By maximizing this final average utilization rate of the RMC, the production scheduling agent is encouraged to minimize RMC idle and configuration time during the scheduling process, thereby shortening the total completion time. This reward function is concise and efficient, adaptable to large-scale complex scheduling problems, and can significantly improve the learning convergence speed and generalization ability of the scheduling strategy.

[0143] In one embodiment, optionally, the preset composite scheduling rule includes at least one of the following: job selection rule, node sorting rule, or node selection rule and RMC allocation rule;

[0144] The job selection rule is used to indicate the jobs that are currently scheduled with priority.

[0145] The AND node sorting rule and the OR node selection rule are used to indicate at least one of the following: the information of the selected process path, the execution order of the process features, and the execution order of the process operations;

[0146] The RMC allocation rule is used to indicate the information of the selected RMC.

[0147] In one embodiment, optionally, the reward function is used to indicate minimizing the difference between the average RMC utilization rate in the first state feature value and the average RMC utilization rate in the second state feature value.

[0148] In summary, the production scheduling method described in this invention collects resource configuration data and operational process structure data to construct a process network graph based on an AND-OR graph. The IPPS-RMCs problem is modeled as a Markov decision process, defining 13 state feature values ​​and 36 composite scheduling rules. A production scheduling agent based on an initial deep reinforcement learning network is constructed, employing soft updates and adaptive exploration strategies to achieve policy learning. Network parameters are updated through an experience replay mechanism, ultimately outputting a global scheduling scheme. This method effectively addresses complex production scenarios involving multiple jobs, multiple process paths, multiple RMC resources, and switching processing capabilities, significantly improving scheduling efficiency and system resource utilization.

[0149] like Figure 5 As shown, this embodiment of the invention also provides a production scheduling device, comprising:

[0150] The first acquisition module 501 is used to acquire resource configuration data of multiple RMCs and job process structure data of jobs to be scheduled.

[0151] The determining module 502 is used to determine the state characteristic value of the input production scheduling agent based on the resource configuration data and the operation process structure data, and to obtain the scheduling scheme of the operation to be scheduled output by the production scheduling agent based on the state characteristic value.

[0152] The scheduling scheme includes at least one of the following: information on the selected process path, the execution order of process features, the execution order of process operations, and information on the selected RMC.

[0153] Optionally, the production scheduling device further includes:

[0154] The second acquisition module is used to acquire RMC resource configuration training data and operation process structure training data;

[0155] The acquisition module is used to obtain a first state feature value based on the resource configuration training data and the operation process structure training data;

[0156] The first training module is used to train the initial reinforcement learning network based on the first state feature values ​​to obtain the second state feature values ​​and the first reinforcement learning network.

[0157] The second training module is used to train the first reinforcement learning network based on the second state feature value until a preset number of training iterations are reached, thereby obtaining the production scheduling agent.

[0158] Optionally, in the production scheduling device, the first training module is specifically used for:

[0159] The initial reinforcement learning network is trained based on the first state feature value and the preset composite scheduling rule to obtain the first composite scheduling action selected by the initial reinforcement learning network.

[0160] The initial reinforcement learning network executes the first composite scheduling action to obtain the second state feature value;

[0161] The first reward value is obtained based on the first state feature value, the second state feature value, and the reward function;

[0162] The network parameters of the initial reinforcement learning network are updated based on the first state feature value, the first composite scheduling action, the first reward value, and the second state feature value to obtain the first reinforcement learning network.

[0163] Optionally, in the production scheduling device, the status characteristic value includes at least one of the following: scheduling progress related characteristic value, RMC related characteristic value, completion time related characteristic value, and delay rate related characteristic value;

[0164] The scheduling progress-related characteristic values ​​include at least one of the following: total number of jobs, total number of process features, average job completion rate, standard deviation of job completion rate, and average completion rate of process features;

[0165] The RMC-related characteristic values ​​include at least one of the following: total number of RMCs, average RMC utilization rate, standard deviation of average RMC utilization rate, average configuration time percentage, and standard deviation of configuration time percentage.

[0166] The completion time-related characteristic values ​​include: the standard deviation of the current completion time of RMC and / or the current average completion time of RMC;

[0167] The procrastination rate-related characteristic values ​​include: the actual average procrastination rate.

[0168] Optionally, in the production scheduling device, the preset composite scheduling rule includes at least one of the following: job selection rule, node sorting rule, or node selection rule and RMC allocation rule;

[0169] The job selection rule is used to indicate the jobs that are currently scheduled with priority.

[0170] The AND node sorting rule and the OR node selection rule are used to indicate at least one of the following: the information of the selected process path, the execution order of the process features, and the execution order of the process operations;

[0171] The RMC allocation rule is used to indicate the information of the selected RMC.

[0172] Optionally, in the production scheduling device, the reward function is used to indicate minimizing the difference between the average RMC utilization rate in the first state feature value and the average RMC utilization rate in the second state feature value.

[0173] It should be noted that the apparatus provided in the embodiments of the present invention can implement all the method steps implemented in the above production scheduling method embodiments and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiments and the beneficial effects will not be described in detail.

[0174] This invention also provides a production scheduling device, such as... Figure 6 As shown, it includes:

[0175] The processor 601, memory 602, transceiver 603, and programs or instructions stored in the memory 602 and executable on the processor 601; when the processor 601 executes the programs or instructions, it implements the various processes of the above-described production scheduling method embodiments and achieves the same technical effect. To avoid repetition, these will not be described again here.

[0176] The transceiver 603 is used to receive and send data under the control of the processor 601.

[0177] Among them, Figure 6 In this context, the bus architecture can include any number of interconnected buses and bridges, specifically connecting various circuits of one or more processors represented by processor 601 and memory represented by memory 602. The bus architecture can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides an interface. Transceiver 603 can be multiple elements, including transmitters and receivers, providing a unit for communicating with various other devices over a transmission medium. For different user equipment, the user interface 604 can also be an interface capable of connecting external or internal devices, including but not limited to keypads, displays, speakers, microphones, joysticks, etc.

[0178] The processor 601 is responsible for managing the bus architecture and general processing, while the memory 602 can store the data used by the processor 601 when performing operations.

[0179] This invention also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the above-described production scheduling method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0180] This invention also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, they implement the various processes of the above-described production scheduling method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0181] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0182] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0183] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

Claims

1. A production scheduling method, characterized in that, include: Acquire resource configuration data and job process structure data of multiple reconfigurable manufacturing cells (RMCs) and jobs to be scheduled; Based on the resource configuration data and the work process structure data, determine the state characteristic value of the input production scheduling agent, and obtain the scheduling scheme of the work to be scheduled output by the production scheduling agent based on the state characteristic value; The scheduling scheme includes at least one of the following: information on the selected process path, the execution order of process features, the execution order of process operations, and information on the selected RMC.

2. The method according to claim 1, characterized in that, Before acquiring resource configuration data from multiple RMCs and job process structure data for the jobs to be scheduled, the method further includes: Obtain RMC resource configuration training data and operational process structure training data; Based on the resource configuration training data and the work process structure training data, the first state feature value is obtained; The initial reinforcement learning network is trained based on the first state feature value to obtain the second state feature value and the first reinforcement learning network. The first reinforcement learning network is trained based on the second state feature value until a preset number of training iterations are reached, thereby obtaining the production scheduling intelligent agent.

3. The method according to claim 2, characterized in that, The initial reinforcement learning network is trained based on the first state feature values ​​to obtain the second state feature values ​​and the first reinforcement learning network, including: The initial reinforcement learning network is trained based on the first state feature value and the preset composite scheduling rule to obtain the first composite scheduling action selected by the initial reinforcement learning network. The initial reinforcement learning network executes the first composite scheduling action to obtain the second state feature value; The first reward value is obtained based on the first state feature value, the second state feature value, and the reward function; The network parameters of the initial reinforcement learning network are updated based on the first state feature value, the first composite scheduling action, the first reward value, and the second state feature value to obtain the first reinforcement learning network.

4. The method according to claim 1, characterized in that, The status feature values ​​include at least one of the following: scheduling progress related feature values, RMC related feature values, completion time related feature values, and delay rate related feature values; The scheduling progress-related characteristic values ​​include at least one of the following: total number of jobs, total number of process features, average job completion rate, standard deviation of job completion rate, and average completion rate of process features; The RMC-related characteristic values ​​include at least one of the following: total number of RMCs, average RMC utilization rate, standard deviation of average RMC utilization rate, average configuration time percentage, and standard deviation of configuration time percentage. The completion time-related characteristic values ​​include: the standard deviation of the current completion time of RMC and / or the current average completion time of RMC; The procrastination rate-related characteristic values ​​include: the actual average procrastination rate.

5. The method according to claim 3, characterized in that, The preset composite scheduling rules include at least one of the following: job selection rules, node sorting rules, or node selection rules and RMC allocation rules; The job selection rule is used to indicate the jobs that are currently scheduled with priority. The AND node sorting rule and the OR node selection rule are used to indicate at least one of the following: the information of the selected process path, the execution order of the process features, and the execution order of the process operations; The RMC allocation rule is used to indicate the information of the selected RMC.

6. The method according to claim 3, characterized in that, The reward function is used to indicate minimizing the difference between the average RMC utilization rate of the first state feature value and the average RMC utilization rate of the second state feature value.

7. A production scheduling device, characterized in that, include: The first acquisition module is used to acquire resource configuration data of multiple RMCs and job process structure data of jobs to be scheduled. The determination module is used to determine the state feature value of the input production scheduling agent based on the resource configuration data and the operation process structure data, and to obtain the scheduling scheme of the operation to be scheduled output by the production scheduling agent based on the state feature value. The scheduling scheme includes at least one of the following: information on the selected process path, the execution order of process features, the execution order of process operations, and information on the selected RMC.

8. A production scheduling device, characterized in that, include: A processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the processor, when executing the program or instructions, implements the production scheduling method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the production scheduling method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the production scheduling method as described in any one of claims 1 to 6.

Images