Workshop scheduling methods, devices, equipment, storage media, and computer program products

Abstract

This application relates to the field of production automation scheduling technology, and discloses a workshop scheduling method, apparatus, equipment, storage medium, and computer program product, including: setting up a unit scheduling agent for each manufacturing unit in the workshop according to a preset workshop model; constructing an initial online model and an initial backup model for the unit scheduling agent based on a recurrent neural network and a convolutional sparse Transformer; training the initial online model through a hybrid network and a recycling pool, and ending the training when the first training loss value of the initial online model converges to obtain a target online model; and performing alternating scheduling based on the target online model and the initial backup model. By constructing an online model and a backup model for alternating scheduling, when the environment changes, the backup model switches to the online model, and after the online model training is completed, it enters the application state for scheduling, achieving more efficient, stable, and sustainable production scheduling, and effectively responding to dynamic changes in the workshop environment.

Description

Technical Field

[0001] This application relates to the field of production automation scheduling technology, and in particular to a workshop scheduling method, apparatus, equipment, storage medium and computer program product. Background Technology

[0002] In actual production, the workshop environment changes over a long period. When the scheduling model interacts with this changed environment and makes scheduling decisions, the scheduling strategies based on past knowledge become inapplicable, leading to a decline in model performance and difficulty in consistently outputting efficient scheduling strategies. At this point, to ensure the effectiveness of the scheduling scheme and workshop production efficiency, the scheduling model needs to be retrained. This consumes a significant amount of time, negatively impacting the continuity of production plans and the real-time performance of the system, ultimately affecting overall production efficiency and cost control. Summary of the Invention

[0003] The main objective of this application is to provide a workshop scheduling method, apparatus, equipment, storage medium, and computer program product, which aims to solve the technical problem that previous scheduling strategies cannot quickly adapt to the new environment and make efficient scheduling decisions when the internal environment of the workshop changes during long-term production processes.

[0004] To achieve the above objectives, this application proposes a workshop scheduling method, which includes:

[0005] Set up a unit scheduling agent for each manufacturing unit in the workshop according to the preset workshop model;

[0006] An initial online model and an initial backup model are constructed for the unit scheduling agent based on recurrent neural networks and convolutional sparse Transformers.

[0007] The initial online model is trained using a hybrid network and a recycling pool, and training ends when the first training loss value of the initial online model converges, thus obtaining the target online model.

[0008] Alternate scheduling is performed based on the target online model and the initial backup model.

[0009] Optionally, the state of the model includes an application state and a training state, wherein the application state is the state when the model is scheduled, and the training state is the state when the model is trained based on the empirical data in the replay pool.

[0010] The step of alternating scheduling based on the target online model and the initial backup model includes:

[0011] The target online model is set to the application state, and the initial backup model is trained based on the empirical data in the replay pool;

[0012] When a change in the current environment is detected, the initial backup model is adjusted based on the collected environmental data to obtain the target backup model;

[0013] Switch the target online model from the application state to the training state, and switch the target backup model from the training state to the application state;

[0014] After the online model training of the unit is completed, the target backup model is switched back to the training state, and the target online model that has been trained is switched back to the application state for scheduling decision.

[0015] Optionally, the step of adjusting the initial backup model based on the collected environmental data to obtain the target backup model when a change in the current environment is detected includes:

[0016] When a change in the current environment is detected, a preset amount of environmental data is collected;

[0017] The parameters of the fully connected layer of the initial backup model are adjusted based on the environmental data, and the second training loss value of the initial backup model is monitored in real time.

[0018] Training ends when the second training loss value converges, and the target backup model is obtained.

[0019] Optionally, the step of training the initial online model using a hybrid network and a recycling pool, and ending training when the first training loss value of the initial online model converges to obtain the target online model, includes:

[0020] Obtain the unit observation space from the manufacturing unit, and control the unit scheduling agent to generate corresponding agent actions based on the unit observation space;

[0021] Based on the unit observation space and the agent's actions, the historical experience generated by the interaction between the unit scheduling agent and the workshop environment is recorded in the playback pool;

[0022] The initial online model is trained using the hybrid network and the replay pool, and the first training loss value of the initial online model is continuously monitored. When the first training loss value converges, the training ends, and the target online model is obtained.

[0023] Optionally, the step of obtaining the unit observation space from the manufacturing unit and controlling the unit scheduling agent to generate corresponding agent actions based on the unit observation space includes:

[0024] The unit observation space is obtained based on the manufacturing unit;

[0025] Determine the exploration factor and the list of all selectable actions for the unit scheduling agent;

[0026] Obtain the current training exploration rate and compare the exploration factor with the exploration rate.

[0027] When the exploration factor is greater than the exploration rate, the corresponding action decision is selected from the action list based on the unit observation space as the corresponding agent action;

[0028] When the exploration factor is less than the exploration rate, a random action decision is generated as the corresponding agent action.

[0029] Optionally, the step of setting up a unit scheduling agent for each manufacturing unit in the workshop according to a preset workshop model includes:

[0030] The state space is set based on the preset workshop model;

[0031] Design scheduling rules based on a preset set of scheduling schemes, and determine the action space based on the scheduling rules;

[0032] A reward function is designed based on the average daily movement steps of equipment and the average product processing cycle of the preset workshop model.

[0033] A unit scheduling agent is set up for each manufacturing unit in the workshop based on the state space, the action space, and the reward function.

[0034] Furthermore, to achieve the above objectives, this application also proposes a workshop scheduling device, which includes:

[0035] The agent setting module is used to set up a unit scheduling agent for each manufacturing unit in the workshop according to a preset workshop model.

[0036] The model building module is used to construct an initial online model and an initial backup model for the unit scheduling agent based on recurrent neural networks and convolutional sparse Transformers.

[0037] The model training module is used to train the initial online model through a hybrid network, and to end the training when the training loss value of the initial online model converges, thereby obtaining the target online model.

[0038] The model scheduling module is used to perform alternating scheduling based on the target online model and the initial backup model.

[0039] In addition, to achieve the above objectives, this application also proposes a workshop scheduling device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the workshop scheduling method described above.

[0040] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the workshop scheduling method described above.

[0041] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the workshop scheduling method described above.

[0042] This application discloses a method for setting up a unit scheduling agent for each manufacturing unit within a workshop based on a pre-defined workshop model; constructing an initial online model and an initial backup model for the unit scheduling agent based on a recurrent neural network and a convolutional sparse Transformer; training the initial online model using a hybrid network and a recycling pool, and ending training when the first training loss value of the initial online model converges to obtain the target online model; and performing alternating scheduling based on the target online model and the initial backup model. By constructing initial online and backup models for each unit scheduling agent, and then performing alternating scheduling based on the target online model and the initial backup model, production efficiency and flexibility are ensured while ensuring the rational allocation of production resources, achieving more efficient, stable, and sustainable production scheduling, and effectively responding to dynamic changes in the workshop environment. Attached Figure Description

[0043] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0044] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a flowchart illustrating the first embodiment of the workshop scheduling method of this application;

[0046] Figure 2 This is a structural diagram of the MiniFab verification production line model in the embodiments of this application;

[0047] Figure 3 This is a schematic diagram of the online model structure of the intelligent agent in this application.

[0048] Figure 4 This is a schematic diagram of the backup model structure of the intelligent agent in this application.

[0049] Figure 5 This is a flowchart of the scheduling model switching mechanism in this application;

[0050] Figure 6 This is a flowchart illustrating the second embodiment of the workshop scheduling method of this application.

[0051] Figure 7 This is a flowchart illustrating the third embodiment of the workshop scheduling method of this application;

[0052] Figure 8 A schematic diagram of weighted EDD data for a Transformer backup model;

[0053] Figure 9 This is a schematic diagram of the weighted fitting data for the rule-based SRPT based on the Transformer backup model.

[0054] Figure 10 A schematic diagram of weighted EDD data for a CS Transformer backup model;

[0055] Figure 11 A schematic diagram of the weighted data fitting for the rule-based SRPT of the CS Transformer backup model;

[0056] Figure 12 A comparison chart of the loss value changes during the backup model training process for this application;

[0057] Figure 13 A comparison chart of the loss value changes during the backup model fine-tuning process of this application;

[0058] Figure 14 This is a line graph comparing the average processing cycle in the workshop of this application.

[0059] Figure 15 This is a comparison chart of the average processing cycle of box types in the workshop of this application;

[0060] Figure 16 This is a line graph comparing the average daily movement steps of manufacturing units A and B in this application.

[0061] Figure 17 This is a comparison chart of the average daily movement steps of manufacturing units A and B in this application.

[0062] Figure 18 This is a schematic diagram of the modular structure of the workshop scheduling device according to an embodiment of this application;

[0063] Figure 19 This is a schematic diagram of the equipment structure of the hardware operating environment involved in the workshop scheduling method in this application embodiment.

[0064] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0065] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0066] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0067] The main solution of this application embodiment is as follows: set up a unit scheduling agent for each manufacturing unit in the workshop according to a preset workshop model; construct an initial online model and an initial backup model for the unit scheduling agent based on a recurrent neural network and a convolutional sparse Transformer; train the initial online model through a hybrid network and a recycling pool, and end the training when the first training loss value of the initial online model converges to obtain the target online model; and perform alternating scheduling based on the target online model and the initial backup model.

[0068] As intelligent manufacturing evolves towards personalization and complexity, and the production process becomes increasingly uncertain, higher demands are placed on the real-time response capabilities and flexibility of scheduling systems. However, traditional scheduling optimization methods lack flexibility, have poor real-time response capabilities, and struggle to provide efficient scheduling solutions. Current research in reinforcement learning employs scheduling methods where the scheduling model is trained using data generated from the interaction between an agent and a fixed workshop environment, and the model's parameters remain unchanged after training. However, in actual production, the workshop environment may change over a long period. When the scheduling model interacts with the changed workshop environment and makes scheduling decisions, the scheduling strategies derived from previous knowledge become inapplicable, leading to a decline in model performance and difficulty in consistently outputting efficient scheduling strategies. In this case, to ensure the effectiveness of the scheduling solution and workshop production efficiency, the scheduling model needs to be retrained, which consumes a significant amount of time, negatively impacting the continuity of production plans and the real-time performance of the system, thereby affecting overall production efficiency and cost control.

[0069] Therefore, this application provides an intelligent workshop adaptive scheduling method that can quickly adapt to new environments and make efficient scheduling decisions when the environment changes, reducing the reduction in production efficiency caused by the deterioration of model performance in new environments. While ensuring production efficiency and flexibility, it ensures the rational allocation of production resources, achieving more efficient, stable and sustainable production scheduling, and effectively responding to dynamic changes in the workshop environment.

[0070] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a computer, or an electronic device capable of performing the above functions. The following description uses a workshop scheduling system as an example to illustrate this embodiment and the subsequent embodiments.

[0071] Based on this, the embodiments of this application provide a workshop scheduling method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the workshop scheduling method of this application.

[0072] In this embodiment, the workshop scheduling method includes:

[0073] Step S10: Set up a unit scheduling agent for each manufacturing unit in the workshop according to the preset workshop model.

[0074] It should be noted that a pre-defined workshop model is a pre-defined abstract description or framework of the workshop production system. It includes key information such as the basic structure of the workshop, equipment layout, production process, and material flow paths. It serves as the foundational framework for the entire workshop scheduling method, guiding and constraining subsequent settings and operations, such as the MiniFab model for semiconductor manufacturing workshops. A manufacturing cell is a production area or module within the workshop with relatively independent functions. A cell scheduling agent is an intelligent software or algorithmic entity assigned to each manufacturing cell, responsible for making production scheduling decisions based on the actual situation of that manufacturing cell and the overall goals of the workshop.

[0075] Furthermore, to enable the unit scheduling agent to make scientific and efficient decisions during the workshop scheduling process, thereby improving the overall production efficiency and operational efficiency of the workshop, step S10 may include:

[0076] A state space is set based on the preset workshop model; scheduling rules are designed according to the preset scheduling scheme set, and the action space is determined based on the scheduling rules; a reward function is designed based on the average daily movement steps of equipment and the average processing cycle of products in the preset workshop model; and a unit scheduling agent is set for each manufacturing unit in the workshop according to the state space, the action space and the reward function.

[0077] It should be noted that the state space encompasses various types of information closely related to production within the manufacturing unit. For example, equipment status includes real-time operating parameters, availability, maintenance cycles, and historical maintenance records. Material status includes the inventory quantity, storage location, and quality status of raw materials, semi-finished products, and finished products. The action space clarifies the specific operations or decisions that an agent can take during shop floor scheduling.

[0078] For ease of understanding, the following examples are provided, but are not intended to limit the invention. In one example, reference is made to... Figure 2 , Figure 2 This is a structural diagram of the MiniFab verification production line model in an embodiment of this application. Assume there are I manufacturing units in the smart workshop, and the i-th, i∈[1,…,I] manufacturing unit has M... i Device m, m∈[1,…,M] i The number of devices is m i And there are P types of products in the production task, which can be represented by p, p∈[1,…,p]. Taking the classic semiconductor manufacturing workshop MiniFab model as an example, such as Figure 2 As shown. MiniFab has six processing steps after resources enter the system, producing three types of products, five machines (A, B, C, D, and E), and three manufacturing units. The manufacturing units include a diffusion zone, an ion implantation zone, and a photolithography zone. Equipment A and B are used in the diffusion zone, performing steps 1 and 5; equipment C and D are used in the ion implantation zone, performing steps 2 and 4; and equipment E is used in the photolithography zone, performing steps 3 and 6. The simulation platform used is Plant Simulation, and the experimental platform was implemented using Python. The experimental environment consisted of an AMD R7-4800 CPU, 16GB of memory, and a Windows 10 operating system. The daily material input (Lot / day) for the three products in the workshop is evenly distributed within the range [16, 24].

[0079] When designing the state space of the scheduling agent in the design unit, the global workshop state set is shown in Table 1:

[0080] Table 1 Global Workshop Status Set

[0081] Serial Number Status Name describe 1 <![CDATA[W p ]]> Quantity of work-in-process for product p 2 <![CDATA[L i ]]> Length of the queue to be processed in manufacturing unit i 3 <![CDATA[F p ]]> The number of newly produced products (all processes completed) for product p

[0082] The state space of the design and manufacturing unit is shown in Table 2:

[0083] Table 2 Manufacturing Unit Status Set

[0084] Serial Number Status Name describe 1 <![CDATA[W p,i,m ]]> The quantity of work-in-process for product p in the m-th equipment of manufacturing unit i 2 <![CDATA[L i ]]> Length of the queue to be processed in manufacturing unit i 3 <![CDATA[F i,p ]]> The number of new finished products p in manufacturing unit i

[0085] When designing the action space of the intelligent agent, the scheduling scheme set adopted is a combined scheduling rule composed of three heuristic rules: Earliest Due Date (EDD), Shortest Remaining Processing Time (SRPT), and Critical Ratio (CR). For a unit-scheduled intelligent agent at the t-th decision step, there is... x tc Describes the weight of rule c and satisfies In particular, when x t1 When = 1, Represents rule EDD, when x t2 When x = 1, it represents the rule SRPT, when x t3 When = 1, Represents rule CR.

[0086] When designing and establishing the reward functions, it is necessary to first set the unit objective as increasing the average daily movement steps of equipment (MOV), and the shop floor objective as reducing the average processing cycle time of products (MCT). Based on MOV and MCT, design the unit reward function corresponding to maximizing the average daily movement steps of equipment and the global reward function corresponding to minimizing the average processing cycle time of products.

[0087] The reward function lc of manufacturing unit i i The formula is described as follows:

[0088]

[0089] in, MOV is the baseline average daily movement steps for manufacturing unit i. i Let i be the current average daily movement steps of manufacturing unit i, if This is considered as the completion of the unit objective, i.e., lc i =1, otherwise lc i =0. Subsequently, the completion flag for this unit's objective will play a role in the loss function to optimize the expected future rewards.

[0090] The formula describing the global reward function r is:

[0091]

[0092] Where, η r MCT is the workshop reward value balancing coefficient, used to maintain reward values ​​within a specific range.b The baseline average processing cycle can be obtained by running an average processing rule in the same workshop environment. The average processing rule is as follows: After determining the state space, action space, and reward function, a unit scheduling agent can be built for each manufacturing unit in the smart workshop.

[0093] Step S20: Construct an initial online model and an initial backup model for the unit scheduling agent based on a recurrent neural network and a convolutional sparse Transformer.

[0094] Understandably, the initial online model, constructed with a multi-layered neural network centered on a recurrent neural network, effectively retains training experience and accelerates training speed, resulting in faster convergence and greater flexibility during the online scheduling model training process. The initial backup model uses an improved Transformer algorithm to instantiate the backup model. The Transformer network, through its self-attention mechanism, can more accurately capture the mapping relationship and dynamic changes between states and decisions within the manufacturing unit, thereby improving the accuracy of scheduling decisions within the unit. Simultaneously, the efficient parallel processing capability of multi-head self-attention allows it to adapt to the needs of production environments of manufacturing units of different sizes, thus exhibiting superior model adaptability. This results in the unit backup model possessing better data learning capabilities and faster fine-tuning speed.

[0095] In one example, reference Figure 3 and Figure 4 , Figure 3 This is a schematic diagram of the online model structure of the intelligent agent in this application. Figure 4 This is a schematic diagram of the backup model structure of the intelligent agent in this application.

[0096] The online model for unit scheduling agents is based on RNN (Recurrent Neural Network) and includes an evaluation network and a target network, such as... Figure 3 As shown in the diagram, the agent evaluation network and the agent target network have the same structure: the first layer is the input layer, the second layer is a hidden layer, the third layer is an RNN layer, the fourth layer is a hidden layer, and the fifth layer is the output layer. The o in the upper left corner of the diagram... i The state data within the unit serves as the input to the entire decision-making process. The agent evaluation network receives the current state perception and decision feedback, uses the e-value to determine whether to randomly select or generate an action, and outputs the decision after adjustment. The input layer of the agent evaluation network receives state perception information, which is processed by the hidden layer and then enters the RNN layer. The RNN layer processes the time series information, and after further processing by the hidden layer, it outputs to the output layer. The output layer's output... Indicates the state o at the current moment iThe evaluation value of the action taken is calculated. Simultaneously, the result of the agent evaluation network is copied with a delay into the agent target network to output the evaluation value of the action taken in the next time step. The simultaneous evaluation hybrid network and the target hybrid network calculate losses based on the outputs of the agent evaluation network and the agent target network, respectively, and update the agent evaluation network based on the calculation results. The experience replay pool stores historical experience data, including information such as state, action, and reward. This data can be used by the agent evaluation network and the agent target network for learning and updating. The specific network structure of the agent online model is shown in Table 3.

[0097] Table 3. Dimension of the Network Structure of the Online Scheduling Model

[0098] Input feature dimension 36 Output feature dimension 2 RNN network dimension 256 Linear input / output layer dimension 64

[0099] The unit scheduling agent backup model is based on the CS Transformer (Convolution-Sparse Transformer), such as... Figure 4 As shown. The specific process of the backup model from input to output is as follows:

[0100] After being filtered, the unit state data enters the convolutional layer, followed by three sparse multi-head self-attention layers for data derivation. In the sparse self-attention layers, the manufacturing unit states with input dimension d are... They are transformed into H different query matrices respectively. Key-value matrix and value matrix Where h = 1, ..., H, Y represents the self-attention input of the multi-head self-attention layer. When this multi-head self-attention layer is the first layer, we have When the multi-head self-attention layer is not the first layer, Y is the output of the previous multi-head self-attention layer. Meanwhile... and As learnable parameters, after linear mapping and scaling, the output formula for calculating dot product self-attention is as follows:

[0101]

[0102] Set the upper triangular elements of the mask matrix M to -∞ to filter attention to the right of the current computation position, and output matrices O1, O2, ..., O to the sparse multi-head self-attention layer. H After concatenation, a linear mapping is performed again, and a fully connected layer is added to transform the dimensions based on the attention output, thus obtaining the model output.

[0103] Step S30: Train the initial online model using a hybrid network and a recycling pool, and terminate the training when the first training loss value of the initial online model converges to obtain the target online model.

[0104] It's important to note that training loss is a metric that measures the difference between the model's predictions and the actual (or target) results. In machine learning-based shop floor scheduling models, the loss is typically calculated based on the difference between the model's output scheduling decision (e.g., action selection) and the optimal scheduling decision. Common loss functions include mean squared error (MSE) and cross-entropy loss.

[0105] As we can understand, convergence refers to the state where the training loss value gradually decreases and tends to stabilize during training. Convergence can be determined by observing the curve of the training loss value changing with the number of training epochs (or iterations). When the loss value no longer decreases significantly in multiple consecutive training epochs, i.e., the change is less than a certain preset threshold, the training loss value can be considered to have converged.

[0106] Step S40: Alternate scheduling is performed based on the target online model and the initial backup model.

[0107] It is understood that the alternating scheduling based on the target online model and the initial backup model can be performed according to the performance of the models; it can also be performed when changes are detected in the current workshop environment, such as the introduction of new equipment, equipment failure, sudden changes in order demand (changes in product type, quantity, delivery time, etc.), changes in raw material supply (supplier changes, supply interruptions, raw material quality fluctuations, etc.), or improvements in production processes; or it can be performed after the model training is completed.

[0108] In one example, reference Figure 5 , Figure 5This is a flowchart of the scheduling model switching mechanism in this application. First, the online scheduling model interacts with the environment for training, and the historical experience data generated during this process is properly stored in the replay pool. Next, the training loss value of the online scheduling model is checked for convergence. If it has not converged, the model will continue training until convergence. Once converged, the online scheduling model enters the application state and begins to play a scheduling role in actual production or operational scenarios. Simultaneously, after the online scheduling model enters the application state, the backup model is trained using data from the replay pool. After the online scheduling model enters the application state, it is determined whether the environment has changed. If the environment has not changed, the online scheduling model will continue to be used for scheduling operations, while the backup model is trained using data from the replay pool. If the environment has changed, the backup model will undergo a brief fine-tuning process before entering the application state to cope with the new environmental conditions. When the backup model enters the application state, the online scheduling model will interact with the new environment and retrain, and the generated new historical experience will also be stored in the replay pool. After training is complete... The online scheduling model will switch from the training state back to the application state and resume the actual scheduling work, while the backup model will switch from the application state back to the training state and continue to learn and optimize using the replay pool data.

[0109] In this embodiment, a unit scheduling agent is set up for each manufacturing unit in the workshop according to a preset workshop model; an initial online model and an initial backup model are constructed for the unit scheduling agent based on a recurrent neural network and a convolutional sparse Transformer; the initial online model is trained through a hybrid network and a recycling pool, and training ends when the first training loss value of the initial online model converges, resulting in a target online model; scheduling is performed alternately based on the target online model and the initial backup model. By constructing initial online and backup models for each unit scheduling agent, and then performing alternating scheduling based on the target online model and the initial backup model, production efficiency and flexibility are ensured while ensuring the rational allocation of production resources, achieving more efficient, stable, and sustainable production scheduling, and effectively responding to dynamic changes in the workshop environment.

[0110] Reference Figure 6 , Figure 6 This is a flowchart illustrating the second embodiment of the workshop scheduling method of this application. Based on the first embodiment described above, a second embodiment of the workshop scheduling method of this application is proposed.

[0111] In the second embodiment, step S30 includes:

[0112] Step S301: Obtain the unit observation space from the manufacturing unit, and control the unit scheduling agent to generate corresponding agent actions based on the unit observation space.

[0113] It should be noted that the cell observation space is a comprehensive data representation of the internal state of a manufacturing cell. It is constructed by collecting data from various aspects within the manufacturing cell, such as equipment operating status, material inventory and flow, personnel working status, and production environment conditions.

[0114] It is understood that the control unit can schedule the agent to generate the corresponding agent action by determining the best action plan according to preset rules; or by determining the best action plan according to the previous decision; or by randomly generating the corresponding action for selection.

[0115] Furthermore, to enhance the model's scheduling capability under different conditions and avoid training getting stuck in local optima, step S301 may include: obtaining a unit observation space based on the manufacturing unit; determining the exploration factor and a list of all selectable actions for the unit scheduling agent; obtaining the current training exploration rate and comparing the exploration factor and the exploration rate; when the exploration factor is greater than the exploration rate, selecting the corresponding action decision from the action list based on the unit observation space as the corresponding agent action; when the exploration factor is less than the exploration rate, randomly generating an action decision as the corresponding agent action.

[0116] For ease of understanding, the following examples are provided, but are not intended to limit the invention. In one example, the list of all selectable actions for the unit scheduling agent i is set as follows: Its length is Indicates that there is There are several action combinations to choose from, so the selected action value can be abstracted as a positive integer. Instead, the unit scheduling agent's decision-making is determined by a randomly generated exploration factor e, while the exploration rate ε decreases linearly during training. The specific decay process is as follows:

[0117]

[0118] Where N decay ε represents the number of exploration rate decays, λ represents the decay rate, t represents the number of steps, and ε represents the number of steps. min This represents the minimum exploration rate, ensuring that the unit-scheduled agent always generates random actions during training. When e > ε, actions are selected by scheduling the agent using an RNN network as the core unit; otherwise, actions are generated randomly. The calculation process is as follows:

[0119]

[0120] Where rand represents random generation. Represents a function based on maximizing local Q-values. Choose the action that maximizes the Q-value from all possible actions. (For unit decision-making) Theoretically, the unit scheduling agent i can obtain the expected Q-value vector of this decision as follows:

[0121]

[0122] in, This indicates that under policy π, the unit scheduling agent i targets its observation o. i With action a i The expected cumulative discount reward, where γ is the discount factor and π is the current scheduling policy responsible for making the current action, represents the neural network structure of the entire unit scheduling agent. In practical applications, the process of calculating the Q-value can be expressed as:

[0123]

[0124] Where f i It is the i-th unit scheduling agent network. These are network parameters.

[0125] Step S302: Based on the unit observation space and the agent's actions, record the historical experience generated by the interaction between the unit scheduling agent and the workshop environment in the playback pool.

[0126] It should be understood that the data in the replay pool breaks the temporal correlation between data, thus preventing overfitting during model training. For example, when training a machine learning-based shop floor scheduling model, directly using continuous real-time data may cause the model to become overly reliant on the order and pattern of the current data. The randomly sampled data in the replay pool can solve this problem.

[0127] Step S303: Train the initial online model based on the hybrid network and the replay pool, and continuously monitor the first training loss value of the initial online model. When the first training loss value converges, end the training to obtain the target online model.

[0128] It should be noted that the hybrid network consists of the weights and biases output by the hypernetwork from the workshop state at a given time, and is responsible for updating the network parameters of the scheduling agents of each unit.

[0129] Understandably, the first training loss value is the loss value obtained by training the initial online model based on historical interaction data in the recycling pool. It measures the difference between the initial online model's predictions and the actual (or expected) results. When the first training loss value converges, it means that the initial online model has learned enough knowledge from the training data to make relatively stable and accurate scheduling decisions. The target online model obtained at this point can be applied to actual workshop production scheduling to improve production efficiency and resource utilization.

[0130] In one example, the relevant parameters during the training process of the initial online model are shown in Table 4.

[0131] Table 4 Training Parameters for Online Scheduling Model

[0132]

[0133] Among them, the maximum number of steps in a single run, t max The learning rate determines the magnitude of each parameter update, which is the maximum number of actions that can be performed in a single training session. The discount factor is used to measure the importance of future rewards to the current decision.

[0134] In this embodiment, a unit observation space is obtained from the manufacturing unit, and the control unit schedules the agent to generate corresponding agent actions based on the unit observation space. Historical experiences generated by the interaction between the unit scheduling agent and the workshop environment are recorded in a replay pool based on the unit observation space and agent actions. An initial online model is trained using a hybrid network and the replay pool, and the first training loss value of the initial online model is continuously monitored. Training ends when the first training loss value converges, yielding the target online model. This allows the model to continuously optimize during training, accurately learning the patterns and optimal scheduling strategies in workshop production, thereby improving the rationality and efficiency of workshop scheduling.

[0135] Reference Figure 7 , Figure 7 This is a flowchart illustrating the third embodiment of the workshop scheduling method of this application. Based on the second embodiment described above, a third embodiment of the workshop scheduling method of this application is proposed.

[0136] In the third embodiment, step S40 includes:

[0137] Step S401: Set the target online model to the application state and train the initial backup model based on the experience data in the replay pool.

[0138] It should be noted that the application state refers to the current state of the model during haptic scheduling, meaning that the model is being used for actual workshop scheduling decisions. The model also includes a training state, representing that the model is being trained based on empirical data in the playback pool.

[0139] Understandably, learning the historical interaction data between the online scheduling model of the learning unit and the environment enables it to make decisions about the state of the unit.

[0140] For ease of understanding, the following examples are provided, but are not intended to limit the invention. In one example, reference is made to... Figure 8 , Figure 9 , Figure 10 , Figure 11 and Figure 12 . Figure 8 This is a schematic diagram of the weighted data fitting for rule-based EDD (Electronic Data Decomposition) based on the Transformer backup model. Figure 9 This is a schematic diagram illustrating the weighted fitting data for the rule-based SRPT based on the Transformer backup model. Figure 10 This is a schematic diagram illustrating the weighted data fitting for rule-based EDD (Electronic Data Distribution) based on the CSTransformer backup model. Figure 11 This is a schematic diagram of the weight fitting data for the rule-based SRPT of the CS Transformer backup model.

[0141] Backup the data output by the model Label data of samples in the playback pool In comparison, the loss function formula is as follows:

[0142]

[0143] Where n is the sequence length of the model training data. After obtaining the loss, the model is updated using the stochastic gradient descent algorithm to minimize the output data. With tag data The gap is maintained until the change in the loss value converges, thus achieving the training of the backup model. The specific network parameter settings for the backup model are shown in Table 5.

[0144] Table 5 Backup Model Network Parameters Table

[0145]

[0146] Based on the above parameters, Transformer without sparse convolution mechanism and CS Transformer can be trained separately, and the training effect in the later stage can be sampled and compared to generate a comparison chart.

[0147] Figure 8 The root mean square error of the medium sample is 1.221%, which shows that the Transformer algorithm can roughly fit the overall trend of the data, but there is still a gap in the specific numerical values. Figure 9The root mean square error of the medium sample is 1.023%. Similar to the fitting of regular EDD, the Transformer algorithm can still fit the overall trend of data change, but it lacks accuracy. Figure 10 The root mean square error of the samples is 0.832%, indicating that the CS Transformer algorithm has higher fitting accuracy than the Transformer algorithm. Figure 11 The root mean square error of the medium sample is 0.758%. In the fitting of the same regular SRPT, the CSTransformer algorithm has higher fitting accuracy than the Transformer algorithm and is better able to learn the overall trend of data change.

[0148] Figure 12 This is a comparison chart showing the change in loss values ​​during the training process of the backup model in this application. The chart shows that in the later stages of training, both models converge and the loss values ​​tend to stabilize. Therefore, the focus is on the changes in the loss curves of the two models in the early stages of training. Initially, in the initial training phase, the improved method has a lower initial loss value. Meanwhile, as the model begins to update, the unimproved method maintains a certain advantage. Subsequently, during the update process, the loss of the improved method gradually decreases compared to the unimproved method and remains lower. Finally, in the convergence phase, the improved method has a lower convergence value, demonstrating better training performance. The loss value of the improved method stabilizes at around 0.8%, while the unimproved method stabilizes at around 1.2%. Therefore, both convergence speed and training results demonstrate the superiority of the improved method.

[0149] Step S402: When a change in the current environment is detected, the initial backup model is adjusted according to the collected environmental data to obtain the target backup model.

[0150] It should be noted that events that require scheduling model updates occur when the current environment changes, such as a fault alarm signal for a critical piece of equipment, a notification that material inventory is below the safety stock level, or the addition of new production equipment.

[0151] It is understandable that adjusting the initial backup model can involve modifying its parameters, adjusting its structure (such as adding new network layers or modifying the connections of existing layers), or adjusting the model's results and parameters.

[0152] Furthermore, in order to adapt the shop floor scheduling model to the new production environment with lower computational cost and ensure smooth production, step S402 may include:

[0153] When a change in the current environment is detected, a preset amount of environmental data is collected; the parameters of the fully connected layer of the initial backup model are adjusted based on the environmental data, and the second training loss value of the initial backup model is monitored in real time; when the second training loss value converges, training ends and the target backup model is obtained.

[0154] In one example, reference Figure 13 , Figure 13 This is a comparison chart showing the changes in loss values ​​during the fine-tuning process of the backup model in this application. The formula for fine-tuning the backup model is:

[0155]

[0156] in, These are the fine-tuned model parameters. These are the model parameters before fine-tuning. The target dataset used for fine-tuning, i.e. the training data, is a small amount of new environmental data.

[0157] Model fine-tuning was performed on the trained Backup Transformer Scheduling Model (BTSM) and Backup CS Transformer Scheduling Model (BCSTSM). The fine-tuning process involved freezing all Transformer layers and modifying only the parameters of the final fully connected layer. The training data consisted of a small amount of new environmental data, and the loss function design for fine-tuning was the same as for training. The specific changes in the loss value are shown in the figure. Based on the figure, it can be seen that the loss of both methods gradually converged during model fine-tuning. BTSM converged more slowly but achieved a higher final convergence value; in contrast, BCSTSM showed stronger adaptability, a faster convergence speed, and maintained a lower convergence value. This demonstrates that the improvement achieved by introducing a convolutional sparsity mechanism is effective.

[0158] Step S403: Switch the target online model from the application state to the training state, and switch the target backup model from the training state to the application state.

[0159] It should be understood that while the target online model performed well in the previous environment, it needs to be relearned and optimized to adapt to the new environment. Switching it to training mode allows for retraining using data generated in the new environment, enabling it to continuously improve its scheduling performance in the new environment. Through this switching between the target online model and the target backup model, the workshop production scheduling system can better adapt to environmental changes, ensuring production continuity and stability while continuously optimizing production efficiency and resource utilization.

[0160] Understandably, before switching, it's crucial to ensure that the relevant data generated by the target online model in its application state is properly saved and processed for subsequent training. This data can be stored in an experience replay pool, serving as a vital data source for retraining the target online model. Before switching the target backup model to its application state, it needs thorough validation and evaluation. Validation methods can include testing with simulated environment data to ensure the model makes reasonable scheduling decisions under various possible new environmental scenarios.

[0161] Step S404: After the online model training of the unit is completed, the target backup model is switched back to the training state, and the target online model after training is switched back to the application state for scheduling decision.

[0162] Understandably, once the target online model has finished training, the target backup model needs to switch back to training mode. After switching back, the target backup model can adjust and optimize its parameters based on new data. By using the new training data, the target backup model can learn production scheduling strategies in the new environment, thereby improving its performance.

[0163] In one example, reference Figure 14 , Figure 15 , Figure 16 and Figure 17 . Figure 14 This is a line graph comparing the average processing cycle in the workshop for this application. Figure 15 This is a comparison chart of the average processing cycle for box types in the workshop of this application. Figure 16 This is a line graph comparing the average daily movement steps of manufacturing units A and B in this application. Figure 17 This is a comparison chart of the average daily movement steps of manufacturing units A and B in this application.

[0164] Based on the constructed experimental environment, four types of models were compared: the outdated online scheduling model (OOSM), BTSM, BCSTSM, and the updated online scheduling model (UOSM). Specifically, when the cell scenario changed, the outdated online scheduling model was OOSM, the updated online scheduling model was UOSM, and BTSM and BCSTSM were fine-tuned versions of the cell backup models. BTSM lacked a convolutional sparsity mechanism, while BCSTSM employed one. Therefore, the comparison between BTSM and OOSM reflects the effectiveness of the proposed cell adaptive scheduling method; the comparison between BTSM and BCSTSM demonstrates the effectiveness of the improvement to the Transformer algorithm in the cell adaptive scheduling problem. Finally, UOSM, as a control after the update of OOSM, indicates that the entire workshop has been updated, completing the closed loop of the proposed cell adaptive scheduling method.

[0165] Depend on Figure 14 It can be seen that UOSM has the best global objective performance, possessing the ability to outperform other models from the very beginning. BCSTSM has the second best performance, occasionally falling behind BTSM; OOSM performs the worst, but shows the best performance in the initial scheduling phase, subsequently falling behind the other three methods. Figure 15 It can be seen that BTSM improves overall performance by 0.53% compared to OOSM, while BCSTSM improves by 3.52% compared to OOSM. BTSM improves by a median of 3.35% compared to OOSM, while BCSTSM improves by 5.96% compared to OOSM. This indicates that in terms of average processing cycle performance, BCSTSM can significantly improve the adaptive scheduling capability of the unit, quickly and efficiently adapting to new environments.

[0166] Depend on Figure 16 It can be seen that OOSM consistently exhibits the lowest average daily movement steps over extended periods (3-7, 10-17, and 24-32 days), demonstrating the effectiveness of the cell adaptive method. Meanwhile, BTSM and BCSTSM do not show a clear advantage for either, exhibiting their respective strengths and weaknesses throughout the processing cycle. Figure 17 In the data, BTSM showed an average performance improvement of 0.66% compared to OOSM, while BCSTSM showed an improvement of 0.886% compared to OOSM. The median improvement of BTSM compared to OOSM was 1.245%, while BCSTSM showed an improvement of 1.257%. This indicates that the switching mechanism between the backup model and the online scheduling model can effectively improve the scheduling performance of the unit and adapt to the new environment within the unit, in terms of the average daily movement steps of unit A and B.

[0167] Table 6 shows a comparison of the training times for each model. The training data for BTSM and BCSTSM are the historical training data of the online scheduling model. Similarly, the online scheduling model updates are based on the online scheduling model already in use in the workshop. The table shows that the training time for BTSM and BCSTSM is significantly shorter than the update time for the online scheduling model, demonstrating the effectiveness of the proposed method and theory.

[0168] Table 6. Timetable for training each model

[0169]

[0170] The results in Table 6 show that the training time of BTSM and BCSTSM is significantly shorter than that of the online scheduling model, which demonstrates the effectiveness of the proposed method and theory. Therefore, the proposed BCSTSM method can quickly adapt to the new environment of the unit when the workshop unit environment changes, effectively improving the adaptive level of the intelligent workshop manufacturing unit.

[0171] In this embodiment, the target online model is set to the application state, and the initial backup model is trained based on empirical data in the replay pool. When a change in the current environment is detected, the initial backup model is adjusted according to the collected environmental data to obtain the target backup model. The target online model is switched from the application state to the training state, and the target backup model is switched from the training state to the application state. After the unit online model training is completed, the target backup model is switched back to the training state, and the trained target online model is switched back to the application state for scheduling decisions. When the environment changes, the backup model and the online model switch, the online model enters the training phase from the application state, and after training is completed, it transitions from the training state to the application state, achieving the purpose of adaptive scheduling.

[0172] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the workshop scheduling method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0173] This application also provides a workshop scheduling device, please refer to... Figure 18 The workshop scheduling device includes:

[0174] The intelligent agent setting module 10 is used to set up a unit scheduling intelligent agent for each manufacturing unit in the workshop according to a preset workshop model;

[0175] Model building module 20 is used to build an initial online model and an initial backup model for the unit scheduling agent based on recurrent neural networks and convolutional sparse Transformers;

[0176] The model training module 30 is used to train the initial online model through a hybrid network, and to end the training when the training loss value of the initial online model converges, thereby obtaining the target online model.

[0177] The model scheduling module 40 is used to perform alternating scheduling based on the target online model and the initial backup model.

[0178] The workshop scheduling device provided in this application, employing the workshop scheduling method described in the above embodiments, can solve the technical problem that previous scheduling strategies cannot quickly adapt to new environments and make efficient scheduling decisions when the internal environment of the workshop changes during long-term production processes. Compared with the prior art, the beneficial effects of the workshop scheduling device provided in this application are the same as those of the workshop scheduling method provided in the above embodiments, and other technical features in the workshop scheduling device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0179] This application provides a workshop scheduling device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the workshop scheduling method in Embodiment 1 above.

[0180] The following is for reference. Figure 19 The diagram illustrates a structural schematic suitable for implementing the workshop scheduling equipment of the embodiments of this application. The workshop scheduling equipment in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), vehicle terminals (e.g., vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 19 The workshop scheduling equipment shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0181] like Figure 19As shown, the shop floor scheduling equipment may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the shop floor scheduling equipment. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the shop floor scheduling equipment to communicate wirelessly or wiredly with other equipment to exchange data. Although the figure shows shop floor scheduling equipment with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0182] The workshop scheduling equipment provided in this application, employing the workshop scheduling method described in the above embodiments, can solve the technical problem that previous scheduling strategies cannot quickly adapt to new environments and make efficient scheduling decisions when the internal environment of the workshop changes during long-term production processes. Compared with the prior art, the beneficial effects of the workshop scheduling equipment provided in this application are the same as those of the workshop scheduling method provided in the above embodiments, and other technical features of this workshop scheduling equipment are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0183] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0184] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the workshop scheduling method in the above embodiments.

[0185] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0186] The aforementioned computer-readable storage medium may be included in the shop floor scheduling equipment; or it may exist independently and not assembled into the shop floor scheduling equipment. The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the shop floor scheduling equipment, cause the shop floor scheduling equipment to perform the shop floor scheduling method described above.

[0187] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0188] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described shop floor scheduling method. This solves the technical problem that previous scheduling strategies cannot quickly adapt to new environments and make efficient scheduling decisions when the internal environment of the shop changes over a long period of production. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the shop floor scheduling method provided in the above embodiments, and will not be repeated here.

[0189] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the workshop scheduling method described above.

[0190] The computer program product provided in this application can solve the technical problem that previous scheduling strategies cannot quickly adapt to the new environment and make efficient scheduling decisions when the internal environment of the workshop changes during long-term production processes. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the workshop scheduling method provided in the above embodiments, and will not be repeated here.

[0191] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A workshop scheduling method, characterized in that, The workshop scheduling method includes: Set up a unit scheduling agent for each manufacturing unit in the workshop according to the preset workshop model; An initial online model and an initial backup model are constructed for the unit scheduling agent based on recurrent neural networks and convolutional sparse Transformers. The initial online model is trained using a hybrid network and a recycling pool, and training ends when the first training loss value of the initial online model converges, thus obtaining the target online model. Alternate scheduling is performed based on the target online model and the initial backup model; The step of training the initial online model using a hybrid network and a recycling pool, and ending training when the first training loss value of the initial online model converges to obtain the target online model, includes: Obtain the unit observation space from the manufacturing unit, and control the unit scheduling agent to generate corresponding agent actions based on the unit observation space; Based on the unit observation space and the agent's actions, the historical experience generated by the interaction between the unit scheduling agent and the workshop environment is recorded in the playback pool; The initial online model is trained using the hybrid network and the replay pool, and the first training loss value of the initial online model is continuously monitored. When the first training loss value converges, the training ends and the target online model is obtained. The step of obtaining the unit observation space from the manufacturing unit and controlling the unit scheduling agent to generate corresponding agent actions based on the unit observation space includes: The unit observation space is obtained based on the manufacturing unit; Determine the exploration factor and the list of all selectable actions for the unit scheduling agent; Obtain the current training exploration rate and compare the exploration factor with the exploration rate. When the exploration factor is greater than the exploration rate, the corresponding action decision is selected from the action list based on the unit observation space as the corresponding agent action; When the exploration factor is less than the exploration rate, a random action decision is generated as the corresponding agent action. The step of setting up a unit scheduling agent for each manufacturing unit in the workshop according to a preset workshop model includes: The state space is set based on the preset workshop model; Design scheduling rules based on a preset set of scheduling schemes, and determine the action space based on the scheduling rules; A reward function is designed based on the average daily movement steps of equipment and the average product processing cycle of the preset workshop model. A unit scheduling agent is set up for each manufacturing unit in the workshop based on the state space, the action space, and the reward function.

2. The workshop scheduling method as described in claim 1, characterized in that, The model's state includes an application state and a training state. The application state is the state when the model is scheduled, and the training state is the state when the model is trained based on the empirical data in the replay pool. The step of alternating scheduling based on the target online model and the initial backup model includes: The target online model is set to the application state, and the initial backup model is trained based on the empirical data in the replay pool; When a change in the current environment is detected, the initial backup model is adjusted based on the collected environmental data to obtain the target backup model; Switch the target online model from the application state to the training state, and switch the target backup model from the training state to the application state; After the target online model is trained, the target backup model is switched back to the training state, and the trained target online model is switched back to the application state for scheduling decisions.

3. The workshop scheduling method as described in claim 2, characterized in that, The step of adjusting the initial backup model based on the collected environmental data to obtain the target backup model when a change in the current environment is detected includes: When a change in the current environment is detected, a preset amount of environmental data is collected; The parameters of the fully connected layer of the initial backup model are adjusted based on the environmental data, and the second training loss value of the initial backup model is monitored in real time. Training ends when the second training loss value converges, and the target backup model is obtained.

4. A workshop scheduling device, characterized in that, The workshop scheduling device includes: The agent setting module is used to set up a unit scheduling agent for each manufacturing unit in the workshop according to a preset workshop model. The model building module is used to construct an initial online model and an initial backup model for the unit scheduling agent based on recurrent neural networks and convolutional sparse Transformers. The model training module is used to train the initial online model through a hybrid network, and to end the training when the training loss value of the initial online model converges, thereby obtaining the target online model. The model scheduling module is used to perform alternating scheduling based on the target online model and the initial backup model; The model training module is further configured to obtain the unit observation space from the manufacturing unit, and control the unit scheduling agent to generate corresponding agent actions based on the unit observation space; record the historical experience generated by the interaction between the unit scheduling agent and the workshop environment in the replay pool based on the unit observation space and the agent actions; train the initial online model according to the hybrid network and the replay pool, and continuously monitor the first training loss value of the initial online model; when the first training loss value converges, end the training to obtain the target online model. The model training module is further configured to: acquire a unit observation space based on the manufacturing unit; determine the exploration factor and a list of all selectable actions for the unit scheduling agent; acquire the current training exploration rate and compare the exploration factor and the exploration rate; when the exploration factor is greater than the exploration rate, select the corresponding action decision from the action list based on the unit observation space as the corresponding agent action; when the exploration factor is less than the exploration rate, randomly generate an action decision as the corresponding agent action. The agent setting module is further configured to set a state space based on the preset workshop model; design scheduling rules according to a preset scheduling scheme set, and determine an action space based on the scheduling rules; design a reward function based on the average daily movement steps of equipment and the average processing cycle of products in the preset workshop model; and set a unit scheduling agent for each manufacturing unit in the workshop according to the state space, the action space, and the reward function.

5. A workshop scheduling device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the workshop scheduling method as described in any one of claims 1 to 3.

6. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the workshop scheduling method as described in any one of claims 1 to 3.

7. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the workshop scheduling method as described in any one of claims 1 to 3.

Images