Parallel variable speed machine batch mixed flow shop scheduling execution method and system

Abstract

The present invention relates to the technical field of workshop scheduling, and in particular to a method and system for executing workshop scheduling of parallel variable-speed machine batch mixed flow. The method comprises the following steps: S1, constructing a workshop scheduling knowledge graph; S2, constructing and training a workshop scheduling model; S3, saving the trained workshop scheduling training model, retrieving work tasks from the workshop scheduling knowledge graph, and inputting the workshop scheduling training model to generate the optimal workshop scheduling result, and executing production tasks according to the workshop scheduling result. The method can realize the integration and management of multi-source heterogeneous data, workshop scheduling and dynamic event processing, improve the intelligence and responsiveness of workshop scheduling, improve the production efficiency of the workshop, shorten the order delivery time, and reduce the production cost.

Description

Technical Field

[0001] The invention relates to the technical field of workshop scheduling, in particular to a method and system for executing scheduling of a parallel variable-speed machine batch mixed flow workshop. Background Art

[0002] In the current global market, the manufacturing industry faces fierce competition. To maintain competitiveness, manufacturing companies need to improve production efficiency, reduce costs, and focus on energy conservation and emission reduction. Among them, workshop scheduling, as a key link in the manufacturing process, is crucial to improving efficiency, reducing delivery time, reducing operating costs and saving energy. The Hybrid Flow-Shop Scheduling Problem with Batch and ParallelHeterogeneous Machines (HFSPPBPH) is one of the common workshop scheduling modes and is widely used in various manufacturing industries, such as electronics, papermaking, textiles, machinery, aviation, shipbuilding, and heavy energy equipment. Taking the production scheduling of high-temperature heating panel type tube panels of power station boilers as an example, it usually includes multiple processing stages, such as tube cutting, chamfering, internal boring, polishing, butt welding, tube bending and assembly welding. In order to improve the transfer efficiency and subsequent processing needs, the workpieces will be transferred in batches according to orders between the processing stages. Each stage may contain one or more parallel machines with different processing speeds, which provides higher production efficiency and flexibility than traditional flow workshops. However, the complexity of the problem is also much higher than that of traditional flow workshops, and it is a typical NP-hard problem. In addition, various dynamic events such as equipment maintenance, emergency orders and production execution deviations may occur during the production process. The traditional scheduling method that relies on manual experience is insufficient to cope with complex and unexpected situations, resulting in low scheduling efficiency and frequent delays in order delivery. It can be seen that the dynamic scheduling and multi-objective optimization of HFSPBPH are of great significance to enterprises in improving scheduling and production efficiency, reducing order delays and reducing production costs.

[0003] Before conducting workshop scheduling, enterprises need to process relevant order data. This process involves many departments and complex data, that is, the data involved in the scheduling process comes from many sources and in different formats. At the same time, there are certain information islands between departments, which makes collaborative work difficult, information transparency is low, and unified management is difficult. In the traditional model, relying on manual experience, it is easy to make mistakes when processing a large number of orders, processes and workshop data from different sources and formats. In addition, the workshop situation is complex and there are many emergencies. The planner needs to manually process a large amount of production data and dynamic information, and perform complex processing and adjustments. This makes it difficult to evaluate the rationality of the production schedule, aggravates the uncertainty in production, and makes the scheduling efficiency, planning accuracy and production efficiency low. It can be seen that realizing the integration of scheduling data from multi-source heterogeneous data, assisting scheduling personnel in processing orders, processes and workshop data from different sources and formats, frequent dynamic events in the workshop and scheduling results, is of great significance for enterprises to achieve unified management of production materials and improve the accuracy and efficiency of workshop production planning and scheduling.

[0004] The current storage and distribution of workshop scheduling plans also rely on traditional models, which are stored in the form of Excel tables and tasks are distributed in printed paper versions, which can easily lead to management confusion and execution deviations. This leads to difficulties in management, frequent material flipping and searching, waiting for materials, production stagnation, and excessive inventory on the line, which in turn leads to long maximum order completion time and failure to meet delivery dates. It can be seen that the management and rapid distribution of scheduling results are of great significance to enterprises in accelerating the execution of workshop production tasks and thus improving overall production efficiency.

[0005] However, the existing scheduling algorithms have many shortcomings in dealing with the dynamic scheduling needs in complex workshop environments. First, traditional scheduling systems rely on group intelligence optimization algorithms, such as genetic algorithms, ant colony algorithms, and particle swarm algorithms. They have slow convergence speed when solving large-scale and complex problems, poor stability of results, high complexity of parameter adjustment, and performance that often cannot be linearly expanded. They also need to recompile algorithms for specific scenarios. Therefore, they cannot fully meet the needs of dynamic scheduling of multi-objective mixed flow workshops in many complex environments; they are difficult to adapt to dynamic events that frequently occur in the production process, such as equipment failures and emergency order insertions, which leads to poor scheduling flexibility and inability to respond in a timely manner. In addition, the data management method in the traditional scheduling system is relatively backward, and different departments rely on different SQL databases, which have poor horizontal scalability, limited ability to process unstructured data, and performance bottlenecks for complex queries; and data processing and scheduling execution are independent of each other, and real-time data sharing and feedback cannot be achieved, which greatly reduces scheduling efficiency and makes it difficult to guarantee the execution effect of scheduling plans. Therefore, there is an urgent need for a scheduling system that can effectively integrate multi-source heterogeneous data, support dynamic event processing, achieve multi-objective optimization, and manage and issue scheduling results, so as to improve the overall production efficiency of the workshop, reduce order delays and optimize resource utilization. Summary of the invention

[0006] The purpose of the present invention is to overcome the above-mentioned defects of the prior art, and proposes a parallel speed machine batch mixed flow shop scheduling execution method and system, which can realize the integration and management of multi-source heterogeneous data, shop scheduling and dynamic time processing, enhance the intelligence and responsiveness of shop scheduling, improve the production efficiency of the shop, shorten the order delivery time, and reduce the production cost.

[0007] The technical solution of the present invention is: a method for scheduling and executing a parallel variable speed machine batch mixed flow shop, which includes the following steps:

[0008] S1. Build a knowledge graph for workshop scheduling;

[0009] S2. Build and train the workshop scheduling model;

[0010] S3. Save the trained workshop scheduling training model, retrieve the work tasks from the workshop scheduling knowledge graph, and input them into the workshop scheduling training model to generate the optimal workshop scheduling results, and execute the production tasks according to the workshop scheduling results.

[0011] In the present invention, step S1 includes the following specific execution process:

[0012] S1.1. Construct each ontology in the workshop scheduling knowledge graph, including workshop resource ontology, process flow ontology, order information ontology, dynamic event ontology and scheduling result ontology, determine the classes and some instances in each ontology, determine the association relationship between classes, between instances and between classes, and determine the specific information of the instance;

[0013] S1.2. Construct a unified ontology layer: By sharing the same International Resource Identifier (IRI) in the same classes or instances in different ontologies, the same classes or instances between these ontologies are unified into shared classes;

[0014] S1.3, set the relationship between each class;

[0015] S1.4, filling and completing instance layer data: Based on the constructed ontology layer, align the class to which the instance layer data belongs through the association relationship set in step S1.3, and complete the correlation relationship between each instance;

[0016] S1.5. Verify whether the function of the unified ontology layer is correct. If correct, import the unified ontology layer into the graph database;

[0017] S1.6. Verify whether the function of the graph database is correct. If correct, realize the visualization of the workshop scheduling knowledge graph.

[0018] Step S2 includes the following specific execution process:

[0019] S2.1, Environmental state extraction;

[0020] S2.2, input the state into the policy network to generate actions;

[0021] S2.3, input the action generated in step S2.2 into the scheduling simulation environment, perform the actual operation, feedback the reward according to the execution result, and update the status;

[0022] S2.4, evaluate the current action through the value network to obtain the evaluation Q value;

[0023] S2.5, generate the target value Q_target through the target network, and optimize and update the policy network and value network by minimizing the error between the evaluation Q value and the target value Q_target;

[0024] S2.6, repeating steps S2.1 to S2.5, during the iterative process, continuously optimizing the strategy network and the value network, and the iterative process continues until the number of remaining workpieces in the state vector is 0, thus achieving scheduling optimization within one production round;

[0025] S2.7, after the scheduling optimization process of a production round is completed, before starting another new production round, the scheduling simulation environment is initialized;

[0026] Repeat the above steps S2.1 to S2.7 to train the workshop scheduling model through the scheduling optimization process of several production rounds.

[0027] In step S2.1, the current state vector is extracted from the scheduling simulation environment ,

[0028] ,

[0029] in, represents the current time step, Indicates the order number. Indicates the current processing stage. Indicates the order product type. Indicates the remaining number of workpieces. represents the normalized order deadline,

[0030] ,

[0031] in, Indicates the actual deadline of the order. represents the time scale factor.

[0032] In step S2.2, the network structure of the policy network includes:

[0033] Input layer, accepts the state vector , the dimension is ;

[0034] LSTM layer, including one LSTM layer, the number of hidden units is 256, the LSTM layer is for the input sequence The processing formula is:

[0035] ,

[0036] in, is the hidden state, is the state of memory cells;

[0037] The fully connected layer maps the output of LSTM to 128 dimensions. The output of the fully connected layer passes through the ReLU activation function, and the processing formula is:

[0038] ,

[0039] in, is the output of the activation function, is the weight matrix, is the bias vector;

[0040] Dropout layer, add Dropout after the fully connected layer, the processing formula of the Dropout layer is:

[0041] ,

[0042] Among them, the Dropout rate The value of .

[0043] Output layer, output action after activation function , the dimension is , the processing formula of the output layer is:

[0044] ,

[0045] in, Indicates the maximum value of the action, is the hyperbolic tangent activation function, is the weight matrix, is the bias vector;

[0046] The action value that will be generated Converted to action vector,

[0047] ,

[0048] in, Represents the equipment assignment decision, selecting the machine number to be used in the current processing stage; Indicates the number of workpiece batches, the number of workpieces processed on the machine; the specific conversion process is as follows:

[0049] S2.2.1. Set the action value Scale to the range [0,1];

[0050] S2.2.2. The number of batches of workpieces for:

[0051] ,

[0052] Machine Selection for:

[0053] ,

[0054] in, and is the original continuous value output by the action network;

[0055] Actual processing quantity for:

[0056] ,

[0057] in, is the maximum number of workpieces that can be produced in a single time step, Express The obtained value is rounded to the nearest integer;

[0058] S2.2.3. Formulate an indexing method for the machine.

[0059] ,

[0060] in, Indicates the number of machines available at the current processing stage; mod represents the modulo operation to ensure that the machine index is within the legal range;

[0061] Select a specific machine based on the index method;

[0062] ,

[0063] in, Represents a list of all machines in the current processing stage. The actual machine number is obtained through the index index.

[0064] In step S2.3, the status update includes order status update and machine status update;

[0065] The order status update includes the update of the remaining number of workpieces and the update of the processing stage. The remaining number of workpieces in the updated order is ,

[0066] ,

[0067] If the current processing stage All workpieces have been processed, that is, , the order enters processing stages;

[0068] When the machine status is updated, Selected Machine , it is necessary to update the start time and completion time of its processing tasks, as well as the available time of the machine:

[0069] Processing start time The calculation method is:

[0070] ,

[0071] in, Indicates orderi In the previous processing stage The completion time, if ,but =0; Indicates the processing stage Medium Machine Current available time;

[0072] Processing completion time The calculation method is:

[0073] ,

[0074] in, is the processing time, calculated as:

[0075] ,

[0076] in, It is an order i Product Type In the processing stage Basic processing time; It is the processing stage Medium Machine The machine speed factor.

[0077] In step S2.3, the execution result is rewarded through a reward function, which includes a step reward function and a round reward function;

[0078] The step reward function includes the processing time difference and the total energy consumption of the current step , where the processing time difference The calculation formula is:

[0079] ,

[0080] in, Indicates the current time after the current action is executed. Indicates the time before the current action is executed;

[0081] Total energy consumption of the current step The calculation formula is:

[0082] ,

[0083] in, Indicates the number of actions in this time step. l The energy consumption of each processing task, and the total energy consumption is the accumulation of the energy consumption of all processing tasks;

[0084] Step reward function The calculation formula is:

[0085] ,

[0086] in, is the processing time weight coefficient, is the energy consumption weight coefficient;

[0087] If no effective action is taken for all tasks in the current time step, that is, all processing quantities are zero, then:

[0088] ,

[0089] in, is a positive number greater than 10000;

[0090] For each order in a production round, if the actual completion time exceeds the deadline, the delay time is calculated, otherwise it is zero. The total delay time of the order is The calculation formula is:

[0091] ,

[0092] in, It is an order The actual completion time; It is an order Deadline:

[0093] Episode Reward Function The calculation formula is:

[0094] ,

[0095] in, Indicates the penalty coefficient for delay time; Represents the sum of step rewards in all steps in this production round.

[0096] In step S2.4, the new state vector obtained by state update in step S2.3 is and current action , input two value networks for value evaluation at the same time, and the two value networks output the value evaluation values ​​of the current action respectively, and take the smallest value evaluation value as the final evaluation Q value;

[0097] The structure of the value network includes:

[0098] Input layer, accepting states and actions The splicing dimension is ;

[0099] The first hidden layer is a fully connected layer with 256 hidden units. , the calculation formula of the first hidden layer is:

[0100] ,

[0101] in, represents the weight matrix, represents the bias vector;

[0102] The second hidden layer has 256 hidden units and the calculation formula for the second hidden layer is:

[0103] ,

[0104] in, represents the weight matrix, represents the bias vector;

[0105] Output layer, the calculation formula of this layer is:

[0106] ,

[0107] in, represents the weight matrix, represents the bias vector;

[0108] In step S2.5, the specific optimization and updating process is as follows:

[0109] S2.5.1. Experience replay: Use the experience replay buffer to store past experience values ;

[0110] S2.5.2, Sample batch data: Randomly sample a batch of data from the experience replay buffer;

[0111] S2.5.3. Calculate target action :

[0112] ,

[0113] in, represents the policy target network; represents the added noise; represents the noise clipping threshold;

[0114] S2.5.4. Calculate the target value Q_target:

[0115] ,

[0116] in, represents the discount factor; e represents the serial number of the value network; Represents the value target network;

[0117] S2.5.5. Update value network parameters:

[0118] for , the loss function of the value network for:

[0119] ,

[0120] in, Represents the set of parameters in the value network;

[0121] Continuously optimize the value network loss function In the process, the parameters of the value network are updated;

[0122] S2.5.6. Delayed update of policy network:

[0123] Each interval Step 2, update the policy network, the loss function of the policy network for:

[0124] ,

[0125] in, Represents the set of parameters of the policy network; represents the policy network;

[0126] Continuously optimize the policy network loss function In the process, the parameters of the strategy network are updated;

[0127] S2.5.7, Soft Update Target Network:

[0128] For the policy target network and value target network, perform soft update:

[0129] ,

[0130] in, represents the soft update coefficient; Represents online network parameters, including strategy network parameters and value network parameters; Indicates target network parameters, including strategy target network parameters and value target network parameters; Indicates the target network parameters obtained after soft update.

[0131] When the time step When a dynamic event occurs, do the following:

[0132] When machine maintenance is required, the machine to be repaired Set the availability status to unavailable and update the machine collection:

[0133] : ,

[0134] Among them, the symbol Represents the difference operation of sets, that is, removing certain elements from a set; Indicates the containing element A collection of Indicates the machine to be removed;

[0135] When an urgent order is inserted, add the urgent order to the order list , update the order quantity , and initialize the parameters of the new order;

[0136] When deviations occur in production execution, adjust the workpiece quantity of the affected orders , simulate workpiece damage, rework or quantity changes: ;

[0137] Save the scheduling results and current status before the dynamic event time point, re-extract the environment status, and generate new scheduling decisions based on the latest status, adjust resource allocation, integrate the new scheduling strategy after the dynamic event with the previous scheduling results, and form the final scheduling plan.

[0138] The present application also includes a system for executing the scheduling method of a batch mixed flow shop using the above-mentioned parallel variable speed machines, the system including a shop scheduling knowledge graph and a shop scheduling processing module;

[0139] Workshop scheduling knowledge graph module: used to connect to the graph database to achieve management, query and visualization of the graph database;

[0140] Workshop scheduling processing module: Integrates the workshop scheduling optimization algorithm model to process the order information queried and saved in the graph database, and outputs the corresponding scheduling result Gantt chart and plan form.

[0141] The beneficial effects of the present invention are:

[0142] (1) This application can achieve efficient scheduling of parallel variable speed machine batch production and maximize production efficiency. In particular, in the production process, the system can quickly identify and respond to dynamic events, effectively reduce production delays caused by equipment maintenance or emergency order insertion, and ensure the continuity and efficiency of the production process;

[0143] (2) This application achieves comprehensive integration and management of multi-source heterogeneous data in the production process by introducing knowledge graphs, breaking the data silos between departments in traditional manufacturing enterprises; various types of production data such as order information, process flow, workshop resources and scheduling results can be uniformly managed and represented through knowledge graphs, thereby improving data transparency and traceability; combined with automatic reasoning and data completion functions, the system can automatically generate accurate scheduling decisions, reduce human intervention, and avoid the inefficiency and error risks of manual data processing in traditional scheduling methods;

[0144] (3) This application realizes multi-objective optimization in scheduling, solving the problems of slow convergence speed and poor solution consistency of traditional scheduling algorithms in complex workshop environments; this application can reduce machine energy consumption while minimizing order delays and maximizing production efficiency, significantly reducing the operating costs of enterprises; especially when dealing with the scheduling problem of parallel variable-speed machine batch mixed flow workshop, the system can intelligently adjust the scheduling strategy according to the real-time data and dynamic events in actual production through the improved reinforcement learning algorithm, ensuring efficient use of resources and continuous optimization of scheduling plans;

[0145] (4) This application manages scheduling results through knowledge graphs, which simplifies the traditional scheduling plan that relies on paper documents to transmit, avoiding production delays caused by information lag or manual operation errors; the system can automatically generate detailed scheduling plans and display them to production operators in a visual way through Gantt charts or form files, facilitating rapid execution; at the same time, the scheduling results can also be directly sent to the equipment control layer through the SCADA system, realizing the production process control of CNC equipment, greatly reducing human intervention and execution errors;

[0146] (5) This application implements a real-time data sharing and feedback mechanism, solving the problems of data processing lag and untimely scheduling adjustment in various departments in the traditional scheduling system. By integrating multi-source data, when dynamic events occur in the workshop (such as equipment maintenance, production execution deviations, and new emergency orders, etc.), it can immediately feedback and adjust the scheduling plan, thereby improving the flexibility and response speed of scheduling. Through the combination of knowledge graphs and scheduling algorithms, production data can be seamlessly shared among departments, ensuring the rapid adjustment and optimization of production plans, effectively improving the overall production efficiency of the enterprise. BRIEF DESCRIPTION OF THE DRAWINGS

[0147] Figure 1 It is a flow chart for constructing the workshop scheduling knowledge graph;

[0148] Figure 2 It is the training flow chart of the workshop scheduling model. DETAILED DESCRIPTION

[0149] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0150] In the following description, specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in a variety of other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0151] The present application discloses a method for executing scheduling of a batch mixed flow shop with parallel variable speed machines, which comprises the following steps.

[0152] The first step is to build a workshop scheduling knowledge graph.

[0153] The ontology layer of the knowledge graph defines the concepts, relationships, and attributes in the entire graph. In order to construct the ontology layer, the basic structure of the workshop scheduling knowledge graph is designed to form a framework so that the knowledge involved in the production scheduling process can be organized and represented in a structured form, and finally the ontology layer is established in an ontology modeling tool such as protégé. The ontology layer in this application includes workshop resource ontology, process flow ontology, order information ontology, dynamic event ontology, and scheduling result ontology. By establishing several ontologies, different concepts in the production scheduling process are described respectively, and mutual relationships are established between ontologies. The specific implementation process of this step is described as follows: Figure 1 shown.

[0154] First, construct each ontology in the workshop scheduling knowledge graph separately.

[0155] (I) Define concepts, clarify the main concepts of the workshop scheduling knowledge graph of this application, including workshop resources, process flow, order information, dynamic events and scheduling results, and determine the classes and some instances in each ontology.

[0156] (ii) Establishing relationships: Determine the relationships between concepts, that is, determine the object attributes of classes and instances in each ontology.

[0157] For example, the material preparation stage has a processing machine: Material Preparation Line No. 1 Machine, Material Preparation Line No. 1 Machine has an operator, whose worker code is 11, and the machine speed coefficient of worker 11 is 1.1. In addition, the machine power consumption of Material Preparation Line No. 1 is 12. When establishing the relationship between the above-mentioned entities, it includes associating the processing stage with its corresponding machine, associating the machine with the code of its operator, associating the operator with the speed coefficient of operating the machine, and associating the machine with its power consumption index.

[0158] In the above manner, association relationships are established between classes and between instances.

[0159] (III) Attribute setting: Determine the specific information of the instance, that is, determine the data property (Data Property) of the specific instance corresponding to the class of each ontology. It is usually used to describe the specific information of the instance, such as the name of the operator, the speed factor of the machine, etc. This information is usually name or value, etc., which are all basic data types. For example, the data property "speed_factor_value" of the "11 speed factor" instance is "1.1".

[0160] By setting attributes, the specific characteristics of the instance can be described.

[0161] Second, build a unified ontology layer.

[0162] After completing the modeling of each ontology, the same classes or instances in different ontologies are unified into shared classes by sharing the same International Resource Identifier (IRI). By associating object attributes and data attributes, a consistent semantic network across ontologies is constructed to form a complete unified ontology to ensure data consistency and semantic intercommunication between different ontologies.

[0163] Third, define the association relationship between each class to realize the automatic reasoning and expansion of new knowledge in the workshop scheduling knowledge graph.

[0164] According to the concepts and attributes of different classes in the unified ontology layer, SWRL reasoning rules are written in protégé. The reasoning rules are used to define the association relationship between classes and support the automatic reasoning and expansion of new knowledge in the shop scheduling knowledge graph.

[0165] For example, when adding an order for a specific month, the month of a certain instance is "January". By formulating inference rules, the month instance is automatically associated with three order instances, namely "Order 0", "Order 1" and "Order 2".

[0166] Fourth, fill and complete the instance layer data.

[0167] Based on the constructed ontology layer, the inference rules are used to align the classes to which the instance layer data belongs, and the correlation between the instances is completed. The instance layer data in this application includes the processing machines, workers, process flow, order information and scheduling results of the production workshop, etc. The instance data is supplemented to the corresponding class to ensure that the instance layer data is complete and accurate.

[0168] Fifth, verify whether the functions of the unified ontology layer are correct.

[0169] After the instance layer data is filled and completed, based on the ontology layer structure and instance layer data, query rules can be written using query languages ​​such as SQWRL to verify whether the functions of the unified ontology layer are correctly implemented.

[0170] For example, when querying the basic information of orders in January, determine whether it can correctly output relevant information including the order, order name, order product quantity, and order deadline.

[0171] Sixth, import the unified ontology layer constructed through the above steps into the graph database.

[0172] In actual engineering applications, in order to more quickly apply the ontology and data in the knowledge graph and realize its visual presentation, a graph database is used to represent the knowledge graph and complete the storage and application of data, where the data is organized in the form of nodes (entities) and edges (relationships).

[0173] In this application, the ontology, relationships, attributes and other information in the unified ontology layer established through the above steps are exported as RDF files, the Cypher language and neosemantics import command are used to load the data, the data is mapped in the Neo4j graph database, and the basic structure of the production scheduling optimization knowledge graph is built.

[0174] Then, according to the ontology framework established in the previous steps, the instances that meet the requirements are mapped to the corresponding ontology. In order to store instance data from different sources, data can be obtained in a table format, and the corresponding ontology, instance filling and relationship completion of each column of data can be completed.

[0175] Seventh, verify whether the graph database functions correctly.

[0176] In this application, the functionality of the knowledge graph is verified through Cypher query statements to ensure the correctness of each node and its relationship in the graph database, and to lay the foundation for subsequent data filling and query.

[0177] For example, by querying all the production materials information related to the orders placed in January, the query function of the production materials entity including orders, workshops and processes can be verified.

[0178] Eighth, visualization of workshop scheduling knowledge graph.

[0179] Since the visualization capability of Neo4j graph database is limited, we extract graph data from Neo4j, use NetworkX to process the data, and then visualize the graph as an HTML page through PyVis. We also use Flask to provide Web services, thus realizing the complete process from data acquisition, processing to display, and providing an interactive workshop scheduling knowledge graph display platform.

[0180] The second step is to build and train the workshop scheduling model.

[0181] Conduct in-depth research on the production scheduling rules of the workshop and analyze the various constraints and optimization goals in actual production. The workpiece needs to go through multiple processing stages, each stage may have multiple machines with different processing speeds, and the workpiece can be processed and transported in batches. In addition, dynamic disturbance events may occur during the production process, such as machine maintenance, emergency orders, and production execution deviations.

[0182] Based on this, the workshop scheduling model constructed and trained in this step takes minimizing the maximum completion time, total machine energy consumption and order delay time as the optimization goal.

[0183] In this application, it is necessary to train the workshop scheduling model through many training rounds. In the process of each training round, several processing steps are included, that is, a complete training round is achieved through several processing steps. After a training round, the scheduling environment is reset and a new training round is started, thereby achieving iterative optimization of the scheduling model. In this embodiment, after 800 rounds of iteration, the optimization effect tends to be stable, the model has completed training, and the training model is saved for subsequent scheduling task calls.

[0184] The specific implementation process of this step is as follows: Figure 2 shown.

[0185] First, environmental status extraction.

[0186] Extract the current state vector from the scheduling simulation environment , including key information such as order number, processing stage, product type, remaining processing quantity, order deadline, etc. These states are the initial input data of the scheduling task and directly reflect the dynamic information of the current workshop.

[0187] In this application, the state space is described in detail by a six-dimensional array, the dimensions of which include the current time step , which represents the state t Processing steps, order number , current processing stage , order product type , the remaining number of workpieces , normalized order deadline This refined state representation method allows the model to accurately capture every detail of the production environment, including the operating status of the machine, the specific information of the product batch, as well as the processing capacity and the priority of urgent orders. By combining these dimensions, the model can evaluate all possible actions at each decision point to optimize the scheduling decision. That is:

[0188] ,

[0189] In the state space, the order deadline uses normalized data. That is, the order deadline is scaled to a suitable range to make it close to the numerical scale of other state variables to avoid numerical instability caused by excessive values. The calculation method of the normalized order deadline is:

[0190] .

[0191] in, is the actual deadline for the order, is the time scale factor.

[0192] Second, the state is input into the policy network model to generate actions.

[0193] Extraction status The input is fed into the policy network model Actor. The policy network combines the long short-term memory network LSTM module to process time series data, capture the impact of task sequence and dynamic events, and generate actions. The action includes the specific decision on the processing machine number assigned to the workpiece and the processing quantity.

[0194] The policy network Actor aims to Output corresponding action In order to handle the sequence data and long-term dependencies in the scheduling problem, the policy network Actor adopts the LSTM module and introduces Dropout in the network to prevent overfitting and improve generalization ability.

[0195] In this application, the network structure of the policy network Actor includes: an input layer, which accepts the state vector , the dimension is ; LSTM layer, including one LSTM layer with 256 hidden units; Fully connected layer: maps the output of LSTM to 128 dimensions through the fully connected layer; Dropout layer, adds Dropout after the fully connected layer, with a Dropout rate of ; Output layer, output action after activation function , the dimension is .

[0196] The LSTM layer is used for the input sequence The processing formula is:

[0197] ,

[0198] in, is the hidden state, Memory cell state.

[0199] The output of the fully connected layer passes through the ReLU activation function, and its processing formula is:

[0200] ,

[0201] in, is the output of the activation function, is the weight matrix, is the bias vector.

[0202] The processing formula of the Dropout layer is:

[0203] ,

[0204] In this embodiment, the Dropout rate The value of .

[0205] The processing formula of the output layer is:

[0206] ,

[0207] in, is the maximum value of the action, is the hyperbolic tangent activation function, is the weight matrix, is the bias vector.

[0208] Next, the generated action value Converted into actual operation instructions that can be understood by the environment, such as allocating processing machines, determining processing order and quantity.

[0209] In this application, the action of each training round is decomposed into several time steps, and the current state corresponding to each time step is specifically represented by the state array. A corresponding action vector is designed for each state vector. The action array includes all action vectors of each time step. The action vectors in this application include: device assignment decision , select the machine number used in the current processing stage; the number of workpiece batches , the number of workpieces processed on this machine, so:

[0210] .

[0211] In this application, in order to convert continuous action values To convert it into actual operation instructions that the environment can understand, the action value needs to be scaled and discretized. The specific steps are as follows.

[0212] 1. Set the action value Scaling to the range [0,1]:

[0213] For the workpiece processing quantity part, that is, the batch quantity of the workpiece:

[0214] ,

[0215] For machine selection:

[0216] ,

[0217] in, and is the raw continuous value output by the action network.

[0218] 2. Calculate the actual processing quantity :

[0219] ,

[0220] in, is the maximum number of workpieces that can be produced in a single time step, Express The obtained value is rounded to the nearest integer.

[0221] 3. Set an indexing method for the machine and select a specific machine based on the indexing method:

[0222] First, develop a specific indexing method for the machine:

[0223] ,

[0224] in: is the number of machines available at the current processing stage. It is multiplied by 100 to increase the value range, improve randomness and resolution. mod means modulus operation to ensure that the machine index is within the legal range.

[0225] Then, select the specific machine according to the above index method:

[0226] ,

[0227] in, It is a list of all machines in the current processing stage. The actual machine number is obtained through the index index.

[0228] Through the processing of the above steps, the continuous action values ​​are effectively mapped into actual production operation instructions.

[0229] Third, the actions generated in the previous steps are input into the scheduling simulation environment to perform actual operations. Rewards are fed back based on the execution results, and the status is updated. This step realizes the transformation of data from decision output to actual production.

[0230] In order to effectively simulate the scheduling environment of an actual production workshop, this application constructs a scheduling simulation environment based on the Markov Decision Process (MDP). The status is updated and the reward is fed back in the scheduling simulation environment.

[0231] In this application, status updates include order status updates and machine status updates.

[0232] The order status update includes the update of the remaining number of workpieces and the update of the processing stage. The updated remaining number of workpieces for the order is , Indicates order i The number of workpieces planned to be processed.

[0233] ,

[0234] If the current processing stage All workpieces have been processed, that is, , the order enters processing stage.

[0235] When the machine status is updated, Selected Machine , it is necessary to update the start time and completion time of its processing tasks, as well as the available time of the machine.

[0236] Processing start time The calculation method is:

[0237] ,

[0238] in: It is an order i In the previous processing stage completion time; if ,but =0; It is the processing stage Medium Machine The current available time.

[0239] Processing completion time The calculation method is:

[0240] ,

[0241] in, is the processing time, calculated as:

[0242] ,

[0243] in, It is an order i Product Type In the processing stage Basic processing time; It is the processing stage Medium Machine The machine speed factor.

[0244] The reward function setting in this application covers the following key aspects: (1) Efficiency penalty: In order to encourage the model to optimize the production process and reduce the total production cycle, a penalty is imposed on actions with longer production cycles. This means that any decision that leads to reduced production efficiency will receive a negative reward, prompting the model to find a more efficient scheduling solution. (2) Delay penalty: Timely delivery of orders is one of the important goals of production scheduling. Therefore, any action that causes order delays will be penalized to ensure that the model fully considers the order deadline when making decisions, thereby minimizing order delays. (3) Energy consumption penalty: Given the importance of energy efficiency in modern manufacturing, this application imposes negative rewards on actions with higher energy consumption through energy consumption penalties.

[0245] The reward function in this application includes step reward and round reward. The calculation method of step reward is as follows.

[0246] At each time step In each case, take the corresponding action After that, the scheduling simulation environment will feedback a step reward The reward takes into account the impact of this action on production time and energy consumption, including the processing time difference and the total energy consumption of the current step .

[0247] Processing time difference The calculation formula is:

[0248] ,

[0249] in, is the current time after the current action is executed, Is the time before the current action is executed.

[0250] Total energy consumption of the current step The calculation formula is:

[0251] ,

[0252] in, Indicates the number of steps in the action l The energy consumption of each processing task is the energy consumption of each processing task, and the total energy consumption is the accumulation of the energy consumption of all processing tasks.

[0253] In summary, the step reward function The calculation formula is:

[0254] ,

[0255] in, is the processing time weight coefficient, is the energy consumption weight coefficient.

[0256] If no valid actions are taken for all tasks in the current step, that is, all processing quantities are zero, a large negative reward is given to punish this behavior:

[0257] ,

[0258] in, is a sufficiently large positive number, for example, a positive number greater than 10000.

[0259] After a complete processing round is completed, the round reward can be calculated. At this time, the step reward needs to be adjusted to consider the impact of order delay time on the overall reward.

[0260] Total order delay time The calculation formula is:

[0261] ,

[0262] in, It is an order The actual completion time; It is an order Deadline.

[0263] For each order, if the actual completion time exceeds the deadline, the delay time is calculated, otherwise it is zero.

[0264] Episode Reward Function The calculation formula is:

[0265] ,

[0266] in: is the penalty coefficient for delay time; It is the sum of the step rewards of all steps in this round.

[0267] By adjusting , Different parameter combinations can result in four scheduling strategies: balanced mode, efficiency priority, energy saving priority, and delivery date priority, to achieve scheduling solutions under different requirements.

[0268] Fourth, evaluate the current action through the value network.

[0269] The state vector and current action , and input two value networks for value evaluation at the same time. The two value network models in this application include value network Critic1 and value network Critic2. The two value networks output the value of the current action respectively, among which value network Critic1 outputs the value of the current action , the value network Critic2 outputs the value of the current action ,Pick and The minimum value in is taken as the final evaluation Q value. By adopting two value networks, the deviation of value estimation can be reduced.

[0270] In this application, the value network is used to evaluate the value of a given state and action and guide the optimization of the strategy network. A dual network structure is adopted, that is, two independent value networks are introduced. and , to reduce the problem of strategy over-estimation and improve the accuracy of value estimation.

[0271] The structure of each value network includes: input layer, receiving state and actions The splicing dimension is ; The first hidden layer is a fully connected layer with 256 hidden units; the second hidden layer is also a fully connected layer with 256 hidden units; the output layer outputs the corresponding Q value .

[0272] For input , the calculation formula of the first hidden layer is:

[0273] ,

[0274] in, is the weight matrix, is the bias vector.

[0275] The calculation formula for the second hidden layer is:

[0276] ,

[0277] in, is the weight matrix, is the bias vector.

[0278] The calculation formula of the output layer is:

[0279] .

[0280] in, is the weight matrix, is the bias vector.

[0281] Fifth, optimize and update the strategy network and value network through the target network.

[0282] Through the target network, combined with the historical data stored in the experience replay buffer, including the training process , , , , generating the target value Q_target. In this application, three target networks are set, namely, the target strategy network Actor', the target value network Critic1', and the target value network Critic2'.

[0283] In this embodiment, a target network is maintained for each of the two value networks. and , used to calculate the target value Q_target and stabilize the training process. is the parameter set in the value network Critic1, is the parameter set in the value network Critic2.

[0284] The target value is used to guide the training of the value network, minimize the error between the current evaluation Q value and the target value Q_target, and ensure that the value evaluation is more accurate.

[0285] The specific optimization and updating process is described as follows.

[0286] 1. Experience replay.

[0287] Use the experience replay buffer to store past experience values , breaking data correlation and improving training stability.

[0288] (ii) Sampling batch data: Randomly sample a batch of data from the experience replay buffer.

[0289] 3. Calculating target actions .

[0290] ,

[0291] in, For strategic target networks; is the added noise. In this embodiment, the noise standard deviation Set to 0.2 for smoothing of target motion; is the noise clipping threshold. In this embodiment, Set to 0.443 to limit the amplitude of the noise.

[0292] 4. Calculate the target value Q_target.

[0293] ,

[0294] in, is the discount factor. In this embodiment, It is set to 0.999, emphasizing long-term benefits; e is the serial number of the value network; Represents the value target network.

[0295] 5. Update value network parameters.

[0296] for , the loss function of the value network for:

[0297] ,

[0298] in, is the parameter set in the value network. Continuously optimize the value network loss function In the process, the parameters of the value network are updated.

[0299] (vi) Delayed updates of policy networks.

[0300] Each interval Step 2: Update the policy network. The update frequency of the policy network is lower than that of the value network. By delaying the update of the policy network, the instability caused by excessive updates of the policy network is avoided, thus ensuring the stability of training.

[0301] In this embodiment, Set to 4 to delay the update of the policy network. The loss function of the policy network for:

[0302] ,

[0303] in, is the parameter set of the policy network; For the policy network. Continuously optimize the policy network loss function In the process, the parameters of the policy network are updated.

[0304] 7. Soft update of target network.

[0305] For the policy target network and value target network, perform soft update:

[0306] ,

[0307] in, is the soft update coefficient. Set it to 0.01 to speed up the update of the target network; Online network parameters, including strategy network parameters and value network parameters; The target network parameters include the strategy target network parameters and the value target network parameters; Indicates the target network parameters obtained after soft update.

[0308] Sixth, loop iteration.

[0309] After completing the optimization and update of the strategy network and the value network, repeat the first to fifth steps, that is, extract the next state from the scheduling environment , and use it as the input for the next round of production; generate new actions through the policy network , and execute it in the scheduling simulation environment, feedback rewards based on the execution results, and update the status; and once again optimize the policy network and value network.

[0310] During each iteration, the feedback reward for action execution and the new state , and is continuously stored in the experience replay buffer for subsequent training. The iteration process continues until the number of remaining workpieces in the state vectors of all environmental feedback is 0, thus achieving a complete scheduling optimization process within a production round.

[0311] At each time step When all workpieces of all orders have been processed, the remaining quantity of all workpieces will be determined. satisfy:

[0312] ,

[0313] Then the scheduling process of a production round ends.

[0314] Seventh, schedule the initialization of the simulation environment.

[0315] After the scheduling optimization process of a production round is completed, before starting another new production round, the scheduling simulation environment needs to be initialized. The initialization work mainly includes order initialization, machine initialization and state initialization.

[0316] Among them, order initialization includes: setting the order quantity Initialize the number of pieces for each order , Order Type , Deadline And other information.

[0317] Machine initialization includes: , set the number of machines ; For each machine Set the speed factor and power consumption ; Initialize the machine's available time to 0.

[0318] State initialization includes: current time step ; Processing stage of the order ; Construct the initial state vector

[0319] Repeat the above steps 1 to 7 to train the workshop scheduling model through the scheduling optimization process of several production rounds.

[0320] The third step is to save the workshop scheduling training model after the workshop scheduling training model is trained. The work tasks are retrieved from the workshop scheduling knowledge graph and input into the workshop scheduling training model. The optimal workshop scheduling result is generated through the workshop scheduling training model.

[0321] The workshop scheduling results can be displayed in the form of Gantt charts or tables to intuitively display the time schedule of production tasks, and can also be directly sent to the equipment layer through the SCADA system to automatically execute production tasks. Through the above steps, a complete closed loop process from data extraction, decision generation to result execution is achieved.

[0322] When dynamic events occur during the execution of the initial scheduling results of the workshop, such as urgent order insertion, equipment maintenance, etc., the system saves the scheduling results and current status before the dynamic event time point, re-extracts the environment status, and generates new scheduling decisions based on the latest status to adjust resource allocation. The new scheduling strategy after the dynamic event is integrated with the previous scheduling results to form a complete final scheduling plan.

[0323] When the time step When a dynamic event occurs, do the following:

[0324] (I) When machine maintenance is required, the machine to be repaired Set the availability status to unavailable and update the machine collection:

[0325] : ,

[0326] Among them, the symbol Represents the difference operation of sets, that is, removing certain elements from a set. Represents a file that contains only elements The collection of machines to be removed .

[0327] (ii) When an urgent order is inserted, add the urgent order to the order list , update the order quantity , and initialize the parameters of the new order wait.

[0328] (III) When deviations occur in production execution, adjust the number of workpieces for the affected orders , simulate workpiece damage, rework or quantity changes, etc.: .in, Indicates the number of workpieces damaged or reworked.

[0329] The present application also includes a parallel variable-speed machine batch mixed flow shop scheduling execution system, which includes a shop scheduling knowledge graph and a shop scheduling processing module.

[0330] Among them, the workshop scheduling knowledge graph module is used to connect to the graph database to realize the management, query and visualization of the graph database.

[0331] The workshop scheduling processing module is used to integrate the workshop scheduling optimization algorithm model, process the order information queried and saved in the graph database, and output the corresponding scheduling result Gantt chart and plan form.

[0332] The above is a detailed introduction to the parallel variable-speed machine batch mixed flow shop scheduling execution method and system provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method and core ideas of the present invention. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, the present invention can also be improved and modified in a number of ways, and these improvements and modifications also fall within the scope of protection of the claims of the present invention. The above description of the disclosed embodiments enables professionals and technicians in this field to implement or use the present invention. Various modifications to these embodiments will be obvious to professionals and technicians in this field, and the general principles defined in this article can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown in this article, but will conform to the widest range consistent with the principles and novel features disclosed herein.

Claims

1. A method for scheduling and executing a parallel variable speed machine batch mixed flow shop, characterized in that: The following steps are involved: S1. Build a knowledge graph for workshop scheduling; S2. Build and train the workshop scheduling model; S3. Save the trained workshop scheduling training model, retrieve the work tasks from the workshop scheduling knowledge graph, and input them into the workshop scheduling training model to generate the optimal workshop scheduling results, and execute the production tasks according to the workshop scheduling results; Step S2 includes the following specific execution process: S2.1, Environmental state extraction; S2.2, input the state into the policy network to generate actions; S2.3, input the action generated in step S2.2 into the scheduling simulation environment, perform the actual operation, feedback the reward according to the execution result, and update the status; S2.4, evaluate the current action through the value network, obtain the evaluation Q value, and optimize and update the strategy network; S2.5, generate a target value Q_target through the target network, and optimize and update the value network by minimizing the error between the evaluation Q value and the target value Q_target; S2.6, repeat steps S2.1 to S2.5, and continuously optimize the policy network and the value network during the iterative process, and the iterative process continues until the number of remaining workpieces in the state vector is 0, thus achieving scheduling optimization within one training round; S2.7, after the scheduling optimization process of a training round is completed, before starting another new training round, the scheduling simulation environment is initialized; Repeat the above steps S2.1 to S2.7 to train the workshop scheduling model through several training rounds of scheduling optimization process.

2. The method for scheduling and executing a parallel variable speed machine batch hybrid flow shop according to claim 1, characterized in that: Step S1 includes the following specific execution process: S1.

1. Construct each ontology in the workshop scheduling knowledge graph, including workshop resource ontology, process flow ontology, order information ontology, dynamic event ontology and scheduling result ontology, determine the classes and some instances in each ontology, determine the association relationship between classes, between instances and between classes, and determine the specific information of the instance; S1.

2. Construct a unified ontology layer: By sharing the same International Resource Identifier (IRI) in the same classes or instances in different ontologies, the same classes or instances between these ontologies are unified into shared classes; S1.3, set the relationship between each class; S1.4, filling and completing instance layer data: Based on the constructed ontology layer, align the class to which the instance layer data belongs through the association relationship set in step S1.3, and complete the correlation relationship between each instance; S1.

5. Verify whether the function of the unified ontology layer is correct. If correct, import the unified ontology layer into the graph database; S1.

6. Verify whether the function of the graph database is correct. If correct, realize the visualization of the workshop scheduling knowledge graph.

3. The method for scheduling and executing a parallel variable speed machine batch hybrid flow shop according to claim 1, characterized in that: In step S2.1, the current state vector is extracted from the scheduling simulation environment , ， in, represents the current time step, Indicates the order number. Indicates the current processing stage. Indicates the order product type. Indicates the remaining number of workpieces. represents the normalized order deadline, ， in, Indicates the actual deadline of the order. represents the time scale factor.

4. The method for scheduling and executing a parallel variable speed machine batch hybrid flow shop according to claim 1, characterized in that: In step S2.2, the network structure of the policy network includes: Input layer, accepts the state vector , the dimension is ; LSTM layer, including one LSTM layer, the number of hidden units is 256, the LSTM layer is for the input sequence The processing formula is: ， in, is the hidden state, is the state of memory cells; The fully connected layer maps the output of LSTM to 128 dimensions. The output of the fully connected layer passes through the ReLU activation function, and the processing formula is: ， in, is the output of the activation function, is the weight matrix, is the bias vector; Dropout layer, add Dropout after the fully connected layer, the processing formula of the Dropout layer is: ， Among them, the Dropout rate The value of ; Output layer, output action after activation function , the dimension is , the processing formula of the output layer is: ， in, Indicates the maximum value of the action, is the hyperbolic tangent activation function, is the weight matrix, is the bias vector; The action value that will be generated Converted to action vector, ， in, Represents the equipment assignment decision, selecting the machine number to be used in the current processing stage; Indicates the number of workpiece batches, the number of workpieces processed on the machine; the specific conversion process is as follows: S2.2.1、The action value Scale to the range [0,1]; S2.2.

2. The number of batches of workpieces for: ， Machine Selection for: ， in, and is the original continuous value output by the action network; Actual processing quantity for: ， in, is the maximum number of workpieces that can be produced in a single time step, Express The obtained value is rounded to the nearest integer; S2.2.

3. Formulate an indexing method for the machine. ， in, Indicates the number of machines available at the current processing stage; mod represents the modulo operation to ensure that the machine index is within the legal range; Select a specific machine based on the index method; ， in, Represents a list of all machines in the current processing stage. The actual machine number is obtained through the index index.

5. The method for scheduling and executing a parallel variable speed machine batch hybrid flow shop according to claim 1, characterized in that: In step S2.3, the status update includes order status update and machine status update; The order status update includes the update of the remaining number of workpieces and the update of the processing stage. The remaining number of workpieces in the updated order is , Indicates order i The number of workpieces planned to be processed, ， If the current processing stage All workpieces have been processed, that is, , the order enters processing stages; When the machine status is updated, Selected Machine , it is necessary to update the start time and completion time of its processing tasks, as well as the available time of the machine: Processing start time The calculation method is: ， in, Indicates order i In the previous processing stage The completion time, if ,but =0; Indicates the processing stage Medium Machine Current available time; Processing completion time The calculation method is: ， in, is the processing time, calculated as: ， in, It is an order i Product Type In the processing stage Basic processing time; It is the processing stage Medium Machine The machine speed factor.

6. The method for scheduling and executing a parallel variable speed machine batch hybrid flow shop according to claim 1, characterized in that: In step S2.3, the reward function is used to provide feedback reward for the execution result, and the reward function includes a step reward function and a round reward function; The step reward function includes the processing time difference and the total energy consumption of the current step , where the processing time difference The calculation formula is: ， in, Indicates the current time after the current action is executed. Indicates the time before the current action is executed; Total energy consumption of the current step The calculation formula is: ， in, Indicates the number of actions in this time step. l The energy consumption of each processing task, and the total energy consumption is the accumulation of the energy consumption of all processing tasks; Step reward function The calculation formula is: ， in, is the processing time weight coefficient, is the energy consumption weight coefficient; If no effective action is taken for any task in the current time step, that is, all processing quantities are zero, then: ， in, is a positive number greater than 10000; For each order in a production round, if the actual completion time exceeds the deadline, the delay time is calculated, otherwise it is zero. The total delay time of the order is The calculation formula is: ， in, It is an order The actual completion time; It is an order Deadline: Episode Reward Function The calculation formula is: ， in, Indicates the penalty coefficient for delay time; Represents the sum of step rewards in all steps in this production round.

7. The method for scheduling and executing a parallel variable speed machine batch hybrid flow shop according to claim 1, characterized in that: In step S2.4, the state vector and current action , input two value networks for value evaluation at the same time, and the two value networks output the value evaluation values ​​of the current action respectively, and take the smallest value evaluation value as the final evaluation Q value; The structure of the value network includes: Input layer, accepting states and actions The splicing dimension is ; The first hidden layer is a fully connected layer with 256 hidden units. , the calculation formula of the first hidden layer is: ， in, represents the weight matrix, represents the bias vector; The second hidden layer has 256 hidden units and the calculation formula for the second hidden layer is: ， in, represents the weight matrix, represents the bias vector; Output layer, the calculation formula of this layer is: ， in, represents the weight matrix, represents the bias vector; In step S2.5, the specific optimization and updating process is as follows: S2.5.

1. Experience replay: Use the experience replay buffer to store past experience values ; S2.5.2, Sample batch data: Randomly sample a batch of data from the experience replay buffer; S2.5.

3. Calculate target action : ， in, represents the policy target network; represents the added noise; represents the noise clipping threshold; S2.5.

4. Calculate the target value Q_target: ， in, represents the discount factor; e represents the serial number of the value network; Represents the value target network; S2.5.

5. Update value network parameters: for , the loss function of the value network for: ， in, Represents the set of parameters in the value network; Continuously optimize the value network loss function In the process, the parameters of the value network are updated; S2.5.

6. Delayed update of policy network: Each interval Step 2, update the policy network, the loss function of the policy network for: ， in, Represents the set of parameters of the policy network; represents the policy network; Continuously optimize the policy network loss function In the process, the parameters of the strategy network are updated; S2.5.7, Soft Update Target Network: For the policy target network and value target network, perform soft update: ， in, represents the soft update coefficient; Represents online network parameters, including strategy network parameters and value network parameters; Indicates target network parameters, including strategy target network parameters and value target network parameters; Indicates the target network parameters obtained after soft update.

8. The method for scheduling and executing a parallel variable speed machine batch hybrid flow shop according to claim 1, characterized in that: When the time step When a dynamic event occurs, do the following: When machine maintenance is required, the machine to be repaired Set the availability status to unavailable and update the machine collection: ： ， Among them, the symbol Represents the difference operation of sets, that is, removing certain elements from a set; Indicates the containing element A collection of Indicates the machine to be removed; When an urgent order is inserted, add the urgent order to the order list , update the order quantity , and initialize the parameters of the new order; When deviations occur in production execution, adjust the workpiece quantity of the affected orders , simulate workpiece damage, rework or quantity changes: , Indicates the number of damaged or reworked workpieces; Save the scheduling results and current status before the dynamic event time point, re-extract the environment status, and generate new scheduling decisions based on the latest status, adjust resource allocation, integrate the new scheduling strategy after the dynamic event with the previous scheduling results, and form the final scheduling plan.

9. A system using the method for executing the scheduling of a parallel variable speed machine batch hybrid flow shop according to any one of claims 1 to 8, characterized in that: Includes workshop scheduling knowledge graph and workshop scheduling processing module; Workshop scheduling knowledge graph module: used to connect to the graph database to achieve management, query and visualization of the graph database; Workshop scheduling processing module: Integrates the workshop scheduling optimization algorithm model to process the order information queried and saved in the graph database, and outputs the corresponding scheduling result Gantt chart and plan form.

Images