Computer-implemented method that, in execution, plans a production schedule for a specific production process in a production system and controls and / or regulates the production system according to the production schedule, and computer program for a production planning and control device of a production system

Abstract

A computer-implemented method that, in execution, plans a production schedule for a specific production process (P3) in a production system and controls and / or regulates the production system according to the production schedule, the method comprising the steps of: inserting a graph neural network (4) trained to minimize a cost function of a production process, comprising at least one or more classification layers, wherein the trained graph neural network (4) receives the generated graphs (G) of the existing production processes as input and transformed graphs (G) as output, and the classification layers receive the transformed graph (G) as input and provide a probability distribution over production sequence lots as output using the knowledge transfer from the existing production processes (P1, P2) to the specific production process (P3) (V5);Input of the created digital production model (M4) for the specific production process (P3) into a production planning and control device of the production system and control and / or regulation of the production system by means of the production planning and control device (V7).;

Description

[0001] The invention relates to a computer-implemented method that, when implemented, plans a production plan for a specific production process in a production system and controls and / or regulates the production system according to the production plan. Furthermore, the invention relates to a computer program for a production planning and control device of a production system.

[0002] The following definitions apply to the entire content of the disclosure.

[0003] During production, products, including tangible goods and services, are created based on production factors, including materials and resources. A multi-stage production process or a multi-stage production system can comprise one or more production stages. The production system is a system in which something is produced, for example a business, a factory, a workshop or one or more production lines. The production system can also comprise an IT system for the design and monitoring of production. The production stages can be carried out sequentially or in parallel. The production stages can each comprise one or more machines or production lines. Buffers, for example material buffers, or transport systems can be located between the production stages. Methods for production planning and / or control are known in the prior art.

[0004] Production in modern industrial manufacturing facilities today is often dependent on a multitude of external factors. These include globally interconnected supply chains, a high number of product variants, an unmanageable volume of purchased parts, and highly dynamic customer orders. The influence of these factors creates a highly volatile production environment, making manual production planning extremely difficult, if not impossible.

[0005] Production planning encompasses strategic considerations and preparations of all key variables of a company's manufacturing process or work procedure for a future period before it begins. For example, production planning encompasses the operational, temporal, quantitative, and spatial planning and management of all processes necessary for the production of goods and products.

[0006] The production sequence is a sequence of production batches per machine or production line. A production batch, also called a lot (production), batch, or lot, comprises the type of material to be produced and / or the number of parts to be produced or the required quantity. The production process is regulated and / or controlled based on the production sequence.

[0007] In this context, computer-aided solutions make it possible to create optimized plans in the shortest possible time.

[0008] The subject matters disclosed in DE 10 2020 203 716 A1 and DE 10 2020 203 718 A1, in particular the methods and systems disclosed therein, are called adaptive production optimization, abbreviated APO, and are part of the present disclosure by this explicit reference.

[0009] In particular, optimization approaches based on reinforcement learning using graph neural networks, abbreviated GRPO, as disclosed in German application 10 2022 211 446.9, which is incorporated into the present disclosure by this explicit reference, show great potential. GRPO uses a production simulation and a database with historical production data records of any multi-stage production process. This data is used by machine learning to train a specific production model for the process. The trained model can be used to generate highly optimized assembly sequences in real time for new data from the same production process. GRPO thus enables rapid response to disruptions in the supply chain and consequently saves significant amounts of time, money, and resources.

[0010] A disadvantage of this solution is the extremely high time and computational effort required to train such a model. This has a significant impact on the rollout and scaling of such methods across different production areas, as each new training requires a significant amount of time and thus incurs high costs.

[0011] The object of the invention was to find out how the GRPO disclosed in the German application 10 2022 211 446.9 can be improved, in particular how time and / or computational effort can be reduced in order to control and / or regulate a production system in an optimized manner.

[0012] The subject matter of the independent claim and the subordinate claims each solves this problem. Advantageous embodiments of the invention emerge from the definitions, the subclaims, the drawings, and the description of preferred embodiments.

[0013] The invention is first summarized for the sake of clarity. All features mentioned therein, individually and in combination with one another, are part of the inventive approach. The features and definitions mentioned in this summary apply in combination and in connection with the claimed subject matter.

[0014] The solution approach disclosed here recognized that specialized production models of a production process can be transferred to other related problems through knowledge transfer. To solve the aforementioned problem, knowledge transfer learning was implemented by training a graph neural network. The graph neural network trained in this way makes it possible, in practice, to determine an optimized production sequence by minimizing a cost function of the production process and to control and / or regulate the production system according to this production sequence. This makes it possible to significantly reduce the required computational effort while simultaneously achieving high accuracy, saving a large portion of costs and computing resources, and thus training models more sustainably.

[0015] A comprehensive system, called the Graph Neural Network and Reinforcement Learning-based Production Optimizer (GRPO), is provided for the detailed planning of a complex, particularly a complex multi-stage, production process. The GRPO is continuously improved using a machine learning method based on reinforcement learning.

[0016] The core of the GRPO is a graph neural network (GNN), which determines the production sequence as output based on the state of the production simulation, i.e., the current production status. The GRPO minimizes a cost function that can be variably adjusted by a user, for example, a human controller of the production processes.

[0017] In graph theory, a graph is an abstract structure that represents a set of objects along with the connections between these objects. The mathematical abstractions of the objects are called nodes of the graph. The pairwise connections between nodes are called edges. Edges can be directed or undirected. The preceding three sentences are part of a Wikipedia text under a CC-BY-SA license. In the GRPO, all entities of the production state—including production sections, materials, and production lines—are encoded as nodes, and all relations, such as predecessor material, successor buffer, and line capability, are encoded as edges.

[0018] A GNN is an artificial neural network that operates on graph data. The transformations performed by the network preserve the symmetry of the graph, meaning the transformations are invariant under permutations. The network can perform transformations on nodes, edges, and / or the global context of the graph. The network comprises one or more layers. Each layer receives a graph as input and produces a new graph as output. Nodes, edges, and / or global properties of the input graph are updated in each layer using graph-attribute-separated functions, resulting in a new graph that has the same connectivity as the input graph.

[0019] The current production status can include one or more production parameters. Furthermore, the production status can include requirements, such as customer requirements, such as which material is in demand at what time or a weighting of the requirements. Furthermore, the production status can include production layout, material flows, and / or process times.

[0020] The cost function can, for example, consist of a combination of total delay time of requirements, total setup time, utilization / output and minimum stock levels of certain final or intermediate products.

[0021] A further advantage is that it achieves a high degree of adaptability to the respective production area. GRPO continuously improves through its reinforcement learning approach.

[0022] Another advantage is the high degree of variability. By using the graph structure and the GNN, the trained network is able to use relations and dependencies instead of hard-coded features. This means that the network can achieve good results without retraining, even when production changes occur, such as the addition or modification of materials and / or production lines.

[0023] The inventive solution to the above-mentioned problem is provided by the following aspects: According to one aspect, the invention provides a computer-implemented method which, when implemented, plans production planning for a specific production process in a production system and controls and / or regulates the production system according to the production planning. At the start of the method, no digital production model exists for the specific production process. By implementing the method, a new digital production model is created for the specific production process, based on which the specific production process can then be controlled and / or regulated in an optimized manner. By means of the knowledge transfer from existing production processes carried out according to the method, the method enables the creation of production models for the detailed planning of new, multi-stage production processes.The new production model has higher performance compared to existing digital production models thanks to knowledge transfer, while at the same time significantly reducing processing times.

[0024] In one step of the process, digital production models are imported from a first database. The first database is a model zoo database in which existing digital production models are managed, for example, using special data management software. A digital production model creates production sequences for existing production processes in a computer-implemented simulation of the production system based on sensor-recorded data on production states. The production system includes sensors that record the production states. A production state includes production requirements, for example, delivery times for specific orders. The production requirements can be recorded using time sensors, for example. The production state can also include production parameters, such as employee situation, machine capabilities, and / or material availability.The production parameters can be recorded using camera sensors, for example. The camera sensors are located, for example, along production lines. The production status can also include a virtual representation of the production process, such as production layout, material flows, process times, and / or expected deliveries. The virtual representation can be recorded using virtual sensors, for example. When executing the computer-implemented simulation, the data recorded by sensors during the creation of the production sequences is evaluated.

[0025] By evaluating the sensory data acquired during the execution of the computer-implemented simulation, an interaction with external physical reality occurs, at least at the input level. The new digital production model created in the simulation predicts production sequences for the specific production process and regulates and / or controls it. Thus, the computer-implemented simulation makes a technical contribution.

[0026] In a further step of the method, input data on production states of the specific production process is entered via a human-machine interface or a machine-machine interface. Based on this input data, the new digital production model is created according to the following process steps. Through knowledge transfer, this production model is specialized for the specific production process. Via the human-machine interface, a human user, for example a manufacturing expert or production planning expert, can enter a data set or data sets on production states of the specific production process. Via the machine-machine interface, for example a data transfer interface, one or more data sets on production states of the specific production process, which can be generated, for example, using a database management device, can be entered.

[0027] In a further step of the process, the digital production models are analyzed for similarity between the production states of the existing production processes and the input data on the production states of the specific production process. Based on this analysis, one or more of the digital production models that exceed a predefined similarity threshold are identified for knowledge transfer from the existing production processes to the specific production process. The analysis and / or determination is carried out using an evaluation device. If a predefined similarity threshold is exceeded, the production models are related. By transferring knowledge from related production models during the computer-implemented simulation, better results are found more quickly. This results in a fewer number of iterations required to create a new, high-quality digital production model.Knowledge transfer is beneficial, for example, for all new production processes for which there is not yet sufficient data available for specialized production models.

[0028] Knowledge transfer at the production model level also enables knowledge to be exchanged between different production systems, for example, those located at different production sites, or between different companies, without having to exchange the underlying production data. This enables federated learning, in which a production model is generated for different production systems. Each participating production system has its own local production data sets, which do not need to be exchanged with other production systems at other locations. The process does not require a central production data set.

[0029] According to one aspect, the similarity of the digital production models is analyzed using a model analyzer. The model analyzer can be a human expert, a software module, or a hardware module. The model analyzer can provide a set of strategies for comparing the digital production models and identifying suitable candidates for knowledge transfer.

[0030] In a further step of the process, graphs are determined for the production states of the specific digital production model(s). Each production state of a production process within the production system, along with subsequent production batches, is represented as a graph. Entities of the production process, including production sections of the production system, production lines, and / or materials, are represented as nodes, and relations between the entities, including predecessor material, successor buffers, and / or line capability, are represented as edges. This creates the data structure of the graph for the production states. This data structure is optimized for evaluation in the subsequent steps of the process using the graph neural network.

[0031] Using a graph neural network trained to minimize a cost function of a production process, comprising at least one or more classification layers, wherein the graph neural network receives the generated graphs of the existing production processes as input and transformed graphs as output, and the classification layers receive the transformed graph as input and provide a probability distribution over production sequence lots as output using the knowledge transfer from the existing production processes to the specific production process.

[0032] The graph neural network can be a so-called message-passing graph neural network, which uses message-passing technology to update each node using information contained in every neighboring node. This means that each production section of the production system, production line, and / or material buffer in the graph neural network receives information from previous production sections, production lines, and / or material buffers in the production sequence. Furthermore, information about the relations between the nodes embedded in the edges of the graph can also be passed on accordingly. The graph neural network can also be a so-called graph convolutional neural network, which applies mathematical convolution to the embeddings of the nodes and / or edges of the graph.The graph neural network can also be a so-called graph attention neural network, in which attention technology is used to weight the importance of neighboring nodes in the graph during the information transfer phase, i.e., during message passing. An attention value is calculated for each pair of nodes, which is used to weight the individual nodes.

[0033] The graph neural network can be trained on training data for the inference of a production sequence. The training data comprises graphs that represent production states together with production sequence batches. Production sections of the production system, production lines, and / or materials are represented as nodes, and relations between the entities, including predecessor material, successor buffers, and / or line capability, are represented as edges. To generate this training data, the production process is simulated using computer implementation. An initial state of the simulation is a production state based on an initial production sequence. Based on this, production sequence batches are determined that are available at a simulation time in which a production line of the production system has processed the production sequence batches assigned to it.At this simulation time, a Monte Carlo tree search is performed, obtaining a probability distribution across these production sequence batches. A production sequence batch is then randomly selected at this simulation time according to the probability distribution. The preceding steps are repeated at each subsequent simulation time, starting from the simulation state obtained from the production sequence extended by this production sequence batch, until a predefined simulation termination criterion is reached. Upon reaching the simulation termination criterion, a relative value is determined that relates a first result value obtained from the simulation data, which characterizes the quality of the simulated production process, to a second result value, which characterizes the quality of the production process.In an adaptive production optimization, in the real production process, material requirements in the production stages are prioritized depending on an influence on an optimization of a cost function of the production system, one of the material requirements is selected in the order of prioritization, at least one required quantity and / or a required time of materials in preceding production stages is adjusted to fulfill the material requirement and the materials and the respectively adjusted required quantity and / or required time are reserved, another of the material requirements is selected and the preceding steps are repeated until the materials and the respectively adjusted required quantities and / or required times are reserved for all of the prioritized material requirements; whereby a production sequence is obtained, and the second result value for the obtained production sequence is determined.Subsequently, the graph representing the state of the production process at the simulation termination point is labeled with the relative value and the probability distribution obtained at this simulation point using Monte Carlo tree search. By repeating the previous steps, appropriately labeled graphs are obtained, which form the training data.

[0034] The graph neural network is trained on this training data. At the end of the training, the trained graph neural network used in the method disclosed here is obtained. The graph representing the state of the production process at a current simulation time point along with the production sequences is input into the graph neural network, and a transformed graph is obtained in a forward path of the graph neural network. One or more first classification layers of the graph neural network determine a probability distribution over the production sequences from the transformed graph, and one or more second classification layers of the graph neural network determine a quality value characterizing the quality of the production process at the respective simulation time point.The production sequence lot with the highest probability value is then selected from the probability distribution, and the preceding steps are repeated at each subsequent simulation time point, starting from the simulation state obtained from the production sequence each time extended by this production sequence lot, until a predefined termination criterion for the simulation is reached. When the simulation termination is reached, a first difference between the quality value and a relative value of one of the labeled graphs is determined, and a second difference between the probability distribution obtained using the graph neural network and a probability distribution of the labeled graph obtained using a Monte Carlo tree search is determined. In a backward path of the graph neural network, the first and second differences are error-fed back through the graph neural network.

[0035] In another aspect, the graph neural network with the best inference performance is replaced by the currently trained graph neural network to generate new graphs. In this context, this is called reinforcement learning.

[0036] In a further step of the process, the production sequence lot with the highest probability value is selected from the probability distribution for the specific production process. The steps of generating graphs and inserting the trained graph into the neural network are repeated starting from a state of a computer-implemented simulation of the specific production process obtained from the production sequence each time extended by this production sequence lot. This iteration continues until a predefined termination criterion for the computer-implemented simulation is reached. The each extended production sequence, which is optimized with respect to the cost function, changes the specific production process.

[0037] A digital production model is created for the specific production process. This means that a specific digital production model is obtained for the specific production process using the graph neural network.

[0038] The trained graph neural network is deployed on a high-performance computing platform. The high-performance computing platform has specialized hardware for processing the trained graph neural network and at least one graphics processor. Through knowledge transfer, the new digital production model is advantageously achieved very quickly while saving computing resources.

[0039] In a further step of the process, the created digital production model for the specific production process is entered into a production planning and control device of the production system, and the production system is then regulated and / or controlled using the production planning and control device. This allows the production system to be adapted and optimized, even in a volatile production environment that depends, for example, on interconnected supply chains, a high number of product variants, a complex quantity of purchased parts, and highly dynamic customer requests.

[0040] According to a further aspect, the invention provides a computer program for a production planning and control device of a production system. The computer program comprises instructions that cause a high-performance computing platform to execute the steps of the method disclosed herein when the computer program is loaded or executed by the high-performance computing platform.

[0041] The method and / or the computer program can be carried out locally in a production system or remotely in a cloud environment, wherein the production system comprises an IT infrastructure via which data exchange with the cloud is realized and carried out.

[0042] In one embodiment, the created digital production model is applied to new data from the same specific production process, generating production planning in real time. This advantageously allows highly optimized production plans to be generated in real time based on new data. This allows for very rapid response to supply chain disruptions, thus saving resources, time, and money.

[0043] In another embodiment, the created digital production model is exported to the first database. This allows the first database to be iteratively expanded with new digital production models.

[0044] In a further embodiment, metadata is stored in the first database. The metadata is used to identify and compare the digital production models with each other. The metadata includes at least the name of the production model, the version of the production model, the creation date of the production model, the model type of the production model, a reference to similar production models, a reference to input data used, a reference to training data, and / or past performance data of the production models that indicate the performance of the digital production models on test or validation data. This allows the digital production models stored in the first database to be advantageously managed and compared. Model types include, for example, a best-fit model, a consensus model, and / or a transferred model.

[0045] In a further embodiment, the one of the digital production models intended for knowledge transfer from the existing production process to the specific production process is determined based on past performance data indicating the performance of the digital production model on test or validation data. For this purpose, the digital production models stored in the first database can be applied to perform planning using a suitable validation dataset. The models with the best planning results are then selected for further processing. Or, the several digital production models intended for knowledge transfer from the existing production processes to the specific production process are combined into a consensus model.

[0046] According to one aspect, the consensus model is formed by using several of the digital production models together in the simulation to select the next production sequence lot. To do this, each production model is first applied individually to find the best value of the production sequence. Then, the production sequence lot that receives the most votes according to a majority vote is selected. Each production model can have one or more votes to introduce a weighting for the majority vote.

[0047] According to another aspect, the consensus model is formed by combining the production models into a new production model, which makes the sole decision. This production model is also called a mixture model. For this purpose, individual production sequences, also called scheduling policies (policies for short), can be combined into a common policy, for example, using a weighted average: Let P i (X) a policy distribution of production models i and λ i the corresponding weight, then the common policy P G (X): PG(X)=∑i=1nλi∑inλiPi(X).

[0048] In a further embodiment, the digital production models are created from the first database by the trained graph neural network. For this purpose, data on the existing production processes is stored in a second database. Data is selected from the second database using a database management device. The trained graph neural network receives the selected data in graph form as input. By processing the selected data, the graph neural network creates the digital production model. The created digital production model is exported to the first database.

[0049] According to one aspect, the database management device comprises a data selector. The data selector can be a software or hardware module or a human-machine interface through which a human expert can provide input. The data selector analyzes the second database to select one or more data sets to be used to create or optimize a production model. Different metrics can be applied for this purpose, for example, multivariate statistical methods, such as principal component analysis, or dimensionality reduction to a latent feature space using encoder-decoder techniques, such as autoencoders. The metrics are used to analyze the similarities between the different production processes.For example, the metadata of the various production processes, which can be obtained via data exchange with the first database or stored directly in the second database, can be compared. The more similar these data sets are, the better suited the corresponding data sets are for creating or improving a new digital production model using the trained graph neural network.

[0050] The data from the second database can be historical production records of any multi-stage production process.

[0051] In a further embodiment, the selected data from the second database is combined, and a digital production model is created based on the combined data. The combined data can be achieved using a software or hardware module, also called a data blender.

[0052] According to one aspect, the data is combined by using an equal number of production states from different data sets. This prevents, for example, one production process from being over-emphasized. This strategy is therefore advantageous for creating generic digital production models that can serve as a general basis for knowledge transfer. This can also be used to generate foundation models.

[0053] Another approach involves combining the data by weighting the similarity of the production process. This involves using more data points from production processes with higher similarity. This strategy is more suitable for creating a specialized production model for a new production process.

[0054] The invention is explained by way of example with reference to the following figures. They show: Fig. 1 is an overview of a production optimization system used to carry out the method disclosed here, Fig. 2 a diagrammatic comparison of the method disclosed here with the methods known from the prior art based on the accuracy achieved as a function of the iteration steps required for this purpose of the method disclosed here, Fig. 3 an embodiment of a graph, Fig. 4 a schematic representation of a graph neural network and Fig. 5 a flowchart of the method disclosed here.

[0055] In the figures, identical reference symbols designate identical or functionally similar objects. To avoid repetition, only the relevant objects are identified by reference symbols.

[0056] Fig. Figure 1 shows a production optimization system (TGRPO) that can be used to implement the method disclosed here. TGRPO stands for Transferable Graph Neural Network and Reinforcement Learning-based Production Optimizer. TGRPO consists of seven parts, which are described below.

[0057] Second Database 1: The second database 1 contains and manages data sets S1, S2 for various specialized production processes P1, P2. Each data set S1, S2 is a production data set and consists of a series of production states, i.e., snapshots of the production system, containing all the necessary data of the production system, for example, a factory. This data serves as input for the production simulation, which is used as the basis for creating a digital production model M4 by the trained graph neural network 4.A production state can, for example, include: production requirements, for example delivery times for certain orders, production parameters, for example employee situation, machine capabilities, material availability, a virtual representation of the production process, for example production layout, material flow, process times, expected deliveries, and / or planning strategy, for example demand priorities, specifications.

[0058] Data Selector 2: The data selector analyzes the second database 1 to select one or more data sets S1, S2 to be used to create or improve a production model M1, M2, M3, M4. A selection of different metrics is available for this purpose, which analyze the similarities between the different production processes P1, P2. For example, metadata of the various production processes P1, P2 are compared in the second database 2. The more similar these are, the more suitable the corresponding data sets S1, S2 are for creating or improving a new production model M1, M2, M3, M4 using the graph neural network.

[0059] Data Mixer 3: Data Mixer 3 combines selected production data sets S1 and S2 to create or improve a production model M4 using the graph neural network 4. For this purpose, the following strategies are optionally used: a. Equal number of production states from different data sets S1, S2. This prevents a production process P1, P2 from being overweighted. Therefore, this strategy is suitable for creating generic models that can serve as a general basis for knowledge transfer. b. Weighting based on the similarity of production processes P1 and P2. For example, if a new production process P3 is analyzed in data selector 2 and similarities are found with production processes P1 and P2, for example, if a metric is above a certain threshold, then data mixer 3 will feed data from the two respective production processes P1 and P2 into a mixed data set S3. This is usually done by weighting the similarity values. These can also involve different aspects of a new production process P3.

[0060] The data S3 obtained by the data selector 2 and / or the data mixer 3 for the specific production process P3, i.e., the production data S3, are input into the graph neural network 4. The graph neural network 4 determines the digital production model M4. The graph neural network 4, for example, iteratively performs the following steps until the production model M4 no longer improves. The first step is a production simulation using Monte Carlo Tree Search. A large number of simulation runs, for example, more than 1000 simulation runs using adaptive production optimization, abbreviated to APO, as disclosed in DE 10 2020 203 716 A1 and DE 10 2020 203 718 A1, are run through and compared. A scheduling policy, or policy for short, is created for each production step.This policy indicates how often a production batch has led to a better solution than calculated by APO. For each step, the policy and the associated production data are then stored as a graph G. The policy and graph G are then used in the second step to train the production model M4 using the graph neural network 4. The resulting production model M4 can be used in the next iteration to create an initial policy for each step, which can then be further improved. This gradually helps the production model learn which decisions lead to a good assembly sequence.

[0061] The final digital production model M4 can be exported to the first database. At the first database level, only production models M1, M2, M3, and M4 are exchanged, not production data.

[0062] First database 5, also called the model zoo, for managing pre-trained production models M1, M2, and M3. In addition to the actual production models M1, M2, and M3, the first database 5 can also store metadata to identify and compare the production models M1, M2, and M3. The metadata can include, for example, the name, version, creation date, model type, reference to related models, reference to the input data 10 used, reference to the training data used, and / or past performance data of the model.

[0063] Model Analyzer 6: Model Analyzer 6 provides a number of strategies for comparing production models M1, M2, and M3 and identifying suitable candidates for potential knowledge transfer. Several approaches are available for this purpose: a. Selection of one or more production models M1, M2, M3 based on the similarity of the production processes P1, P2, P3 identified by the data selector 2; b. Selection based on performance data using a validation dataset. For this purpose, the stored production models M1, M2, and M3 from Model Zoo 5 are applied to perform planning using a suitable validation dataset. The production models M1, M2, and M3 with the best planning results are then selected for further processing.

[0064] The model analyzer 6 can also determine a best-fit model MF.

[0065] Model Mixer 7: Model Mixer 7 combines selected production models M1, M2, and M3 into a consensus Model MK, which is generally superior to individual production models M1, M2, and M3. Various approaches can be used optionally: a. First, the production models M1, M2, and M3 are used together in the simulation to select the next action. To do this, each production model M1, M2, and M3 is first applied individually to find the best policy value. Then, the action that receives the most votes is selected. Each production model M1, M2, and M3 can have one vote or several, to introduce weighting. b. Alternatively, the production models M1, M2, and M3 are combined into a new one, which makes the sole decision, also called a mixture model. For this purpose, for example, the individual policies are combined into a single one.

[0066] Fig. 2 is based on real production data and proves the superiority of the method disclosed here over the known state of the art based on the accuracy achieved as a function of the iteration steps required.

[0067] Fig. Figure 3 shows an overview of a graph G that encodes a production state. In this overview, all nodes of the same type are grouped together to simplify representation. This graph G contains both static and dynamic data. Static data is the same for each iteration. For example, the number of lines and the bill of materials are the same for each iteration. However, both the number of nodes and edges (for example, the "plannable" node describes a batch that can be started on the related line at this production time) and the edge / node data (for example, the current stock of a material in a buffer) can change.

[0068] Fig. Figure 4 schematically shows the graph neural network 4, which receives a graph G as input.

[0069] Fig. 5 shows the steps of the method disclosed here, which, in execution, plans a production plan for the specific production process P3 in a production system and controls and / or regulates the production system according to the production plan.

[0070] In a step V1, the digital production models M1, M2, M3 are imported from the first database 5. In a step V2, the input data 10 relating to production states of the specific production process P3 are entered. In a step V3, the digital production models M1, M2, M3 are analyzed for similarity between the production states of the existing production processes P1, P2 and the input data 10 relating to production states of the specific production process P3. In doing so, one or more of the digital production models M1, M2, M3 that exceed a predetermined similarity threshold are identified for knowledge transfer from the existing production processes P1, P2 to the specific production process P3. In a step V4, graphs G are generated for the production states of the specific digital production model(s) M1, M2, M3.Each production state of a production process of the production system is represented together with production sequences as a graph G, see also . Fig.3. In step V5, the graph neural network 4 trained to minimize a cost function of a production process is deployed. In step V6, the production sequence lot with the highest probability value is selected from the probability distribution for the specific production process P3. The steps of generating the graphs G and inserting the trained graph neural network 4, starting from a state of a computer-implemented simulation of the specific production process P3 obtained from the production sequence each extended by this production sequence lot, are repeated until a predefined termination criterion for the computer-implemented simulation is reached. In the process, the digital production model M4 for the specific production process P3 is created. The trained graph neural network 4 is deployed on a high-performance computing platform.The high-performance computing platform comprises at least one graphics processor for processing the trained graph neural network 4. In a step V7, the created digital production model M4 for the specific production process P3 is input into a production planning and control device of the production system, and the production system is regulated and / or controlled by means of the production planning and control device. Reference symbol TGRPO production optimization system for transferable detailed planning for multi-stage production processes using reinforcement learning trained graphs of neural networks 1 second database 2 Data selector 3 data mixers 4 graph neural network 5 first database 6 Model analyzer 7 model mixers 10 Input data GGraph P1, P2 existing production processes S1, S2 Data on existing production processes P3 special production process S3 data on the specific production process M1-M3 digital production models M4 digital production model for the special production process MF Best-Fit Model MK consensus model V1-V7 procedural steps QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] DE 10 2020 203 716 A1 [0008, 0060] DE 10 2020 203 718 A1 [0008, 0060]

Claims

[1] Computer-implemented method which, in execution, plans a production plan for a specific production process (P3) in a production system and controls and / or regulates the production system according to the production plan, the method comprising the steps: • Importing digital production models (M1, M2, M3) from a first database (5), wherein a digital production model (M1, M2, M3) in a computer-implemented simulation of the production system creates production sequences for existing production processes (P1, P2) based on sensor-recorded data on production states, wherein the production system comprises sensors that record the production states, and when executing the computer-implemented simulation, the sensor-recorded data is evaluated when creating the production sequences (V1); • Entering input data (10) on production states of the special production process (P3) by means of a human-machine interface or a machine-machine interface (V2); • Analyzing the digital production models (M1, M2, M3) for similarity of the production states of the existing production processes (P1, P2) with the input data (10) on production states of the special production process (P3) and determining one or more of the digital production models (M1, M2, M3) that exceed a predetermined similarity threshold for a knowledge transfer from the existing production processes (P1, P2) to the special production process (P3), wherein the analyzing and / or determining are carried out by means of an evaluation device (V3); • Generating graphs (G) for the production states of the specific digital production model(s) (M1, M2, M3), wherein in each case a production state of a production process of the production system together with production sequence lots is represented as a graph (G) and entities of the production process comprising production sections of the production system, production lines and / or materials are represented as nodes and relations between the entities comprising predecessor material, successor buffers and / or line capability are represented as edges (V4); • Using a graph neural network (4) trained to minimize a cost function of a production process, comprising at least one or more classification layers, wherein the trained graph neural network (4) receives the generated graphs (G) for the existing production processes as input and transformed graphs (G) as output, and the classification layers receive the transformed graph as input and provide a probability distribution over production sequence lots as output using the knowledge transfer from the existing production processes (P1, P2) to the specific production process (P3) (V5); • for the special production process (P3) selecting the production sequence lot with the highest probability value from the probability distribution and repeating the steps of generating graphs (G) and inserting the trained graph neural network (4) starting from a state of a computer-implemented simulation of the special production process (P3) obtained from the production sequence each extended by this production sequence lot, until a predetermined termination criterion for the computer-implemented simulation is reached, wherein the respectively extended production sequence is optimized with respect to the cost function and changes the special production process (P3), and creating a digital production model (M4) for the special production process (P3),wherein the trained graph neural network (4) is used on a high-performance computer platform and the high-performance computer platform comprises at least one graphics processor (V6) for processing the trained graph neural network (4);, • Entering the created digital production model (M4) for the specific production process (P3) into a production planning and control device of the production system and controlling and / or regulating the production system by means of the production planning and control device (V7). [2] Method according to claim 1, wherein the created digital production model (M4) is used on new data of the same specific production process (P3) and generates a production plan in real time. [3] Method according to one of the preceding claims, wherein the created digital production model (M4) is exported to the first database (5). [4] Method according to one of the preceding claims, wherein metadata is stored in the first database (5) and the digital production models (M1, M2, M3) are identified and compared with one another by means of the metadata, wherein the metadata comprises at least the name of the production model, the version of the production model, the creation date of the production model, the model type of the production model, a reference to similar production models, a reference to input data used, a reference to training data and / or past performance data of the production models (M1, M2, M3) which indicate the performance of the digital production models on test or validation data. [5] Method according to one of the preceding claims, wherein the one of the digital production models (M1, M2, M3) which is intended for a knowledge transfer from the existing production process (P1, P2) to the special production process (P3) is determined based on past performance data which indicate the performance of the digital production model (M1, M2, M3) on test or validation data, or wherein the plurality of digital production models (M1, M2, M3) which are intended for a knowledge transfer from the existing production processes (P1, P2) to the special production process (P3) are combined to form a consensus model (MK). [6] Method according to one of the preceding claims, wherein the digital production models (M1, M2, M3) are created from the first database (5) by the trained graph neural network (4), wherein data (S1, S2) on the existing production processes (P1, P2) are stored in a second database (1), data from the second database (1) are selected by means of a database management device, the trained graph neural network (4) receives the selected data in the form of graphs (G) as input and creates the digital production model (M1, M2, M3, M4) by processing the selected data, and the created digital production model (M4) is exported to the first database (5). [7] Method according to claim 6, wherein selected data from the second database (1) are combined and a digital production model (M1, M2, M3, M4) is created on the combined data. [8] A computer program for a production planning and control device of a production system, comprising instructions that cause a high-performance computing platform to carry out the steps of the method according to any one of the preceding claims when the computer program is loaded or executed by the high-performance computing platform.

Images