Computer-implemented method that, in execution, plans a production schedule for a production process in a production system and controls and / or regulates the production system according to the production schedule, and computer program for a production planning and control device of a production system

Abstract

A computer-implemented method that, in execution, plans a production schedule for a production process (P1, P2) in a production system and controls and / or regulates the production system according to the production schedule, the method comprising the steps: providing an initial database (5) comprising digital production models (M1, M2, M3), selecting one of the temporally sequential digital production models (M1, M2, M3) based on performance data, inferring the production sequences using the basic production model (B) and the selected production model (M1, M2, M3), inserting a graph neural network (4) trained to minimize a cost function of a production process, inputting the improved digital production model (M4) into a production planning and control device of the production system, and controlling and / or regulating the production system using the production planning and control device.

Description

[0001] The invention relates to a computer-implemented method that, when implemented, plans a production plan for a production process in a production system and controls and / or regulates the production system according to the production plan. Furthermore, the invention relates to a computer program for a production planning and control device of a production system.

[0002] The following definitions apply to the entire content of the disclosure.

[0003] During production, products, including tangible goods and services, are created based on production factors, including materials and resources. A multi-stage production process or a multi-stage production system can comprise one or more production stages. The production system is a system in which something is produced, for example a business, a factory, a workshop or one or more production lines. The production system can also comprise an IT system for the design and monitoring of production. The production stages can be carried out sequentially or in parallel. The production stages can each comprise one or more machines or production lines. Buffers, for example material buffers, or transport systems can be located between the production stages. Methods for production planning and / or control are known in the prior art.

[0004] Production in modern industrial manufacturing facilities today is often dependent on a multitude of external factors. These include globally interconnected supply chains, a high number of product variants, an unmanageable volume of purchased parts, and highly dynamic customer orders. The influence of these factors creates a highly volatile production environment.

[0005] Production planning encompasses strategic considerations and preparations of all key variables of a company's manufacturing process or work procedure for a future period before it begins. For example, production planning encompasses the operational, temporal, quantitative, and spatial planning and management of all processes necessary for the production of goods and products.

[0006] The production sequence is a sequence of production batches per machine or production line. A production batch, also called a lot (production), batch, or lot, comprises the type of material to be produced and / or the number of parts to be produced or the required quantity. The production process is regulated and / or controlled based on the production sequence.

[0007] In this context, computer-aided solutions make it possible to create optimized plans in the shortest possible time using digital production models.

[0008] The subject matters disclosed in DE 10 2020 203 716 A1 and DE 10 2020 203 718 A1, in particular the methods and systems disclosed therein, are called adaptive production optimization, abbreviated APO, and are part of the present disclosure by this explicit reference.

[0009] In particular, optimization approaches based on reinforcement learning using graph neural networks, abbreviated GRPO, as disclosed in German application 10 2022 211 446.9, which is incorporated into the present disclosure by this explicit reference, show great potential. GRPO uses a production simulation and a database with historical production data records of any multi-stage production process. This data is used by machine learning to train a specific production model for the process. The trained model can be used to generate highly optimized assembly sequences in real time for new data from the same production process. GRPO thus enables rapid response to disruptions in the supply chain and consequently saves significant amounts of time, money, and resources.

[0010] A disadvantage of this solution is that it does not take into account how well a production model's results compare to other production models and how the production model's performance changes over time, especially when changes occur in the input data or due to peculiarities in the production process. This can lead to poor production results and increased costs.

[0011] The object of the invention was to find out how detailed planning, in particular continuous adaptive detailed planning, can be implemented for production processes, in particular multi-stage production processes, in order to improve the control and / or regulation of a production system based on this.

[0012] The subject matter of the independent claim and the subordinate claims each solves this problem. Advantageous embodiments of the invention emerge from the definitions, the subclaims, the drawings, and the description of preferred embodiments.

[0013] The invention is first summarized for the sake of clarity. All features mentioned therein, individually and in combination with one another, are part of the inventive approach. The features and definitions mentioned in this summary apply in combination and in connection with the claimed subject matter.

[0014] A comprehensive system, called the Continuous Adapting Graph Neural Network and Reinforcement Learning-based Production Optimizer (CARPO), is provided for the continuous adaptive detailed planning of a complex, particularly a complex multi-stage, production process. The CARPO is continuously improved using a machine learning method based on reinforcement learning.

[0015] The input for CARPO can be a production data record, for example, a continuous data stream from a specific production area. It can be a snapshot of factory data required to create a production sequence, such as an assembly sequence. This can include production requirements, such as delivery times for specific orders; production parameters, such as employee situation, machine capabilities, material availability; a virtual representation of the production process, including, for example, production layout, material flow, process times, expected deliveries; and / or planning strategy, such as demand priorities. CARPO captures additional metadata, which is used to classify data records chronologically and / or functionally, such as various timestamps, the length of the planning horizon, inventory buildup, and / or line utilization.

[0016] At the core of CARPO is a graph neural network (GNN), which determines the production sequence as output based on the state of the production simulation, i.e., the current production status. CARPO minimizes a cost function that can be variably adjusted by a user, for example, a human controller of the production processes.

[0017] In graph theory, a graph is an abstract structure that represents a set of objects along with the connections between these objects. The mathematical abstractions of the objects are called nodes of the graph. The pairwise connections between nodes are called edges. Edges can be directed or undirected. The preceding three sentences are part of a Wikipedia text under a CC-BY-SA license. In the GRPO, all entities of the production state—including production sections, materials, and production lines—are encoded as nodes, and all relations, such as predecessor material, successor buffer, and line capability, are encoded as edges.

[0018] A GNN is an artificial neural network that operates on graph data. The transformations performed by the network preserve the symmetry of the graph, meaning the transformations are invariant under permutations. The network can perform transformations on nodes, edges, and / or the global context of the graph. The network comprises one or more layers. Each layer receives a graph as input and produces a new graph as output. Nodes, edges, and / or global properties of the input graph are updated in each layer using graph-attribute-separated functions, resulting in a new graph that has the same connectivity as the input graph.

[0019] The current production status can include one or more production parameters. Furthermore, the production status can include requirements, such as customer requirements, such as which material is in demand at what time or a weighting of the requirements. Furthermore, the production status can include production layout, material flows, and / or process times.

[0020] The cost function can, for example, consist of a combination of total delay time of requirements, total setup time, utilization / output and minimum stock levels of certain final or intermediate products.

[0021] A further advantage is that it achieves a high degree of adaptability to the respective production area. CARPO continuously improves through its reinforcement learning approach.

[0022] Another advantage is the high degree of variability. By using the graph structure and the GNN, the trained network is able to use relations and dependencies instead of hard-coded features. This means that the network can achieve good results without retraining, even when production changes occur, such as the addition or modification of materials and / or production lines.

[0023] Advantages of the invention include, in summary: • Automatic collection of performance metrics for early detection of declining model performance, for example due to model drifts; • improved transparency and increased explainability of model quality; • automatic generation of new data points to improve models (active learning); • improved model performance through continuous improvements and fine-tuning over time (Continuous Learning / Incremental Learning); • Time and cost savings through increased model performance

[0024] The inventive solution to the above-mentioned problem is provided by the following aspects: According to one aspect, the invention provides a computer-implemented method which, when implemented, plans a production plan for a production process in a production system and controls and / or regulates the production system according to the production plan. By carrying out the method, a new, improved, finely tuned digital production model is created for the production process, based on which the production process can then be controlled and / or regulated in an optimized manner. By means of the fine tuning carried out according to the method, the method enables the creation of improved production models for the detailed planning of new, multi-stage production processes. Due to the fine tuning, the new production model has a relatively higher performance while simultaneously significantly reducing processing times.

[0025] In one step of the method, a first database is provided. The first database contains digital production models. In a computer-implemented simulation of the production system, a digital production model infers production sequences for existing production processes based on sensor-recorded data on production states. The production system comprises sensors that record the production states. When the computer-implemented simulation is executed, the sensor-recorded data is evaluated when creating the production sequences. For each existing production process, a digital base production model and several chronologically successive digital production models are stored in the first database. The first database is a model zoo database in which existing digital production models are managed, for example using special data management software, and validated.Furthermore, the first database can include a release mechanism for the production models. A digital production model creates production sequences for existing production processes in a computer-implemented simulation of the production system based on sensor-recorded data on production states. The production system includes sensors that record the production states. A production state includes production requirements, for example, delivery times for specific orders. The production requirements can be recorded, for example, using time sensors. The production state can also include production parameters, for example, employee situation, machine capabilities, and / or material availability. The production parameters can be recorded, for example, using camera sensors. The camera sensors are arranged, for example, on production lines.The production state can also include a virtual representation of the production process, for example, production layout, material flows, process times, and / or expected deliveries. The virtual representation can be captured using virtual sensors, for example. When the computer-implemented simulation is executed, the sensor-captured data is evaluated when creating the production sequences.

[0026] By evaluating the sensory data acquired during the execution of the computer-implemented simulation, an interaction with external physical reality occurs, at least at the input level. The new digital production model created in the simulation predicts production sequences for the specific production process and regulates and / or controls it. Thus, the computer-implemented simulation makes a technical contribution.

[0027] In a further step of the process, one of the consecutive digital production models is selected at a first point in time based on performance data. The performance data indicates the performance of the consecutive digital production models on test or validation data. For example, the performance data can be recorded over time, and accuracy, mean error, and other statistical parameters can be evaluated. Production models are selected for inference within a specific production area. This can be done based on performance data as well as on functional criteria.

[0028] For example, a manual selection of one or more production models aggregated in the first database can be made by a human expert, which is also called human in the loop. According to a further aspect, one or more production models, for example those aggregated in the first database, can be selected automatically, for example based on performance data. The selection can be made automatically, with a human continuously monitoring the system, which is also called human on the loop. By integrating a human expert, human supervision of the system is realized. The human interaction can take place, for example, via a human-machine interface. The human supervision serves, among other things, to prevent or minimize safety risks.

[0029] According to another aspect, the selection can also be carried out completely autonomously without human intervention, which is also called human out of the loop.

[0030] In a further step of the process, input data relating to one of the existing production processes is entered at the first point in time using a human-machine interface or a machine-machine interface into the corresponding base production model and the selected production model of this existing production process. Based on this input data, the new digital production model is created according to the following process steps. A human user, for example, a manufacturing expert or production planning expert, can enter a data set or data sets relating to production states of the specific production process via the human-machine interface.One or more data records relating to production states of the specific production process, which can be generated, for example, by means of a database management device, can be entered via the machine-to-machine interface, for example a data transmission interface.

[0031] In a further step of the method, the production sequences are inferred from the input data using the base production model and the selected production model of this existing production process. The base production model is used to measure and compare the performance data of the selected production models and those stored in the first database. If, in a given case, all of the production models used exhibit poorer performance than the base production model, the result of the base production model can be used. According to a further aspect, the selected production models are aggregated in this method step, for example, using weighting, their results are aggregated, and the best results are selected.

[0032] In a further step of the process, the production model from the previous step is selected whose inferred production sequences result in the comparatively best production performance. According to one aspect, a performance monitor is used for this purpose. The performance monitor measures and compares performance data of the production models. By comparing the currently selected production models with the baseline production model, a possible model drift can advantageously be identified. Model drifts can lead to decreasing model performance. Early detection of model drifts can ensure high model performance. Furthermore, a fine-tuned production model can be compared with a currently used production model to determine whether the fine-tuning has led to improved performance. Fine-tuned production models can also be compared with the baseline production model.

[0033] In a further step of the process, graphs are generated for production states of the previously selected production model. Each production state of a production process within the production system is represented as a graph along with subsequent production batches. Entities of the production process include, for example, production sections of the production system, production lines, and / or materials, and are represented as nodes. Relations between the entities, including predecessor material, successor buffers, and / or line capability, are represented as edges. This creates the data structure of the graph for the production states. This data structure is optimized for evaluation in the subsequent steps of the process using the graph neural network.

[0034] Implementing a graph neural network trained to minimize a cost function of a production process, comprising at least one or more classification layers, wherein the graph neural network receives the generated graphs of the existing production processes as input and transformed graphs as output, and the classification layers receive the transformed graph as input and provide a probability distribution over production sequences as output. Improved production sequences are obtained by the production model used, selected from the base production model and the selected production model.

[0035] The graph neural network can be a so-called message-passing graph neural network, which uses message-passing technology to update each node using information contained in every neighboring node. This means that each production section of the production system, production line, and / or material buffer in the graph neural network receives information from previous production sections, production lines, and / or material buffers in the production sequence. Furthermore, information about the relations between the nodes embedded in the edges of the graph can also be passed on accordingly. The graph neural network can also be a so-called graph convolutional neural network, which applies mathematical convolution to the embeddings of the nodes and / or edges of the graph.The graph neural network can also be a so-called graph attention neural network, in which attention technology is used to weight the importance of neighboring nodes in the graph during the information transfer phase, i.e., during message passing. An attention value is calculated for each pair of nodes, which is used to weight the individual nodes.

[0036] The graph neural network can be trained on training data for the inference of a production sequence. The training data comprises graphs that represent production states together with production sequence batches. Production sections of the production system, production lines, and / or materials are represented as nodes, and relations between the entities, including predecessor material, successor buffers, and / or line capability, are represented as edges. To generate this training data, the production process is simulated using computer implementation. An initial state of the simulation is a production state based on an initial production sequence. Based on this, production sequence batches are determined that are available at a simulation time in which a production line of the production system has processed the production sequence batches assigned to it.At this simulation time, a Monte Carlo tree search is performed, obtaining a probability distribution across these production sequence batches. A production sequence batch is then randomly selected at this simulation time according to the probability distribution. The preceding steps are repeated at each subsequent simulation time, starting from the simulation state obtained from the production sequence extended by this production sequence batch, until a predefined simulation termination criterion is reached. Upon reaching the simulation termination criterion, a relative value is determined that relates a first result value obtained from the simulation data, which characterizes the quality of the simulated production process, to a second result value, which characterizes the quality of the production process.In an adaptive production optimization, in the real production process, material requirements in the production stages are prioritized depending on an influence on an optimization of a cost function of the production system, one of the material requirements is selected in the order of prioritization, at least one required quantity and / or a required time of materials in preceding production stages is adjusted to fulfill the material requirement and the materials and the respectively adjusted required quantity and / or required time are reserved, another of the material requirements is selected and the preceding steps are repeated until the materials and the respectively adjusted required quantities and / or required times are reserved for all of the prioritized material requirements; whereby a production sequence is obtained, and the second result value for the obtained production sequence is determined.Subsequently, the graph representing the state of the production process at the simulation termination point is labeled with the relative value and the probability distribution obtained at this simulation point using Monte Carlo tree search. By repeating the previous steps, appropriately labeled graphs are obtained, which form the training data.

[0037] The graph neural network is trained on this training data. At the end of the training, the trained graph neural network used in the method disclosed here is obtained. The graph representing the state of the production process at a current simulation time point along with the production sequences is input into the graph neural network, and a transformed graph is obtained in a forward path of the graph neural network. One or more first classification layers of the graph neural network determine a probability distribution over the production sequences from the transformed graph, and one or more second classification layers of the graph neural network determine a quality value characterizing the quality of the production process at the respective simulation time point.The production sequence lot with the highest probability value is then selected from the probability distribution, and the preceding steps are repeated at each subsequent simulation time point, starting from the simulation state obtained from the production sequence each time extended by this production sequence lot, until a predefined termination criterion for the simulation is reached. When the simulation termination is reached, a first difference between the quality value and a relative value of one of the labeled graphs is determined, and a second difference between the probability distribution obtained using the graph neural network and a probability distribution of the labeled graph obtained using a Monte Carlo tree search is determined. In a backward path of the graph neural network, the first and second differences are error-fed back through the graph neural network.

[0038] In another aspect, the graph neural network with the best inference performance is replaced by the currently trained graph neural network to generate new graphs. In this context, this is called reinforcement learning.

[0039] In a further step of the method, the production sequence lot with the highest probability value is selected from the probability distribution for the respective existing production process. The steps of generating graphs and implementing the trained graph neural network are repeated starting from a state of a computer-implemented simulation of this production process obtained from the production sequence each time extended by this production sequence lot, until a predetermined termination criterion for the computer-implemented simulation is reached. The each extended production sequence is optimized with respect to the cost function and changes this production process. A digital production model with improved production performance is obtained. The trained graph neural network is deployed on a high-performance computing platform.The high-performance computing platform includes at least one graphics processor for processing the trained graph neural network. The high-performance computing platform is hardware-specifically specialized for processing the trained graph neural network.

[0040] In a further step of the process, the improved digital production model is input into a production planning and control device of the production system. The production system is then regulated and / or controlled using the production planning and control device. This allows the production system to be adapted and optimized, even in a volatile production environment that depends, for example, on interconnected supply chains, a high number of product variants, a complex quantity of purchased parts, and highly dynamic customer requests.

[0041] According to a further aspect, the invention provides a computer program for a production planning and control device of a production system.

[0042] The computer program includes instructions that cause a high-performance computing platform to perform the steps of the method disclosed herein when the computer program is loaded or executed by the high-performance computing platform.

[0043] The method and / or the computer program can be carried out locally in a production system or remotely in a cloud environment, wherein the production system comprises an IT infrastructure via which data exchange with the cloud is realized and carried out.

[0044] According to one aspect, the improved digital production model is exported to the first database. This expands the first database with fine-tuned production models. According to another aspect, the improved digital production models are validated by a hardware and / or software module, called a model validator, before being exported to the first database. For example, the obtained improved digital production models are manually selected by a domain expert using human in-the-loop and, after selection, exported to the first database. The selection can also be performed using human on-the-loop or human out-of-the-loop. A further comparison with the base production model can also be performed.

[0045] According to a further aspect, the steps of selecting one of the consecutive digital production models until the improved digital production model is entered are repeated at subsequent points in time. This enables continuous fine-tuning.

[0046] According to a further aspect, data records relating to the respective existing production processes are stored in a second database. The data records each comprise several chronologically successive data collections. The data collections represent changes in the respective existing production processes. Data is selected from the second database using a database management device. The trained graph neural network receives the selected data in the form of graphs as input. The digital production model thus obtained from the trained graph neural network is improved and fine-tuned with regard to production performance.

[0047] A data selector, for example, in the form of a hardware and / or software module, can be used to store the data records in the second database. The data selector can technically curate and annotate newly generated production data records from each production process. For example, curation can be performed by a human expert with domain knowledge via human in the loop. Data processing and / or data management can also be performed using a release mechanism via human on the loop, or fully automated data processing and / or data management can be performed using predefined criteria via human out of the loop.

[0048] The second database is thus a data-curated database. Individual data records are grouped into collections that fit together chronologically or technically. This ensures that changes in the production process are also reflected in the data. The collections can be used to create initial production models or to continuously improve or fine-tune existing production models.

[0049] The invention is explained by way of example with reference to the following figures. They show: Fig. 1 is an overview of a production optimization system used to carry out the method disclosed here, Fig. 2 an embodiment of a graph, Fig. 3 a schematic representation of a graph neural network and Fig. 4 a flowchart of the method disclosed here.

[0050] In the figures, identical reference symbols designate identical or functionally similar objects. To avoid repetition, only the relevant objects are identified by reference symbols.

[0051] Fig. Figure 1 shows a production optimization system, CARPO, that can be used to implement the method disclosed here. CARPO is the abbreviation for Continuous Adapting Graph Neural Network and Reinforcement Learning Based Production Optimizer. CARPO can consist of ten modules, which are described below. However, CARPO can also be successfully operated with a subset of these modules.

[0052] Input Data 10: Input data 10 for CARPO is a production data set P1 or a continuous stream of this data from a specific production area. This is a snapshot of the factory data required to create a corresponding assembly sequence. This includes, among others: • Production requirements (e.g. delivery times for certain orders, etc.); • Production parameters (e.g. employee situation, machine capabilities, material availability, etc.); • virtual representation of the production process (e.g. production layout, material flow, process times, expected deliveries, etc.); • Planning strategy (e.g. demand priorities, specifications, etc.).

[0053] In addition, further metadata can be recorded that allows the data records to be classified chronologically and subject-wise: • different timestamps; • Length of the planning horizon; • additional information on the planning strategy: ◯ Inventory building, ◯ full line capacity, ◯ typical KPIs that evaluate the production process.

[0054] Data Collector 2: A data collector that curates, annotates, and stores newly generated production data records from each production area in a second database 1. There are several approaches to this: • Curation by human experts with domain knowledge; • automatic data processing and data management with approval mechanism by experts; • fully automated data processing and data management using predefined criteria.

[0055] Second Database 1: A database that contains and manages data records for various specialized production processes P1, P2. The second database 1 includes temporal components and curation by the data collector. Individual data records are summarized here into collections K11, K12, K13, K21, K22, K23 that fit together chronologically or technically. This ensures that changes to the production process P1, P2 are also reflected in the data. These collections K11, K12, K13, K21, K22, K23 can then be used to create initial models or to continuously improve or fine-tune existing production models M1, M2, M3.

[0056] First database 5: A database for managing pre-trained production models M1, M2, and M3 for various production processes P1, P2. The first database 5 contains a temporal component and a validation of the models M1, M2, and M3, as well as an approval mechanism, the model validator 7. The production model M1 was obtained at time t1, the production model M2 at a subsequent time t2, and the production model M3 at a subsequent time t3.

[0057] Model Selector 3: A selection mechanism for models selected for inference in a specific production area. This can be done either based on specific performance characteristics or business criteria. Several approaches exist for this purpose: • manual selection of one or more aggregated models by human expert Exp; • Automatic selection of a model, multiple models, or an aggregated model based on certain performance characteristics that describe how well a model performs on test or validation data.

[0058] Inference Inf: An inference mechanism that can consist of three parts: • a baseline heuristic B, which is used to measure and compare the performance data of the trained production models M1, M2, M3, see also Performance Monitor 6. If in a case all of the used production models M1, M2, M3 show a worse performance than the baseline B, the result of the baseline B is automatically adopted as a fallback; • one or more models M1, M2, M3 used to perform optimized sequence planning. There are different strategies for using multiple models M1, M2, M3: ◯ (weighted) aggregation of models M1, M2, M3, ◯ (weighted) aggregation of the results, ◯ Selecting the best result.

[0059] Performance Monitor 6: A Performance Monitor 6 used to continuously compare the performance data of the base production model B or the baseline and the used models M1, M2, and M3. Performance Monitor 6 performs the following tasks: • Comparison of the currently used model M1, M2, M3 and the baseline B to determine a possible model drift; • Comparison of a fine-tuned M4 and the currently used model M1, M2, M3 to find out whether the fine-tuning of M4 has led to improved performance; • Comparison of a fine-tuned model M4 and the baseline B.

[0060] To compare performance data, several methods can be used, for example (mean) change of performance data over time, e.g. accuracy, mean error.

[0061] Model Tuner 9: Model Tuner 9 can improve and fine-tune an existing model M1, M2, or M3. There are several ways to initiate a fine-tuning operation: • manual control by human expert Exp; • automatic fine-tuning due to deteriorated model performance; • Continuous fine-tuning with new data over time.

[0062] Advanced Data Selector 8: A mechanism for selecting data used to improve and fine-tune models M1, M2, and M3. Several approaches are available for this purpose: • manual selection; • Selection based on professional criteria; • automatic selection of new data records; • Extension of the data sets with synthetic data (optional).

[0063] Model Validator 7: A mechanism for validating fine-tuned M4 models before use. There are several approaches to this: • manual selection by a domain expert; • automatic selection based on predefined metrics; • Comparison with the baseline heuristic B.

[0064] Fig. Figure 2 shows an overview of a graph G that encodes a production state. In this overview, all nodes of the same type are grouped together to simplify representation. This graph G contains both static and dynamic data. Static data is the same for each iteration. For example, the number of lines and the bill of materials are the same for each iteration. However, both the number of nodes and edges (for example, the "plannable" node describes a batch that can be started on the related line at this production time) and the edge / node data (for example, the current stock of a material in a buffer) can change.

[0065] Fig. Figure 3 schematically shows the graph neural network 4, which receives a graph G as input.

[0066] Fig.4 shows the steps of the method disclosed herein, which, in execution, plans a production schedule in a production system and controls and / or regulates the production system according to the production schedule.

[0067] In a step V1 of the method, a first database 5 comprising digital production models M1, M2, M3 is provided, wherein a digital production model M1, M2, M3 infers production sequences for existing production processes P1, P2 in a computer-implemented simulation of the production system based on sensor-recorded data on production states. The production system comprises sensors that record the production states. When the computer-implemented simulation is executed, the sensor-recorded data are evaluated when creating the production sequences, wherein a digital base production model B and several chronologically successive digital production models M1, M2, M3 are stored in the first database 5 for each existing production process.

[0068] In a step V2, at a first time t1, one of the temporally successive digital production models M1, M2, M3 is selected based on performance data that indicate the performance of the temporally successive digital production models M1, M2, M3 on test or validation data.

[0069] In a step V3, at the first time t1, input data 10 for one of the existing production processes P1, P2 are entered into the associated basic production model B and the selected production model M1, M2, M3 of this existing production process P1, P2 by means of a human-machine interface or a machine-machine interface.

[0070] In a step V4, the production sequences are inferred from the input data 10 using the base production model B and the selected production model M1, M2, M3 of this existing production process P1, P2.

[0071] In a step V5, the production model B, M1, M2, M3 is selected from the previous step, whose inferred production sequences comparatively result in the best production performance.

[0072] In step V6, graphs G are generated for production states of the previously selected production model B, M1, M2, M3, wherein in each case a production state of a production process of the production system together with production sequence lots is represented as graph G and entities of the production process comprising production sections of the production system, production lines and / or materials are represented as nodes and relations between the entities comprising predecessor material, successor buffer and / or line capability are represented as edges.

[0073] In a step V7, a graph neural network 4 trained to minimize a cost function of a production process is used, comprising at least one or more classification layers, wherein the trained graph neural network 4 receives the generated graphs G as input and transformed graphs G as output, and the classification layers receive the transformed graph as input and provide a probability distribution over production sequence lots as output, wherein improved production sequences are obtained by the selected production model B, M1, M2, M3 from the base production model B and the selected production model M1, M2, M3.

[0074] In a step V8, the production sequence lot with the highest probability value is selected from the probability distribution for the respective existing production process P1, P2 and the steps of generating graph G and inserting the trained graph neural network 4 are repeated starting from a state of a computer-implemented simulation of this production process P1, P2 obtained from the production sequence each extended by this production sequence lot, until a predetermined termination criterion for the computer-implemented simulation is reached, wherein the respectively extended production sequence is optimized with respect to the cost function and changes this production process P1, P2.A digital production model M4 improved in terms of production performance is obtained, wherein the trained graph neural network 4 is used on a high-performance computing platform and the high-performance computing platform comprises at least one graphics processor for processing the trained graph neural network 4.

[0075] In a step V9, the improved digital production model M4 is input into a production planning and control device of the production system and the production system is controlled and / or regulated by means of the production planning and control device. Reference symbol CARPO production optimization system for continuous adaptive detailed planning for multi-stage production processes using deep reinforcement learning trained graph neural networks 1 second database 2 Data collector 3 Model selector 4 graph neural network 5 first database 6 Performance Monitor 7 Model Validator 8 advanced data selector 9 model tuners 10 Input data GGraph P1, P2 existing production processes S1, S2 Data on existing production processes M1-M3 digital production models M4 improved digital production model K11, K12, K13 data collection K21, K22, K23 data collection Exp Expert Inf Inference t1-t3 time points V1-V9 procedural steps QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] DE 10 2020 203 716 A1

[0008] DE 10 2020 203 718 A1

[0008]

Claims

[1] Computer-implemented method which, in execution, plans a production plan for a production process (P1, P2) in a production system and controls and / or regulates the production system according to the production plan, the method comprising the steps: • Providing a first database (5) comprising digital production models (M1, M2, M3), wherein a digital production model (M1, M2, M3) in a computer-implemented simulation of the production system infers production sequences for existing production processes (P1, P2) based on sensor-recorded data on production states, wherein the production system comprises sensors that record the production states, and when the computer-implemented simulation is carried out, the sensor-recorded data are evaluated when creating the production sequences, wherein for the existing production processes (P1, P2) in each case a digital basic production model (B) and several chronologically successive digital production models (M1, M2, M3) are stored in the first database (5) (V1); • at a first point in time (t1), selecting one of the chronologically successive digital production models (M1, M2, M3) based on performance data indicating the performance of the chronologically successive digital production models (M1, M2, M3) on test or validation data (V2); • at the first time (t1) entering input data (10) for one of the existing production processes (P1, P2) by means of a human-machine interface or a machine-machine interface into the associated basic production model (B) and the selected production model (M1, M2, M3) of this existing production process (P1, P2) (V3); • on the input data (10) inferring the production sequences using the base production model (B) and the selected production model (M1, M2, M3) of this existing production process (P1, P2) (V4); • Selecting the production model (B, M1, M2, M3) from the previous step whose inferred production sequences comparatively result in the best production performance (V5); • Generating graphs (G) for production states of the previously selected production model (B, M1, M2, M3), wherein in each case a production state of a production process of the production system is represented together with production sequence lots as a graph (G) and entities of the production process comprising production sections of the production system, production lines and / or materials are represented as nodes and relations between the entities comprising predecessor material, successor buffer and / or line capability are represented as edges (V6); • Using a graph neural network (4) trained to minimize a cost function of a production process, comprising at least one or more classification layers, wherein the trained graph neural network (4) receives the generated graphs (G) as input and transformed graphs (G) as output, and the classification layers receive the transformed graph as input and provide a probability distribution over production sequences as output, wherein improved production sequences are obtained by the selected production model (B, M1, M2, M3) used from the base production model (B) and the selected production model (M1, M2, M3) (V7); • for the respective existing production process (P1, P2), selecting the production sequence lot with the highest probability value from the probability distribution and repeating the steps of generating graphs (G) and inserting the trained graph neural network (4) starting from a state of a computer-implemented simulation of this production process (P1, P2) obtained from the production sequence each extended by this production sequence lot, until a predetermined termination criterion for the computer-implemented simulation is reached, wherein the respectively extended production sequence is optimized with respect to the cost function and changes this production process (P1, P2), and obtaining a digital production model (M4) improved with respect to production performance,wherein the trained graph neural network (4) is used on a high-performance computing platform and the high-performance computing platform comprises at least one graphics processor (V8) for processing the trained graph neural network (4);, • Inputting the improved digital production model (M4) into a production planning and control device of the production system and controlling and / or regulating the production system by means of the production planning and control device (V9). [2] The method of claim 1, wherein the improved digital production model (M4) is exported to the first database (5). [3] Method according to one of the preceding claims, wherein at further times the steps of selecting one of the temporally successive digital production models (M1, M2, M3) until inputting the improved digital production model (M4) are repeated. [4] Method according to one of the preceding claims, wherein data records (S1, S2) relating to the respective existing production processes (P1, P2) are stored in a second database (1), wherein the data records (S1, S2) each comprise a plurality of temporally successive data collections (K11, K12, K13, K21, K22, K23) and the data collections (K11, K12, K13, K21, K22, K23) depict changes in the respective existing production processes (P1, P2), data are selected from the second database (1) by means of a database management device, the trained graph neural network (4) receives the selected data in the form of graphs (G) as input and the digital production model (M4) obtained from the trained graph neural network (4) is thus improved and fine-tuned with regard to production performance. [5] A computer program for a production planning and control device of a production system, comprising instructions which cause a high-performance computer platform to carry out the steps of the method according to any one of the preceding claims when the computer program is loaded or executed by the high-performance computer platform.

Images