Multi-scale digital twinning system of combustion explosive production line

Abstract

The embodiment of the invention provides a multi-scale digital twin system of a combustion explosive production line. Comprising the steps that a digital twin data layer represents the manufacturing process and dynamic evolution of a combustion explosive production line through production line geometric model data, production line physical model data, production line behavior model data and production line rule model data of the combustion explosive; the digital twinning communication unit is used for realizing data interaction between the digital twinning virtual production line and the physical production line and data interaction between different intelligent agents in the digital twinning analysis layer; the digital twinning analysis layer is used for completing collaborative analysis of models in the multi-scale digital twinning analysis layer by adopting a multi-agent deep reinforcement learning technology; and the digital twinning application layer is used for realizing multi-scale digital twinning application. According to the method, analysis and decision making are carried out by combining the high-risk characteristics of the combustion explosive production line and calculation of the equipment operation state and the safety risk state, and the characterization and prediction capability of the safety risk of the combustion explosive production line is improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of industrial digitization and intelligent technology, and specifically to a multi-scale digital twin system of a combustion and explosive production line. Background Art

[0002] In recent years, the concept, framework and technology of digital twins have gradually been studied and applied in the manufacturing industry. Digital twin technology achieves accurate digital mapping by constructing physical objects in real time in digital space, and forms the best decision based on analysis and prediction, thereby optimizing the entire business process of the manufacturing process. At present, some researchers have proposed the concept and system architecture of digital twin workshops, and divided digital twin workshops into virtual workshops, physical workshops, workshop service systems and workshop digital twin data. By constructing multi-dimensional workshop models such as geometry, physics, behavior, and rules, the physical workshop is truly portrayed and described. Other researchers have proposed the connotation and characteristics of multi-dimensional and multi-scale intelligent manufacturing space, and combined with the implementation logic of digital twin technology, studied the modeling method of intelligent manufacturing space and the virtual-real mapping method in complex multi-dimensional space-time domains. Other researchers have proposed corresponding digital twin model paradigms in combination with aerospace testing and automobile manufacturing. In terms of production and manufacturing, some researchers have developed a prototype system for discrete workshops based on the knowledge-driven multi-dimensional and multi-scale digital twin modeling method of manufacturing processes. Other researchers have analyzed the characteristics of digital twin systems for human-machine interaction, established a reference architecture for human-machine-environment collaborative analysis, and key technologies such as three-dimensional geometric modeling based on generative artificial intelligence and optimal control strategy generation based on human-machine collaboration. Other researchers have studied the simulation optimization of manufacturing processes through evolutionary algorithms, deep learning technology, and multi-agent technology. The above research mainly simulates and optimizes production scheduling problems. Other researchers have proposed a simulation of the intelligent test process of spacecraft based on the combination of multi-agent and digital twin technology.

[0003] Although digital twin technology has made a lot of research progress in many fields, since the production line of incendiary explosives is subject to high safety and complex production requirements, the product manufacturing process involves physical and chemical reactions, the internal mechanism is complex, and it involves a large number of process parameters. The multi-parameter coupling process is extremely complex. Therefore, it is difficult to achieve accurate modeling of digital twins. Further research and application are also needed in terms of product manufacturing process characterization, multi-source data fusion, and model collaborative analysis. In addition, the manufacturing process of the incendiary explosive production line has the characteristics of easy combustion and explosion, high dynamics, and nonlinear coupling of safety risks. Its digital twin model needs to consider the safety and reliability of the production process. Therefore, the digital twin technology in the field of incendiary explosive production lines currently lacks a complete system and architecture. Summary of the invention

[0004] An embodiment of the present invention provides a multi-scale digital twin system for a combustible explosive production line, which integrates multi-agent deep reinforcement learning and edge and cloud collaborative analysis technologies. It targets the high-risk, high-dynamic, and multi-factor coupled characteristics of combustible explosive production lines, and solves the problem of the current lack of a complete digital twin system and architecture technology in this field.

[0005] In a first aspect, an embodiment of the present invention provides a multi-scale digital twin system for a combustion and explosive production line, including:

[0006] Digital twin data layer, digital twin communication unit, digital twin analysis layer, digital twin application layer; among them, the digital twin data layer, digital twin analysis layer, and digital twin application layer are divided into three scales: equipment level, process unit level, and production line level;

[0007] The digital twin data layer is used to characterize the data of the production line geometry model, the data of the production line physical model, the data of the production line behavior model, and the data of the production line rule model in the multi-scale digital twin system of the combustion and explosive production line;

[0008] The digital twin communication unit is used to realize data interaction between the virtual production line and the physical production line in the multi-scale digital twin system of the combustion and explosive production line, and data interaction between different intelligent agents in the digital twin analysis layer;

[0009] The digital twin analysis layer is used to use multi-agent deep reinforcement learning technology to complete collaborative analysis between the digital twin analysis layers at the equipment level, process unit level, and production line level; wherein different agents can be called individually, or can be implemented through mutual communication, interaction, collaboration, etc., and complete complex system-level decisions;

[0010] The digital twin application layer is used to realize various digital twin applications at the equipment level, process unit level, and production line level.

[0011] In one possible implementation, the data of the production line geometry model, the data of the production line physical model, the data of the production line behavior model, and the data of the production line rule model are used to form a unified digital twin data base through multi-source data fusion technology, and to realize collaborative analysis and optimization based on the data of the production line geometry model, the data of the production line physical model, the data of the production line behavior model, and the data of the production line rule model in a multi-scale digital twin system.

[0012] In a possible implementation, the production line geometry model is used to characterize the physical production line geometry of the incendiary explosive production line, construct the model from the geometric shape and physical constraint levels, and perform realistic rendering on the model;

[0013] The production line physical model is used to describe the physical characteristics of the production line and equipment;

[0014] The production line behavior model is used to describe the dynamic behavior of the production line operation; a mathematical model is established for the dynamic evolution of the manufacturing elements of the production line over time to dynamically describe the manufacturing elements, organizational structure and operation mechanism of the production line manufacturing process;

[0015] The production line rule model is used to determine the constraint rules that the physical entities of the production line and equipment need to follow during the production process; wherein the constraint rules include production line safety constraint rules and process constraint rules, and the process constraint rules include process flow, process control method and process instrument flow chart, material balance, heat balance, boundary condition requirements, raw material and public engineering specifications and consumption requirements, three waste emissions and treatment methods, safety risks and countermeasures, main equipment principles and specifications and design conditions, plane layout suggestions, equipment start-up and shutdown and operation requirements, and staffing and quantitative requirements.

[0016] In a possible implementation, the production line geometric model includes the geometric dimensions, tolerances, and spatial layout, pipeline connection relationships, and associated attributes of the production line buildings and equipment, and the associated attributes include equipment specifications, technical requirements, process notes, and materials, versions, and dates;

[0017] The physical model of the production line includes an analytical dynamics model, a thermodynamics model, a fluid mechanics model, a stress analysis model, and a kinematics model.

[0018] In one possible implementation, the digital twin analysis layer includes: equipment agent, process unit agent and decision agent.

[0019] In a possible implementation, the equipment agent is used to analyze the spatial interference of physical production line equipment of a combustion and explosive production line, equipment fault diagnosis, equipment fault prediction, and equipment hazard assessment;

[0020] The process unit intelligent agent, based on the virtual-real synchronization mechanism of the production line, uses the real-time data of the incendiary and explosive production line to drive the calculation of the process bottleneck in the process unit, which is then used to calculate and adjust the production line optimization scheduling plan in the decision-making intelligent agent, and perform simulation analysis and verification based on the virtual production line environment in the multi-scale digital twin system of the incendiary and explosive production line;

[0021] The decision-making agent is used to collect global information of the manufacturing process, use the global information to set the goals to be analyzed and optimized in the digital twin analysis layer, perform global planning, evaluate the execution of the process unit agent, evaluate the tasks of the digital twin analysis layer, assign tasks to the computing models and nodes in the digital twin analysis layer, and update the status of the equipment agent and the process unit agent.

[0022] In one possible implementation, the digital twin communication unit uses edge computing technology to achieve adaptive update of the operating status of the virtual production line of the incendiary and explosive production line, so as to optimize data transmission efficiency and maintain the virtual-real synchronization of the digital twin system; through long short-term memory networks and autoencoders, edge calculation and alarm are performed on the process parameters and equipment fault status signals of the physical production line.

[0023] In one possible implementation, the multi-agent deep reinforcement learning technology includes a proximal policy optimization reinforcement learning method.

[0024] In a possible implementation, the proximal policy optimization reinforcement learning method defines a state space, an action space, a global reward function, and an instantaneous reward function;

[0025] The state space is used to represent the environment in which the multi-agents are located;

[0026] The action space is used to represent the set of actions that the multi-agent can take at the corresponding moment in each state space;

[0027] The global reward function is used in the proximal policy optimization reinforcement learning method, where the multi-agents learn and optimize by interacting with the environment. At each moment, the multi-agents update their states based on the interaction with the environment, thereby achieving the goal to be analyzed and optimized in the digital twin analysis layer.

[0028] The instantaneous reward function is used to reward the calculation result at the current moment.

[0029] In a possible implementation, the device-level digital twin application is used for equipment risk assessment, equipment failure analysis and diagnosis, and equipment failure early warning;

[0030] The digital twin application at the process unit level is used for process bottleneck analysis;

[0031] The production line-level digital twin application is used for production rhythm optimization and abnormal operating condition simulation.

[0032] The embodiments of the present invention disclose the following technical effects:

[0033] 1) Aiming at the multi-factor, coupled, process mechanism complexity and difficulty in characterization characteristics of the incendiary explosive production line, the present invention realizes the characterization of multi-dimensional and multi-factor manufacturing process data of the incendiary explosive production line from four dimensions: production line geometric model, production line physical model (spatial constraints, physical field), production line behavior model and production line rule model, and combines the application scenarios at the equipment level, process unit level and production line level, thus providing support for the realization of digital twin applications at the equipment level, process unit level and production line level.

[0034] 2) The embodiment of the present invention uses a digital twin communication unit to realize data interaction between the virtual production line and the physical production line in the multi-scale digital twin system of the incendiary explosive production line, as well as data interaction between different intelligent agents in the digital twin analysis layer. Based on the collaborative computing of the edge and the cloud, the results of equipment fault warning and process safety risk status are transmitted to the physical production line for alarm prompts. Improve the computing efficiency of digital twin application scenarios with high real-time requirements.

[0035] 3) The present invention adopts multi-agent deep reinforcement learning to realize the coupling analysis between multi-scale models at the equipment level, process unit level, and production line level, deduce the dynamic evolution law of the production line over time, and realize the analysis and decision-making of the production line based on the real-time operation status of the production line. Among them, the state space representation method and reward mechanism of multi-agent reinforcement learning proposed in the present invention combine the high-risk and high-dynamic characteristics of the production line of incendiary explosives, and consider the dynamic evolution of the equipment operation status and process safety risk status on the basis of factors such as equipment availability and workpiece status, thereby improving the ability to predict and warn of production line safety risks and promoting the adaptability of the multi-agent system in the complex environment of the production line.

[0036] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 An architecture diagram of a multi-scale digital twin system for a combustion and explosive production line provided by an embodiment of the present invention;

[0039] Figure 2 A schematic diagram for characterizing the manufacturing process of a combustion explosive production line from four dimensions provided by an embodiment of the present invention;

[0040] Figure 3a A schematic diagram of an edge computing technology provided by an embodiment of the present invention;

[0041] Figure 3b A schematic diagram of a cloud computing technology provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0044] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0045] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if determining" or "if monitoring (stated condition or event)" may be interpreted as "when determining" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0046] The production line of incendiary explosives has the characteristics of multiple factors, coupling, complex manufacturing process mechanisms, and difficulty in characterizing the manufacturing process. Since the production line of incendiary explosives is subject to high safety and complex production requirements, the product manufacturing process includes physical and chemical reactions, complex internal mechanisms, and involves a large number of process parameters. The multi-parameter coupling process is extremely complex. Therefore, it is difficult to achieve accurate modeling of digital twins. Further research and application are also needed in terms of product manufacturing process characterization, multi-source data fusion, and model collaborative analysis. In addition, the manufacturing process of the production line of incendiary explosives has the characteristics of being easy to burn and explode, and its digital twin model needs to consider the safety and reliability of the production process. Therefore, the digital twin technology in the field of incendiary explosive production lines currently lacks a complete system and architecture.

[0047] In view of this, an embodiment of the present invention provides a multi-scale digital twin system for a combustion explosive production line. The system combines a multi-agent approach. The system includes: a digital twin data layer, a digital twin communication unit, a digital twin analysis layer, and a digital twin application layer. The specific architecture is as follows: Figure 1 As shown in the figure, the digital twin data layer, digital twin analysis layer, and digital twin application layer are respectively divided into three scales: equipment level, process unit level, and production line level. Therefore, the multi-scale digital twin system includes the equipment-level digital twin system, the process unit-level digital twin system, and the production line-level digital twin system. Among them, the equipment-level digital twin system includes the equipment-level digital twin data layer, the equipment-level digital twin analysis layer, and the equipment-level digital twin application layer. The process unit-level digital twin system includes the process unit-level digital twin data layer, the process unit-level digital twin analysis layer, and the process unit-level digital twin application layer. The production line-level digital twin system includes the production line-level digital twin data layer, the production line-level digital twin analysis layer, and the production line-level digital twin application layer. The equipment-level digital twin system, the process unit-level digital twin system, and the production line-level digital twin system share a digital twin communication unit.

[0048] The multi-scale digital twin system of the above-mentioned incendiary explosive production line and the effects that can be further produced are described in detail below in conjunction with the embodiments of the present invention.

[0049] First, the “digital twin data layer” in the multi-scale digital twin system of the above-mentioned incendiary explosive production line is described in detail in combination with the embodiments of the present invention.

[0050] In the embodiment of the present invention, refer to Figure 2, the manufacturing process of the production line is characterized from four dimensions: production line geometry model, production line physical model (spatial constraints, physical fields), production line behavior model, and production line rule model, realizing the characterization of manufacturing factor data of the incendiary explosive production line, and providing a reference for related research in the field of incendiary explosive production lines. The multi-scale digital twin data layer constructed by the present invention has the beneficial effect of realizing the characterization of manufacturing factor data of the incendiary explosive production line at the equipment level, process unit level, and production line level based on the multi-factor, coupling, process mechanism complexity, and difficulty in characterization characteristics of the incendiary explosive production line. Provide support for the realization of digital twin analysis and digital twin applications.

[0051] Among them, the geometric model of the incendiary explosive production line constructed in the embodiment of the present invention is implemented by using three-dimensional modeling software, and the geometric model of the production line is optimized by using a lightweight three-dimensional model. The beneficial effect is that the smooth loading of the geometric model of the production line under the condition of large data volume is ensured. The production line geometric model is used to characterize the physical production line entity geometry structure of the incendiary explosive production line, and the model is constructed from the geometric shape and physical constraint levels, and the model is realistically rendered. It should be noted that the production line geometry model includes the geometric dimensions, tolerances, spatial layout, pipeline connection relationships and associated attributes of the production line buildings and equipment. Associated attributes include equipment specifications, technical requirements, process notes and materials, versions, and dates.

[0052] The physical model of the production line of incendiary explosives is used to describe the physical characteristics of the production line and equipment. It should be noted that the physical model of the production line includes analytical dynamics model, thermodynamics model, fluid mechanics model, stress analysis model, and kinematics model.

[0053] The behavioral model of the incendiary explosive production line is used to describe the dynamic behavior of the production line operation; a mathematical model is established for the dynamic evolution of the manufacturing elements of the production line over time to dynamically describe the manufacturing elements, organizational structure and operation mechanism of the production line manufacturing process. It should be noted that the production line behavior model includes the robot arm kinematic model and the production line process parameterization model.

[0054] The rule model of the incendiary explosive production line is used to determine the constraint rules that the physical entities of the production line and equipment need to follow during the production process. It should be noted that the constraint rules include production line safety constraint rules and process constraint rules. Among them, the production line safety constraint rules refer to the constraint conditions of the online charge quantity of the production line. In the manufacturing process of the production line of the embodiment of the present invention, the online charge quantity is obtained by reading the number of pallets and the number of products in the programmable logic controller (PLC) of the control system through conversion. Among them, the explosion-proof requirements and explosion distance requirements are met. In the process of safety risk assessment of equipment or production units, the combustion and explosion of dangerous goods are the main source of damage. According to the damage to personnel and the destructiveness to buildings caused by the shock wave of the combustion and explosion, the consequences caused by the risk are defined and calculated.

[0055] The calculation formula for the explosion distance of air burst shock wave is:

[0056]

[0057] in, Δp is the peak overpressure of the shock wave of the explosion of the dangerous goods at the target, r is the distance between the target and the explosion center, w is the TNT equivalent of the burning explosive, is the proportional distance. a, b, c are calculation parameters, which are taken under different boundary conditions.

[0058] Process constraint rules refer to the constraints constructed based on the process, equipment, materials, process parameters, quality parameters, etc. of the production line of incendiary explosives. It should be noted that process constraint rules include process flow, process control method and process instrument flow chart, material balance, heat balance, boundary condition requirements, raw materials and public engineering specifications and consumption requirements, three wastes emission and treatment methods, safety risks and countermeasures, main equipment principles and specifications and design conditions, plane layout suggestions, equipment start-up and shutdown and operation requirements, and staffing and quantitative requirements.

[0059] In an embodiment of the present invention, the digital twin data layer is used to characterize the data of the production line geometry model, the data of the production line physical model, the data of the production line behavior model, and the data of the production line rule model in the multi-scale digital twin system of the combustion and explosive production line. It should be noted that the data of the production line geometry model, the data of the production line physical model, the data of the production line behavior model, and the data of the production line rule model can form a unified digital twin data base through multi-source data fusion technology, and its beneficial effect is to make full use of the multi-dimensional data of the manufacturing process, comprehensively characterize the status of equipment, processes and other elements in the digital twin system, and use the complementary characteristics between multi-dimensional information to improve the accuracy of multi-agent decision-making. Based on the multi-scale digital twin data layer constructed by the present invention, the collaborative analysis and optimization of the data of the production line geometry model, the data of the production line physical model, the data of the production line behavior model, and the data of the production line rule model can be realized in the multi-scale digital twin system.

[0060] The “digital twin communication unit” in the multi-scale digital twin system of the above-mentioned incendiary explosive production line is described in detail below in conjunction with an embodiment of the present invention.

[0061] In an embodiment of the present invention, the digital twin communication unit is used to realize the data interaction between the virtual production line and the physical production line in the multi-scale digital twin system of the incendiary explosive production line, as well as the data interaction between different intelligent agents in the digital twin analysis layer. In terms of data interface, there are many specific data interfaces used for data interaction between the virtual production line and the physical production line and data interaction between different intelligent agents, which may include: Open Platform Communications Unified Architecture (OPC UA) interface, Socket interface, modbus, Transmission Control Protocol (TCP) and other standard communication interfaces and communication protocols.

[0062] It should be noted that the digital twin communication unit uses Figure 3a The edge computing technologies shown are Figure 3bThe cloud computing technology shown cooperates to realize the adaptive update of the operating status of the virtual production line to optimize the data transmission efficiency and maintain the virtual-real synchronization of the digital twin system; a mathematical model is established through a long short-term memory network and an autoencoder to perform edge-end calculation and alarm on the process parameters and equipment fault status signals of the physical production line. Specifically, referring to the calculation process of each parameter in the state space, it involves the digital twin communication unit transmitting the data of the virtual production line to the physical production line, and transmitting the results of the analysis and early warning of the multi-scale digital twin system of the virtual production line to the physical production line, and issuing an alarm prompt on the physical production line. The multi-scale digital twin analysis layer constructed by the present invention has the beneficial effect that it adopts a multi-agent deep reinforcement learning method to construct the targets that need to be analyzed and optimized in the multi-scale digital twin analysis layer, and realizes collaborative analysis between models through the method of data and analysis models of different scales at the equipment level, process unit level, and production line level.

[0063] The “digital twin analysis layer” in the multi-scale digital twin system of the above-mentioned incendiary explosive production line is described in detail below in conjunction with an embodiment of the present invention.

[0064] In an embodiment of the present invention, the digital twin analysis layer is used to use multi-agent deep reinforcement learning technology to complete the collaborative analysis between the digital twin analysis layers at the equipment level, process unit level, and production line level. Different agents can be called individually, or they can be implemented through mutual communication, interaction, collaboration, etc., and complete system-level decisions. It should be noted that system-level decisions are achieved through collaborative analysis between the digital twin analysis layers at the equipment level, process unit level, and production line level. In the digital twin analysis layer, the present invention uses multi-agent deep reinforcement learning technology to construct the target to be analyzed and optimized in the multi-scale digital twin analysis layer, and completes the collaborative analysis between the digital twin analysis layers at the equipment level, process unit level, and production line level. Among them, the multi-agent deep reinforcement learning technology includes a proximal policy optimization reinforcement learning method. The proximal policy optimization reinforcement learning method defines a state space, an action space, a global reward function, and an instantaneous reward function. The multi-scale digital twin analysis layer proposed by the present invention has the beneficial effect of combining the high-risk characteristics of the combustion and explosive production line, adding online drug quantity and other production safety constraints in the process unit agent to build the production line rule model, and combining equipment health status, equipment availability, process parameter trend prediction and other factors in the equipment agent to update the multi-agent reinforcement learning state space, and then perform multi-agent analysis and optimization. Instead of only building the state space through equipment utilization and workpiece status, it is more in line with the characteristics of the combustion and explosive production line manufacturing process.

[0065] The state space is used to characterize the environment in which the multi-agent is located. For the embodiment of the present invention, the state space characterizes the state of the equipment, workpiece, and process. In the equipment agent, the equipment availability h1 and the current equipment operation state h2 are combined. The workpiece information includes the workpiece state information W1 that the agent has not processed, and the workpiece state information W2 that the agent can currently choose to process; the process state includes the online dosage state V1 of the current processing process, the current safety risk state V2 of the process and other factors to update the multi-agent reinforcement learning state space, and then perform multi-agent analysis and optimization. Instead of only constructing the state space through the utilization rate of the equipment and the state of the workpiece, it is more in line with the characteristics of the manufacturing process of the combustion and explosive production line. The state space characterization method proposed in the present invention has a beneficial effect in that it combines the high-risk characteristics of the combustion and explosive production line, realizes the calculation of the equipment operation state h2 and the process safety risk state V2, instead of only considering the static characteristics of the equipment and process, thereby improving the ability to predict and warn of equipment and process safety risks.

[0066] Specifically, in the state space designed by the embodiment of the present invention, the equipment information includes the equipment availability h1 and the current equipment operation status h2. The workpiece information includes the workpiece status information W1 that the agent has not processed and the workpiece status information W2 that the agent can currently choose to process; the process status includes the online dosage status V1 of the current processing process and the process safety risk status V2. The current state of the state space is represented by S = (h1, h2, W1, W2, V1, V2). The calculation of each parameter in the state space is as follows:

[0067] Equipment availability h1 = equipment average failure interval / (equipment average failure interval + average repair time), where the current equipment operating state h2 refers to the equipment failure state, which is calculated by the equipment failure prediction model based on autoencoders and long short-term memory networks (LSTM). The equipment failure prediction model based on autoencoders and LSTM used in the present invention is constructed using an autoencoder model and an LSTM model.

[0068] The online drug volume V1 of each process is updated in real time according to the sensing monitoring data of the number of WIP in the workshop (for example, the number of pallets). Figure 2 The technical architecture uses sensors deployed on the physical production line to perceive and monitor in real time, obtain the number of pallets through the data interface, and then convert it into online medicine dosage.

[0069] The process safety risk status V2 establishes a safety risk parameter system through safety risk hidden danger identification, builds a model through long short-term memory network (LSTM), autoencoders and other methods, and obtains it through real-time access to process parameters and equipment vibration parameters through real-time calculation at the edge.

[0070] The action space is used to represent the set of actions that multiple agents can take at the corresponding moment in each state space, represented by A.

[0071] At each scheduling time node, the multi-agent needs to make independent decisions based on the current observations. This decision is the workpiece that the multi-agent should choose to process at the current moment. The result of this decision is called an action. In the embodiment of the present invention, the action of the multi-agent at the current moment consists of the equipment action space M, the workpiece action space, and the process action space. The greedy strategy ε-greedy is used for action selection to screen out the current set of equipment, workpieces, and processes that meet the conditions.

[0072] It should be noted that in the embodiment of the present invention, it is necessary to establish a communication mechanism between intelligent agents through a digital twin communication unit, and to realize real-time updating of the data, equipment, process, and production plan status of the digital twin virtual production line based on collaborative analysis between the edge and the cloud. Equipment fault diagnosis and early warning are realized based on the equipment intelligent agent, and the production rhythm of the process unit is analyzed based on the process unit intelligent agent, and the status of the equipment, process, and workpiece is updated. The digital twin communication unit proposed in the present invention has the beneficial effect of establishing a mathematical model through a long short-term memory network and an autoencoder, and performing edge calculation and alarm on the process parameters and equipment fault status signals of the physical production line, thereby enhancing the prediction ability of the digital twin model and system. In addition, through the implementation of edge computing, the alarm response speed is effectively improved, providing support for improving the safety of the production process.

[0073] The global reward function is used in the proximal policy optimization reinforcement learning method. Multi-agents achieve learning and optimization through interaction with the environment. At each moment, multi-agents update their states based on the interaction with the environment, thereby achieving the goals to be analyzed and optimized in the digital twin analysis layer.

[0074] The goal of the global reward function of the present invention is to achieve the goal of optimizing the production line scheduling, that is, to minimize the total delay. A full cooperation mechanism is adopted between the multiple agents in the present invention, and this study designs a global reward function to optimize the total delay. In the reinforcement learning algorithm, the goal is to maximize the cumulative reward, so the increasing direction of the reward function value should be consistent with the decreasing direction of the optimization target value. Therefore, the present invention adopts the Sigmoid function to design a global reward mechanism. In this way, the reward function designed by the present invention can effectively achieve the multi-agent optimization goal. The calculation formula of the reward function at time t is as follows:

[0075] In the multi-agent deep reinforcement learning process, the present invention uses Markov game to solve the optimal strategy, and the optimization target is calculated using the following formula:

[0076]

[0077] Among them, V is the value function of multi-agent reinforcement learning, t refers to the intermediate moment from the initial operation to the termination of the iteration of the digital twin system; S refers to the state space, which changes with time; R is the reward function value of the kth agent (when k = 0, it corresponds to the equipment agent, when k = 1, it corresponds to the process unit agent, when k = 2, it corresponds to the decision-making agent) at time t, and Υ refers to the discount coefficient, which is used to calculate the importance of the reward function value of the agent at time t. The larger the discount coefficient, the lower the importance.

[0078] The goal of the global reward function in the embodiment of the present invention is to achieve the goal of optimizing the scheduling of the production line, that is, to minimize the total delay. A full cooperation mechanism is adopted between the multiple agents in the embodiment of the present invention. This study designs a global reward function to optimize the total delay. In the reinforcement learning algorithm, the goal is to maximize the cumulative reward, so the increasing direction of the reward function value should be consistent with the decreasing direction of the optimization target value. Therefore, the present invention adopts the Sigmoid function to design a global reward mechanism. In this way, the global reward function designed by the present invention can effectively achieve the multi-agent optimization goal. The calculation formula of the global reward function at time t is as follows:

[0079]

[0080] in, P is the type of product; G is the total delay of the product; D i is the delivery date of product i; E i is the completion time of product i. t refers to the intermediate time from the initial operation to the termination of the iteration of the digital twin system; r is the reward function value of the kth agent at time t.

[0081] It should be noted that the embodiment of the present invention uses the key parameters such as the initial planning schedule, equipment operation status, workpiece status, process status, etc. to construct an action selection strategy, and completes the collaborative analysis between the digital twin analysis layers of the equipment level, process unit level, and production line level through multi-agent deep reinforcement learning. Under the framework of the multi-agent deep reinforcement learning algorithm, each agent realizes the learning and optimization of the strategy through continuous interaction with the environment. During each interaction, the agent first observes the local state information of the environment, makes decisions based on the current strategy framework, and performs corresponding actions. After the action is executed, the agent evaluates the effectiveness of its decision based on the reward signal fed back by the environment. Subsequently, the agent updates the global reward function based on the reward value obtained, so as to make better decisions in subsequent interactions. Through this iterative interaction and state update mechanism, the agent gradually improves and improves its strategy by continuously interacting with the environment and updating the state, and finally converges to the optimal decision strategy.

[0082] The action selection strategy proposed in the embodiment of the present invention has the beneficial effect of being able to dynamically select the equipment to execute the rejection process in the process unit intelligent body under the premise of satisfying the production line process constraints, online drug dosage constraints and other rules, combined with the failure conditions of equipment operation, thereby improving the utilization efficiency of equipment resources and obtaining the optimal action selection strategy.

[0083] In view of the multi-agent collaborative analysis and decision-making process, an embodiment of the present invention designs and implements a conflict handling mechanism. Specifically, after completing the action selection, each agent marks the equipment, process, and workpiece corresponding to the selected action space as unavailable to prevent repeated selection in subsequent calculations. In addition, for equipment with a fault warning in the equipment health status, it is also set to an unavailable state to ensure the safety and efficiency of the system. In order to further improve the utilization efficiency of the equipment and achieve the balance of the task load, the embodiment of the present invention uses the equipment utilization rate and the equipment health status as the key basis for the equipment selection sorting. By optimizing the equipment selection strategy, the utilization rate of idle equipment can be effectively improved, avoiding the selection of equipment with potential faults, while ensuring the balanced distribution of task loads. In the design of the reward mechanism, the embodiment of the present invention associates instantaneous rewards and cumulative rewards with the idle time of the equipment. The beneficial effect of the present invention is that by optimizing the action selection and reward mechanism of the agent, the efficiency of the multi-agent system can be significantly improved.

[0084] In summary, the conflict handling mechanism and optimized reward mechanism proposed in the embodiment of the present invention provide an effective strategy for the multi-agent system in the collaborative analysis and decision-making process, which can achieve efficient use of equipment and balanced distribution of task loads. The beneficial effect of the reward mechanism designed in the embodiment of the present invention is that it can promote the stability and adaptability of the multi-agent system in a complex environment, and provide a reference for research and application in related fields.

[0085] The instantaneous reward function is used to reward the calculation result at the current moment.

[0086] Among them, the digital twin analysis layer includes equipment agents, process unit agents and decision agents. It should be noted that the equipment agent is used to analyze the spatial interference of physical production line equipment, and combined with the high-risk characteristics of the incendiary explosive production line, it carries out mechanism- and data-driven equipment fault diagnosis, equipment fault prediction, and equipment hazard assessment. Based on the virtual-real synchronization mechanism of the production line, the process unit agent uses the real-time data of the incendiary explosive production line to drive the calculation of the process bottleneck in the process unit, which is then used to calculate and adjust the production line optimization scheduling plan in the decision agent, and simulate and verify the virtual production line environment in the multi-scale digital twin system of the incendiary explosive production line. The decision agent is used to collect global information of the manufacturing process, use the global information to set the target to be analyzed and optimized in the digital twin analysis layer, perform global planning, and evaluate the execution of the process unit agent, evaluate the tasks of the digital twin analysis layer, assign tasks to the calculation models and nodes in the digital twin analysis layer, and update the status of the equipment agent and the process unit agent. The aforementioned global information can be production scheduling and production plan information.

[0087] The “digital twin application layer” in the multi-scale digital twin system of the above-mentioned incendiary explosive production line is described in detail below in conjunction with an embodiment of the present invention.

[0088] In the embodiment of the present invention, the digital twin application layer is used to implement various digital twin applications at the equipment level, process unit level, and production line level. Among them, the digital twin application at the equipment level is used for equipment hazard assessment, equipment fault analysis and diagnosis, and equipment fault warning. The digital twin application at the process unit level is used for process bottleneck analysis. The digital twin application at the production line level is used for production cycle optimization and abnormal working condition deduction.

[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, rather than to limit them. Although the embodiments of the present invention have been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-scale digital twin system for a combustion and explosive production line, characterized in that: include: Digital twin data layer, digital twin communication unit, digital twin analysis layer, digital twin application layer; among them, the digital twin data layer, digital twin analysis layer, and digital twin application layer are divided into three scales: equipment level, process unit level, and production line level; The digital twin data layer is used to characterize the data of the production line geometry model, the data of the production line physical model, the data of the production line behavior model, and the data of the production line rule model in the multi-scale digital twin system of the combustion and explosive production line; The digital twin communication unit is used to realize data interaction between the virtual production line and the physical production line in the multi-scale digital twin system of the combustion and explosive production line, and data interaction between different intelligent agents in the digital twin analysis layer; The digital twin analysis layer is used to complete the collaborative analysis between the equipment-level, process unit-level, and production line-level digital twin analysis layers using multi-agent deep reinforcement learning technology; wherein different agents can be called separately, and can be implemented through mutual communication, interaction, and collaboration, and complete system-level decision-making; The digital twin application layer is used to realize various digital twin applications at the equipment level, process unit level, and production line level.

2. The system according to claim 1, characterized in that The data of the production line geometry model, the data of the production line physical model, the data of the production line behavior model, and the data of the production line rule model are used to form a unified digital twin data base through multi-source data fusion technology, and to realize collaborative analysis and optimization based on the data of the production line geometry model, the data of the production line physical model, the data of the production line behavior model, and the data of the production line rule model in the multi-scale digital twin system.

3. The system according to claim 1 or 2, characterized in that: The production line geometry model is used to characterize the physical production line geometry of the incendiary explosives production line, construct the model from the geometric shape and physical constraint levels, and render the model realistically; The production line physical model is used to describe the physical characteristics of the production line and equipment; The production line behavior model is used to describe the dynamic behavior of the production line operation; a mathematical model is established for the dynamic evolution of the manufacturing elements of the production line over time to dynamically describe the manufacturing elements, organizational structure and operation mechanism of the production line manufacturing process; The production line rule model is used to determine the constraint rules that the physical entities of the production line and equipment need to follow during the production process; wherein the constraint rules include production line safety constraint rules and process constraint rules, and the process constraint rules include process flow, process control method and process instrument flow chart, material balance, heat balance, boundary condition requirements, raw material and public engineering specifications and consumption requirements, three waste emissions and treatment methods, safety risks and countermeasures, main equipment principles and specifications and design conditions, plane layout suggestions, equipment start-up and shutdown and operation requirements, and staffing and quantitative requirements.

4. The system according to claim 3, characterized in that The production line geometric model includes the geometric dimensions, tolerances, spatial layout, pipeline connection relationships and associated attributes of the production line buildings and equipment, and the associated attributes include equipment specifications, technical requirements, process notes and materials, versions, and dates; The physical model of the production line includes an analytical dynamics model, a thermodynamics model, a fluid mechanics model, a stress analysis model, and a kinematics model.

5. The system according to claim 1, characterized in that The digital twin analysis layer includes: equipment agent, process unit agent and decision-making agent.

6. The system according to claim 5, characterized in that The equipment intelligent agent is used to analyze the spatial interference of physical production line equipment of the combustion and explosive production line, equipment fault diagnosis, equipment fault prediction, and equipment hazard assessment; The process unit intelligent agent, based on the virtual-real synchronization mechanism of the production line, uses the real-time data of the incendiary and explosive production line to drive the calculation of the process bottleneck in the process unit, which is then used to calculate and adjust the production line optimization scheduling plan in the decision-making intelligent agent, and perform simulation analysis and verification based on the virtual production line environment in the multi-scale digital twin system of the incendiary and explosive production line; The decision-making agent is used to collect global information of the manufacturing process, use the global information to set the goals to be analyzed and optimized in the digital twin analysis layer, perform global planning, evaluate the execution of the process unit agent, evaluate the tasks of the digital twin analysis layer, assign tasks to the computing models and nodes in the digital twin analysis layer, and update the status of the equipment agent and the process unit agent.

7. The system according to claim 1, characterized in that The digital twin communication unit uses edge computing technology to achieve adaptive updates of the operating status of the virtual production line of the incendiary and explosive production line, so as to optimize data transmission efficiency and maintain the virtual-real synchronization of the digital twin system; through long short-term memory networks and autoencoders, edge calculations and alarms are performed on the process parameters and equipment fault status signals of the physical production line.

8. The system according to claim 1, characterized in that The multi-agent deep reinforcement learning technology includes a proximal strategy optimization reinforcement learning method.

9. The system according to claim 8, characterized in that The proximal policy optimization reinforcement learning method defines a state space, an action space, a global reward function and an instantaneous reward function; The state space is used to represent the environment in which the multi-agents are located; The action space is used to represent the set of actions that the multi-agent can take at the corresponding moment in each state space; The global reward function is used in the proximal policy optimization reinforcement learning method, where the multi-agents learn and optimize by interacting with the environment. At each moment, the multi-agents update their states based on the interaction with the environment, thereby achieving the goal to be analyzed and optimized in the digital twin analysis layer. The instantaneous reward function is used to reward the calculation result at the current moment.

10. The system according to claim 1, characterized in that The device-level digital twin application is used for equipment risk assessment, equipment failure analysis and diagnosis, and equipment failure warning; The digital twin application at the process unit level is used for process bottleneck analysis; The production line-level digital twin application is used for production rhythm optimization and abnormal operating condition simulation.

Images