A multi-scale digital twin system for a combustion and explosives production line

Abstract

An embodiment of the present invention provides a multi-scale digital twin system for a combustible explosive production line, comprising: a digital twin data layer that characterizes the manufacturing process and its dynamic evolution of the combustible explosive production line through geometric model data, physical model data, behavioral model data, and rule model data of the combustible explosive production line; a digital twin communication unit that enables data interaction between the digital twin virtual production line and the physical production line, as well as data interaction between different agents in the digital twin analysis layer; a digital twin analysis layer that uses multi-agent deep reinforcement learning technology to complete collaborative analysis of models in the multi-scale digital twin analysis layer; and a digital twin application layer that implements multi-scale digital twin applications. The present invention combines the high-risk characteristics of combustible explosive production lines with the calculation of equipment operating status and safety risk status for analysis and decision-making, thereby improving the ability to characterize and predict safety risks of combustible explosive production lines.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of industrial digitalization and intelligent technology, and specifically to a multi-scale digital twin system for a combustion and explosive production line. Background Art

[0002] In recent years, the concept, framework, and technology of digital twins have been gradually researched and applied in the manufacturing industry. Digital twin technology achieves precise digital mapping of physical objects in real time within a digital space, enabling optimal decision-making based on analysis and prediction, thereby optimizing the entire manufacturing process. Currently, some researchers have proposed the concept and system architecture of a digital twin workshop, dividing it into the virtual workshop, the physical workshop, the workshop service system, and the workshop digital twin data. By constructing a multi-dimensional workshop model encompassing geometry, physics, behavior, and rules, the physical workshop is realistically depicted and described. Other researchers have proposed the connotations and characteristics of multi-dimensional and multi-scale intelligent manufacturing spaces. Incorporating the implementation logic of digital twin technology, they have studied modeling methods for intelligent manufacturing spaces and virtual-to-physical mapping methods in complex multi-dimensional spatiotemporal domains. Other researchers have proposed corresponding digital twin model paradigms based on applications in aerospace testing and automotive manufacturing. In the manufacturing sector, some researchers have developed a prototype system for discrete workshops based on a knowledge-driven multi-dimensional and multi-scale digital twin modeling approach for manufacturing processes. Other researchers have analyzed the characteristics of digital twin systems for human-machine interaction and established a reference architecture for human-machine-environment collaborative analysis. They have also developed key technologies, including 3D geometric modeling based on generative AI and optimal control strategies based on human-machine collaboration. Other researchers have studied manufacturing process simulation optimization using evolutionary algorithms, deep learning, and multi-agent technology. These studies primarily address production scheduling issues. Other researchers have proposed simulating spacecraft intelligent testing processes using a combination of multi-agent and digital twin technologies.

[0003] While digital twin technology has made significant research progress in various fields, achieving accurate digital twin modeling is challenging due to the high safety and complex production requirements of incendiary explosive production lines. The manufacturing process involves physical and chemical reactions, complex internal mechanisms, numerous process parameters, and extremely complex multi-parameter coupling. Further research and application are needed in areas such as manufacturing process characterization, multi-source data fusion, and model collaborative analysis. Furthermore, the manufacturing process of incendiary explosive production lines is characterized by high flammability and explosion, high dynamics, and nonlinear coupling with safety risks. Therefore, digital twin models for these production lines must consider the safety and reliability of the production process. Therefore, digital twin technology for incendiary explosive production lines currently lacks a comprehensive system and architecture. Summary of the Invention

[0004] An embodiment of the present invention provides a multi-scale digital twin system for a combustible explosive production line, integrating multi-agent deep reinforcement learning and edge and cloud collaborative analysis technologies. Targeting the high-risk, high-dynamic, and multi-factor coupled characteristics of combustible explosive production lines, it addresses the current lack of a complete digital twin system and architecture technology in this field.

[0005] In a first aspect, an embodiment of the present invention provides a multi-scale digital twin system for a combustion and explosive production line, comprising:

[0006] Digital twin data layer, digital twin communication unit, digital twin analysis layer, and digital twin application layer; the digital twin data layer, digital twin analysis layer, and digital twin application layer are divided into three scales: equipment level, process unit level, and production line level;

[0007] The digital twin data layer is used to characterize the data of the production line geometry model, the data of the production line physical model, the data of the production line behavior model, and the data of the production line rule model in the multi-scale digital twin system of the combustible explosive production line;

[0008] The digital twin communication unit is used to realize data interaction between the virtual production line and the physical production line in the multi-scale digital twin system of the incendiary explosive production line, as well as data interaction between different intelligent agents in the digital twin analysis layer;

[0009] The digital twin analysis layer is used to implement collaborative analysis between the equipment-level, process unit-level, and production line-level digital twin analysis layers using multi-agent deep reinforcement learning technology. Different agents can be called individually or through mutual communication, interaction, and collaboration to complete complex system-level decisions.

[0010] The digital twin application layer is used to implement various digital twin applications at the equipment level, process unit level, and production line level.

[0011] In one possible implementation, the data of the production line geometric model, the data of the production line physical model, the data of the production line behavior model, and the data of the production line rule model are used to form a unified digital twin data base through multi-source data fusion technology, and to realize collaborative analysis and optimization based on the data of the production line geometric model, the data of the production line physical model, the data of the production line behavior model, and the data of the production line rule model in a multi-scale digital twin system.

[0012] In one possible implementation, the production line geometric model is used to represent the physical production line geometry of the incendiary explosives production line, construct the model from the perspective of geometric shape and physical constraints, and perform realistic rendering on the model;

[0013] The production line physical model is used to describe the physical characteristics of the production line and equipment;

[0014] The production line behavior model is used to describe the dynamic behavior of the production line operation; a mathematical model is established for the dynamic evolution of the production line's manufacturing elements over time to dynamically describe the manufacturing elements, organizational structure and operating mechanism of the production line manufacturing process;

[0015] The production line rule model is used to determine the constraint rules that the physical entities of the production line and equipment need to follow during the production process; wherein, the constraint rules include production line safety constraint rules and process constraint rules, and the process constraint rules include process flow, process control methods and process instrument flow charts, material balance, heat balance, boundary condition requirements, raw material and public engineering specifications and consumption requirements, three waste emissions and treatment methods, safety risks and response measures, main equipment principles and specifications and design conditions, floor plan recommendations, equipment start-up and shutdown and operation requirements, and staffing and quantity requirements.

[0016] In one possible implementation, the production line geometric model includes the geometric dimensions and tolerances of the production line buildings and equipment, as well as the spatial layout, pipeline connection relationships, and associated attributes. The associated attributes include equipment specifications, technical requirements, process notes, materials, versions, and dates.

[0017] The physical model of the production line includes an analytical dynamics model, a thermodynamics model, a fluid dynamics model, a stress analysis model, and a kinematics model.

[0018] In one possible implementation, the digital twin analysis layer includes: an equipment agent, a process unit agent, and a decision-making agent.

[0019] In one possible implementation, the equipment agent is used to analyze spatial interference, equipment fault diagnosis, equipment fault prediction, and equipment hazard assessment of physical production line equipment in a combustion and explosives production line;

[0020] The process unit agent, based on the virtual-real synchronization mechanism of the production line, uses real-time data from the incendiary and explosive production line to drive the calculation of process bottlenecks in the process unit. This data is then used in the decision-making agent to calculate and adjust the production line optimization scheduling plan, and simulation analysis and verification are performed based on the virtual production line environment in the multi-scale digital twin system of the incendiary and explosive production line.

[0021] The decision-making agent is used to collect global information of the manufacturing process, use the global information to set the goals to be analyzed and optimized in the digital twin analysis layer, perform global planning, evaluate the execution of the process unit agent, evaluate the tasks of the digital twin analysis layer, assign tasks to the computing models and nodes in the digital twin analysis layer, and update the status of the equipment agent and the process unit agent.

[0022] In one possible implementation, the digital twin communication unit uses edge computing technology to achieve adaptive updates of the operating status of the virtual production line of the incendiary explosives production line, so as to optimize data transmission efficiency and maintain the virtual-real synchronization of the digital twin system; through long-short-term memory networks and autoencoders, edge computing and alarms are performed on the process parameters and equipment fault status signals of the physical production line.

[0023] In one possible implementation, the multi-agent deep reinforcement learning technology includes a proximal policy optimization reinforcement learning method.

[0024] In one possible implementation, the proximal policy optimization reinforcement learning method defines a state space, an action space, a global reward function, and an instantaneous reward function;

[0025] The state space is used to represent the environment in which the multi-agent is located;

[0026] The action space is used to represent the set of actions that the multi-agent can take at each moment in the state space;

[0027] The global reward function is used in the proximal policy optimization reinforcement learning method, where multiple agents learn and optimize through interaction with the environment. At each moment, the multi-agent updates its state based on the interaction with the environment, thereby achieving the goal to be analyzed and optimized in the digital twin analysis layer;

[0028] The instantaneous reward function is used to reward the calculation result at the current moment.

[0029] In one possible implementation, the device-level digital twin application is used for equipment risk assessment, equipment failure analysis and diagnosis, and equipment failure early warning;

[0030] The digital twin application at the process unit level is used for process bottleneck analysis;

[0031] The production line-level digital twin application is used for production rhythm optimization and abnormal operating condition simulation.

[0032] The embodiments of the present invention disclose the following technical effects:

[0033] 1) Aiming at the multi-factor, coupled, process mechanism complexity and difficulty in characterization characteristics of the incendiary explosive production line, the present invention realizes the characterization of multi-dimensional and multi-factor manufacturing process data of the incendiary explosive production line from four dimensions: production line geometric model, production line physical model (spatial constraints, physical field), production line behavior model and production line rule model, and combines the application scenarios at the equipment level, process unit level and production line level, providing support for the realization of digital twin applications at the equipment level, process unit level and production line level.

[0034] 2) This embodiment of the present invention uses a digital twin communication unit to enable data exchange between the virtual and physical production lines within a multi-scale digital twin system for an explosives production line, as well as between different agents within the digital twin analysis layer. Through collaborative computing between the edge and the cloud, equipment failure warnings and process safety risk status are transmitted to the physical production line for alert notification. This improves computing efficiency in digital twin application scenarios with high real-time requirements.

[0035] 3) The present invention uses multi-agent deep reinforcement learning to implement coupling analysis between multi-scale models at the equipment level, process unit level, and production line level, deduce the dynamic evolution of the production line over time, and implement production line analysis and decision-making based on the real-time operating status of the production line. Among them, the state space representation method and reward mechanism of multi-agent reinforcement learning proposed in the present invention combines the high-risk and high-dynamic characteristics of the combustion and explosive production line. Based on factors such as equipment availability and workpiece status, it considers the dynamic evolution of equipment operating status and process safety risk status, thereby improving the ability to predict and warn of production line safety risks and promoting the adaptability of the multi-agent system in the complex environment of the production line.

[0036] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 This is an architectural diagram of a multi-scale digital twin system for a combustion and explosives production line provided by an embodiment of the present invention;

[0039] Figure 2 A schematic diagram illustrating the manufacturing process of an incendiary explosive production line from four dimensions provided by an embodiment of the present invention;

[0040] Figure 3a A schematic diagram of edge computing technology provided by an embodiment of the present invention;

[0041] Figure 3b A schematic diagram of the cloud computing technology provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0043] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "an", "the" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise.

[0044] It should be understood that the term "and / or" as used herein is merely a description of the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the associated objects are in an "or" relationship.

[0045] The word "if," as used herein, may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if monitoring (stated condition or event)" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)," depending on the context.

[0046] Incendiary explosive production lines are characterized by multi-factor, coupled nature, and complex manufacturing processes, making them difficult to characterize. Due to the high safety and complex production requirements, the manufacturing process involves physical and chemical reactions, complex internal mechanisms, and a large number of process parameters. The multi-parameter coupling process is extremely complex, making accurate digital twin modeling challenging. Further research and application are needed in areas such as product manufacturing process characterization, multi-source data fusion, and model collaborative analysis. Furthermore, the manufacturing process of incendiary explosive production lines is prone to flammability and explosion, so digital twin models must consider the safety and reliability of the production process. Therefore, digital twin technology for incendiary explosive production lines currently lacks a comprehensive system and architecture.

[0047] In view of this, the embodiment of the present invention provides a multi-scale digital twin system for a combustion explosive production line. The system combines a multi-agent approach. The system includes: a digital twin data layer, a digital twin communication unit, a digital twin analysis layer, and a digital twin application layer. The specific architecture is as follows: Figure 1 As shown in the figure, the digital twin data layer, digital twin analysis layer, and digital twin application layer are divided into three scales: device level, process unit level, and production line level. Therefore, the multi-scale digital twin system includes the device-level digital twin system, the process unit-level digital twin system, and the production line-level digital twin system. The device-level digital twin system includes the device-level digital twin data layer, the device-level digital twin analysis layer, and the device-level digital twin application layer. The process unit-level digital twin system includes the process unit-level digital twin data layer, the process unit-level digital twin analysis layer, and the process unit-level digital twin application layer. The production line-level digital twin system includes the production line-level digital twin data layer, the production line-level digital twin analysis layer, and the production line-level digital twin application layer. The device-level digital twin system, the process unit-level digital twin system, and the production line-level digital twin system share a digital twin communication unit.

[0048] The multi-scale digital twin system of the above-mentioned incendiary explosive production line and the effects that can be further produced are described in detail below in conjunction with the embodiments of the present invention.

[0049] First, the “digital twin data layer” in the multi-scale digital twin system of the above-mentioned incendiary explosive production line is described in detail in combination with the embodiments of the present invention.

[0050] In the embodiment of the present invention, see Figure 2, the production line manufacturing process is characterized from four dimensions: production line geometry model, production line physical model (spatial constraints, physical field), production line behavior model, and production line rule model, realizing the characterization of manufacturing factor data of the combustible explosive production line, and providing a reference for related research in the field of combustible explosive production lines. The multi-scale digital twin data layer constructed by the present invention has the beneficial effect of realizing the characterization of manufacturing factor data of the combustible explosive production line at the equipment level, process unit level, and production line level based on the multi-factor, coupling, process mechanism complexity, and difficulty in characterization characteristics of the combustible explosive production line. It provides support for the realization of digital twin analysis and digital twin applications.

[0051] The geometric model of the incendiary explosives production line constructed in this embodiment of the present invention is implemented using 3D modeling software. This optimization utilizes lightweight 3D models to ensure smooth loading of the production line geometric model even with large amounts of data. The production line geometric model is used to characterize the physical geometry of the incendiary explosives production line. The model is constructed from the perspectives of geometric shape and physical constraints, and the model is realistically rendered. It should be noted that the production line geometric model includes the geometric dimensions, tolerances, spatial layout, pipeline connections, and associated attributes of the production line buildings and equipment. Associated attributes include equipment specifications, technical requirements, process notes, materials, versions, and dates.

[0052] The physical model of an incendiary explosives production line is used to describe the physical characteristics of the production line and its equipment. It should be noted that the production line physical model includes analytical dynamics models, thermodynamic models, fluid dynamics models, stress analysis models, and kinematic models.

[0053] The behavioral model of an incendiary explosives production line describes the dynamic behavior of the production line. A mathematical model is established for the dynamic evolution of the production line's manufacturing elements over time, dynamically describing the manufacturing elements, organizational structure, and operational mechanisms of the production process. It should be noted that the production line behavioral model includes a kinematic model of the robotic arm and a parametric model of the production line's process flow.

[0054] The rule model of the production line of incendiary explosives is used to determine the constraint rules that the physical entities of the production line and equipment need to follow during the production process. It should be noted that the constraint rules include production line safety constraint rules and process constraint rules. Among them, the production line safety constraint rules refer to the constraint conditions of the online charge quantity of the production line. In the embodiment of the present invention, the online charge quantity in the production line manufacturing process is obtained by reading the data of the number of pallets and the number of products in the programmable logic controller (PLC) of the control system through conversion. Among them, in order to meet the explosion-proof requirements and explosion distance requirements, in the process of safety risk assessment of equipment or production units, the combustion and explosion of dangerous goods are the main sources of damage. According to the damage to personnel and the destructiveness to buildings caused by the shock wave of the combustion and explosion, the consequences caused by the risk are defined and calculated.

[0055] The calculation formula for the explosion distance of the airburst shock wave is:

[0056]

[0057] in, Δp is the peak overpressure of the shock wave caused by the explosion of dangerous goods at the target, r is the distance between the target and the explosion center, and w is the TNT equivalent of the burning explosive. is the proportional distance. a, b, and c are calculation parameters, which are determined under different boundary conditions.

[0058] Process constraints refer to constraints established based on the procedures, equipment, materials, process parameters, and quality parameters of the explosives production line. They include process flow charts, process control methods, instrument flow charts, material balances, heat balances, boundary conditions, raw material and utility specifications and consumption requirements, waste gas emissions and treatment methods, safety risks and recommended countermeasures, key equipment principles, specifications, and design requirements, layout recommendations, equipment startup and shutdown, and operational requirements, and staffing and quantity requirements.

[0059] In an embodiment of the present invention, the digital twin data layer is used to characterize the data of the production line geometry model, the data of the production line physical model, the data of the production line behavior model, and the data of the production line rule model in the multi-scale digital twin system of the incendiary explosive production line. It should be noted that the data of the production line geometry model, the data of the production line physical model, the data of the production line behavior model, and the data of the production line rule model can form a unified digital twin data base through multi-source data fusion technology. Its beneficial effect is to make full use of the multi-dimensional data of the manufacturing process, comprehensively characterize the status of equipment, processes and other elements in the digital twin system, and utilize the complementary characteristics between multi-dimensional information to improve the accuracy of multi-agent decision-making. The multi-scale digital twin data layer constructed based on the present invention can realize collaborative analysis and optimization based on the data of the production line geometry model, the data of the production line physical model, the data of the production line behavior model, and the data of the production line rule model in the multi-scale digital twin system.

[0060] The “digital twin communication unit” in the multi-scale digital twin system of the above-mentioned incendiary explosive production line is described in detail below in conjunction with an embodiment of the present invention.

[0061] In an embodiment of the present invention, a digital twin communication unit is used to enable data exchange between the virtual and physical production lines within a multi-scale digital twin system for an explosives production line, as well as data exchange between different agents within the digital twin analysis layer. In terms of data interfaces, the data exchange between the virtual and physical production lines, as well as between different agents, utilizes a variety of specific data interfaces, including standard communication interfaces and protocols such as the Open Platform Communications Unified Architecture (OPC UA) interface, Socket interface, Modbus, and Transmission Control Protocol (TCP).

[0062] It should be noted that the digital twin communication unit uses Figure 3a The edge computing technology shown and Figure 3bThe cloud computing technology shown cooperates to realize the adaptive update of the operating status of the virtual production line to optimize the data transmission efficiency and maintain the virtual-real synchronization of the digital twin system; a mathematical model is established through a long short-term memory network and an autoencoder to perform edge-end calculation and alarm on the process parameters and equipment fault status signals of the physical production line. Specifically, referring to the calculation process of each parameter in the state space, it involves the digital twin communication unit transmitting the data of the virtual production line to the physical production line, and transmitting the results of the analysis and early warning of the multi-scale digital twin system of the virtual production line to the physical production line, and issuing an alarm prompt on the physical production line. The multi-scale digital twin analysis layer constructed by the present invention has the beneficial effect of adopting a multi-agent deep reinforcement learning method to construct the targets that need to be analyzed and optimized in the multi-scale digital twin analysis layer, and the data and analysis models of different scales at the equipment level, process unit level, and production line level are realized through this method. Collaborative analysis between models is achieved.

[0063] The “digital twin analysis layer” in the multi-scale digital twin system of the above-mentioned incendiary explosive production line is described in detail below in conjunction with an embodiment of the present invention.

[0064] In an embodiment of the present invention, the digital twin analysis layer is used to use multi-agent deep reinforcement learning technology to complete the collaborative analysis between the digital twin analysis layers at the equipment level, process unit level, and production line level. Different agents can be called individually, or they can be implemented through mutual communication, interaction, collaboration, etc., and complete system-level decisions. It should be noted that system-level decisions are achieved through collaborative analysis between the digital twin analysis layers at the equipment level, process unit level, and production line level. In the digital twin analysis layer, the present invention uses multi-agent deep reinforcement learning technology to construct the target to be analyzed and optimized in the multi-scale digital twin analysis layer, and completes the collaborative analysis between the digital twin analysis layers at the equipment level, process unit level, and production line level. Among them, the multi-agent deep reinforcement learning technology includes a proximal policy optimization reinforcement learning method. The proximal policy optimization reinforcement learning method defines a state space, an action space, a global reward function, and an instantaneous reward function. The multi-scale digital twin analysis layer proposed in this paper has the beneficial effect of incorporating the high-risk characteristics of explosive and combustible production lines into the process unit agent, adding production safety constraints such as online charge quantity to construct a production line rule model. In the equipment agent, factors such as equipment health, equipment availability, and process parameter trend prediction are incorporated to update the multi-agent reinforcement learning state space, thereby performing multi-agent analysis and optimization. This approach, rather than constructing the state space solely based on equipment utilization and workpiece status, better reflects the characteristics of the explosive and combustible production line manufacturing process.

[0065] The state space is used to characterize the environment in which multiple agents reside. In the embodiments of the present invention, the state space represents the status of equipment, workpieces, and processes. In the equipment agent, the equipment availability h1 and the current equipment operating status h2 are combined. Workpiece information includes the state information W1 of workpieces that the agent has not yet processed, and the state information W2 of workpieces that the agent can currently process. The process status includes the online charge status V1 of the current processing process, the current safety risk status V2 of the process, and other factors. The multi-agent reinforcement learning state space is updated based on these factors, and then multi-agent analysis and optimization is performed. Rather than constructing the state space solely based on equipment utilization and workpiece status, this method is more consistent with the characteristics of the manufacturing process of incendiary explosive production lines. The state space characterization method proposed in the present invention has the beneficial effect of incorporating the high-risk characteristics of incendiary explosive production lines to achieve the calculation of the equipment operating status h2 and the process safety risk status V2, rather than simply considering the static characteristics of the equipment and processes, thereby improving the ability to predict and warn of equipment and process safety risks.

[0066] Specifically, in the state space designed by the embodiment of the present invention, equipment information includes equipment availability h1 and current equipment operating status h2. Workpiece information includes workpiece status information W1 that the agent has not processed and status information W2 of workpieces that the agent can currently process. Process status includes the online dosage status V1 of the current processing step and the process safety risk status V2. The current state of the state space is represented by S = (h1, h2, W1, W2, V1, V2). The calculation of each parameter in the state space is as follows:

[0067] Equipment availability h1 = equipment mean time between failures / (equipment mean time between failures + mean time to repair), where the current equipment operating status h2 refers to the equipment failure status. This is calculated using an equipment failure prediction model based on autoencoders and long short-term memory networks (LSTMs). The autoencoder- and LSTM-based equipment failure prediction model used in this invention is constructed using an autoencoder model and an LSTM model.

[0068] The online dosage V1 of each process is updated in real time based on the monitoring data of the number of work-in-progress in the workshop (for example, the number of pallets). Figure 2 The technical architecture uses sensors deployed on the physical production line for real-time perception and monitoring, obtains the number of pallets through the data interface, and then converts it into online medicine dosage.

[0069] Process safety risk status V2 establishes a safety risk parameter system through safety risk hidden danger identification, builds a model through methods such as Long Short Term Memory Network (LSTM) and autoencoders, and obtains it in real-time calculation at the edge through real-time access to process parameters and equipment vibration parameters.

[0070] The action space is used to represent the set of actions that multiple agents can take at the corresponding moment in each state space, represented by A.

[0071] At each scheduling time point, multiple agents must independently make decisions based on current observations. This decision determines which workpiece the multi-agents should process at that moment. The result of this decision is called an action. In this embodiment of the present invention, the multi-agent action at that moment consists of the equipment action space M, the workpiece action space, and the process action space. A greedy strategy, ε-greedy, is used for action selection to select the set of equipment, workpieces, and processes that meet the current requirements.

[0072] It should be noted that, in the embodiment of the present invention, it is necessary to establish a communication mechanism between intelligent bodies through a digital twin communication unit, and to realize real-time updating of the data, equipment, process, and production plan status of the digital twin virtual production line based on collaborative analysis between the edge and the cloud. Equipment fault diagnosis and early warning are realized based on the equipment intelligent body, and the production rhythm of the process unit is analyzed based on the process unit intelligent body, and the status of the equipment, process, and workpiece is updated. The digital twin communication unit proposed in the present invention has the beneficial effect of establishing a mathematical model through a long short-term memory network and an autoencoder, and performing edge-end calculation and alarm on the process parameters and equipment fault status signals of the physical production line, thereby enhancing the predictive ability of the digital twin model and the system. In addition, the implementation of edge-end computing effectively improves the alarm response speed, providing support for improving the safety of the production process.

[0073] The global reward function is used in the proximal policy optimization reinforcement learning method. Multi-agents achieve learning and optimization through interaction with the environment. At each moment, multi-agents update their states based on the interaction with the environment, thereby achieving the goals to be analyzed and optimized in the digital twin analysis layer.

[0074] The goal of the global reward function of the present invention is to achieve the production line optimization scheduling goal, that is, to minimize the total delay. The multiple agents in the present invention adopt a full cooperation mechanism. This study designs a global reward function to optimize the total delay. In the reinforcement learning algorithm, the goal is to maximize the cumulative reward, so the increasing direction of the reward function value should be consistent with the decreasing direction of the optimization target value. Therefore, the present invention adopts the Sigmoid function to design a global reward mechanism. In this way, the reward function designed by the present invention can effectively achieve the multi-agent optimization goal. The calculation formula of the reward function at time t is as follows:

[0075] In the multi-agent deep reinforcement learning process, the present invention uses Markov game to solve the optimal strategy, and the optimization target is calculated using the following formula:

[0076]

[0077] Among them, V is the value function of multi-agent reinforcement learning, t refers to the intermediate moment from the initial operation to the termination of the iteration of the digital twin system; S refers to the state space, which changes with time; R is the reward function value of the kth agent (when k = 0, it corresponds to the equipment agent, when k = 1, it corresponds to the process unit agent, and when k = 2, it corresponds to the decision-making agent) at time t, and Υ refers to the discount coefficient, which is used to calculate the importance of the reward function value of the agent at time t. The larger the discount coefficient, the lower the importance.

[0078] The goal of the global reward function in the embodiment of the present invention is to achieve the goal of optimizing the scheduling of the production line, that is, to minimize the total delay. A full cooperation mechanism is adopted between the multiple agents in the embodiment of the present invention. This study designs a global reward function to optimize the total delay. In the reinforcement learning algorithm, the goal is to maximize the cumulative reward, so the increasing direction of the reward function value should be consistent with the decreasing direction of the optimization target value. Therefore, the present invention adopts the Sigmoid function to design a global reward mechanism. In this way, the global reward function designed by the present invention can effectively achieve the multi-agent optimization goal. The calculation formula of the global reward function at time t is as follows:

[0079]

[0080] in, P is the type of product; G is the total delay of the product; D i is the delivery time of product i; E i is the completion time of product i. t refers to the intermediate time from the initial operation to the end of the iteration of the digital twin system; r is the reward function value of the kth agent at time t.

[0081] It should be noted that the embodiment of the present invention uses key parameters such as the initial planning schedule, equipment operating status, workpiece status, and process status to construct an action selection strategy, and completes the collaborative analysis between the digital twin analysis layers at the equipment level, process unit level, and production line level through multi-agent deep reinforcement learning. Under the multi-agent deep reinforcement learning algorithm framework, each agent learns and optimizes the strategy through continuous interaction with the environment. During each interaction, the agent first observes the local state information of the environment, makes decisions based on the current policy framework, and performs corresponding actions. After the action is executed, the agent evaluates the effectiveness of its decision based on the reward signal fed back by the environment. Subsequently, the agent updates the global reward function based on the reward value obtained, so as to make better decisions in subsequent interactions. Through this iterative interaction and state update mechanism, the agent gradually improves and improves its strategy by continuously interacting with the environment and updating its state, and finally converges to the optimal decision-making strategy.

[0082] The action selection strategy proposed in the embodiment of the present invention has the beneficial effect of being able to dynamically select the equipment to execute the rejection process in combination with the failure conditions of the equipment operation, on the premise of meeting the production line process constraints, online drug dosage constraints and other rules, thereby improving the utilization efficiency of equipment resources and obtaining the optimal action selection strategy.

[0083] In view of the collaborative analysis and decision-making process of multiple agents, an embodiment of the present invention designs and implements a conflict handling mechanism. Specifically, after completing the action selection, each agent marks the equipment, process, and workpiece corresponding to the selected action space as unavailable to prevent repeated selection in subsequent calculations. In addition, for equipment with a fault warning in the equipment health status, it is also set to an unavailable state to ensure the safety and efficiency of the system. In order to further improve the utilization efficiency of equipment and achieve a balanced task load, the embodiment of the present invention uses equipment utilization and equipment health status as the key basis for equipment selection and sorting. By optimizing the equipment selection strategy, the utilization rate of idle equipment can be effectively improved, and the selection of equipment with potential faults can be avoided, while ensuring the balanced distribution of task loads. In the design of the reward mechanism, the embodiment of the present invention associates instantaneous rewards and cumulative rewards with the idle time of the equipment. The beneficial effect of the present invention is that by optimizing the action selection and reward mechanism of the agent, the efficiency of the multi-agent system can be significantly improved.

[0084] In summary, the conflict resolution mechanism and optimized reward mechanism proposed in the embodiments of the present invention provide an effective strategy for multi-agent systems in the collaborative analysis and decision-making process, enabling efficient equipment utilization and balanced distribution of task loads. The beneficial effect of the reward mechanism designed in the embodiments of the present invention is that it can promote the stability and adaptability of multi-agent systems in complex environments, providing a reference for research and application in related fields.

[0085] The instantaneous reward function is used to reward the calculation results at the current moment.

[0086] The digital twin analysis layer includes an equipment agent, a process unit agent, and a decision-making agent. The equipment agent analyzes the spatial interference of physical production line equipment and, taking into account the high-risk characteristics of the explosives production line, conducts mechanism- and data-driven equipment fault diagnosis, equipment failure prediction, and equipment hazard assessment. The process unit agent, based on the production line's virtual-real synchronization mechanism, uses real-time data from the explosives production line to drive the calculation of process bottlenecks within the process unit. This is then used within the decision-making agent to calculate and adjust the production line's optimal scheduling plan. This plan is simulated, analyzed, and verified within the virtual production line environment within the multi-scale digital twin system of the explosives production line. The decision-making agent aggregates global information about the manufacturing process and uses this information to set the objectives to be analyzed and optimized within the digital twin analysis layer. It also performs global planning, evaluates the performance of the process unit agents, assesses tasks within the digital twin analysis layer, and assigns tasks to computational models and nodes within the digital twin analysis layer. It also updates the status of the equipment and process unit agents. This global information can be production scheduling and production planning information.

[0087] The following describes in detail the “digital twin application layer” in the multi-scale digital twin system of the above-mentioned incendiary explosive production line in conjunction with an embodiment of the present invention.

[0088] In this embodiment of the present invention, the digital twin application layer is used to implement various digital twin applications at the device, process unit, and production line levels. Specifically, device-level digital twin applications are used for equipment hazard assessment, equipment fault analysis and diagnosis, and equipment fault early warning. Process unit-level digital twin applications are used for process bottleneck analysis. Production line-level digital twin applications are used for production cycle optimization and abnormal operating condition simulation.

[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, rather than to limit them. Although the embodiments of the present invention have been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-scale digital twin system for a combustion and explosives production line, characterized by: include: Digital twin data layer, digital twin communication unit, digital twin analysis layer, and digital twin application layer; the digital twin data layer, digital twin analysis layer, and digital twin application layer are divided into three scales: equipment level, process unit level, and production line level; The digital twin data layer is used to characterize the data of the production line geometry model, the data of the production line physical model, the data of the production line behavior model, and the data of the production line rule model in the multi-scale digital twin system of the combustible explosive production line; The digital twin communication unit is used to realize data interaction between the virtual production line and the physical production line in the multi-scale digital twin system of the incendiary explosive production line, as well as data interaction between different intelligent agents in the digital twin analysis layer; The digital twin analysis layer is used to complete collaborative analysis between the device-level, process unit-level, and production line-level digital twin analysis layers using multi-agent deep reinforcement learning technology. Different agents can be called independently and can be implemented through mutual communication, interaction, and collaboration to complete system-level decision-making. The multi-agent deep reinforcement learning technology includes a proximal policy optimization reinforcement learning method. The proximal policy optimization reinforcement learning method defines a state space, an action space, a global reward function, and an instantaneous reward function. The state space is used to represent the environment in which the multi-agent is located; The action space is used to represent the set of actions that the multi-agent can take at each moment in the state space; The global reward function is used in the proximal policy optimization reinforcement learning method, where multiple agents learn and optimize through interaction with the environment. At each moment, the multi-agent updates its state based on the interaction with the environment, thereby achieving the goal to be analyzed and optimized in the digital twin analysis layer; The instantaneous reward function is used to reward the calculation result at the current moment; The calculation formula of the global reward function is as follows: in, P is the type of product; G is the total delay of the product; D i is the delivery time of product i; E i is the completion time of product i; t refers to the intermediate time from the initial operation to the end of the iteration of the digital twin system; r is the reward function value of the kth agent at time t; The digital twin analysis layer includes: equipment agent, process unit agent and decision agent; The equipment agent is used to analyze the spatial interference of physical production line equipment in the combustion and explosive production line, diagnose equipment faults, predict equipment faults, and assess equipment hazard; The digital twin application layer is used to implement various digital twin applications at the equipment level, process unit level, and production line level.

2. The system according to claim 1, wherein: The data of the production line geometric model, the data of the production line physical model, the data of the production line behavior model, and the data of the production line rule model are used to form a unified digital twin data base through multi-source data fusion technology, and to realize collaborative analysis and optimization based on the data of the production line geometric model, the data of the production line physical model, the data of the production line behavior model, and the data of the production line rule model in a multi-scale digital twin system.

3. The system according to claim 1 or 2, characterized in that The production line geometric model is used to represent the physical production line geometry of the incendiary explosives production line, construct the model from the geometric shape and physical constraint levels, and perform realistic rendering of the model; The production line physical model is used to describe the physical characteristics of the production line and equipment; The production line behavior model is used to describe the dynamic behavior of the production line operation; a mathematical model is established for the dynamic evolution of the production line's manufacturing elements over time to dynamically describe the manufacturing elements, organizational structure and operating mechanism of the production line manufacturing process; The production line rule model is used to determine the constraint rules that the physical entities of the production line and equipment need to follow during the production process; wherein, the constraint rules include production line safety constraint rules and process constraint rules, and the process constraint rules include process flow, process control methods and process instrument flow charts, material balance, heat balance, boundary condition requirements, raw material and public engineering specifications and consumption requirements, three waste emissions and treatment methods, safety risks and response measures, main equipment principles and specifications and design conditions, floor plan recommendations, equipment start-up and shutdown and operation requirements, and staffing and quantity requirements.

4. The system according to claim 3, characterized in that The production line geometric model includes the geometric dimensions and tolerances of the production line buildings and equipment, as well as the spatial layout, pipeline connection relationships and associated attributes. The associated attributes include equipment specifications, technical requirements, process notes and materials, versions, and dates. The physical model of the production line includes an analytical dynamics model, a thermodynamics model, a fluid dynamics model, a stress analysis model, and a kinematics model.

5. The system according to claim 1, wherein: The process unit agent, based on the virtual-real synchronization mechanism of the production line, uses real-time data from the incendiary and explosive production line to drive the calculation of process bottlenecks in the process unit. This data is then used in the decision-making agent to calculate and adjust the production line optimization scheduling plan, and simulation analysis and verification are performed based on the virtual production line environment in the multi-scale digital twin system of the incendiary and explosive production line. The decision-making agent is used to collect global information of the manufacturing process, use the global information to set the goals to be analyzed and optimized in the digital twin analysis layer, perform global planning, evaluate the execution of the process unit agent, evaluate the tasks of the digital twin analysis layer, assign tasks to the computing models and nodes in the digital twin analysis layer, and update the status of the equipment agent and the process unit agent.

6. The system according to claim 1, wherein: The digital twin communication unit uses edge computing technology to achieve adaptive updates of the operating status of the virtual production line of the incendiary explosives production line, so as to optimize data transmission efficiency and maintain the virtual-real synchronization of the digital twin system; through long-short-term memory networks and autoencoders, edge computing and alarms are performed on the process parameters and equipment fault status signals of the physical production line.

7. The system according to claim 1, wherein: The device-level digital twin application is used for equipment risk assessment, equipment failure analysis and diagnosis, and equipment failure warning; The digital twin application at the process unit level is used for process bottleneck analysis; The production line-level digital twin application is used for production rhythm optimization and abnormal operating condition simulation.

Images