Factory monitoring method and system based on digital twinning

Abstract

The invention provides a factory monitoring method and system based on digital twinning, belongs to the field of intelligent control, and solves the problems that video monitoring in a factory has a monitoring dead angle and also has a large amount of non-key information, and the monitoring efficiency is poor. If undifferentiated analysis is carried out on video data in real time, a large number of computing resources are consumed on meaningless data, and in addition, the problem that personnel video information and a digital twinborn model are difficult to fuse in video monitoring is solved. The method comprises the following steps: acquiring a manual control signal of operation equipment in a factory; based on the manual control signal, determining manual operation information currently implemented by an operator for the operation equipment; and deducing the current operation action of the operator in the digital twin model through the manual operation information so as to simulate the three-dimensional simulation dynamic effect of the operator corresponding to the operation action.

Description

Technical Field

[0001] The present application relates to the field of intelligent control technology, and in particular to a plant monitoring method and system based on digital twins. Background Art

[0002] Digital twin is a method of creating virtual models of physical entities or systems through digital means. These virtual models are not just static representations of physical objects, but dynamic systems that can reflect their status and behavior in real time. Digital twin technology integrates advanced technologies such as the Internet of Things, big data, cloud computing and artificial intelligence, and can provide comprehensive monitoring, prediction and optimization of physical entities.

[0003] With the development of the Internet of Things and big data technology, digital twins have gradually been introduced into the manufacturing industry. Early applications mainly focused on equipment monitoring and predictive maintenance, using sensors to collect and analyze equipment data in real time to predict equipment failures and performance changes.

[0004] In recent years, with the popularization of Industry 4.0 and intelligent manufacturing concepts, digital twin technology has been widely used and developed. It is not only used for monitoring and optimization of equipment and production lines, but also applied to the digital management of the entire factory and supply chain.

[0005] However, personnel monitoring currently still relies on video surveillance products, and the integration of video surveillance information and digital twin technology is still relatively limited. Summary of the invention

[0006] The embodiment of the present application provides a factory monitoring method and system based on digital twins, which can solve the problem that there are blind spots in video surveillance within the factory and a large amount of non-critical information in video surveillance. If the video data is analyzed in real time without difference, a large amount of computing resources will inevitably be consumed on meaningless data. In addition, the present application will also solve the problem that it is difficult to integrate personnel video information and digital twin models in video surveillance.

[0007] A first aspect of an embodiment of the present application provides a plant monitoring method based on digital twins, including: Obtain manual control signals for operating equipment in the factory; Determining the manual operation information currently performed by the operator on the operating equipment based on the manual control signal; The operator's current operating actions are deduced in the digital twin model through the manual operation information to simulate the three-dimensional simulation animation of the operator corresponding to the operating actions.

[0008] Optionally, the deducing the operator's current operation action in the digital twin model through the manual operation information to simulate the operator's three-dimensional simulation dynamic effect corresponding to the operation action includes: Deducing the operator's current operation actions in the digital twin model through the manual operation information; The corresponding position of the operating equipment in the BIM model simulates the three-dimensional simulation effect of the operator corresponding to the operating action.

[0009] Optionally, it also includes: Get the current viewpoint information of the 3D scene roaming; Determine the visible area range in the BIM model corresponding to the current viewpoint information; The three-dimensional simulation animation and equipment information of the operator associated with the operating equipment within the visible area are displayed in the three-dimensional scene.

[0010] Optionally, it also includes: Determine the operating equipment and manual control stations for the remaining processes based on the operator's current operating actions, associated operating equipment, and factory standard operating procedures; Generate three-dimensional scene roaming path planning information based on the operating equipment of the remaining process, the manual control station and the operating sequence; A suggested viewpoint moving direction is displayed in the current viewpoint based on the path planning information.

[0011] Optionally, the three-dimensional scene roaming includes three-dimensional scene display roaming and three-dimensional scene monitoring roaming, and the method further includes: When the three-dimensional scene roaming is a three-dimensional scene display roaming and there are currently no operators and operating actions for the operating equipment of the remaining processes associated with the visible area after the viewpoint moves, a three-dimensional virtual animation of the operator is generated at the corresponding manual control workstation according to the operating sequence of the operating equipment of the remaining processes, wherein the display effect of the three-dimensional virtual animation is different from that of the three-dimensional simulation animation.

[0012] Optionally, also include: After determining the manual operation information currently implemented by the operator on the working equipment based on the manual control signal, analyzing the manual operation information; When it is determined that the operation action associated with the manual operation information is an operation action that requires safety measures, obtaining and analyzing video surveillance information of the area where the operating equipment is located, so as to generate an alarm when the video surveillance information indicates that the operator is not equipped with safety measures; and / or, When it is determined that the operation action associated with the manual operation information is an operation action that requires safety measures, the target operator information currently associated with the operating equipment in the scheduling information is queried to send a reminder message for configuring safety measures to the smart mobile terminal associated with the target operator information.

[0013] Optionally, also include: After determining the manual operation information currently implemented by the operator on the working equipment based on the manual control signal, analyzing the manual operation information; When it is determined that the operation action associated with the manual operation information requires an executing caregiver, acquiring imaging data of a head-mounted imaging device of the associated executing caregiver in real time; When it is determined through analysis that the operator does not exist in the imaging data or that the operator is not in the visual focus of the caretaker, an alarm is generated to the operator or the smart mobile terminal of the executive caretaker, and / or the operating status of the corresponding operating equipment is shut down based on the digital twin model.

[0014] A second aspect of the embodiment of the present application provides a plant monitoring system based on digital twins, including: An acquisition unit, used to acquire manual control signals of operating equipment in the factory; a determination unit, configured to determine, based on the manual control signal, manual operation information currently implemented by the operator on the operating equipment; The simulation unit is used to deduce the operator's current operation action in the digital twin model through the manual operation information to simulate the three-dimensional simulation animation of the operator corresponding to the operation action.

[0015] A third aspect of an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the processor is used to implement the steps of the above-mentioned digital twin-based factory monitoring method when executing a computer program stored in the memory.

[0016] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned digital twin-based factory monitoring method are implemented.

[0017] In summary, the plant monitoring method based on digital twins provided in the embodiment of the present application obtains the manual control signal of the operating equipment in the factory; determines the manual operation information currently implemented by the operator for the operating equipment based on the manual control signal; and deduces the current operation action of the operator in the digital twin model through the manual operation information to simulate the three-dimensional simulation effect of the operator corresponding to the operation action. All controlled devices can be fully covered by the device control signal to avoid the physical limitations of video monitoring. Any device operation can be captured and analyzed without worrying about the problem of physical blind spots. Only meaningful control signals are collected and processed instead of the entire video stream, reducing the processing of irrelevant data. The data volume is small and highly correlated, saving computing resources and storage space. The operation action is deduced by the device control signal, which does not involve direct video images, and can protect the privacy of personnel. It avoids direct infringement of personnel privacy and provides effective monitoring of operation behavior. The control signal directly reflects the device status, and the deduced operation action is highly consistent with the device operation. Achieve seamless docking between the device status and the operation action, and enhance the accuracy and real-time performance of the digital twin model. The acquisition and processing of control signals are highly real-time, and the operation actions can be quickly deduced and reflected in the simulation model. High real-time performance can respond and reflect the behavior of operators in a timely manner, improving the efficiency of factory management.

[0018] Correspondingly, the system, electronic device, and computer-readable storage medium provided by the embodiments of the present invention also have the above-mentioned technical effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic flow chart of a possible plant monitoring method based on digital twins provided in an embodiment of the present application; Figure 2 A schematic structural block diagram of a possible plant monitoring system based on digital twins provided in an embodiment of the present application; Figure 3 A schematic diagram of the hardware structure of a possible digital twin-based plant monitoring system provided in an embodiment of the present application; Figure 4 A schematic structural block diagram of a possible electronic device provided in an embodiment of the present application; Figure 5 A schematic structural block diagram of a possible computer-readable storage medium provided for an embodiment of the present application. DETAILED DESCRIPTION

[0020] The embodiments of the present application provide a factory monitoring method and related equipment based on digital twins, which can solve the problem that there are blind spots in video surveillance within the factory and a large amount of non-critical information in video surveillance. If the video data is analyzed in real time without difference, a large amount of computing resources will inevitably be consumed on meaningless data. In addition, the present application will also solve the problem that it is difficult to integrate personnel video information and digital twin models in video surveillance.

[0021] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0022] See also Figure 1 , which is a flow chart of a plant monitoring method based on digital twins provided in an embodiment of the present application, may specifically include: S110-S130.

[0023] S110, obtaining manual control signals of operating equipment in the factory.

[0024] For example, the manual control signals of the operating equipment in the factory can be collected in real time through sensors and data acquisition systems. For example, PLC (Programmable Logic Controller), SCADA (Supervisory Control and Data Acquisition System), sensors and other equipment can be used to monitor the operating status and control signals of the equipment in real time, such as buttons, switches, touch screen inputs, etc.

[0025] S120, determining the manual operation information currently performed by the operator on the operating equipment based on the manual control signal; For example, based on the collected control signals, the specific operation performed by the operator on the operating equipment is analyzed and determined. The control signal is converted into specific operation information through data analysis and processing algorithms. For example, analyzing the signal of a button press can determine that the operator has started a certain device.

[0026] S130, deduce the operator's current operation action in the digital twin model through the manual operation information to simulate the three-dimensional simulation animation of the operator corresponding to the operation action.

[0027] For example, the determined manual operation information can be used to deduce the operator's specific operation actions in the digital twin model. Through the action reasoning algorithm, the operation information is mapped to the operator's action. For example, the operator pressing a button means the action of pressing the hand down. A three-dimensional simulation model corresponding to the operation action is constructed in the digital twin model to simulate the operator's action. Using three-dimensional modeling and animation technology, a three-dimensional simulation effect is generated based on the operation action. Virtual reality (VR) and augmented reality (AR) technology can be used to provide intuitive three-dimensional simulation.

[0028] In summary, the plant monitoring method based on digital twins provided in the above-mentioned embodiment of the present application obtains the manual control signal of the operating equipment in the factory; determines the manual operation information currently implemented by the operator for the operating equipment based on the manual control signal; and deduces the current operation action of the operator in the digital twin model through the manual operation information to simulate the three-dimensional simulation effect of the operator corresponding to the operation action. All controlled devices can be fully covered by the device control signal to avoid the physical limitations of video monitoring. Any device operation can be captured and analyzed without worrying about physical blind spots. Only meaningful control signals are collected and processed instead of the entire video stream, reducing the processing of irrelevant data. The data volume is small and highly correlated, saving computing resources and storage space. The operation action is deduced by the device control signal, which does not involve direct video images, and can protect the privacy of personnel. It avoids direct infringement of personnel privacy and provides effective monitoring of operation behavior. The control signal directly reflects the device status, and the deduced operation action is highly consistent with the device operation. Achieve seamless docking between the device status and the operation action, and enhance the accuracy and real-time performance of the digital twin model. The acquisition and processing of control signals are highly real-time, and the operation actions can be quickly deduced and reflected in the simulation model. High real-time performance can respond and reflect the behavior of operators in a timely manner, improving the efficiency of factory management.

[0029] In some examples, the deducing the operator's current operation action in the digital twin model through the manual operation information to simulate the operator's three-dimensional simulation animation corresponding to the operation action includes: Deducing the operator's current operation actions in the digital twin model through the manual operation information; The corresponding position of the operating equipment in the BIM model simulates the three-dimensional simulation effect of the operator corresponding to the operating action.

[0030] For example, by installing various types of sensors (such as button sensors, torque sensors, position sensors, etc.) on the equipment, various operation signals of the operator on the equipment are collected in real time. The collected control signals are parsed into specific operation information by using data parsing and processing systems (such as PLC, SCADA systems). This step can be completed by preset rules or algorithms to ensure that the operation behavior corresponding to each operation signal can be accurately identified. After receiving the specific operation information, the digital twin system uses the action reasoning algorithm to convert this information into the specific actions of the operator. These algorithms can be deduced based on a predefined action library or through a machine learning model. The action reasoning algorithm needs to consider the operator's workflow and the physical characteristics of the equipment to ensure that the deduced actions are true and accurate. The three-dimensional simulation effects of the operator are simulated at the corresponding position in the BIM model, and the deduced operation actions are converted into three-dimensional animations. By using three-dimensional modeling and animation technology, a high-fidelity operator model is created to show the operator's hand movements and body postures. Virtual reality (VR) and augmented reality (AR) technologies can further enhance the simulation effect and provide intuitive visualization experience for operators and managers. For example, an operator uses a wrench to tighten a bolt on an assembly line, and the sensor on the wrench records the torque signal of tightening the bolt. The control system collects the torque signal of the wrench through the wireless sensor network. The data analysis system analyzes the torque signal into the operation information of "tightening the bolt". The digital twin system receives the operation information of "tightening the bolt" and deduce the operator's action of tightening the bolt with the wrench through the action reasoning algorithm. The digital twin system simulates the three-dimensional simulation effect of the operator tightening the bolt at the corresponding position in the BIM model, showing the operator's hand movements and body postures when using the wrench.

[0031] In some examples, this also includes: Get the current viewpoint information of the 3D scene roaming; Determine the visible area range in the BIM model corresponding to the current viewpoint information; The three-dimensional simulation animation and equipment information of the operator associated with the operating equipment within the visible area are displayed in the three-dimensional scene.

[0032] Exemplarily, the current viewpoint information of the user when roaming in the three-dimensional scene is obtained. The viewpoint position and direction of the user can be captured in real time through viewpoint tracking technology and user interface. Based on the current viewpoint information of the user, the visible area range of the user's current viewpoint in the BIM model is determined. The visible area of ​​the user's current viewpoint can be calculated by the viewpoint calculation algorithm in combination with the three-dimensional spatial data of the BIM model. Within the visible area range of the user's current viewpoint, the three-dimensional simulation dynamic effects and equipment information of the operator associated with the operating equipment are displayed. The three-dimensional rendering and information display technology can be used to display the simulation actions of the relevant operators and the detailed information of the equipment within the user's viewpoint. Through the viewpoint tracking technology and user interface, the viewpoint position and direction of the user in the three-dimensional scene are captured in real time to ensure that each perspective of the user in the scene is accurately recorded. Through the viewpoint calculation algorithm, combined with the three-dimensional spatial data of the BIM model, the visible area range of the user's current viewpoint is calculated. The algorithm needs to consider factors such as the height, direction and distance of the viewpoint to ensure that the calculated visible area is accurate. Within the visible area range of the user's current viewpoint, the simulation actions of the operator and the detailed information of the equipment are dynamically displayed using three-dimensional rendering and information display technology. Ensure that users can intuitively see the operator's actions and equipment status within the current viewpoint range, improve visualization effects and transparency of operations. For example, users roam in a three-dimensional scene to view equipment operations on a production line. Obtain the user's current viewpoint information and determine the range of its visible area. In the visible area, display the operator's three-dimensional simulation of starting the equipment and the equipment's operating status information. Users check the equipment maintenance status in a safe area in a three-dimensional scene. The system obtains the user's current viewpoint information and calculates the visible area range. In the visible area, display the operator's simulation of unlocking the equipment and the equipment's maintenance record. Users view the installation process of parts on an assembly line in a three-dimensional scene. Obtain the user's viewpoint information and determine the visible area range. In the visible area, display the operator's three-dimensional simulation of tightening bolts with a wrench and the progress of parts installation.

[0033] In some examples, this also includes: Determine the operating equipment and manual control stations for the remaining processes based on the operator's current operating actions, associated operating equipment, and factory standard operating procedures; Generate three-dimensional scene roaming path planning information based on the operating equipment of the remaining process, the manual control station and the operating sequence; A suggested viewpoint moving direction is displayed in the current viewpoint based on the path planning information.

[0034] For example, the factory's standard operating process database and current operation information can be used to analyze and determine the equipment and control stations required for subsequent processes. The path planning algorithm can be used to combine the equipment position and process sequence in the three-dimensional scene to generate the optimal roaming path. The recommended viewpoint movement direction guidance can be dynamically displayed in the user's viewpoint interface through augmented reality (AR) or other navigation technologies. Take the installation of parts on the assembly line as an example: Get manual control signal: The operator tightens the bolt using a wrench.

[0035] Determine the manual operation information: interpret it as the "tighten the bolt" operation.

[0036] Simulate the operator's operating actions: simulate the tightening action.

[0037] 3D simulation of operators: display tightening actions in BIM models.

[0038] Get the current viewpoint information of the 3D scene roaming: capture the user's viewpoint.

[0039] Determine the visible area range corresponding to the current viewpoint information: calculate the visible area.

[0040] Display the operating equipment and simulation effects within the visible area: display the tightening operation and equipment information.

[0041] Determine the operating equipment and control stations for the remaining processes: based on current operations and work processes.

[0042] Generate 3D scene roaming path planning information: Generate the roaming path for the remaining processes.

[0043] Show suggested viewpoint movement direction: Display the movement direction in the user viewpoint interface.

[0044] Therefore, in the user's current viewpoint interface, based on the path planning information, the suggested viewpoint movement direction is dynamically displayed through augmented reality (AR) technology. Through arrows or other navigation instructions, users can know which direction to move the viewpoint next, so as to better view and monitor the operation of the remaining processes. It is convenient for users or customers who are not familiar with the factory environment and operating rules to fully, accurately and completely display the operator's work.

[0045] In some examples, the 3D scene roaming includes 3D scene display roaming and 3D scene monitoring roaming, and the method further includes: When the three-dimensional scene roaming is a three-dimensional scene display roaming and there are currently no operators and operating actions for the operating equipment of the remaining processes associated with the visible area after the viewpoint moves, a three-dimensional virtual animation of the operator is generated at the corresponding manual control workstation according to the operating sequence of the operating equipment of the remaining processes, wherein the display effect of the three-dimensional virtual animation is different from that of the three-dimensional simulation animation.

[0046] For example, in the three-dimensional scene display roaming, its role is more to let users understand the factory's working processes and rules, rather than to monitor personnel. When a set of processes requires the same operator to operate, and the operator's current working environment takes a long time, the operator's three-dimensional virtual animation can be generated at the corresponding manual control station according to the operating sequence of the operating equipment of the remaining processes. The system generates a three-dimensional virtual animation of the operator at the corresponding manual control station according to the operating sequence of the remaining processes. These virtual animations are different from the simulated animations of real operators in display effects, and can be distinguished by color, transparency or other visual effects, so that users can clearly distinguish between virtual animations and real operations.

[0047] In some examples, this also includes: After determining the manual operation information currently implemented by the operator on the working equipment based on the manual control signal, analyzing the manual operation information; When it is determined that the operation action associated with the manual operation information is an operation action that requires safety measures, obtaining and analyzing video surveillance information of the area where the operating equipment is located, so as to generate an alarm when the video surveillance information indicates that the operator is not equipped with safety measures; and / or, When it is determined that the operation action associated with the manual operation information is an operation action that requires safety measures, the target operator information currently associated with the operating equipment in the scheduling information is queried to send a reminder message for configuring safety measures to the smart mobile terminal associated with the target operator information.

[0048] Take the maintenance operation of acid mist absorber as an example: 1. Obtain manual control signal: The operator maintains the acid mist absorber.

[0049] 2. Determine the manual operation information: Analyze it as the "maintain acid mist absorber" operation.

[0050] 3. Analyze manual operation information: Check the operation information and determine whether safety measures (such as protective gloves and protective masks) are required when repairing the acid mist absorber.

[0051] 4. Obtain and analyze video surveillance information: Extract video surveillance data of the equipment area and analyze whether the operator is wearing protective gloves and protective masks.

[0052] 5. Generate an alarm: If the operator does not wear protective gloves and a protective mask, an alarm message is generated and relevant personnel are notified.

[0053] Therefore, by combining equipment control signals and video surveillance information, we ensure that all operations and equipment information can be fully monitored and displayed, improving monitoring efficiency and coverage. By detecting safety measures for key operating steps, we ensure that each operator meets safety standards when performing dangerous operations, reducing the occurrence of safety accidents. By using computer vision technology and automatic alarm mechanisms, we can achieve automation and real-time safety monitoring, reduce human intervention, and improve response speed. We provide detailed operator actions and safety measures information to provide managers with more powerful decision-making support capabilities and improve factory management efficiency and safety levels.

[0054] Take hazardous chemicals operation as an example: 1. Obtain manual control signal: The operator opens the container of hazardous chemicals.

[0055] 2. Determine the manual operation information: interpreted as the “open hazardous chemical container” operation.

[0056] 3. Analyze manual operation information: Check the operation information to determine whether safety measures (such as protective masks) are required when opening containers of hazardous chemicals.

[0057] 4. Query the shift information: obtain the operator information currently operating the hazardous chemical container.

[0058] 5. Send reminder messages for safety measures: Send reminder messages to the operator’s smart mobile terminal to remind them to wear protective masks.

[0059] Therefore, by combining equipment control signals, video surveillance information, and shift information, it is ensured that all operations and equipment information can be fully monitored and displayed, improving monitoring efficiency and coverage. By accessing the shift system, the operator information of the current equipment can be obtained to ensure that the prompt message can be accurately sent to the relevant personnel. Combined with the shift information, the accuracy of the prompt message is ensured to improve the effectiveness of the prompt. The prompt message is sent to the operator's smart mobile terminal through SMS, email or mobile application to remind him to equip the necessary safety measures. The operator is reminded in real time through the smart terminal to ensure that he is equipped with the necessary safety measures when performing dangerous operations, thereby improving the safety prevention effect.

[0060] In some examples, this also includes: After determining the manual operation information currently implemented by the operator on the working equipment based on the manual control signal, analyzing the manual operation information; When it is determined that the operation action associated with the manual operation information requires an executing caregiver, acquiring imaging data of a head-mounted imaging device of the associated executing caregiver in real time; When it is determined through analysis that the operator does not exist in the imaging data or that the operator is not in the visual focus of the caretaker, an alarm is generated to the operator or the smart mobile terminal of the executive caretaker, and / or the operating status of the corresponding operating equipment is shut down based on the digital twin model.

[0061] Take device operation as an example. 1. Obtaining a manual control signal: The operator starts the equipment, e.g., the sludge feed pump.

[0062] 2. Determine the manual operation information: interpreted as the "start sludge feed pump" operation.

[0063] 3. Analyze manual operation information: Check the operation information and make sure that an executive supervisor is required to start the sludge feed pump.

[0064] 4. Obtain the imaging data of the caregiver in real time: obtain the real-time imaging data of the caregiver through a head-mounted imaging device.

[0065] 5. Analyze imaging data: Determine whether there is an operator in the imaging data and whether the operator is in the visual focus of the caregiver.

[0066] 6. Generate an alarm and / or shut down the operating equipment: If the operator is not in the visual focus of the caregiver, send an alarm message to the operator or the caregiver's smart mobile terminal and shut down the corresponding equipment.

[0067] Therefore, the caregiver's perspective data is collected in real time through the head-mounted imaging device to ensure real-time monitoring of the operator. Provide real-time monitoring data from the first perspective to ensure that the operator is adequately supervised when performing dangerous or complex operations. Use computer vision and image processing technology to analyze imaging data and determine the presence and location of the operator. Automatically analyze the operation status from the caregiver's perspective and / or whether the caregiver's attention is on the operator, so as to use the system to monitor the caregiver's attention for the machine, real-time alarm and automatic shutdown mechanism to ensure safe operation and reduce the risk of human negligence.

[0068] For example, the neutralization reaction mixer, acidification reduction reaction tank, acidification reduction mixer, vortex flotation, comprehensive reaction tank, hardness removal reaction mixer, coagulation reaction mixer, pH adjustment mixer, flocculation mixer, MBR integrated box, bag filter, MBR suction pump, RO water inlet pump, denitrification mixer, disc aerator, flat MBR membrane box, acid and alkali cleaning barrel, sludge return pump, RO system, Roots blower, air compressor, cold dryer, gas storage tank, acid discharge pump, acid storage tank, acid mist absorber, carbon source dispensing tank, carbon source dosing pump, PAC dispensing tank, PAC dosing pump, PAM dispensing tank, PAM dosing pump, degumming agent dispensing tank, degumming agent dosing pump, polyferric flocculant dispensing tank, polyferric flocculant dosing pump, sodium carbonate mixer, sodium carbonate dosing pump, ferrous sulfate mixer, ferrous sulfate dosing pump, lime silo , lime milk dissolution + storage tank, lime milk transfer pump, lime milk dosing pump, fungicide dispensing tank, fungicide dosing pump, reducing agent dispensing tank, reducing agent dosing pump, scale inhibitor dispensing tank, scale inhibitor dosing pump, sludge conditioning tank, conditioning tank mixer, sludge feed pump, diaphragm filter press, PAM preparation tank, PAM dosing pump, eyewash, dewatering clear liquid tank, dewatering clear liquid return pump, MBR water inlet pump, sludge lifting pump, recycled water pump, sludge tank aeration and stirring system, intermediate water tank aeration and stirring system, water collection tank drainage pump, regulating tank lifting pump, neutralization reaction tank, acid adding pump, solid waste storage and metering system, powder storage silo and conveying system, powder metering system, mixing system, chemical storage and metering system, water storage and metering system, submersible sewage pump, air power system, crusher, hopper, molding machine, electrical control system, etc. are all applicable to the above scheme.

[0069] The above describes the factory monitoring method based on digital twin in the embodiment of the present application. The following describes the factory monitoring system based on digital twin in the embodiment of the present application.

[0070] See also Figure 2 In the embodiments of the present application, an embodiment of a plant monitoring system based on digital twins is described, which may include: An acquisition unit 201 is used to acquire a manual control signal of an operating device in a factory; A determination unit 202, configured to determine, based on the manual control signal, manual operation information currently implemented by the operator on the operating equipment; The simulation unit 203 is used to deduce the operator's current operation action in the digital twin model through the manual operation information to simulate the three-dimensional simulation animation of the operator corresponding to the operation action.

[0071] In summary, the plant monitoring system based on digital twins provided in the above embodiments obtains the manual control signal of the operating equipment in the factory; determines the manual operation information currently implemented by the operator for the operating equipment based on the manual control signal; and deduces the current operation action of the operator in the digital twin model through the manual operation information to simulate the three-dimensional simulation effect of the operator corresponding to the operation action. All controlled devices can be fully covered by the device control signal to avoid the physical limitations of video monitoring. Any device operation can be captured and analyzed without worrying about physical blind spots. Only meaningful control signals are collected and processed instead of the entire video stream, reducing the processing of irrelevant data. The data volume is small and highly correlated, saving computing resources and storage space. The operation action is deduced by the device control signal, which does not involve direct video images, and can protect the privacy of personnel. It avoids direct infringement of personnel privacy and provides effective monitoring of operation behavior. The control signal directly reflects the device status, and the deduced operation action is highly consistent with the device operation. Achieve seamless docking between device status and operation action, and enhance the accuracy and real-time performance of the digital twin model. The acquisition and processing of control signals are highly real-time, and the operation actions can be quickly deduced and reflected in the simulation model. High real-time performance can respond and reflect the behavior of operators in a timely manner, improving the efficiency of factory management.

[0072] above Figure 2 The plant monitoring system based on digital twins in the embodiment of the present application is described from the perspective of modular functional entities. The plant monitoring system based on digital twins in the embodiment of the present application is described in detail from the perspective of hardware processing. Please refer to Figure 3 , an embodiment of a plant monitoring system 300 based on digital twins in an embodiment of the present application includes: An input device 301, an output device 302, a processor 303 and a memory 304, wherein the number of the processor 303 can be one or more. Figure 3 In some embodiments of the present application, the input device 301, the output device 302, the processor 303 and the memory 304 may be connected via a bus or other means, wherein: Figure 3 The example of connecting through bus is taken in the following.

[0073] Wherein, by calling the operation instruction stored in the memory 304, the processor 303 is used to perform the following steps: Obtain manual control signals for operating equipment in the factory; Determining the manual operation information currently performed by the operator on the operating equipment based on the manual control signal; The operator's current operating actions are deduced in the digital twin model through the manual operation information to simulate the three-dimensional simulation animation of the operator corresponding to the operating actions.

[0074] Optionally, the deducing the operator's current operation action in the digital twin model through the manual operation information to simulate the operator's three-dimensional simulation dynamic effect corresponding to the operation action includes: Deducing the operator's current operation actions in the digital twin model through the manual operation information; The corresponding position of the operating equipment in the BIM model simulates the three-dimensional simulation effect of the operator corresponding to the operating action.

[0075] Optionally, it also includes: Get the current viewpoint information of the 3D scene roaming; Determine the visible area range in the BIM model corresponding to the current viewpoint information; The three-dimensional simulation animation and equipment information of the operator associated with the operating equipment within the visible area are displayed in the three-dimensional scene.

[0076] Optionally, it also includes: Determine the operating equipment and manual control stations for the remaining processes based on the operator's current operating actions, associated operating equipment, and factory standard operating procedures; Generate three-dimensional scene roaming path planning information based on the operating equipment of the remaining process, the manual control station and the operating sequence; A suggested viewpoint moving direction is displayed in the current viewpoint based on the path planning information.

[0077] Optionally, the three-dimensional scene roaming includes three-dimensional scene display roaming and three-dimensional scene monitoring roaming, and the method further includes: When the three-dimensional scene roaming is a three-dimensional scene display roaming and there are currently no operators and operating actions for the operating equipment of the remaining processes associated with the visible area after the viewpoint moves, a three-dimensional virtual animation of the operator is generated at the corresponding manual control workstation according to the operating sequence of the operating equipment of the remaining processes, wherein the display effect of the three-dimensional virtual animation is different from that of the three-dimensional simulation animation.

[0078] Optionally, also include: After determining the manual operation information currently implemented by the operator on the working equipment based on the manual control signal, analyzing the manual operation information; When it is determined that the operation action associated with the manual operation information is an operation action that requires safety measures, obtaining and analyzing video surveillance information of the area where the operating equipment is located, so as to generate an alarm when the video surveillance information indicates that the operator is not equipped with safety measures; and / or, When it is determined that the operation action associated with the manual operation information is an operation action that requires safety measures, the target operator information currently associated with the operating equipment in the scheduling information is queried to send a reminder message for configuring safety measures to the smart mobile terminal associated with the target operator information.

[0079] Optionally, also include: After determining the manual operation information currently implemented by the operator on the working equipment based on the manual control signal, analyzing the manual operation information; When it is determined that the operation action associated with the manual operation information requires an executing caregiver, acquiring imaging data of a head-mounted imaging device of the associated executing caregiver in real time; When it is determined through analysis that the operator does not exist in the imaging data or that the operator is not in the visual focus of the caretaker, an alarm is generated to the operator or the smart mobile terminal of the executive caretaker, and / or the operating status of the corresponding operating equipment is shut down based on the digital twin model.

[0080] By calling the operation instructions stored in the memory 304, the processor 303 is also used to execute Figure 1 Any method in the corresponding embodiment.

[0081] See also Figure 4 , Figure 4 A schematic diagram of an embodiment of an electronic device provided in an embodiment of the present application.

[0082] like Figure 4 As shown, an embodiment of the present application provides an electronic device, including a memory 410, a processor 420, and a computer program 411 stored in the memory 410 and executable on the processor 420. When the processor 420 executes the computer program 411, the following steps are implemented: Obtain manual control signals for operating equipment in the factory; Determining the manual operation information currently performed by the operator on the operating equipment based on the manual control signal; The operator's current operating actions are deduced in the digital twin model through the manual operation information to simulate the three-dimensional simulation animation of the operator corresponding to the operating actions.

[0083] Optionally, the deducing the operator's current operation action in the digital twin model through the manual operation information to simulate the operator's three-dimensional simulation dynamic effect corresponding to the operation action includes: Deducing the operator's current operation actions in the digital twin model through the manual operation information; The corresponding position of the operating equipment in the BIM model simulates the three-dimensional simulation effect of the operator corresponding to the operating action.

[0084] Optionally, it also includes: Get the current viewpoint information of the 3D scene roaming; Determine the visible area range in the BIM model corresponding to the current viewpoint information; The three-dimensional simulation animation and equipment information of the operator associated with the operating equipment within the visible area are displayed in the three-dimensional scene.

[0085] Optionally, it also includes: Determine the operating equipment and manual control stations for the remaining processes based on the operator's current operating actions, associated operating equipment, and factory standard operating procedures; Generate three-dimensional scene roaming path planning information based on the operating equipment of the remaining process, the manual control station and the operating sequence; A suggested viewpoint moving direction is displayed in the current viewpoint based on the path planning information.

[0086] Optionally, the three-dimensional scene roaming includes three-dimensional scene display roaming and three-dimensional scene monitoring roaming, and the method further includes: When the three-dimensional scene roaming is a three-dimensional scene display roaming and there are currently no operators and operating actions for the operating equipment of the remaining processes associated with the visible area after the viewpoint moves, a three-dimensional virtual animation of the operator is generated at the corresponding manual control workstation according to the operating sequence of the operating equipment of the remaining processes, wherein the display effect of the three-dimensional virtual animation is different from that of the three-dimensional simulation animation.

[0087] Optionally, also include: After determining the manual operation information currently implemented by the operator on the working equipment based on the manual control signal, analyzing the manual operation information; When it is determined that the operation action associated with the manual operation information is an operation action that requires safety measures, obtaining and analyzing video surveillance information of the area where the operating equipment is located, so as to generate an alarm when the video surveillance information indicates that the operator is not equipped with safety measures; and / or, When it is determined that the operation action associated with the manual operation information is an operation action that requires safety measures, the target operator information currently associated with the operating equipment in the scheduling information is queried to send a reminder message for configuring safety measures to the smart mobile terminal associated with the target operator information.

[0088] Optionally, also include: After determining the manual operation information currently implemented by the operator on the working equipment based on the manual control signal, analyzing the manual operation information; When it is determined that the operation action associated with the manual operation information requires an executing caregiver, acquiring imaging data of a head-mounted imaging device of the associated executing caregiver in real time; When it is determined through analysis that the operator does not exist in the imaging data or that the operator is not in the visual focus of the caretaker, an alarm is generated to the operator or the smart mobile terminal of the executive caretaker, and / or the operating status of the corresponding operating equipment is shut down based on the digital twin model.

[0089] In the specific implementation process, when the processor 420 executes the computer program 411, it can achieve Figure 1 Any implementation manner in the corresponding embodiments.

[0090] Since the electronic device introduced in this embodiment is a device used to implement a factory monitoring system based on digital twins in the embodiment of the present application, based on the method introduced in the embodiment of the present application, the technical personnel in this field can understand the specific implementation mode of the electronic device of this embodiment and its various variations. Therefore, how the electronic device implements the method in the embodiment of the present application is not introduced in detail here. As long as the equipment used by the technical personnel in this field to implement the method in the embodiment of the present application, it belongs to the scope of protection of this application.

[0091] See also Figure 5 , Figure 5 A schematic diagram of an embodiment of a computer-readable storage medium provided in an embodiment of the present application.

[0092] like Figure 5 As shown, this embodiment provides a computer-readable storage medium 500, on which a computer program 511 is stored. When the computer program 511 is executed by a processor, the following steps are implemented: Obtain manual control signals for operating equipment in the factory; Determining the manual operation information currently performed by the operator on the operating equipment based on the manual control signal; The operator's current operating actions are deduced in the digital twin model through the manual operation information to simulate the three-dimensional simulation animation of the operator corresponding to the operating actions.

[0093] Optionally, the deducing the operator's current operation action in the digital twin model through the manual operation information to simulate the operator's three-dimensional simulation dynamic effect corresponding to the operation action includes: Deducing the operator's current operation actions in the digital twin model through the manual operation information; The corresponding position of the operating equipment in the BIM model simulates the three-dimensional simulation effect of the operator corresponding to the operating action.

[0094] Optionally, it also includes: Get the current viewpoint information of the 3D scene roaming; Determine the visible area range in the BIM model corresponding to the current viewpoint information; The three-dimensional simulation animation and equipment information of the operator associated with the operating equipment within the visible area are displayed in the three-dimensional scene.

[0095] Optionally, it also includes: Determine the operating equipment and manual control stations for the remaining processes based on the operator's current operating actions, associated operating equipment, and factory standard operating procedures; Generate three-dimensional scene roaming path planning information based on the operating equipment of the remaining process, the manual control station and the operating sequence; A suggested viewpoint moving direction is displayed in the current viewpoint based on the path planning information.

[0096] Optionally, the three-dimensional scene roaming includes three-dimensional scene display roaming and three-dimensional scene monitoring roaming, and the method further includes: When the three-dimensional scene roaming is a three-dimensional scene display roaming and there are currently no operators and operating actions for the operating equipment of the remaining processes associated with the visible area after the viewpoint moves, a three-dimensional virtual animation of the operator is generated at the corresponding manual control workstation according to the operating sequence of the operating equipment of the remaining processes, wherein the display effect of the three-dimensional virtual animation is different from that of the three-dimensional simulation animation.

[0097] Optionally, also include: After determining the manual operation information currently implemented by the operator on the working equipment based on the manual control signal, analyzing the manual operation information; When it is determined that the operation action associated with the manual operation information is an operation action that requires safety measures, obtaining and analyzing video surveillance information of the area where the operating equipment is located, so as to generate an alarm when the video surveillance information indicates that the operator is not equipped with safety measures; and / or, When it is determined that the operation action associated with the manual operation information is an operation action that requires safety measures, the target operator information currently associated with the operating equipment in the scheduling information is queried to send a reminder message for configuring safety measures to the smart mobile terminal associated with the target operator information.

[0098] Optionally, also include: After determining the manual operation information currently implemented by the operator on the working equipment based on the manual control signal, analyzing the manual operation information; When it is determined that the operation action associated with the manual operation information requires an executing caregiver, acquiring imaging data of a head-mounted imaging device of the associated executing caregiver in real time; When it is determined through analysis that the operator does not exist in the imaging data or that the operator is not in the visual focus of the caretaker, an alarm is generated to the operator or the smart mobile terminal of the executive caretaker, and / or the operating status of the corresponding operating equipment is shut down based on the digital twin model.

[0099] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0100] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0101] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0102] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0103] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0104] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0105] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A plant monitoring method based on digital twins, characterized in that: The method comprises: Obtain manual control signals for operating equipment in the factory; Determining the manual operation information currently performed by the operator on the operating equipment based on the manual control signal; The operator's current operating actions are deduced in the digital twin model through the manual operation information to simulate the three-dimensional simulation animation of the operator corresponding to the operating actions.

2. The method according to claim 1, characterized in that The step of deducing the current operation action of the operator in the digital twin model through the manual operation information to simulate the three-dimensional simulation dynamic effect of the operator corresponding to the operation action includes: Deducing the operator's current operation actions in the digital twin model through the manual operation information; The corresponding position of the operating equipment in the BIM model simulates the three-dimensional simulation effect of the operator corresponding to the operating action.

3. The method according to claim 1, characterized in that Also includes: Get the current viewpoint information of the 3D scene roaming; Determine the visible area range in the BIM model corresponding to the current viewpoint information; The three-dimensional simulation animation and equipment information of the operator associated with the operating equipment within the visible area are displayed in the three-dimensional scene.

4. The method according to claim 1, characterized in that: Also includes: Determine the operating equipment and manual control stations for the remaining processes based on the operator's current operating actions, associated operating equipment, and factory standard operating procedures; Generate three-dimensional scene roaming path planning information based on the operating equipment of the remaining process, the manual control station and the operating sequence; A suggested viewpoint moving direction is displayed in the current viewpoint based on the path planning information.

5. The method according to claim 4, characterized in that The three-dimensional scene roaming includes three-dimensional scene display roaming and three-dimensional scene monitoring roaming, and the method further includes: When the three-dimensional scene roaming is a three-dimensional scene display roaming and there are currently no operators and operating actions for the operating equipment of the remaining processes associated with the visible area after the viewpoint moves, a three-dimensional virtual animation of the operator is generated at the corresponding manual control workstation according to the operating sequence of the operating equipment of the remaining processes, wherein the display effect of the three-dimensional virtual animation is different from that of the three-dimensional simulation animation.

6. The method according to any one of claims 1 to 5, characterized in that Also includes: After determining the manual operation information currently implemented by the operator on the working equipment based on the manual control signal, analyzing the manual operation information; When it is determined that the operation action associated with the manual operation information is an operation action that requires safety measures, obtaining and analyzing video surveillance information of the area where the operating equipment is located, so as to generate an alarm when the video surveillance information indicates that the operator is not equipped with safety measures; and / or, When it is determined that the operation action associated with the manual operation information is an operation action that requires safety measures, the target operator information currently associated with the operating equipment in the scheduling information is queried to send a reminder message for configuring safety measures to the smart mobile terminal associated with the target operator information.

7. The method according to any one of claims 1 to 5, characterized in that Also includes: After determining the manual operation information currently implemented by the operator on the working equipment based on the manual control signal, analyzing the manual operation information; When it is determined that the operation action associated with the manual operation information requires an executing caregiver, acquiring imaging data of a head-mounted imaging device of the associated executing caregiver in real time; When it is determined through analysis that the operator does not exist in the imaging data or that the operator is not in the visual focus of the caretaker, an alarm is generated to the operator or the smart mobile terminal of the executive caretaker, and / or the operating status of the corresponding operating equipment is shut down based on the digital twin model.

8. A plant monitoring system based on digital twins, characterized in that: The system comprises: An acquisition unit, used to acquire manual control signals of operating equipment in the factory; a determination unit, configured to determine, based on the manual control signal, manual operation information currently implemented by the operator on the operating equipment; The simulation unit is used to deduce the operator's current operation action in the digital twin model through the manual operation information to simulate the three-dimensional simulation animation of the operator corresponding to the operation action.

9. An electronic device, characterized in that: The electronic device includes at least one processor and at least one memory connected to the processor, wherein the processor is used to call program instructions in the memory to execute the digital twin-based plant monitoring method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium includes a stored program, wherein, when the program is running, the device where the storage medium is located is controlled to execute the digital twin-based factory monitoring method according to any one of claims 1 to 7.

Images