Smart factory working condition monitoring system based on digital twinborn technology

Abstract

The invention relates to the technical field of working condition monitoring, and discloses a smart factory working condition monitoring system based on a digital twin technology. Comprising a three-dimensional modeling and optimization module, a data integration module, a three-dimensional display module, an anomaly prediction and early warning module and a user interface and interaction module, wherein the three-dimensional modeling and optimization module is used for collecting data, modeling and optimizing a model; the data integration module is used for integrating data; the three-dimensional display module is used for visually displaying the production state of a factory; the abnormity prediction and early warning module is used for predicting the abnormal state of a factory and outputting a working condition score; and the user interface and interaction module is used for providing a user interaction interface and visually displaying the operation condition of the factory. According to the invention, the operation efficiency and decision accuracy of an administrator are improved, and real-time monitoring of the equipment operation parameters and early warning of the equipment state are realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of operating condition monitoring, and specifically relates to a smart factory operating condition monitoring system based on digital twin technology. Background Art

[0002] In modern industrial production, the factory monitoring system is crucial. It can monitor the production process and equipment status in real time, optimize production efficiency, prevent failures, improve safety, and provide data support for management decisions. However, traditional monitoring systems have obvious limitations. First, the data display method is single, mostly using two-dimensional charts, which cannot intuitively present complex spatial relationships and production processes, making it difficult for managers to quickly understand the on-site conditions. Secondly, the real-time performance is insufficient, the data update speed is slow, and it is easy to lag when facing high-frequency collection needs, affecting timely decision-making. In addition, traditional systems have difficulty integrating different data sources, forming data islands, lacking unified data views and in-depth analysis capabilities. At the same time, they usually lack intelligent analysis and prediction functions, and cannot provide early warnings or optimization suggestions, resulting in management relying on manual experience. Finally, the interface interactivity is poor, the operation is inconvenient, and the user experience is affected.

[0003] Therefore, there is an urgent need for a smart factory condition monitoring system based on digital twin technology. Summary of the invention

[0004] The present invention provides a smart factory operating condition monitoring system based on digital twin technology, which solves the technical problems in the related technology that the monitoring system has a single display method, insufficient real-time performance, lacks in-depth analysis capabilities, and cannot provide early warning.

[0005] The present invention provides a smart factory operating condition monitoring system based on digital twin technology, including a three-dimensional modeling and optimization module, a data integration module, a three-dimensional display module, an abnormality prediction and early warning module, and a user interface and interaction module:

[0006] The three-dimensional modeling and optimization module includes: a data acquisition and modeling submodule and a model optimization submodule;

[0007] The data acquisition and modeling submodule is used to collect three-dimensional point cloud data, sensor data and production data of the factory environment and production equipment, and convert the three-dimensional point cloud data into a three-dimensional model;

[0008] The model optimization submodule is used to optimize the three-dimensional model;

[0009] The data integration module is used to integrate sensor data and production data to form unified standardized data;

[0010] The three-dimensional display module is used to map the sensor data onto the three-dimensional model to intuitively display the factory production status;

[0011] The abnormal prediction and warning module includes: an abnormal prediction submodule and an intelligent warning submodule;

[0012] The abnormality prediction submodule is used to analyze the sensor data according to the neural network algorithm, predict the abnormality, and output the working condition score;

[0013] The intelligent early warning submodule is used to automatically trigger an alarm according to the operating condition score and a set score threshold, and execute response measures;

[0014] The user interface and interaction module are used to provide a user interaction interface and to visualize the operation status of the factory.

[0015] Furthermore, 3D point cloud data of the factory environment and the surface of production equipment are obtained through 3D laser scanning technology. The 3D point cloud data includes spatial coordinates and shape information, and point cloud processing software is used to convert the pre-processed 3D point cloud data into a 3D geometric model for further visualization.

[0016] Furthermore, the steps of preprocessing the three-dimensional point cloud data include:

[0017] Step S201, using a statistical filtering algorithm to perform denoising on the point cloud data;

[0018] Step S202, using voxel grid downsampling technology to reduce the amount of point cloud data;

[0019] Step S203, using the ICP algorithm to align the three-dimensional point cloud data sets of different perspectives into a unified coordinate system;

[0020] Step S204: performing density-based deduplication processing on the three-dimensional point cloud data to reduce duplicate points.

[0021] Furthermore, the step S203 specifically includes the following steps:

[0022] Step S2031, initializing an initial transformation matrix T between two three-dimensional point cloud data sets P and Q;

[0023] Step S2032, find the matching pair of each point in P and the nearest point in Q through nearest neighbor search;

[0024] Step S2033, minimize the distance error, the distance error formula is: where p i is a point in P, q i is its nearest neighbor in Q, ||T·p i -q i || represents the point p after transformation by the transformation matrix T i With point qi The Euclidean distance between

[0025] Step S2034, updating the transformation matrix T, and repeating the above steps until the distance error converges.

[0026] Furthermore, the optimization steps in the model optimization submodule include:

[0027] Step S401, setting the position, type and intensity of the virtual light source in the point cloud processing software to simulate the lighting conditions of the real scene;

[0028] Step S402, adjusting the color of the light source, the depth and angle of the shadow, so that the model presents a more realistic effect under different lighting conditions;

[0029] Step S403: Use global illumination technology to fully simulate the light in the model scene.

[0030] Furthermore, the sensor data is sensor data obtained by using sensors in the factory to monitor the operating status of equipment and environmental conditions in real time. The format of the sensor data is original voltage signal and digital coded data. The production data includes the production plan, inventory data, logistics data, and sales data of the enterprise. The format of the production data is a database format. The steps of integrating the sensor data and the production data include:

[0031] Step S501, converting data from different sensors into standardized formats such as JSON and XML;

[0032] Step S502, performing type mapping for different data types during data conversion to ensure consistency of data types in the integrated system;

[0033] Step S503: adding a unified timestamp field to the sensor data and production data.

[0034] Furthermore, an abnormal state prediction model is constructed based on the sensor data collected in real time. The abnormal state prediction model is constructed based on a gated cyclic unit. The input of the abnormal state prediction model is a sensor data sequence collected according to a preset time interval within a preset time period, and the output is an operating condition score within the preset time period.

[0035] Furthermore, manual labeling is used as sample labels for the abnormal state prediction model.

[0036] Furthermore, the loss function of the abnormal state prediction model is a cross entropy loss function.

[0037] Furthermore, the functions of the user interface and interaction module include real-time three-dimensional model display, multi-view and layered display, sensor data display, status alarm and notification, automatic execution of response measures and historical data playback.

[0038] The beneficial effects of the present invention are as follows: the present invention constructs a high-precision three-dimensional model to dynamically display the factory layout and equipment operating status, so that managers can intuitively understand the production process and equipment status, thereby improving operational efficiency and decision-making accuracy; the present invention also integrates sensor data to achieve real-time monitoring of key parameters such as temperature, pressure, and flow, and achieves early warning of equipment status by building an abnormal status prediction model. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a module schematic diagram of a smart factory operating condition monitoring system based on digital twin technology of the present invention;

[0040] Figure 2 It is a flow chart of preprocessing three-dimensional point cloud data according to the present invention.

[0041] In the figure: a three-dimensional modeling and optimization module 101, a data integration module 102, a three-dimensional display module 103, an abnormality prediction and early warning module 104 and a user interface and interaction module 105. DETAILED DESCRIPTION

[0042] The subject matter described herein will now be discussed with reference to example embodiments. It should be understood that the discussion of these embodiments is only to enable those skilled in the art to better understand and implement the subject matter described herein, and the functions and arrangements of the elements discussed may be changed without departing from the scope of protection of the contents of this specification. Each example may omit, replace or add various processes or components as needed. In addition, the features described relative to some examples may also be combined in other examples.

[0043] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in one or more embodiments of the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in one or more embodiments of the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0044] like Figure 1-Figure 2As shown, a smart factory working condition monitoring system based on digital twin technology includes a three-dimensional modeling and optimization module 101, a data integration module 102, a three-dimensional display module 103, an abnormality prediction and early warning module 104 and a user interface and interaction module 105:

[0045] The three-dimensional modeling and optimization module includes: a data acquisition and modeling submodule 1011 and a model optimization submodule 1012;

[0046] The data acquisition and modeling submodule is used to collect three-dimensional point cloud data, sensor data and production data of the factory environment and production equipment, and convert the three-dimensional point cloud data into a three-dimensional model;

[0047] The model optimization submodule is used to optimize the three-dimensional model;

[0048] The data integration module is used to integrate sensor data and production data to form unified standardized data;

[0049] The three-dimensional display module is used to map the sensor data onto the three-dimensional model to intuitively display the factory production status;

[0050] The abnormal prediction and warning module includes: an abnormal prediction submodule 1041 and an intelligent warning submodule 1042;

[0051] The abnormality prediction submodule is used to analyze the sensor data according to the neural network algorithm, predict the abnormality, and output the working condition score;

[0052] The intelligent early warning submodule is used to automatically trigger an alarm according to the operating condition score and a set score threshold, and execute response measures;

[0053] The user interface and interaction module are used to provide a user interaction interface and to visualize the operation status of the factory.

[0054] In one embodiment of the present invention, a three-dimensional laser scanner is used to acquire three-dimensional point cloud data of the factory environment and equipment surface by emitting laser beams and receiving reflected signals. The three-dimensional point cloud data includes spatial coordinates and shape information. The collected point cloud data usually contains noise and redundant data. The data is pre-processed by denoising, downsampling and other means to improve the clarity and accuracy of the data. The Delaunay triangulation algorithm is used to convert the pre-processed point cloud data into a three-dimensional geometric model for further rendering and visualization.

[0055] In another embodiment of the present invention, a drone equipped with a high-definition camera can be used to take aerial photos according to a preset flight route and altitude to obtain image data of the factory environment and production equipment, and the collected image data can be denoised and distortion corrected to ensure image clarity. The structure from motion technology can be used to extract multiple two-dimensional photos taken from different angles based on the image data, and the SIFT image processing algorithm can be used to extract feature points. The camera's motion trajectory and the three-dimensional structure of the scene can be reconstructed by matching these feature points.

[0056] In one embodiment of the present invention, the step of preprocessing the three-dimensional point cloud data includes:

[0057] Step S201, use a statistical filtering algorithm to perform denoising on the point cloud data; the formula is: where d i Represents point p i The average distance, p j Indicates p i points in the neighborhood, k represents the neighborhood size, which determines the number of neighborhood points considered when calculating the average distance, ||p i -p j || represents point p i and point p j The Euclidean distance between

[0058] Step S202, using voxel grid downsampling technology to reduce the amount of point cloud data. Voxel grid downsampling divides the point cloud data into a three-dimensional grid (voxel) of fixed size, and then uses the centroid of each voxel to represent all points in the voxel, thereby reducing the amount of point cloud data. The formula is: V is the set of points in a voxel, |V| represents the number of points in the voxel, and P represents the position of the point;

[0059] Step S203, using the ICP algorithm to align the three-dimensional point clouds of different perspectives into a unified coordinate system;

[0060] Step S204: performing density-based deduplication processing on the three-dimensional point cloud data to reduce duplicate points.

[0061] In one embodiment of the present invention, step S203 specifically includes the following steps:

[0062] Step S2031, initializing an initial transformation matrix T between two three-dimensional point cloud data sets P and Q;

[0063] Step S2032, find the matching pair of each point in P and the nearest point in Q through nearest neighbor search;

[0064] Step S2033, minimize the distance error, the distance error formula is: where p i is a point in P, q i is its nearest neighbor in Q, ||T·p i -q i || represents the point p after transformation by the transformation matrix T i With point q i The Euclidean distance between

[0065] Step S2034, updating the transformation matrix T, and repeating the above steps until the distance error converges.

[0066] In one embodiment of the present invention, the steps of converting the pre-processed 3D point cloud data into a 3D geometric model using a Delaunay triangulation algorithm include:

[0067] Step S301, initialize the three-dimensional point cloud data and represent it as a set P = p1, p2, ..., p n}, where each point p i Contains its three-dimensional coordinates (x i ,y i ,z i );

[0068] Step S302, projecting the three-dimensional point cloud data onto a plane for two-dimensional Delaunay segmentation, and then restoring the three-dimensional structure;

[0069] Step S303: construct a Delaunay triangle mesh according to the set P, wherein the circumscribed circle of each triangle should not contain other points.

[0070] In one embodiment of the present invention, the optimization step in the model optimization submodule includes:

[0071] Step S401, setting the position, type (such as point light source, parallel light, etc.) and intensity of the virtual light source in the point cloud processing software to simulate the lighting conditions of the real scene;

[0072] Step S402, adjusting the color of the light source, the depth and angle of the shadow, so that the model presents a more realistic effect under different lighting conditions. For example, scenes such as lighting in a factory and light transmission through windows can be simulated by adjusting the lighting conditions;

[0073] Step S403, using global illumination (such as ray tracing) technology, comprehensively simulate the light in the model scene, including reflection, refraction, etc., to enhance the three-dimensional sense and visual layering of the model.

[0074] In one embodiment of the present invention, the sensor data is sensor data obtained by using sensors in a factory to monitor the operating status of equipment and environmental conditions in real time. The format of the sensor data is original voltage signal and digital coded data. The production data includes the production plan, inventory data, logistics data, and sales data of the enterprise. The format of the production data is a database format (such as SQL). The steps of integrating the sensor data and the production data include:

[0075] Step S501, converting data from different sensors into standardized formats such as JSON and XML to facilitate subsequent data analysis and processing;

[0076] Step S502, for different data types (such as Boolean, integer, floating point, etc.), type mapping is performed during the data conversion process to ensure the consistency of data types in the integrated system. For example, the real number data in the sensor data is converted to the digital type in the database.

[0077] Step S503, adding a unified timestamp field to the sensor data and production data to ensure that the data is analyzed and displayed on a unified time axis, which facilitates the backtracking and comparison of historical data.

[0078] In one embodiment of the present invention, the core of the process of mapping the sensor data to the three-dimensional model is to map the sensor data collected in real time (such as temperature, pressure, and equipment status) one-to-one with the specific objects in the three-dimensional model, so that the three-dimensional model can dynamically reflect the status changes of the real equipment.

[0079] In one embodiment of the present invention, an abnormal state prediction model is constructed based on sensor data collected in real time. The abnormal state prediction model is constructed based on a gated cycle unit. The input of the abnormal state prediction model is a sensor data sequence collected according to a preset time interval within a preset time period, and the output is an operating condition score within the preset time period.

[0080] In one embodiment of the present invention, manual labeling is used as sample labels for the abnormal state prediction model.

[0081] In one embodiment of the present invention, the loss function of the abnormal state prediction model is a cross entropy loss function.

[0082] In one embodiment of the present invention, according to the operating condition score combined with preset score thresholds of different levels, the system can automatically trigger different response measures to ensure the normal operation of the equipment or production process.

[0083] In one embodiment of the present invention, the functions of the user interface and interaction module include real-time 3D model display, multi-view and layered display, sensor data display, status alarm and notification, automatic execution of response measures and historical data playback. The real-time 3D model display function dynamically displays the factory layout and equipment operation status with a high-precision 3D model, allowing users to intuitively understand the overall situation of the factory; the multi-view and layered display function supports users to view the 3D model from different perspectives, and hide or display specific areas and layers as needed, focusing on the parts that need to be monitored; the sensor data display function displays the temperature, pressure, flow and other data collected by the sensor in real time, and directly presents the current status of the relevant equipment on the 3D model; the status alarm and notification function can automatically generate an alarm on the interface and send a notification to relevant personnel when the equipment or environmental parameters exceed the safety threshold; the automatic execution of response measures function can automatically adjust the equipment parameters or perform shutdown operations according to the preset emergency plan when an abnormality is detected to ensure production safety; the historical data playback function allows users to select a specific time period and playback the equipment status and data changes at that time for fault analysis and operation review.

[0084] The above describes an embodiment of the present embodiment, but the present embodiment is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present embodiment, ordinary technicians in this field can also make many forms, all of which are within the protection of the present embodiment.

Claims

1. A smart factory operating condition monitoring system based on digital twin technology, characterized in that: It includes 3D modeling and optimization module, data integration module, 3D display module, abnormal prediction and warning module and user interface and interaction module: The three-dimensional modeling and optimization module includes: a data acquisition and modeling submodule and a model optimization submodule; The data acquisition and modeling submodule is used to collect three-dimensional point cloud data, sensor data and production data of the factory environment and production equipment, and convert the three-dimensional point cloud data into a three-dimensional model; The model optimization submodule is used to optimize the three-dimensional model; The data integration module is used to integrate sensor data and production data to form unified standardized data; The three-dimensional display module is used to map the sensor data onto the three-dimensional model to intuitively display the factory production status; The abnormal prediction and warning module includes: an abnormal prediction submodule and an intelligent warning submodule; The abnormality prediction submodule is used to analyze the sensor data according to the neural network algorithm, predict the abnormality, and output the working condition score; The intelligent early warning submodule is used to automatically trigger an alarm according to the operating condition score and a set score threshold, and execute response measures; The user interface and interaction module are used to provide a user interaction interface and to visualize the operation status of the factory.

2. According to claim 1, a smart factory operating condition monitoring system based on digital twin technology is characterized in that: The 3D point cloud data of the factory environment and the surface of production equipment are obtained through 3D laser scanning technology. The 3D point cloud data includes spatial coordinates and shape information. The point cloud processing software is used to convert the pre-processed 3D point cloud data into a 3D geometric model for further visualization.

3. According to claim 2, a smart factory operating condition monitoring system based on digital twin technology is characterized in that: The steps for preprocessing 3D point cloud data include: Step S201, using a statistical filtering algorithm to perform denoising on the point cloud data; Step S202, using voxel grid downsampling technology to reduce the amount of point cloud data; Step S203, using the ICP algorithm to align the three-dimensional point cloud data sets of different perspectives into a unified coordinate system; Step S204: performing density-based deduplication processing on the three-dimensional point cloud data to reduce duplicate points.

4. According to claim 3, a smart factory operating condition monitoring system based on digital twin technology is characterized in that: The step S203 specifically includes the following steps: Step S2031, initializing an initial transformation matrix T between two three-dimensional point cloud data sets P and Q; Step S2032, find the matching pair of each point in P and the nearest point in Q through nearest neighbor search; Step S2033, minimize the distance error, the distance error formula is: where p i is a point in P, q i is its nearest neighbor in Q, T·p i -q i ‖ represents the point p after transformation by the transformation matrix T i With dot q i The Euclidean distance between Step S2034, updating the transformation matrix T, and repeating the above steps until the distance error converges.

5. According to claim 1, a smart factory operating condition monitoring system based on digital twin technology is characterized in that: The optimization steps in the model optimization submodule include: Step S401, setting the position, type and intensity of the virtual light source in the point cloud processing software to simulate the lighting conditions of the real scene; Step S402, adjusting the color of the light source, the depth and angle of the shadow, so that the model presents a more realistic effect under different lighting conditions; Step S403: Use global illumination technology to fully simulate the light in the model scene.

6. According to claim 1, a smart factory operating condition monitoring system based on digital twin technology is characterized in that: The sensor data is sensor data obtained by using sensors in the factory to monitor the operating status of equipment and environmental conditions in real time. The format of the sensor data is original voltage signal and digital coded data. The production data includes the production plan, inventory data, logistics data, and sales data of the enterprise. The format of the production data is a database format. The steps of integrating the sensor data and the production data include: Step S501, converting data from different sensors into standardized formats such as JSON and XML; Step S502, performing type mapping for different data types during data conversion to ensure consistency of data types in the integrated system; Step S503: adding a unified timestamp field to the sensor data and production data.

7. According to claim 1, a smart factory operating condition monitoring system based on digital twin technology is characterized in that: According to the sensor data collected in real time, an abnormal state prediction model is constructed. The abnormal state prediction model is constructed based on a gated cyclic unit. The input of the abnormal state prediction model is a sensor data sequence collected according to a preset time interval within a preset time period, and the output is an operating condition score within the preset time period.

8. The smart factory operating condition monitoring system based on digital twin technology according to claim 7 is characterized in that: Manual labeling is used as sample labels for the abnormal state prediction model.

9. The smart factory operating condition monitoring system based on digital twin technology according to claim 7 is characterized in that: The loss function of the abnormal state prediction model is the cross entropy loss function.

10. The smart factory operating condition monitoring system based on digital twin technology according to claim 1 is characterized in that: The functions of the user interface and interaction module include real-time 3D model display, multi-view and layered display, sensor data display, status alarm and notification, automatic execution of response measures and historical data playback.

Images