Industrial robot plate sorting digital twin system and construction method

Abstract

The invention discloses an industrial robot plate sorting digital twin system and a construction method. The digital twin system comprises a physical layer, a data layer, a virtual layer and an application layer. The physical layer is used for equipment operation and twin data acquisition; the data layer is used for carrying out preprocessing and fusion analysis on the twin data; the virtual layer is used for constructing a virtual model and realizing intelligent decision and control according to the virtual model; and the application layer is used for providing user interaction, state detection, visual monitoring and intelligent decision support. According to the invention, the equipment operation state and the data transmission stability in the automatic sorting process can be monitored, so that the industrial sorting system gradually develops from the current automation to the intelligent and informatization management direction of the system.

Description

Technical Field

[0001] This invention belongs to the field of automated board sorting technology in board production, and particularly relates to a digital twin system for industrial robot board sorting and its construction method. Background Technology

[0002] Automated sorting is a key area of ​​transformation in furniture board production. In the field of industrial automation, material sorting has achieved highly efficient automated operation through multi-machine collaborative work, significantly improving production efficiency. However, some manufacturing enterprises have exposed obvious shortcomings in production monitoring and management during the intelligent transformation process. Specifically, existing monitoring methods are still mainly based on traditional video surveillance, which makes it difficult to achieve accurate and panoramic perception of equipment operating status; the visualization effect of the system is not optimized enough, and the user experience needs to be further improved; the system functions are relatively simple, and the interactivity with the monitored physical system needs to be improved.

[0003] To address the aforementioned problems and promote the operation and data management of physical equipment systems, ensuring the normal operation of automated sheet metal sorting systems, the automated sheet metal sorting process can be visualized and monitored. This allows industrial sorting systems to gradually evolve from current automation towards intelligent and information-based management. Digital twins are a technology that integrates multiple physical, multi-scale, and multi-disciplinary attributes, possessing real-time synchronization, faithful mapping, and high fidelity characteristics. It enables the interaction and fusion of the physical and information worlds, providing new insights for industrial equipment monitoring technology. By monitoring the operational status of the sheet metal sorting system in real time, problems can be identified and resolved promptly, preventing downtime and malfunctions. Therefore, this invention proposes establishing a digital twin monitoring system for industrial robot sheet metal sorting scenarios.

[0004] The current implementation of digital twin monitoring systems for automated sorting processes faces several challenges. First, high-precision modeling and real-time data synchronization require detailed 3D modeling of machinery and work environments, and real-time data exchange between physical equipment and the digital twin model. This necessitates a close integration of real-time monitoring and data-driven virtual modeling, ensuring low latency and high reliability in data transmission to guarantee a high degree of consistency between the virtual model and the actual equipment. Second, data acquisition and transmission communication issues exist. Industrial manufacturing environments employ various communication interfaces and protocols, resulting in dispersed and heterogeneous data sources. Furthermore, high data latency makes it difficult to meet real-time requirements. Utilizing high-speed, low-latency transmission protocols can solve the multi-source heterogeneity problem, enabling real-time data processing in both physical and information spaces. Finally, current industrial robot digital twin systems offer relatively limited services, primarily providing data-driven functionality for industrial robots, which is only a part of the overall digital twin system. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention proposes a digital twin system and construction method for industrial robot panel sorting. To promote the operation of the physical system and the management of data, and to ensure the normal operation of the automated sorting system, the system can monitor the equipment operating status and data transmission stability during the automated sorting process. This allows the industrial sorting system to gradually evolve from current automation towards intelligent and information-based management. Because digital twin technology allows for real-time data collection from actual equipment and systems and mapping it to a virtual model, real-time monitoring of the sorting system's operating status enables timely detection and resolution of problems, preventing downtime and malfunctions.

[0006] To achieve the above objectives, the present invention discloses a digital twin system for industrial robot panel sorting, comprising: a physical layer, a data layer, a virtual layer, and an application layer;

[0007] The physical layer is used for device operation and the collection of twin data;

[0008] The data layer is used for preprocessing and fusion analysis of the twin data;

[0009] The virtual layer is used to construct a virtual model, and to realize intelligent decision-making and control based on the virtual model;

[0010] The application layer is used to provide user interaction, status detection, visual monitoring, and intelligent decision support.

[0011] Optionally, the physical layer includes: a device operation interaction unit, a data acquisition unit, and an energy management unit;

[0012] The device operation interaction unit is used to include physical devices and the surrounding environment in the actual working environment. The physical devices are responsible for performing specific sorting tasks and physical operations and interacting with the surrounding environment.

[0013] The data acquisition unit is used to collect equipment operating status information and environmental data using sensors, and to control the operation of the industrial robot according to the received instructions using the control cabinet, thereby realizing the automated operation of the physical equipment, and at the same time transmitting the operating data of the physical equipment to the data layer.

[0014] The energy management unit is used to monitor the energy consumption of physical equipment and adjust the power output according to the operating status to achieve energy-saving operation.

[0015] Optionally, the data layer includes: a data transmission unit, a data preprocessing unit, and a data fusion and analysis unit;

[0016] The data transmission unit is responsible for transmitting the device operation data and environmental data collected by the physical layer to the virtual layer through network protocols;

[0017] The data preprocessing unit is used to perform real-time preprocessing on the collected raw data using edge computing.

[0018] The data fusion and analysis unit is used to establish a multi-source data fusion mechanism, which integrates data from different sources and of different types, and mines the inherent relationships between the data to provide more comprehensive and accurate data support for virtual layer modeling and application layer decision-making.

[0019] Optionally, the virtual layer includes: a virtual model construction unit, a twin data management unit, and an intelligent control unit;

[0020] The virtual model construction unit is used to construct a virtual model of the physical entity. Through real-time data interaction, the operation process of the physical entity is reproduced in the virtual model in real time, realizing real-time monitoring and complete mapping of the operation process of the physical entity, and keeping the virtual model and the physical entity synchronized in real time.

[0021] The twin data management unit is used to store and manage physical entity state data obtained from the data layer, and to map this data to the virtual model in real time, so that the virtual model can be updated in real time according to the twin data.

[0022] The intelligent control unit is used to introduce artificial intelligence-driven virtual model behavior rules, enabling the virtual model to automatically learn and optimize its behavior rules based on historical and real-time data, thereby achieving intelligent sorting operations and equipment control.

[0023] Optionally, the application layer includes: a user interaction unit, a status monitoring unit, a visualization monitoring unit, and an intelligent decision-making unit;

[0024] The user interaction unit is used to provide users with a system login interface and realize multi-user permission management;

[0025] The status monitoring unit is used to monitor and evaluate the health status and energy consumption of the equipment in real time based on the operating status data and twin data of the physical equipment.

[0026] The visualization monitoring unit is used to provide real-time visualization monitoring functions, allowing users to intuitively observe the real-time situation of the sorting site and the operating status of the equipment through virtual reality and augmented reality technologies;

[0027] The intelligent decision-making unit is used to integrate an intelligent decision support system and provide users with intelligent decision-making suggestions based on real-time monitoring data and historical data analysis results.

[0028] This invention also provides a method for constructing a digital twin system for industrial robot panel sorting, comprising:

[0029] Obtain physical entity data;

[0030] Lightweight data is obtained by performing lightweight processing on the physical entity data.

[0031] Based on the aforementioned lightweight data, multiphysics coupling simulation technology is introduced to construct a simulation model;

[0032] Based on the simulation model and lightweight data, a virtual model is constructed, wherein the virtual model is used to realize intelligent decision-making and control.

[0033] Optionally, lightweight processing is performed on the physical entity data to obtain lightweight data, including:

[0034] Based on the physical entity data, each physical device is divided into several independent sub-modules;

[0035] Based on the sub-modules, construct a three-dimensional model;

[0036] The 3D model is then subjected to lightweight processing to obtain lightweight data.

[0037] Optionally, constructing a virtual model based on the simulation model and the real physical scene includes:

[0038] Based on the simulation model and the real physical scenario, construct an initial data twin model;

[0039] Based on the initial data twin model, and combined with the operating status data of the physical equipment, the virtual model is constructed.

[0040] Compared with the prior art, the present invention has the following advantages and technical effects:

[0041] This invention constructs a digital twin system architecture for industrial robot panel sorting, which is mainly divided into four layers: physical layer, data layer, virtual layer, and application layer. The physical layer communicates with the data layer. The physical layer contains all the physical devices of the system. The data layer and physical layer transmit data from the server to the physical devices via a data interface network, and the physical devices operate according to the data. The physical layer and virtual layer are connected through the data layer. The data layer transmits data obtained from the physical layer to the virtual layer, and the physical layer and virtual layer perform bidirectional twin information mapping processing. The data layer and virtual layer communicate with each other, transmitting data from the data layer to the virtual layer via network transmission protocols such as TCP / IP. The virtual model updates in real time based on the twin data to achieve synchronous movement with the physical layer, achieving a seamless virtual-real effect. The data layer is connected to the application layer. The data layer inputs the twin data to the application layer. The application layer performs virtual-real simulation and data monitoring based on the twin data, and the application layer feeds back updated data to the data layer, enabling the data layer to change data in real time. The virtual layer is connected to the application layer. The virtual model in the virtual layer updates in real time based on the twin data. The application layer can visualize and monitor the virtual model in real time. The application layer stores the twin data in a database, and can also reproduce the running state based on this data.

[0042] The construction method of this invention involves consulting data on the physical equipment of a panel sorting system based on real-world physical devices to obtain the shape and size information of the physical entities, and measuring and verifying the entity data of each physical entity. Using the 3D modeling software SolidWorks, based on the obtained equipment entity data, 3D models drawn from multiple independent sub-modules are assembled according to the composition structure of the physical equipment, ensuring consistency with the physical entities in size, proportion, and shape. After saving the 3D models of each physical entity, the models are imported into 3ds Max for lightweight processing to reduce the number of faces and vertices. Using MWorks software, Modelica is used to construct a mechanistic model of an industrial robot, giving it simulation capabilities. The lightweight 3D virtual model is then imported into Unity3D for loading and adjustment. Finally, based on the real-world physical scene, the overall layout of the panel sorting digital twin is completed, and lighting and physical material rendering are added to the panel sorting system. The design of a user login management platform in Unity3D involves establishing a connection between Unity3D and the server database via protocols such as TCP / IP to achieve data-driven functionality. This platform transmits data to the digital twin system in real time through the server connection, enabling real-time 3D monitoring. Attached Figure Description

[0043] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0044] Figure 1 This is an architecture diagram of an industrial robot panel sorting digital twin system according to an embodiment of the present invention;

[0045] Figure 2 This is a schematic diagram of the layout and structure of the industrial robot panel sorting system according to an embodiment of the present invention;

[0046] Figure 3 This is a schematic diagram of three types of book racks in the industrial robot panel sorting system of this invention;

[0047] Figure 4 This refers to the book shelf location number in the industrial robot panel sorting system of this invention.

[0048] Figure 5 This invention relates to a visualization dashboard for book shelf storage in an industrial robot panel sorting system.

[0049] Figure 6 This is a book shelf location information table for an industrial robot panel sorting system according to an embodiment of the present invention;

[0050] Figure 7 This is a roadmap for the construction technology of the digital twin system for industrial robot panel sorting according to an embodiment of the present invention;

[0051] Figure 8 This is a schematic diagram of the industrial robot mechanism according to an embodiment of the present invention;

[0052] Figure 9 This is a schematic diagram of the adjustment of the rotation axes of each joint of the industrial robot according to an embodiment of the present invention;

[0053] Figure 10 This is the user login interface of the industrial robot panel sorting digital twin system according to an embodiment of the present invention;

[0054] Figure 11 This is a diagram illustrating the effect of an industrial robot storing wooden boards in a storage location during real-time linkage between the virtual and physical spaces in the digital twin system for industrial robot board sorting, as described in this embodiment of the invention.

[0055] Among them, 1 is the roller conveyor belt; 2 is the industrial robot; 3 is the A-class book shelf number A01; 4 is the B-class book shelf number A02; 5, 6, 7, 8, 9, and 10 are the C-class book shelves numbered A03, A04, A05, A06, A07, and A08 respectively; 11 is the physical layer; 12 is the data layer; 13 is the virtual layer; and 14 is the application layer. Detailed Implementation

[0056] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0057] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0058] This embodiment proposes a digital twin system for industrial robot panel sorting, such as... Figure 1 As shown, it specifically includes: physical layer 11, data layer 12, virtual layer 13 and application layer 14;

[0059] Physical layer 11 is used for device operation and acquisition of twin data;

[0060] Data layer 12 is used for preprocessing and fusion analysis of twin data;

[0061] Virtual layer 13 is used to construct virtual models and realize intelligent decision-making and control based on the virtual models;

[0062] Application layer 14 is used to provide user interaction, status detection, visual monitoring and intelligent decision support.

[0063] Specifically, physical layer 11 mainly includes physical equipment and surrounding environment involved in the actual working environment, such as industrial robot 2, control cabinet, sensors, bookshelves, conveyor belts, servers, etc. These physical devices are key to building the digital twin system. It is used to build the entities of the automated sorting system and realize the control of the automated sorting system. The server transmits the storage location number of the wooden boards to the robot, and industrial robot 2 stores the wooden boards in the fixed storage location according to the storage location number, ensuring the normal operation and sorting of the automated sorting system. Physical layer 11 is the foundational layer of the system, used to accurately build the entities of the board sorting system and realize the control of the system, ensuring its normal operation and accurate sorting. This layer is the actual operating object of the digital twin system.

[0064] Data layer 12 serves as a bridge connecting physical layer 11 and virtual layer 13. It is used for data acquisition, transmission, and synchronization. Utilizing communication protocols or data transmission networks, it transmits data from the physical entity layer, including the specific storage locations of wooden boards within the bookshelves, to virtual layer 13 in real time. This ensures the consistency and synchronization between the 3D virtual model and the actual automated sorting system. To ensure the stable operation of the digital twin system, a robust data interaction scheme is needed to guarantee the efficiency and security of data acquisition, transmission, and operation.

[0065] The virtual layer 13 includes a virtual model and twin data. The virtual model is used to create a 3D virtual model corresponding to the physical entity in the physical layer 11. It is a precise representation of the physical entity, possessing its relevant physical attributes, mainly including the geometric parameters, behavioral rules, and operating rules of the industrial robot 2, as well as the bookshelf, etc. By receiving data from the data layer 12, the 3D virtual model is dynamically updated to reflect the actual state and operation of the physical entity. The twin data contains the state data of the physical entity obtained by the data layer 12 from the server, including data such as the specific storage location of the wooden boards. It is a key factor in achieving motion synchronization between the virtual model and the physical entity, ensuring that the industrial robot 2 can successfully achieve the expected results. The virtual layer 13 not only completely maps the physical layer 11, but also maintains the real-time isomorphism between the virtual model and the physical entity during synchronized motion, and ensures the virtual model possesses realism, achieving a seamless integration of virtual and real elements.

[0066] Application layer 14 primarily allows users to transparently and comprehensively monitor and analyze physical entities through the 3D virtual model of virtual layer 13 via a user interface. This includes monitoring the physical entity's operational status, viewing operational data, optimizing performance, and obtaining decision support. It also enables historical data playback through data stored in the database. Users register an account and log in through the login interface. Status monitoring can analyze the operational status data of physical equipment, verifying whether the data on the specific storage location of the wooden planks matches reality, and making decisions to determine the health status of the equipment, ensuring stable and healthy operation. Visual monitoring can monitor the movement of industrial robot 2 in real time, achieving comprehensive monitoring of the production site and improving production safety and efficiency. This data can also be stored in the database, enabling historical data playback.

[0067] More specifically, in the process of board sorting, the board sorting system (physical layer 11) is divided into physical equipment, network communication and control and execution design, based on the needs of board handling and sorting. The physical equipment, including roller conveyors, robots, and bookshelves, is the board sorting equipment used for actual sorting operations. Specific wooden boards are transported from roller conveyor 1 to the initial position grasped by the robot. Photoelectric sensors on the conveyor belt detect the boards and send information about their arrival at the initial position to the robot. The robot then grasps the boards at the initial position and, based on the storage location number from the server, the industrial robot 2 stores the boards in the designated storage location. The Ethernet IP protocol, Modbus TCP, and driver interface define the network communication part, used to realize communication and interaction between the board sorting equipment. The server and PLC controller are connected via Modbus TCP, and the PLC controller and robot controller are connected via Ethernet IP, thus enabling the server to connect to the robot. The control and execution design includes the PLC controller, robot control system, and human-machine interface panel, which realizes the control and execution of the corresponding board sorting equipment. The physical layer 11 in this embodiment mainly focuses on the establishment, coordination, and control of the board sorting system. It collects real-time data on board storage and environmental information through network communication protocols and uploads the real-time data to the twin data service system.

[0068] Furthermore, the physical layer 11 includes: a device operation interaction unit, a data acquisition unit, and an energy management unit;

[0069] The equipment operation interaction unit is used to include the physical equipment and surrounding environment in the actual working environment. The physical equipment is responsible for performing specific sorting tasks and physical operations and interacting with the surrounding environment.

[0070] The data acquisition unit is used to collect equipment operating status information and environmental data using sensors, and to control the operation of industrial robot 2 according to the received instructions using the control cabinet, so as to realize the automated operation of physical equipment, and at the same time transmit the operating data of physical equipment to data layer 12.

[0071] The energy management unit is used to monitor the energy consumption of physical equipment and adjust the power output according to the operating status to achieve energy-saving operation.

[0072] Furthermore, data layer 12 includes: a data transmission unit, a data preprocessing unit, and a data fusion and analysis unit;

[0073] The data transmission unit is responsible for transmitting the device operation data and environmental data collected by the physical layer 11 to the virtual layer 13 via network protocols.

[0074] The data preprocessing unit is used to perform real-time preprocessing of the collected raw data using edge computing methods;

[0075] The data fusion and analysis unit is used to establish a multi-source data fusion mechanism, which integrates data from different sources and of different types, and mines the inherent relationships between the data to provide more comprehensive and accurate data support for the modeling of the virtual layer 13 and the decision-making of the application layer 14.

[0076] Specifically, the data layer 12, located between the physical layer 11 and the virtual layer 13, is responsible for data acquisition, transmission, and synchronization. It utilizes the TCP / IP communication protocol and data transmission technology to transmit data from the physical layer 11 server to the virtual layer 13 in real time. The transmitted data mainly includes the specific storage location numbers of the wooden boards, such as shelf number 3 for type A bookshelves, shelf number 3 for type B bookshelves, and shelf numbers 5, 6, 7, 8, 9, and 10 for type C bookshelves, e.g., A01_01_01, ensuring the consistency and synchronization between the virtual model and the actual board sorting system.

[0077] Furthermore, the virtual layer 13 includes: a virtual model construction unit, a twin data management unit, and an intelligent control unit;

[0078] The virtual model building unit is used to build virtual models of physical entities. Through real-time data interaction, the operation process of the physical entity is reproduced in the virtual model in real time, realizing real-time monitoring and complete mapping of the operation process of the physical entity, and keeping the virtual model and the physical entity synchronized in real time.

[0079] The twin data management unit is used to store and manage the physical entity status data obtained from the data layer 12, and to map this data to the virtual model in real time, so that the virtual model can be updated in real time according to the twin data;

[0080] The intelligent control unit is used to introduce artificial intelligence-driven behavior rules for virtual models, enabling virtual models to automatically learn and optimize their behavior rules based on historical and real-time data, thereby achieving intelligent sorting operations and equipment control.

[0081] Specifically, the virtual layer 13 establishes a precise, hierarchical three-dimensional virtual model corresponding to the physical devices of the physical layer 11, such as... Figure 2 As shown, this is the virtual model established for virtual layer 13, which includes an industrial robot 2, a roller conveyor belt 1, and three types of bookshelves: A, B, and C. Figure 3 The image shows three types of bookshelves. Based on the actual bookshelf numbers, in the virtual layer 13, bookshelves of types A, B, and C are assigned storage location numbers according to the bookshelf location number information table, as shown below. Figure 4The diagram shows the shelf location numbering in virtual layer 13. By receiving twin data from data layer 12, the 3D virtual model is dynamically updated to reflect the actual state and operation of the physical entity. A specific wooden board is transported from conveyor belt 1 to the initial position for robot gripping. Photoelectric sensors on the conveyor belt detect the board and send information about its arrival at the initial position to the robot. The robot then grips the board at the initial position and stores it in the specific location based on the twin data of the shelf location number from data layer 12. For example, board A01_01_01 is stored in shelf location 1 on the 1st floor of shelf A01.

[0082] The virtual layer 13 is a precise mapping of the physical layer 11. By realistically reflecting the operation of the physical layer 11 in the virtual layer 13, it provides support for the operation of the physical layer 11. The virtual model in the virtual layer 13 includes the geometry, physical characteristics, behavioral models, and operating parameters of the physical entities. In this embodiment, it aims to simulate the operating state and behavior of the physical equipment for board sorting, thereby achieving three-dimensional visualization monitoring of the board sorting process.

[0083] More specifically, a 3D virtual model refers to a precise digital representation of the physical equipment in a sheet metal sorting system built based on digital twin technology. These 3D virtual models are created using 3D modeling software (such as SolidWorks) and further processed and optimized in the Unity 3D development platform to achieve efficient and accurate operation in a virtual environment. Its functional principle is as follows:

[0084] (1) Loading and displaying three-dimensional virtual models: In Unity3D, the position, rotation and scaling of three-dimensional virtual models are controlled by the Transform component to achieve accurate placement and dynamic performance in virtual space.

[0085] (2) Data-driven dynamic updates: Through real-time data acquisition (such as the storage location number of the wooden board), the actions and status of the three-dimensional virtual model can be updated in real time, reflecting the current status of the physical equipment.

[0086] (3) Interactivity: Users can interact with the 3D virtual model through the interface, such as simulating robot operation and adjusting equipment parameters.

[0087] (4) Visual monitoring: Combining real-time data and model animation, it provides an intuitive monitoring interface, enabling users to observe and analyze the system status from different perspectives.

[0088] Virtual layer 13 first uses 3D modeling software to construct virtual models of each device. Considering that the complex model movement in the twin platform will affect the operating performance of the twin system, the virtual model is reduced in size and weight to reduce the redundancy and complexity of the model. The weighted model is then exported, loaded into the digital twin development platform, the model control script is edited and bound, and the logical processing rules are determined to realize the simulation function of the model.

[0089] The digital twin development platform used in this embodiment is the Unity3D software platform. This platform integrates a rich plugin resource library, intelligent script tools, and a comprehensive technical documentation system, which can significantly accelerate the development process of the board sorting digital twin system. Its modular architecture design supports plug-and-play functionality with multi-source IoT terminals and heterogeneous databases, ensuring low-latency synchronization and intelligent optimization of twin data. Based on an open API ecosystem and an extensible script engine, developers can accurately meet the customized data transmission needs in board sorting scenarios, enabling continuous iterative upgrades to the functionality and performance of the digital twin system. The Unity software's operation panel mainly includes five parts: the Scene panel is the main work area for editing and manipulating virtual models and scenes; the Game panel is used to display the real-time running effect of virtual models; the Hierarchy panel displays the hierarchical structure of all objects in the current scene; the Project panel displays all resources and files in the project; and the Inspector panel is used to view and edit the attributes and components of selected objects, such as position, rotation, and scaling. The integration of these panels in Unity makes building a board sorting digital twin system more efficient and intuitive, enabling accurate simulation and analysis of equipment operation.

[0090] Furthermore, the application layer 14 includes: a user interaction unit, a status monitoring unit, a visualization monitoring unit, and an intelligent decision-making unit;

[0091] The user interaction unit is used to provide users with a system login interface and to implement multi-user permission management.

[0092] The status monitoring unit is used to monitor and evaluate the health status and energy consumption of the equipment in real time based on the operating status data and twin data of the physical equipment.

[0093] The visual monitoring unit provides real-time visual monitoring capabilities, allowing users to intuitively observe the real-time situation and equipment operating status at the sorting site through virtual reality and augmented reality technologies.

[0094] The intelligent decision-making unit is used to integrate an intelligent decision support system and provide users with intelligent decision-making suggestions based on real-time monitoring data and historical data analysis results.

[0095] Specifically, application layer 14 is the topmost user interface, providing an integrated digital twin application system. Through this layer, users can achieve transparent and comprehensive monitoring and analysis of physical equipment through 3D simulation. This includes monitoring the operational status of the physical equipment and viewing the storage locations of the planks. On the digital twin platform, received data is mapped onto the virtual model in real time. By updating the position, speed, and orientation of the virtual model in real time, operators can clearly understand the dynamic behavior of the equipment and make adjustments accordingly to optimize its performance. They can also monitor for operational errors and ensure proper connections between physical equipment by verifying that the storage locations of the planks match reality. Figure 5 This embodiment provides a visualization dashboard for the book shelf storage in the industrial robot 2 board sorting system. The dashboard allows monitoring of the entire system's operation and the status of the stored boards.

[0096] The core of 3D visualization monitoring and operational status assessment lies in mapping the storage location information of wooden planks to a 3D virtual space in real time. This process highly relies on the accurate support of twin data. By integrating multi-source heterogeneous data real-time transmission technology and a distributed data storage architecture based on SQL Server, the system constructs a stable and reliable twin data pipeline, ensuring efficient transmission and persistent storage of massive amounts of data. Figure 6 The image shows the book shelf location information table of the industrial robot 2 board sorting system in the database. In the digital twin platform, the client parses various data streams pushed by the server in real time and maps them to a 3D virtual scene, accurately presenting key production elements such as the real-time operating trajectory of industrial robot 2 and the spatial distribution of board storage locations. This real-time visualization capability not only enhances the system's interactivity, allowing users to interact directly with the virtual model, but also provides strong support for decision-making by intuitively displaying the key operating status of the equipment and the board storage status. Furthermore, it can be used for fault diagnosis, performance prediction, and maintenance optimization, and even as a training tool to improve employees' understanding of complex systems, thereby ensuring the efficient operation of the entire production process and the stability of the equipment.

[0097] By utilizing 3D simulation, the system can be visualized and monitored. Viewpoints can be switched as needed, allowing operators to freely obtain the optimal perspective in virtual space. This enhances the interactivity of the monitoring system, enabling synchronous operation of sorting equipment in both virtual and physical spaces. It achieves comprehensive dynamic monitoring of the sorting scenario, allowing operators to fully understand equipment operation and the surrounding environment. Furthermore, system operation data can be continuously stored in a database, using historical data to drive the virtual model to reproduce historical states, helping users trace the root cause of faults.

[0098] As can be seen from the above embodiments, current industrial monitoring mainly relies on traditional video surveillance, which cannot accurately and comprehensively monitor the operation of system equipment and whether the connections between various physical devices are normal. There are deficiencies in the monitoring and management of automated production, lacking good management methods and monitoring technologies. Furthermore, because different manufacturers and developers design management and control systems that use different standards and protocols, there is a lack of a unified integration platform for information interaction. This results in heterogeneity and diversity in industrial manufacturing information management systems, making information interaction difficult and untimely, and causing serious information silos. It is impossible to collect data, monitor status, and control industrial equipment in real time and accurately, which seriously affects the efficiency of operation.

[0099] In contrast, the industrial robot 2 panel sorting digital twin system in this embodiment utilizes digital twin 3D visualization monitoring technology to provide a more comprehensive real-time view of the panel sorting system's operation. Viewpoints can be switched as needed, allowing for flexible acquisition of the optimal perspective in virtual space to monitor the entire system's operation, enhancing the interactivity of the monitoring system. Real-time monitoring of the sorting system's operation significantly improves the work efficiency of monitoring personnel, enabling timely detection of system anomalies and reducing the probability of errors. During monitoring, this embodiment continuously stores system operation data in a database, allowing technicians to query and access data at any time to analyze the system's operational status. Especially when system failures occur, technicians can use historical data to drive the virtual model to reproduce historical states, helping them quickly locate the cause of the failure and troubleshoot the problem. This continuous data storage method also provides more reliable data support for the long-term operation of the panel sorting system, contributing to improved system reliability and stability.

[0100] Example 2:

[0101] This embodiment also provides a method for constructing a digital twin system for industrial robot panel sorting, including:

[0102] Obtain physical entity data;

[0103] Lightweight data is obtained by performing lightweight processing on physical entity data;

[0104] Based on lightweight data, multiphysics coupling simulation technology is introduced to construct a simulation model;

[0105] A virtual model is constructed based on the simulation model and lightweight data, which is used to realize intelligent decision-making and control.

[0106] Furthermore, lightweight processing is performed on the physical entity data to obtain lightweight data, including:

[0107] Based on the physical entity data, each physical device is divided into several independent sub-modules;

[0108] Build a 3D model based on the submodules;

[0109] The 3D model is lightweighted to obtain lightweight data.

[0110] Furthermore, based on the simulation model and the real physical scene, the virtual model is constructed by including:

[0111] Based on the simulation model and the real physical scenario, construct the initial data twin model;

[0112] Based on the initial data twin model, and combined with the operational status data of the physical equipment, a virtual model is constructed.

[0113] Specifically, such as Figure 7 The construction method of the system shown specifically includes:

[0114] (1) Based on the actual physical equipment, consult the physical equipment data of the board sorting system, obtain the shape and size information of the physical entities, and verify the physical data of each physical entity by measurement. The physical equipment mainly includes industrial robot 2, robot base, roller conveyor belt and three types of bookshelves A, B and C.

[0115] (2) Using the 3D modeling software SolidWorks, based on the acquired equipment entity data, each physical device is divided into multiple independent sub-modules, which are then drawn using SolidWorks. The 3D models drawn from the multiple independent sub-modules are then assembled as a whole according to the composition structure of the physical devices, and are consistent with the physical entities in terms of size, proportion and shape.

[0116] (3) After saving the three-dimensional models of each physical entity, import the models into 3ds Max for lightweight processing to reduce the number of faces and vertices, reduce the redundancy of the model, improve loading speed and performance, and enhance the response speed of the twin system. The coordinate axis center of some three-dimensional virtual models is not at its geometric center, so it is necessary to make appropriate adjustments. Adjust the rotation axis position of each joint of the industrial robot 2 three-dimensional virtual model so that the axis of each joint is on the rotation axis of each joint, so that the robot moves according to each rotation axis.

[0117] (4) Using MWorks software, Modelica was used to construct a mechanistic model of the industrial robot 2, such as... Figure 8 The image shows the Modelica model of industrial robot 2, which enables it to perform simulations.

[0118] (5) Import the lightweight 3D virtual model into Unity3D in .fbx format, load and adjust it, integrate the geometric model, mechanism model and twin data model to complete the construction of the entire board sorting digital twin system, complete the overall layout construction of the board sorting digital twin according to the real physical scene, add lighting and physical material rendering to the board sorting system to make it closer to the real physical equipment, and adjust the layout of the digital twin system to be consistent with the real scene.

[0119] (6) Design the user login management platform using built-in UI components in Unity3D. Next, create a folder named Plugins in the Unity Scene and import the following libraries: I18N.dll, I18N.CJK.dll, I18N.West.dll, Mysql.Data.dll, System.Data.dll, and System.Drawing.dll. Connect Unity3D to the server database to implement data-driven functionality. Transmit data to the digital twin system in real-time via the server connection to achieve real-time 3D monitoring.

[0120] Further in step (3), the lightweighting process and rotation axis setup include:

[0121] 1) Export the 3D model from SolidWorks in .step format and check if the 3D virtual model information is complete;

[0122] 2) Load the model into 3ds Max and use 3ds Max to delete lines and faces that do not participate in actual operation control, simplify the model mesh, and reduce the number of hidden bodies. Taking Industrial Robot 2 as an example, the original model has 21,157 polygon faces and 10,785 vertices. After lightweighting, the model has 12,721 polygon faces and 6,567 vertices.

[0123] 3) After importing the 3D model into ds Max software in .step format, select the robot's base, choose "Affect only axes" in the Hierarchy panel, and then use move and rotate operations to set the axis centers to the corresponding positions. Next, adjust the axis centers of the other joints of industrial robot 2, such as... Figure 9 The image shows the adjusted axis positions of each joint.

[0124] In step (4), the construction of the robot mechanism model includes:

[0125] The 3D robot model drawn in SolidWorks is exported in .step format. The robot model is then imported into MWorks using the CAD toolbox. The robot model is then grouped according to the components of each joint. Rotary joints are added to the model according to the rotation axes of each joint. Finally, rotational drives are added to each joint rotation axis to generate the Modelica model of the robot.

[0126] In step (5), loading and adjustments are performed, including:

[0127] First, import the 3D virtual models of the roller conveyor belt, bookshelf, base, and robot into Unity3D in .fbx format. The robot base's coordinate system is located at the origin of the world coordinate system. Place the industrial robot 2 on the base. Adjust the positions of the roller conveyor belt 1 and bookshelf according to the real-world physical scene to match the layout. Set parent-child relationships for each joint of the robot's virtual model. For example, make the robot base the parent object of joint 1, drag joint 1 to make it a child object of the base, and make joint 1 the parent object of joint 2, and so on. This way, when joint 1 moves, the other joints move with it. Connect all the joints and complete the setup. Adjust the initial state of each joint to match the initial state of the real robot. In Unity3D, add lighting to simulate the real environment and render the materials of each virtual model to make them resemble real physical equipment.

[0128] In addition, unit conversion adjustments are needed when importing 3D virtual models from 3ds Max to Unity. By default, the unit ratio between Unity and 3ds Max is 100:1. The default unit for models in 3ds Max is "centimeters," while the default unit in Unity is "meters." To ensure that the model's size remains consistent during import, the system unit in 3ds Max needs to be set to "meters" to match Unity's properties. Furthermore, virtual models in Unity default to Y-axis pointing upwards, while those in 3ds Max default to Z-axis pointing upwards; therefore, when exporting the model, you must also select Y-axis pointing upwards.

[0129] Specifically, in step (6): a canvas is added in Unity3D using UI components, and a background image, registration, and login buttons are added to the canvas to complete the design of the user login management platform, such as... Figure 10The image shows the system's login interface. The login account and password are saved to the local database. Next, a folder named "Plugins" is created in the Unity Scene. Libraries such as I18N.dll, I18N.CJK.dll, I18N.West.dll, Mysql.Data.dll, System.Data.dll, and System.Drawing.dll are imported into this folder. A connection is established between Unity3D and the server database, and data is retrieved from the server via TCP / IP protocol, implementing a data-driven function. Data is transmitted in real-time to the board sorting digital twin system through the server connection. The system uses the received twin data to drive the robot to store the boards, implementing a real-time 3D monitoring function. The system's operation can be monitored in real-time through the digital twin system's monitoring interface and the storage location visualization dashboard.

[0130] During the verification experiment, it is first necessary to ensure that the board sorting system in the physical space is running, the relevant servers are turned on, and the Unity3D engine is started in virtual layer 13 to establish a communication connection with the server. The board sorting physical equipment stores the boards in specific storage locations according to the storage location numbers transmitted by the server. When data is received from the physical platform server, it is mapped to the virtual model through data parsing, which drives the virtual model to run. In addition, the operation of the board sorting system is monitored in the visual dashboard of the storage locations.

[0131] Once the automated sorting physical platform and the twin system are started, the operation of the automated sorting physical platform can be observed in the traditional video surveillance screen, while the operation of the virtual space can be observed in the automated sorting virtual layer 13 operating interface. Figure 11 This image illustrates the real-time linkage between the virtual and physical spaces in a digital twin system for industrial robot panel sorting, showing an industrial robot storing wooden boards in designated locations. The real-time storage status of the boards matches the data in the digital twin system. Video monitoring provides real-time footage of the physical system's operation, helping users instantly understand its status and compare it with the virtual environment to ensure consistency between the virtual model and the actual system. Furthermore, the dual perspectives of video monitoring and 3D visualization allow users to more accurately diagnose problems. Video monitoring helps identify anomalies in actual operations, while 3D visualization analyzes potential sources and solutions. Combining these two monitoring methods provides more comprehensive data and information, enabling users to conduct integrated analysis and improve the system's monitoring efficiency and accuracy.

[0132] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A digital twin system for sorting industrial robot panels, characterized in that, include: Physical layer, data layer, virtualization layer, and application layer; The physical layer is used for device operation and the collection of twin data; The data layer is used for preprocessing and fusion analysis of the twin data; The virtual layer is used to construct a virtual model, and to realize intelligent decision-making and control based on the virtual model; The application layer is used to provide user interaction, status detection, visual monitoring, and intelligent decision support.

2. The industrial robot panel sorting digital twin system according to claim 1, characterized in that, The physical layer includes: a device operation interaction unit, a data acquisition unit, and an energy management unit; The device operation interaction unit is used to include physical devices and the surrounding environment in the actual working environment. The physical devices are responsible for performing specific sorting tasks and physical operations and interacting with the surrounding environment. The data acquisition unit is used to collect equipment operating status information and environmental data using sensors, and to control the operation of the industrial robot according to the received instructions using the control cabinet, thereby realizing the automated operation of the physical equipment, and at the same time transmitting the operating data of the physical equipment to the data layer. The energy management unit is used to monitor the energy consumption of physical equipment and adjust the power output according to the operating status to achieve energy-saving operation.

3. The industrial robot panel sorting digital twin system according to claim 1, characterized in that, The data layer includes: a data transmission unit, a data preprocessing unit, and a data fusion and analysis unit; The data transmission unit is responsible for transmitting the device operation data and environmental data collected by the physical layer to the virtual layer through network protocols; The data preprocessing unit is used to perform real-time preprocessing on the collected raw data using edge computing. The data fusion and analysis unit is used to establish a multi-source data fusion mechanism, which integrates data from different sources and of different types, and mines the inherent relationships between the data to provide more comprehensive and accurate data support for virtual layer modeling and application layer decision-making.

4. The industrial robot panel sorting digital twin system according to claim 1, characterized in that, The virtual layer includes: a virtual model construction unit, a twin data management unit, and an intelligent control unit; The virtual model construction unit is used to construct a virtual model of the physical entity. Through real-time data interaction, the operation process of the physical entity is reproduced in the virtual model in real time, realizing real-time monitoring and complete mapping of the operation process of the physical entity, and keeping the virtual model and the physical entity synchronized in real time. The twin data management unit is used to store and manage physical entity state data obtained from the data layer, and to map this data to the virtual model in real time, so that the virtual model can be updated in real time according to the twin data. The intelligent control unit is used to introduce artificial intelligence-driven virtual model behavior rules, enabling the virtual model to automatically learn and optimize its behavior rules based on historical and real-time data, thereby achieving intelligent sorting operations and equipment control.

5. The industrial robot panel sorting digital twin system according to claim 1, characterized in that, The application layer includes: a user interaction unit, a status monitoring unit, a visualization monitoring unit, and an intelligent decision-making unit; The user interaction unit is used to provide users with a system login interface and realize multi-user permission management; The status monitoring unit is used to monitor and evaluate the health status and energy consumption of the equipment in real time based on the operating status data and twin data of the physical equipment. The visualization monitoring unit is used to provide real-time visualization monitoring functions, allowing users to intuitively observe the real-time situation of the sorting site and the operating status of the equipment through virtual reality and augmented reality technologies; The intelligent decision-making unit is used to integrate an intelligent decision support system and provide users with intelligent decision-making suggestions based on real-time monitoring data and historical data analysis results.

6. A method for constructing a digital twin system for industrial robot panel sorting according to any one of claims 1-5, characterized in that, include: Obtain physical entity data; Lightweight data is obtained by performing lightweight processing on the physical entity data. Based on the aforementioned lightweight data, multiphysics coupling simulation technology is introduced to construct a simulation model; Based on the simulation model and the real physical scenario, a virtual model is constructed, wherein the virtual model is used to realize intelligent decision-making and control.

7. The method for constructing a digital twin system for industrial robot panel sorting according to claim 6, characterized in that, Lightweighting is performed on the physical entity data to obtain lightweight data, including: Based on the physical entity data, each physical device is divided into several independent sub-modules; Based on the sub-modules, construct a three-dimensional model; The 3D model is then subjected to lightweight processing to obtain lightweight data.

8. The method for constructing a digital twin system for industrial robot panel sorting according to claim 6, characterized in that, Based on the simulation model and the real physical scene, the construction of the virtual model includes: Based on the simulation model and the real physical scenario, construct an initial data twin model; Based on the initial data twin model, and combined with the operating status data of the physical equipment, the virtual model is constructed.

Images