Multi-source heterogeneous data integration and visualization method for ultra-high-speed optical module production line

Abstract

The invention discloses a superspeed optical module production line multi-source heterogeneous data integration and visualization method, and the method comprises the steps: constructing a superspeed optical module production line multi-layer twin model for the production elements of a superspeed optical module, and obtaining a superspeed optical module production line digital twin model; collecting multi-source heterogeneous data in the production process of the ultra-high-speed optical module; preprocessing and standardizing the multi-source heterogeneous data and then storing the data into a database; driving the constructed ultra-high-speed optical module production line digital twinborn model by using data in the database; based on a three-dimensional visualization system, the ultra-high-speed optical module production line and the constructed ultra-high-speed optical module production line digital twinborn model are subjected to deep fusion, and visualization of the digital twinborn model and multi-source heterogeneous data is achieved. The digital twinning technology is introduced, a digital virtual model of the physical production line is constructed, real-time synchronization, dynamic simulation and fine management of data are achieved, and the problems of data integration and visualization are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of digital twin and intelligent manufacturing, and in particular to a multi-source heterogeneous data integration and visualization method for an ultra-high-speed optical module production line driven by a digital twin. Background Art

[0002] With the rapid development of modern optical communication technology, ultra-high-speed optical modules have become core components in optical communication systems. Their production efficiency and quality directly affect the performance and reliability of the entire communication network. The ultra-high-speed optical module production line is a highly complex and sophisticated system involving a variety of equipment and processes, including but not limited to laser welding, fiber alignment, packaging testing and other links. Each link generates a large amount of real-time data, such as equipment status, process parameters, product quality data, environmental monitoring data, etc. These data come from various sources and in different formats, forming a typical multi-source heterogeneous data environment.

[0003] Currently, in traditional data processing and management methods, these heterogeneous data usually need to be processed through manual integration and static analysis, which is time-consuming and prone to errors. The real-time and accuracy of data are often difficult to guarantee, making it difficult to respond to problems quickly during the production process, thus affecting the overall efficiency of the production line and product quality. In addition, traditional data visualization methods are mostly based on static charts and simple monitoring interfaces, which are difficult to provide comprehensive and dynamic production status display and in-depth decision support. Summary of the invention

[0004] The purpose of the present invention is to provide a method for multi-source heterogeneous data integration and visualization of an ultra-high-speed optical module production line. By introducing digital twin technology, a digital virtual model of the physical production line is constructed to achieve real-time synchronization, dynamic simulation and refined management of data, thereby solving the problem of data integration and visualization.

[0005] The first aspect of the present invention provides a method for integrating and visualizing multi-source heterogeneous data of an ultra-high-speed optical module production line, the method comprising:

[0006] Step S1: constructing a multi-level twin model of the ultra-high-speed optical module production line for the production factors of the ultra-high-speed optical module to obtain a digital twin model of the ultra-high-speed optical module production line;

[0007] Step S2: using multiple data sources to collect multi-source heterogeneous data in the actual ultra-high-speed optical module production process;

[0008] Step S3: preprocessing and standardizing the collected multi-source heterogeneous data and storing them in a database;

[0009] Step S4: using the data in the database to drive the constructed digital twin model of the ultra-high-speed optical module production line;

[0010] Step S5: Based on the three-dimensional visualization system, the ultra-high-speed optical module production line and the constructed digital twin model of the ultra-high-speed optical module production line are deeply integrated to realize the visualization of the digital twin model and multi-source heterogeneous data.

[0011] In some embodiments, step S1 comprises:

[0012] A digital twin model of the ultra-high-speed optical module production line is constructed from four dimensions: geometry, information, behavior, and rules. In the geometry dimension, CAD software is used to accurately construct 3D models of all key equipment and components in the production line to ensure that the shape, size, and relative position of each device are consistent with the actual situation. In the information dimension, multi-source heterogeneous data generated during the production of ultra-high-speed optical modules are collected in real time through a variety of sensors and devices, and the data is integrated. In the behavior dimension, the operation process and interaction logic of each device in the production line are modeled through programming or simulation tools, so that the model simulates the dynamic behavior of each link in the actual production process. In the rule dimension, the business logic and control rules of the production line operation are defined and integrated into the digital twin, so that the model can make correct decisions and adjustments based on preset rules.

[0013] In some embodiments, step S2 and step S3 include:

[0014] Deploy a variety of sensors and vision systems on the ultra-high-speed optical module production line to monitor equipment status, production parameters, and environmental conditions. Integrate all sensors with the MAC system to achieve real-time data collection and transmission. At the same time, the vision system uses cameras to capture images and videos of the production process to detect production quality and identify potential defects.

[0015] During the data collection stage, all data are timestamped and the clocks of each data source are synchronized. Next, the collected multi-source heterogeneous data are preprocessed, including data cleaning, removal of outliers, filling in missing data, and standardization of data formats and units. In addition, feature scaling and unified data structure are used to ensure data consistency and comparability. The processed data will be compressed and encoded, and then stored in the database in real time through the data interface.

[0016] In some embodiments, step S4 comprises:

[0017] Connect to the database, create a data parser, format the raw data obtained from the database into a structure that can be used by the digital twin model of the ultra-high-speed optical module production line, and map the equipment operation status to the various components of the virtual model; then, input the formatted data into the digital twin model of the ultra-high-speed optical module production line, and use the data binding and update mechanism to adjust the equipment status and production parameters in the model in real time to ensure that the virtual equipment of the model and the actual production line are synchronized.

[0018] In some embodiments, step S5 comprises:

[0019] The 3D modeling software Unity3D imports a detailed 3D model of the ultra-high-speed optical module production line and connects it with the digital twin model of the ultra-high-speed optical module production line, so that the virtual model accurately reflects the layout, equipment and functions of the production line; creates a virtual environment of the ultra-high-speed optical module production line in Unity3D, and uses 3D modeling tools and physics engines to realistically build a virtual model of the appearance of the ultra-high-speed optical module production line; then, integrates the data in the digital twin model of the ultra-high-speed optical module production line into the Unity project, uses Unity's real-time data binding function to map the data to virtual devices and components, and displays various information of the equipment through Unity's UI elements; sets a data refresh mechanism so that the model updates the production line status in real time, and realizes dynamic loading and display of multi-source heterogeneous data through scripts; implements interactive data analysis functions so that users can intuitively monitor and analyze various data in the production process in a virtual environment; creates visual effects and data visualization tools, including heat maps, trend charts, and equipment status indicators, through Unity's rendering and animation functions to enhance users' understanding of production line operations and performance.

[0020] In some embodiments, the geometric dimensions include:

[0021] First, the geometric data of the production equipment on the high-speed optical module production line is obtained through laser scanning, 3D photogrammetry and CAD files to ensure that the geometric model accurately reflects the actual physical structure of the production line. The production equipment includes placement machines, wire bonding machines, and loading and unloading machines. Then, the 3D modeling software SolidWorks is used to create a geometric model of the production line, focusing on building the spatial layout of the equipment and its relative position relationship, including the distance between placement machines and the distance between placement machines and upper and lower old machines. The data in the CAD file is used for reference to ensure the accuracy of the spatial layout, and the difference between the model and the actual scene is checked in real time during the modeling process. At the same time, the number of polygons of the equipment model is simplified, and only the main shape and functional characteristics of the equipment are retained. After the modeling is completed, the SolidWorks model is exported to the OBJ file format and then imported into the digital twin platform to build the geometric model of the ultra-high-speed optical module production line. Finally, the geometric model is deployed to the actual ultra-high-speed optical module production, and the actual equipment position and size data are used for calibration to ensure the precise alignment of the virtual model with the physical production line.

[0022] In some embodiments, the information dimensions include:

[0023] First, various industrial sensors are deployed at various key nodes covering the ultra-high-speed optical module production line. After the sensors are deployed to start data collection, all key data sources in the ultra-high-speed optical module production line are identified. The data sources include not only sensor data that directly reflects the production environment and equipment status, but also other data closely related to the production process, including logistics data and production data. Secondly, after all key data sources are identified, the data is classified into structured data and semi-structured data. Structured data refers to data with a fixed format, including equipment operating parameters and production schedules. This type of data is stored in a database for easy query and analysis. Semi-structured data refers to data without a strict structure, including log files and equipment alarm records. Next, data preprocessing is performed to remove noise and outliers in the data through mean filtering. Noise refers to invalid information or random errors in the data. Outliers are values ​​that are significantly different from most data points. After data preprocessing, the multi-source data stored in the database are converted into different formats and unified in units. The data are carefully converted and verified in combination with the sensor calibration information and the settings of the data acquisition system. At the same time, the data from different devices and systems are matched and integrated according to key fields to form a unified and complete data set. The key fields include timestamp and device ID. The ant colony algorithm is used to analyze the integrated data and build a production optimization model to identify bottlenecks in the production process, predict equipment failures, optimize production plans and resource scheduling, including reasonable resource allocation and scheduling through analysis of production plans and logistics data. Finally, the information model is deployed in the actual production of ultra-high-speed optical modules. By acquiring the operation data of the production line in real time, the behavior is continuously updated and optimized to ensure that the production line is always in the best operating state.

[0024] In some embodiments, the behavioral dimensions include:

[0025] First, the production process of the ultra-high-speed optical module production line is analyzed in detail, each key step in the production line is identified, and the sequence and dependency between the process steps are clarified. Each step has its specific input, output and process requirements, and the time required for each step, the equipment involved, and the operating conditions and production capacity of each equipment are determined; after completing the production process analysis, since the ultra-high-speed optical module production line involves multiple equipment and complex process steps, the agent-based modeling method is selected to build a behavioral model. The agent-based modeling simulates the independent behavior of each equipment and process and their interactions, adapts to the complex dynamic changes in the production process, and converts the dynamic behavior of the production process into a computable model through modeling; then, the simulation entity and the corresponding behavioral logic are defined. The simulation entity represents the physical and logical objects in the production line, including equipment and workpieces. For each equipment, its behavioral model is established, including various states and transition conditions between states; in addition to equipment behavior, a behavioral model is also established for each process step, including the start conditions, operation time, resource requirements and output status of each process;

[0026] A resource scheduling model is established. The purpose of the resource scheduling model is to reasonably allocate and schedule equipment and workpiece resources in the simulation. The scheduling model considers multiple factors, including resource utilization, waiting time, equipment failure rate, and workpiece priority. When a device failure is detected, the scheduling model triggers the backup equipment or rearranges the production sequence to minimize production interruption time. Similarly, during production peaks or special order requirements, the scheduling model increases the number of operators or adjusts the production speed to meet production requirements. Finally, the behavioral model is deployed to the actual ultra-high-speed optical module production. Through real-time monitoring and feedback, it is ensured that the simulation model is consistent with the dynamic behavior of the actual production process and accurately reflects the operation of the production line.

[0027] In some embodiments, the rule dimensions include:

[0028] First, a standard operating procedure is formulated for the ultra-high-speed optical module production line. The standard operating procedure provides clear steps and guidelines for each link of the production line to ensure that all operators and automated equipment have a unified code of conduct when performing production activities. For the ultra-high-speed optical module production line, the standard operating procedure covers various processes, including chip packaging, fiber fusion splicing, laser etching and aging testing. The standard operating procedure for each process includes preparation before operation, specific operating steps, setting parameters for using equipment, precautions during operation, and cleaning and maintenance steps after the operation is completed. Next, automation rules are formulated. The automation rules include automatic start and stop conditions for equipment, automatic adjustment strategies for production rhythm, and automatic response mechanisms under abnormal conditions. Automation rules are implemented through programming and It is closely integrated with the control system and data acquisition system of the production line to form a closed-loop automated control system. By implementing automated rules and strategies, the automation level and flexibility of production are improved. Secondly, a fault diagnosis and emergency response mechanism is formulated. The fault diagnosis mechanism identifies potential fault hazards by real-time analysis of the operating status and sensor data of the equipment, and initiates the corresponding emergency response mechanism. At the same time, when executing the scheduling strategy, the faulty machine is excluded and the scheduling strategy is executed safely. Finally, the rule model is deployed to the actual production of ultra-high-speed optical modules. The effectiveness of the rules is verified and adjusted through continuous monitoring and feedback. By analyzing the actual data in the production process, the deficiencies in the rule system are discovered, and corresponding optimization and improvement are carried out.

[0029] According to a second aspect of the present invention, there is provided an ultra-high-speed optical module production line multi-source heterogeneous data integration and visualization system for implementing the ultra-high-speed optical module production line multi-source heterogeneous data integration and visualization method according to any one of the first aspects, the system comprising:

[0030] A model building module is used to build a multi-level twin model of the ultra-high-speed optical module production line for the production factors of the ultra-high-speed optical module, and obtain a digital twin model of the ultra-high-speed optical module production line;

[0031] The data acquisition module is used to collect multi-source heterogeneous data in the production process of ultra-high-speed optical modules using multiple data sources;

[0032] The data processing module is used to pre-process and standardize the collected multi-source heterogeneous data and store them in the database;

[0033] A data-driven module, which is used to drive the digital twin model of the ultra-high-speed optical module production line using data in the database;

[0034] The visualization module is used to deeply integrate the ultra-high-speed optical module production line and the constructed digital twin model of the ultra-high-speed optical module production line based on the three-dimensional visualization system, so as to realize the visualization of the digital twin model and multi-source heterogeneous data.

[0035] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0036] (1) The present invention first constructs a digital twin model of the ultra-high-speed optical module production line from four dimensions: geometry, information, behavior, and rules, to ensure a comprehensive and accurate simulation of the real production scenario of the ultra-high-speed optical module production line, while maximizing the use of multi-source heterogeneous data generated in production.

[0037] (2) Acquire real-time multi-source heterogeneous data from various sensors (such as temperature sensors, pressure sensors, vibration sensors, etc.), equipment and systems, and clean, convert and normalize them to improve data accuracy and compatibility, solving the problem of data integration and analysis difficulties caused by diverse data sources and inconsistent formats.

[0038] (3) Real-time data-model mapping: Use data mapping tools on the data processing platform to map the processed multi-source heterogeneous data with the corresponding attributes and locations of the digital twin model in real time, and inject the data into the digital twin model through the API interface to achieve integrated and real-time monitoring of physical factory data. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A flow chart of a multi-source heterogeneous data integration and visualization method for an ultra-high-speed optical module production line provided in an embodiment of the present application;

[0040] Figure 2 An architecture diagram of a multi-source heterogeneous data integration and visualization method for an ultra-high-speed optical module production line provided in an embodiment of the present application;

[0041] Figure 3 A schematic diagram of a visualization result provided in an embodiment of the present application;

[0042] Figure 4 A schematic diagram of a production line model building process provided in an embodiment of the present application;

[0043] Figure 5 A schematic diagram of the architecture of a digital twin model for building an ultra-high-speed optical module production line provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0045] Obviously, the drawings described below are only some examples or embodiments of the present application. For ordinary technicians in this field, the present application can also be applied to other similar scenarios based on these drawings without creative work. In addition, it can also be understood that although the efforts made in this development process may be complicated and lengthy, for ordinary technicians in this field related to the content disclosed in this application, some changes in design, manufacturing or production based on the technical content disclosed in this application are just conventional technical means, and should not be understood as insufficient content disclosed in this application.

[0046] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0047] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation, and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0048] The present application provides a method for integrating and visualizing multi-source heterogeneous data of an ultra-high-speed optical module production line driven by a digital twin. Considering the complexity of the production environment (such as staged operation of equipment, changes in process conditions, etc.), which will make multi-source heterogeneous data integration difficult and real-time data visualization inaccurate, the digital twin model constructed by the method for integrating and visualizing multi-source heterogeneous data of an ultra-high-speed optical module production line driven by a digital twin improves the real-time performance and accuracy of the model by integrating multi-source heterogeneous data. At the same time, the digital twin model and related data are visualized, and a human-computer interaction interface is provided for control.

[0049] Digital twin is a technology that realizes the synchronous operation of physical equipment and virtual models by building a digital virtual model of physical entities, thereby achieving accurate mapping and dynamic monitoring of the physical world. The combination of digital twin and ultra-high-speed optical module production line has important necessity and broad application prospects. First, digital twin technology collects multi-source heterogeneous data from all links of the production line in real time through sensors and IoT devices, and maps them synchronously in the virtual model. This can comprehensively and accurately reflect the operating status of the production line, and timely discover and solve problems in the production process. Secondly, based on the digital twin model, dynamic simulation and predictive analysis of the production process can be carried out. By simulating the impact of changes in different production parameters on the production line, the production process can be optimized and production efficiency can be improved. At the same time, big data and machine learning technologies are used to analyze and mine historical data, predict possible failures and bottlenecks, take preventive measures in advance, and reduce downtime and production losses. In addition, digital twin technology realizes refined management and control of production lines. Managers can monitor the operating status of the production line in real time through a visual interface, make scientific decisions and precise management. Finally, digital twin technology realizes efficient integration and sharing of multi-source heterogeneous data through standardized data interfaces and integration platforms. All types of data can be stored, managed and analyzed on a unified platform to form a unified data view, support cross-departmental and cross-system collaborative work, and improve the overall production management level.

[0050] The multi-source heterogeneous data integration and visualization method of the ultra-high-speed optical module production line driven by the digital twin of the embodiment of the present application is as follows: Figure 1 and Figure 2 As shown, the details are as follows:

[0051] Step S1: For the production factors of the ultra-high-speed optical module, a multi-level twin model of the ultra-high-speed optical module production line is constructed to obtain a digital twin model of the ultra-high-speed optical module production line.

[0052] Step S2: Utilize multiple data sources such as sensors and visual systems to collect multi-source heterogeneous data in the actual ultra-high-speed optical module production process, including equipment status, production parameters, environmental conditions, etc.

[0053] Step S3: Preprocess and standardize the collected multi-source heterogeneous data and store them in the database.

[0054] Step S4: Use the data in the database constructed in step S3 to drive the digital twin model of the ultra-high-speed optical module production line constructed in step S1.

[0055] Step S5: Based on the three-dimensional visualization system, the ultra-high-speed optical module production line and the ultra-high-speed optical module production line digital twin model constructed in step S1 are deeply integrated to realize the visualization of the digital twin model and multi-source heterogeneous data.

[0056] In step S1, a digital twin model of the ultra-high-speed optical module production line is constructed from four dimensions: geometry, information, behavior, and rules. First, in the geometric dimension, CAD software is used to accurately construct 3D models of all key equipment and components in the production line to ensure that the shape, size, and relative position of each device are consistent with the actual situation, providing a basis for subsequent simulation and visualization. In the information dimension, multi-source heterogeneous data generated during the production process of ultra-high-speed optical modules are collected in real time through a variety of sensors and devices, and the data is integrated. Next, in the behavioral dimension, the operating procedures and interaction logic of each device in the production line are modeled through programming or simulation tools, such as the startup sequence of the equipment, data collection and processing procedures, fault response strategies, etc., so that the model can simulate the dynamic behavior of each link in the actual production process. Finally, in the rule dimension, the business logic and control rules of the production line operation are defined, including process specifications, quality control standards, etc., and these rules are integrated into the digital twin so that the model can make correct decisions and adjustments based on preset rules.

[0057] In step S2, a variety of sensors and vision systems are deployed on the ultra-high-speed optical module production line to monitor equipment status, production parameters, and environmental conditions. These sensors are integrated with the MAC system to achieve real-time data collection and transmission. At the same time, the vision system captures images and videos of the production process through the camera to detect production quality and identify potential defects. In the data collection stage, all data are timestamped and the clocks of each data source are synchronized. Next, the collected multi-source heterogeneous data is preprocessed, including data cleaning, removing outliers, filling missing data, and standardizing data formats and units. In addition, the consistency and comparability of the data are ensured by feature scaling and unified data structure. The processed data will be compressed and encoded, and then stored in the database in real time through the data interface.

[0058] In step S3, the local database is connected, a data parser is created, the raw data obtained from the database is formatted into a structure that can be used by the digital twin model of the ultra-high-speed optical module production line, and the equipment operation status is mapped to each component of the virtual model. Then, these formatted data are input into the digital twin model, and the equipment status and production parameters in the model are adjusted in real time using the data binding and update mechanism to ensure that the virtual equipment of the model and the actual production line are synchronized.

[0059] In step S4, the 3D modeling software unity3D imports the detailed 3D model of the ultra-high-speed optical module production line and connects it with the digital twin model so that the virtual model accurately reflects the layout, equipment and functions of the production line, such as Figure 3 and Figure 4As shown. In Unity3D, we create a virtual environment of an ultra-high-speed optical module production line, and use 3D modeling tools and physics engines to realistically build a virtual model of the appearance of the ultra-high-speed optical module production line. Next, the data in the digital twin model (such as sensor readings, production parameters, etc.) are integrated into the Unity project, and the data is mapped to virtual devices and components using Unity's real-time data binding function, and the operating status, temperature changes and other information of the equipment are displayed through Unity's UI elements. Set up a data refresh mechanism so that the model can update the status of the production line in real time, and dynamically load and display multi-source heterogeneous data through scripts. Implement interactive data analysis functions so that users can intuitively monitor and analyze various data in the production process in a virtual environment. Through Unity's rendering and animation functions, create visual effects and data visualization tools such as heat maps, trend charts, and equipment status indicators to enhance users' understanding of production line operations and performance.

[0060] In some of the embodiments, a method for constructing a digital twin model of an ultra-high-speed optical module production line is provided. Figure 5 As shown, including:

[0061] Geometric dimension: First, the geometric data of production equipment (such as placement machines, wire bonding machines, loading and unloading machines, etc.) on the optical module production line are obtained through laser scanning, 3D photogrammetry and CAD files to ensure that the geometric model can accurately reflect the actual physical structure of the production line. Next, the geometric model of the production line is created using 3D modeling software (SolidWorks), focusing on building the spatial layout of the equipment and its relative position relationship. For example, the distance between placement machines and the distance between placement machines and upper and lower old machines. In order to ensure the accuracy of the spatial layout, the data in the CAD file is used for reference, and the difference between the model and the actual scene is checked in real time during the modeling process. At the same time, the number of polygons of the device model is simplified. Since the digital twin model needs to run in a real-time system, an overly complex geometric model will lead to a decrease in system performance, so unnecessary details in the geometric model are reduced. For example, details such as fillets, screw holes, and surface textures of the device housing can be appropriately simplified or ignored, and only the main shape and functional features of the device are retained. After the modeling is completed, the SolidWorks model is exported to the OBJ file format and then imported into the digital twin platform to build a geometric model of the ultra-high-speed optical module production line. Finally, the geometric model is deployed to actual ultra-high-speed optical module production and calibrated using actual device position and dimension data to ensure precise alignment of the virtual model with the physical production line.

[0062] Information dimension: First, in order to obtain real-time production environment data and equipment operation status, we deployed various industrial sensors at various key nodes covering the production line. After deploying sensors to start data collection, all key data sources in the ultra-high-speed optical module production line are identified. These data sources include not only sensor data that directly reflects the production environment and equipment status, but also other data closely related to the production process, such as logistics data and production data. Secondly, after identifying all key data sources, these data are classified, mainly into structured data and semi-structured data. Structured data refers to data with a fixed format, such as equipment operating parameters, production schedules, etc. This type of data is stored in the database for easy query and analysis. Semi-structured data refers to data without a strict structure, such as log files, equipment alarm records, etc. Next, data preprocessing is performed to remove noise and outliers in the data through mean filtering. Noise refers to invalid information or random errors in the data, which may be caused by sensor failure, network transmission problems, or external environmental interference; outliers are values ​​that are significantly different from most data points, usually reflecting emergencies or equipment failures. After completing data preprocessing, we convert the format and unify the units of the multi-source data stored in the database. Since the data comes from different sensors and systems, the units and formats of each data are different. To ensure the consistency and comparability of the data, we convert all the data into a unified format and unit. Combined with the calibration information of the sensor and the settings of the data acquisition system, the data is carefully converted and verified. At the same time, the data from different devices and systems are matched and integrated according to key fields such as timestamps and device IDs to form a unified and complete data set. Using the ant colony algorithm, the integrated data is analyzed to build a production optimization model to identify bottlenecks in the production process, predict equipment failures, and optimize production plans and resource scheduling. For example, by analyzing production plans and logistics data, resources are allocated and scheduled reasonably. Finally, the information model is deployed to the actual ultra-high-speed optical module production. By obtaining the operation data of the production line in real time, it continuously updates and optimizes its own behavior to ensure that the production line is always in the best operating state.

[0063] Behavior dimension: First, the production process of the ultra-high-speed optical module production line is analyzed in detail, each key step in the production line is identified, and the order and dependency between the process steps are clarified. Each step has its specific input, output and process requirements. Determine the time required for each step, the equipment involved (such as wire bonding machine, aging machine, loading and unloading machine, etc.), and the operating conditions and production capacity of each equipment. After completing the production process analysis, since the ultra-high-speed optical module production line involves multiple equipment and complex process steps, we choose to build a behavioral model based on agent modeling. Agent-based modeling can simulate the independent behavior of each device and process and their interaction, adapt to the complex dynamic changes in the production process, and convert the dynamic behavior of the production process into a computable model through modeling. Next, define the simulation entity and the corresponding behavior logic. The simulation entity represents the physical and logical objects in the production line, such as equipment, workpieces, etc. For each device, establish its behavior model, including the states of startup, operation, shutdown, and the transition conditions between these states. For example, for a wire bonding machine, its behavior logic can include startup (when a new workpiece arrives), operation (wire bonding operation), and shutdown (when there is no workpiece or after completing all tasks). In addition to equipment behavior, behavioral models need to be established for each process step. These models include the start conditions, operation time, resource requirements, and output status of each process. For example, in the chip packaging step, the behavioral model can be defined as: when the previous process is completed and the workpiece is transferred to the packaging equipment, the packaging equipment starts the operation; the operation time depends on the complexity of the packaging and the working status of the equipment; after the operation is completed, the equipment outputs the packaged workpiece and is ready to accept the next workpiece. Secondly, in order to further improve the efficiency and resource utilization of the production line, a resource scheduling model is established. The purpose of the resource scheduling model is to reasonably allocate and schedule resources such as equipment and workpieces in the simulation. The scheduling model considers multiple factors, such as resource utilization, waiting time, equipment failure rate, workpiece priority, etc. For example, when a device failure is detected, the scheduling model can trigger backup equipment or reschedule the production sequence to minimize production interruption time. Similarly, during production peaks or special order requirements, the scheduling model can increase the number of operators or adjust the production speed to meet production requirements. Finally, the behavioral model is deployed in the actual production of ultra-high-speed optical modules. Through real-time monitoring and feedback, it is ensured that the simulation model is consistent with the dynamic behavior of the actual production process and can accurately reflect the operation of the production line.

[0064] Rule dimension: First, formulate standard operating procedures for ultra-high-speed optical module production lines. Standard operating procedures provide clear steps and guidelines for each link of the production line to ensure that all operators and automated equipment have unified action guidelines when performing production activities. For ultra-high-speed optical module production lines, standard operating procedures cover various processes, such as chip packaging, fiber fusion, laser etching and aging testing. The standard operating procedure for each process includes preparation before operation, specific operation steps, setting parameters for using equipment, precautions during operation, and cleaning and maintenance steps after the operation is completed. Next, formulate automation rules. Automation rules include automatic start and stop conditions for equipment, automatic adjustment strategies for production rhythm, and automatic response mechanisms under abnormal conditions. For example, set the wire bonding machine to automatically perform self-test and calibration after completing a certain number of operations, and set the aging machine to perform aging operations only after receiving a certain number of workpieces. Automation rules are implemented through programming and are closely integrated with the control system and data acquisition system of the production line to form a closed-loop automation control system. By implementing these automation rules and strategies, manual intervention can be effectively reduced, operational risks can be reduced, and the automation level and flexibility of production can be improved. Secondly, formulate fault diagnosis and emergency response mechanisms. It is inevitable to encounter various equipment failures and abnormal situations during the production process, so it is particularly important to establish a fault diagnosis and emergency handling mechanism. The core of the fault diagnosis mechanism is to identify potential fault hazards by real-time analysis of the equipment's operating status and sensor data. For example, by monitoring the vibration signal and current changes of the wire bonding machine, possible mechanical wear or electrical failures can be discovered in advance. After identifying the fault, the emergency handling mechanism should be started immediately, including shutdown protection, troubleshooting, etc. At the same time, when executing the scheduling strategy, the faulty machine will be excluded and the scheduling strategy will be executed safely. Finally, the rule model is deployed to the actual ultra-high-speed optical module production, and the effectiveness of the rules is verified and adjusted through continuous monitoring and feedback. By analyzing the actual data in the production process, the deficiencies in the rule system are found, and corresponding optimization and improvement are carried out.

[0065] It should be noted that the steps shown in the above process or the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from that here. For example, step S1 can be before step S2 or after step S2.

[0066] The embodiments of the present application also provide a multi-source heterogeneous data integration and visualization system for an ultra-high-speed optical module production line, which are used to implement the above embodiments and preferred implementations, and those that have been described will not be repeated. As used below, the terms "module", "unit", "subunit", etc. can implement a combination of software and / or hardware for a predetermined function. Although the systems described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0067] The ultra-high-speed optical module production line multi-source heterogeneous data integration and visualization system of the visualization method provided in the embodiment of the present application includes:

[0068] A model building module is used to build a multi-level twin model of the ultra-high-speed optical module production line for the production factors of the ultra-high-speed optical module, and obtain a digital twin model of the ultra-high-speed optical module production line;

[0069] The data acquisition module is used to collect multi-source heterogeneous data in the production process of ultra-high-speed optical modules using multiple data sources;

[0070] The data processing module is used to pre-process and standardize the collected multi-source heterogeneous data and store them in the database;

[0071] A data-driven module, which is used to drive the digital twin model of the ultra-high-speed optical module production line using data in the database;

[0072] The visualization module is used to deeply integrate the ultra-high-speed optical module production line and the constructed digital twin model of the ultra-high-speed optical module production line based on the three-dimensional visualization system, so as to realize the visualization of the digital twin model and multi-source heterogeneous data.

[0073] In summary, the present invention first constructs a digital twin model of an ultra-high-speed optical module production line from four dimensions: geometry, information, behavior, and rules, to ensure a comprehensive and accurate simulation of the real production scenario of an ultra-high-speed optical module production line, while maximizing the use of multi-source heterogeneous data generated in production. The present invention obtains real-time multi-source heterogeneous data from various sensors (such as temperature sensors, pressure sensors, vibration sensors, etc.), equipment, and systems, and cleans, converts, and normalizes them to improve data accuracy and compatibility, solving the problem that its data sources are diverse and the format is not uniform, resulting in data integration and analysis difficulties. Finally, the present invention adopts data-model real-time mapping. Using a data mapping tool on a data processing platform, the processed multi-source heterogeneous data is mapped in real time to the corresponding attributes and positions of the digital twin model, and the data is injected into the digital twin model through an API interface to achieve integration and real-time monitoring of physical factory data.

[0074] It should be pointed out that the technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above-mentioned embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification. In addition, according to the needs of implementation, the various steps / components described in this application can be split into more steps / components, and two or more steps / components or partial operations of steps / components can be combined into new steps / components to achieve the purpose of the present invention.

[0075] It is easy for a person skilled in the art to understand that the above-mentioned embodiments only express several implementation methods of the present application, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention patent. It should be pointed out that for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for integrating and visualizing multi-source heterogeneous data in an ultra-high-speed optical module production line, characterized in that: The method includes: Step S1: constructing a multi-level twin model of the ultra-high-speed optical module production line for the production factors of the ultra-high-speed optical module to obtain a digital twin model of the ultra-high-speed optical module production line; Step S2: using multiple data sources to collect multi-source heterogeneous data in the actual ultra-high-speed optical module production process; Step S3: preprocessing and standardizing the collected multi-source heterogeneous data and storing them in a database; Step S4: using the data in the database to drive the constructed digital twin model of the ultra-high-speed optical module production line; Step S5: Based on the three-dimensional visualization system, the ultra-high-speed optical module production line and the constructed digital twin model of the ultra-high-speed optical module production line are deeply integrated to realize the visualization of the digital twin model and multi-source heterogeneous data.

2. The method for integrating and visualizing multi-source heterogeneous data of an ultra-high-speed optical module production line according to claim 1 is characterized in that: Step S1 includes: A digital twin model of the ultra-high-speed optical module production line is constructed from four dimensions: geometry, information, behavior, and rules. In the geometry dimension, CAD software is used to accurately construct 3D models of all key equipment and components in the production line to ensure that the shape, size, and relative position of each device are consistent with the actual situation. In the information dimension, multi-source heterogeneous data generated during the production of ultra-high-speed optical modules are collected in real time through a variety of sensors and devices, and the data is integrated. In the behavior dimension, the operation process and interaction logic of each device in the production line are modeled through programming or simulation tools, so that the model simulates the dynamic behavior of each link in the actual production process. In the rule dimension, the business logic and control rules of the production line operation are defined and integrated into the digital twin, so that the model can make correct decisions and adjustments based on preset rules.

3. The method for integrating and visualizing multi-source heterogeneous data of an ultra-high-speed optical module production line according to claim 1 is characterized in that: Step S2 and step S3 include: Deploy a variety of sensors and vision systems on the ultra-high-speed optical module production line to monitor equipment status, production parameters, and environmental conditions. Integrate all sensors with the MAC system to achieve real-time data collection and transmission. At the same time, the vision system uses cameras to capture images and videos of the production process to detect production quality and identify potential defects. During the data collection stage, all data are timestamped and the clocks of each data source are synchronized. Next, the collected multi-source heterogeneous data are preprocessed, including data cleaning, removal of outliers, filling in missing data, and standardization of data formats and units. In addition, feature scaling and unified data structure are used to ensure data consistency and comparability. The processed data will be compressed and encoded, and then stored in the database in real time through the data interface.

4. The method for multi-source heterogeneous data integration and visualization of an ultra-high-speed optical module production line according to claim 1 is characterized in that: Step S4 includes: Connect to the database, create a data parser, format the raw data obtained from the database into a structure that can be used by the digital twin model of the ultra-high-speed optical module production line, and map the equipment operation status to the various components of the virtual model; then, input the formatted data into the digital twin model of the ultra-high-speed optical module production line, and use the data binding and update mechanism to adjust the equipment status and production parameters in the model in real time to ensure that the virtual equipment of the model and the actual production line are synchronized.

5. The method for multi-source heterogeneous data integration and visualization of an ultra-high-speed optical module production line according to claim 1 is characterized in that: Step S5 includes: The 3D modeling software Unity3D imports a detailed 3D model of the ultra-high-speed optical module production line and connects it with the digital twin model of the ultra-high-speed optical module production line, so that the virtual model accurately reflects the layout, equipment and functions of the production line; creates a virtual environment of the ultra-high-speed optical module production line in Unity3D, and uses 3D modeling tools and physics engines to realistically build a virtual model of the appearance of the ultra-high-speed optical module production line; then, integrates the data in the digital twin model of the ultra-high-speed optical module production line into the Unity project, uses Unity's real-time data binding function to map the data to virtual devices and components, and displays various information of the equipment through Unity's UI elements; sets a data refresh mechanism so that the model updates the production line status in real time, and realizes dynamic loading and display of multi-source heterogeneous data through scripts; implements interactive data analysis functions so that users can intuitively monitor and analyze various data in the production process in a virtual environment; creates visual effects and data visualization tools, including heat maps, trend charts, and equipment status indicators, through Unity's rendering and animation functions to enhance users' understanding of production line operations and performance.

6. The method for integrating and visualizing multi-source heterogeneous data of an ultra-high-speed optical module production line according to claim 2 is characterized in that: The geometric dimensions include: First, the geometric data of the production equipment on the high-speed optical module production line is obtained through laser scanning, 3D photogrammetry and CAD files to ensure that the geometric model accurately reflects the actual physical structure of the production line. The production equipment includes placement machines, wire bonding machines, and loading and unloading machines. Then, the 3D modeling software SolidWorks is used to create a geometric model of the production line, focusing on building the spatial layout of the equipment and its relative position relationship, including the distance between placement machines and the distance between placement machines and upper and lower old machines. The data in the CAD file is used for reference to ensure the accuracy of the spatial layout, and the difference between the model and the actual scene is checked in real time during the modeling process. At the same time, the number of polygons of the equipment model is simplified, and only the main shape and functional characteristics of the equipment are retained. After the modeling is completed, the SolidWorks model is exported to the OBJ file format and then imported into the digital twin platform to build the geometric model of the ultra-high-speed optical module production line. Finally, the geometric model is deployed to the actual ultra-high-speed optical module production, and the actual equipment position and size data are used for calibration to ensure the precise alignment of the virtual model with the physical production line.

7. The method for integrating and visualizing multi-source heterogeneous data of an ultra-high-speed optical module production line according to claim 2 is characterized in that: The information dimensions include: First, various industrial sensors are deployed at various key nodes covering the ultra-high-speed optical module production line. After the sensors are deployed to start data collection, all key data sources in the ultra-high-speed optical module production line are identified. The data sources include not only sensor data that directly reflects the production environment and equipment status, but also other data closely related to the production process, including logistics data and production data. Secondly, after all key data sources are identified, the data is classified into structured data and semi-structured data. Structured data refers to data with a fixed format, including equipment operating parameters and production schedules. This type of data is stored in a database for easy query and analysis. Semi-structured data refers to data without a strict structure, including log files and equipment alarm records. Next, data preprocessing is performed to remove noise and outliers in the data through mean filtering. Noise refers to invalid information or random errors in the data. Outliers are values ​​that are significantly different from most data points. After data preprocessing, the multi-source data stored in the database are converted into different formats and unified in units. The data are carefully converted and verified in combination with the sensor calibration information and the settings of the data acquisition system. At the same time, the data from different devices and systems are matched and integrated according to key fields to form a unified and complete data set. The key fields include timestamp and device ID. The ant colony algorithm is used to analyze the integrated data and build a production optimization model to identify bottlenecks in the production process, predict equipment failures, optimize production plans and resource scheduling, including reasonable resource allocation and scheduling through analysis of production plans and logistics data. Finally, the information model is deployed in the actual production of ultra-high-speed optical modules. By acquiring the operation data of the production line in real time, the behavior is continuously updated and optimized to ensure that the production line is always in the best operating state.

8. The method for integrating and visualizing multi-source heterogeneous data of an ultra-high-speed optical module production line according to claim 2 is characterized in that: The behavioral dimensions include: First, the production process of the ultra-high-speed optical module production line is analyzed in detail, each key step in the production line is identified, and the sequence and dependency between the process steps are clarified. Each step has its specific input, output and process requirements, and the time required for each step, the equipment involved, and the operating conditions and production capacity of each equipment are determined; after completing the production process analysis, since the ultra-high-speed optical module production line involves multiple equipment and complex process steps, the agent-based modeling method is selected to build a behavioral model. The agent-based modeling simulates the independent behavior of each device and process and their interactions, adapts to the complex dynamic changes in the production process, and converts the dynamic behavior of the production process into a computable model through modeling; then, the simulation entity and the corresponding behavioral logic are defined. The simulation entity represents the physical and logical objects in the production line, including equipment and workpieces. For each device, its behavioral model is established, including various states and transition conditions between states; in addition to equipment behavior, a behavioral model is also established for each process step, including the start conditions, operation time, resource requirements and output status of each process; A resource scheduling model is established. The purpose of the resource scheduling model is to reasonably allocate and schedule equipment and workpiece resources in the simulation. The scheduling model considers multiple factors, including resource utilization, waiting time, equipment failure rate, and workpiece priority. When a device failure is detected, the scheduling model triggers the backup equipment or rearranges the production sequence to minimize production interruption time. Similarly, during production peaks or special order requirements, the scheduling model increases the number of operators or adjusts the production speed to meet production requirements. Finally, the behavioral model is deployed to the actual ultra-high-speed optical module production. Through real-time monitoring and feedback, it is ensured that the simulation model is consistent with the dynamic behavior of the actual production process and accurately reflects the operation of the production line.

9. The method for integrating and visualizing multi-source heterogeneous data of an ultra-high-speed optical module production line according to claim 2, characterized in that: The rule dimensions include: First, a standard operating procedure is formulated for the ultra-high-speed optical module production line. The standard operating procedure provides clear steps and guidelines for each link of the production line to ensure that all operators and automated equipment have a unified code of conduct when performing production activities. For the ultra-high-speed optical module production line, the standard operating procedure covers various processes, including chip packaging, fiber fusion splicing, laser etching and aging testing. The standard operating procedure for each process includes preparation before operation, specific operating steps, setting parameters for using equipment, precautions during operation, and cleaning and maintenance steps after the operation is completed. Next, automation rules are formulated. The automation rules include automatic start and stop conditions for equipment, automatic adjustment strategies for production rhythm, and automatic response mechanisms under abnormal conditions. Automation rules are implemented through programming and It is closely integrated with the control system and data acquisition system of the production line to form a closed-loop automated control system. By implementing automated rules and strategies, the automation level and flexibility of production are improved. Secondly, a fault diagnosis and emergency response mechanism is formulated. The fault diagnosis mechanism identifies potential fault hazards by real-time analysis of the operating status and sensor data of the equipment, and initiates the corresponding emergency response mechanism. At the same time, when executing the scheduling strategy, the faulty machine is excluded and the scheduling strategy is executed safely. Finally, the rule model is deployed to the actual production of ultra-high-speed optical modules. The effectiveness of the rules is verified and adjusted through continuous monitoring and feedback. By analyzing the actual data in the production process, the deficiencies in the rule system are discovered, and corresponding optimization and improvement are carried out.

10. An ultra-high-speed optical module production line multi-source heterogeneous data integration and visualization system for implementing the ultra-high-speed optical module production line multi-source heterogeneous data integration and visualization method according to any one of claims 1 to 9, characterized in that: The system includes: A model building module is used to build a multi-level twin model of the ultra-high-speed optical module production line for the production factors of the ultra-high-speed optical module, and obtain a digital twin model of the ultra-high-speed optical module production line; The data acquisition module is used to collect multi-source heterogeneous data in the production process of ultra-high-speed optical modules using multiple data sources; The data processing module is used to pre-process and standardize the collected multi-source heterogeneous data and store them in the database; A data-driven module, which is used to drive the digital twin model of the ultra-high-speed optical module production line using data in the database; The visualization module is used to deeply integrate the ultra-high-speed optical module production line and the constructed digital twin model of the ultra-high-speed optical module production line based on the three-dimensional visualization system, so as to realize the visualization of the digital twin model and multi-source heterogeneous data.

Images