An Industrial Internet of Things Sub-service Platform Data Aggregation System and Method

Abstract

This solution belongs to the technical field of industrial Internet of Things, and specifically relates to a data aggregation system and method for an industrial Internet of Things sub-service platform. The method includes the following steps: Step 1: Multiple collection devices in the object platform collect industrial data of multiple industrial devices with multiple communication methods and multiple protocol specifications and transmit it to the sensor network sub-platform; Step 2: The sensor network sub-platform performs protocol conversion on the industrial data, and then after packet processing, transmits it to the sensor network main platform; Step 3: The sensor network main platform uploads the industrial data uploaded in packets to the data aggregation interface through the UDP network transmission protocol of the data collection interface. This solution can meet the high-concurrency access of large-scale sensing devices in different application fields, with multiple communication channels and multiple communication protocols, and realize the real-time parallel transmission and processing of massive collected data.

Description

Technical Field

[0001] This solution belongs to the technical field of industrial Internet of Things, and specifically relates to a data aggregation system and method for an industrial Internet of Things sub-service platform. Background Art

[0002] Compared with traditional Internet big data, industrial big data has stronger characteristics such as professionalism, relevance, processability, timeliness, and analyzability. The collection of industrial big data mainly uses industrial Internet technology for remote data collection and visual collection, and through collection software, it conducts wide-area, large-scale, and real-time collection and transmission in the industrial Internet. After multiple aggregations and forwards, it is finally sent to the big data platform as the data source for analysis and application.

[0003] Industrial big data mainly includes three types of data. One is all production data collected from industrial control systems, the second is the monitoring data directly collected from intelligent sensor PLC devices, and the third is the data of management systems inside and outside the factory, such as ERP systems, customer relationship management systems, and sales systems. Among the data objects currently collected for industrial big data, the first and second types of data, that is, the data of various control systems and a large number of intelligent devices, have the largest data volume, the fastest growth rate, relatively high real-time requirements, and greater difficulty. The third type of data is mainly collected through the interfaces between management software. Industrial control systems generally provide on-site data in the industrial Ethernet through protocols or specifications. Especially in the power industry, modbus protocol, CDT specification, 101 specification, 104 specification, DNP protocol, or other proprietary protocols are often used. Most of these protocols belong to request / response technical protocols and can only be used to obtain instantaneous data and are commonly used in industrial Ethernet. Another common way to provide data in industrial control systems is to provide a protocol that supports both real-time data generated immediately and historical real-time data. The most representative and extensive of these is the OPC protocol. Intelligent sensor PLCs generally provide data through proprietary protocols. Currently, most intelligent sensors support OPC protocol for network data communication or serial RS232 / 485 protocol communication.

[0004] However, in the actual use process, industrial intelligent devices involve multiple communication methods and multiple protocol specifications. Originally, data was fused and then stored and published through multiple methods and devices, resulting in high costs, serious resource consumption, and high maintenance costs. Therefore, how to fuse and aggregate data of multiple communication methods and protocol specifications through a management system is particularly important. Summary of the Invention

[0005] This solution provides a data aggregation system and method for an industrial Internet of Things sub-service platform that can aggregate data of multiple communication methods and protocol specifications.

[0006] To achieve the above object, this solution provides a data aggregation method for an industrial Internet of Things sub-service platform, including the following steps:

[0007] Step 1: Multiple collection devices in the object platform collect industrial data of multiple industrial devices with multiple communication methods and multiple protocol specifications and transmit it to the sensor network sub-platform;

[0008] Step 2: The sensor network sub-platform performs protocol conversion on the industrial data, and then after packet processing, it is transmitted to the sensor network main platform;

[0009] Step 3: The sensor network main platform uploads the industrial data uploaded in packets to the data aggregation interface through the UDP network transmission protocol of the data collection interface; the data aggregation interface uploads the industrial data uploaded in packets as the first reported data to the main management network database through the TCP network transmission protocol;

[0010] Step 4: After receiving the industrial data transmitted by the sensor network main platform, the main management network database converts the first reported data according to the product template pre-stored in the main management network database that matches the product information of the Internet of Things terminal device to obtain the second reported data that meets the target format of the data aggregation platform, and then transmits the second reported data to the management sub-platform database for processing to form the third reported data and then transmits it to the main management network database and the sub-service database;

[0011] Step 5: The main service database is used to receive the third reported data sent by all sub-service databases. The sub-service database generates a product template for the third reported data according to the reported template parameters defined by the user on the product template definition interface, and saves the product template to the data aggregation to the main service database to generate the aggregated data information;

[0012] Step 6: The user views various device data aggregated on the service platform through the user platform.

[0013] Beneficial effects of this solution: Combining the actual situation of the current industrial Internet of Things (diverse data sources), using multiple collection devices, it can be compatible with and collect industrial device data with different communication methods and protocol specifications, ensuring the comprehensiveness and diversity of data collection.

[0014] In the sensor network sub-platform, protocol conversion is performed on the collected industrial data to make it conform to the standard format of network transmission, and through packet processing, integrity and security are provided for data transmission. Through the combined use of UDP and TCP protocols, efficient and reliable transmission of data between the sensor network main platform and the data aggregation interface is achieved.

[0015] In the main management network database, the reported data is formatted according to the pre-stored product templates to match the target format requirements of the data aggregation platform. The management sub-platform database further processes the converted data to form the third reported data, and collaborates with the sub-service database to generate product templates to meet the reported template parameters defined by the user. The main service database aggregates all the data sent by the sub-service databases to form the final aggregated data information for the user to view and analyze through the user platform.

[0016] By supporting multiple communication methods and protocol specifications, the compatibility of the system with data from different industrial devices and the data acquisition ability are improved. Protocol conversion and packet processing optimize the data transmission process, reduce data loss and errors during transmission, and improve the efficiency of data processing. The combination of UDP and TCP protocols ensures high reliability and low latency of data transmission. Through data format conversion and generation of product templates, data consistency and adaptability to user requirements are ensured, and data availability is improved. The intuitive display on the user platform enables users to conveniently view and analyze the aggregated device data, optimizing the user experience. The aggregated data can be used for big data analysis and machine learning to support intelligent decision-making and automated control in the industrial Internet of Things.

[0017] By supporting multiple communication methods and protocol specifications, the present invention can achieve efficient integration of data from different devices and systems in an industrial environment, ensuring the comprehensiveness and consistency of data. Through real-time monitoring and dynamic adjustment of data aggregation strategies, network bandwidth is effectively utilized, network congestion and latency caused by data transmission are reduced, and the utilization efficiency of network resources is improved.

[0018] While ensuring data transmission efficiency, the present invention reduces unnecessary data transmission through intelligent data aggregation strategies, thereby reducing the cost of data transmission. By using different network transmission protocols (UDP and TCP) between the sensor network sub-platform and the sensor network main platform, reliable data transmission is ensured, and the integrity and accuracy of data can be guaranteed even under unstable network conditions. The main management network database converts data according to the pre-stored product templates, enabling the data to meet the requirements of different application scenarios and providing flexible data processing capabilities. By applying the reported template parameters defined by the user in the sub-service database, product templates are intelligently generated, realizing personalized aggregation and analysis of data.

[0019] Furthermore, it further includes step A: before step one, configure the multiple collection devices to determine the corresponding communication methods and protocol specifications for each collection device, as well as the data collection frequency and priority.

[0020] Further, it also includes step B: In step two, the sub-platform of the sensing network also preprocesses the collected data, including data cleaning, data deduplication, and data compression, to optimize the efficiency of data transmission.

[0021] Further, it also includes step C: In step three, the main platform of the sensing network also monitors the industrial data in real time to evaluate the bandwidth usage and latency of data transmission; it also includes step D: After step three, according to the results of real-time monitoring, dynamically adjust the data aggregation strategy to optimize bandwidth usage and reduce latency.

[0022] Further, step D includes:

[0023] Step D1: When it is detected that the bandwidth utilization rate exceeds the preset threshold, reduce the collection frequency of non-critical data or suspend the collection of some non-critical data;

[0024] Step D2: When it is detected that the latency exceeds the preset threshold, give priority to transmitting critical data and temporarily reduce the data collection accuracy to reduce the packet size;

[0025] Step D3: When it is detected that both the bandwidth utilization rate and the latency are within the acceptable range, keep the current data aggregation strategy unchanged;

[0026] Step D4: Based on steps D1 and D2, use machine learning algorithms to analyze the historical data transmission patterns to predict future bandwidth requirements and latency changes, so as to adjust the data aggregation strategy in advance;

[0027] Step D5: Based on step D3, according to the classification of critical data and non-critical data defined by the user, and the specific requirements of the user for data real-time performance, dynamically adjust the data aggregation strategy to meet the needs of different users;

[0028] Step D6: Based on steps D4 and D5, provide a user interface that allows the user to manually set the thresholds for bandwidth usage and latency, as well as the collection frequency and accuracy of data according to the actual application scenario.

[0029] Further, in step one and step two, data integrity detection is performed according to preset rules. When data loss is detected, the following steps are executed:

[0030] Step E1: Use preset data quality rules to monitor the collected data in real time to identify missing data events;

[0031] Step E2: Determine the recovery priority of the missing data according to the type and severity of the missing data;

[0032] Step E3: For different types of missing data, apply corresponding data recovery algorithms, including but not limited to interpolation algorithms, machine learning prediction models, or rule-based inference systems, to generate estimated values of the missing data;

[0033] Step E4: Perform protocol conversion and packet transmission on the estimated values together with the actually collected industrial data to ensure data integrity and consistency.

[0034] Furthermore, Step E3 includes:

[0035] Step E3a: For temporary missing data, adopt an interpolation algorithm based on time series analysis to predict possible values of the missing data;

[0036] Step E3b: For long-term missing data, adopt a machine learning model to learn the data generation pattern based on historical data and relevant data to predict the estimated values of the missing data;

[0037] Step E3c: For the missing of critical data, adopt a rule-based inference system to infer possible values of the missing data according to predefined business logics and rules.

[0038] Furthermore, in Step Four, when the data received by the main management network database contains missing values, perform the following steps:

[0039] Step F1: Identify the missing parts in the data and mark the positions of the missing data;

[0040] Step F2: Classify the missing data according to the rules in the product template to determine which data recovery strategy to use;

[0041] Step F3: Apply corresponding data recovery strategies, including but not limited to using the mean, median, mode of historical data or inferring through data of similar devices, to fill in the missing data;

[0042] Step F4: Convert the filled complete data into the target format and upload it to the main service database.

[0043] Furthermore, in Step Five and Step Six, when the data requested by the user contains missing values, perform the following steps:

[0044] Step G1: The user specifies the preference settings for data recovery through the user platform, including but not limited to the priority of data recovery, the algorithms or models used, and the accuracy requirements for data recovery;

[0045] Step G2: The sub-service database selects a data recovery strategy according to the user's preference settings;

[0046] Step G3: Generate the final aggregated data information from the restored data and the actually collected industrial data, and provide it to the user platform for viewing.

[0047] An industrial Internet of Things sub-service platform data aggregation system, characterized in that it includes a sensor network platform, a management platform, and a service platform that interact in sequence.

[0048] The sensor network platform is arranged in a front sub-platform manner. The front sub-platform arrangement means that the sensor network platform includes a sensor network main platform and multiple independent sensor network sub-platforms. The multiple sensor network sub-platforms are data collection modules in different communication networks, and the sensor network main platform is the total data collection module for all the communication networks. The sensor network sub-platforms collect industrial data of industrial equipment through sensors, perform protocol conversion on the industrial data, and transmit the packet to the sensor network main platform.

[0049] The sensor network main platform uploads the industrial data uploaded in packets to the data aggregation interface through the UDP network transmission protocol of the data collection interface; the data aggregation interface uploads the industrial data uploaded in packets to the management platform as the first reported data through the TCP network transmission protocol.

[0050] The management platform is arranged in a middle split structure. The management platform includes a main management network database and multiple independent management sub-platforms. Each management sub-platform is provided with a corresponding sub-management database. The main management network database is used to uniformly manage the data of all the communication networks. After receiving the industrial data transmitted by the sensor network platform, the main management network database converts the first reported data according to the product template pre-stored in the main management network database and matching the product information of the Internet of Things terminal device to obtain the second reported data in the target format that meets the data aggregation platform, and then transmits the second reported data to the management sub-platform database for processing to form the third reported data and then transmits it to the main management network database. The main management network database transmits the third reported data to the service platform.

[0051] The service platform includes a sub-service database, a service sub-platform, and a main service database. The sub-service database is used to receive the third reported data sent by the corresponding management platform, and the sub-service database only conducts information interaction with the corresponding management platform; the service sub-platform is used to receive the third reported data sent by the corresponding sub-service database, and the service sub-platform only conducts information interaction with the corresponding sub-service database; all sub-service databases conduct information interaction with the main service database. The main service database is used to receive the third reported data sent by all sub-service databases. The sub-service database generates a product template for the third reported data according to the reported template parameters defined by the user on the product template definition interface, and saves the product template to the data aggregation to the main service database to generate aggregated data information.

[0052] The sensing network platform in this solution adopts a pre-sub-platform structure. The logical relationship is that the sensing information directly reaches the corresponding sub-platform for processing and operation management from bottom to top first, and then the sub-platform (including its own database) further uploads the processed data to the total database, and the total database conducts data interaction with the upper platform. This structure can share the computing pressure of the total database, and thus is applicable to the data aggregation of industrial intelligent devices involving multiple communication methods and multiple protocol protocols, and can ensure that industrial data can be transmitted in real time and securely, thereby effectively alleviating the pressure on data acquisition interfaces and network transmissions.

[0053] The management platform adopts a middle-partitioned structure form, deploying the total database or data center + sub-platform. One total database receives and aggregates all data, and the further operation management of the data is implemented by the sub-platform. Information interaction with the upper and lower functional platforms is all through the total database, which is conducive to the centralized control and aggregation of data, convenient for management, and realizes the integration of multiple communication protocols and protocol protocols.

[0054] The service platform adopts a combined pre-sub-platform structure form. The total database, sub-platform database, and sub-platform are deployed in the functional platform. The sub-platform database shares the computing pressure of the total database, can meet the high-concurrency access of large-scale sensing devices in different application fields, multiple communication channels, and multiple communication protocols, realizes the real-time parallel transmission and processing of massive collected data, and supports the fusion of heterogeneous data and the management of big data, providing high-quality data support in standard format for Internet of Things business applications and guaranteeing the development of Internet of Things-related services. Brief Description of the Drawings

[0055] Figure 1 It is a schematic diagram of the system architecture of Embodiment 1 of this application;

[0056] Figure 2 It is a schematic diagram of the method steps of Embodiment 1 of this application.

[0057] Figure 3 It is the logic block diagram of the system in Embodiment 1 of this application. Specific implementation mode

[0058] The following is a further detailed description through specific implementation modes:

[0059] Embodiment 1: Basically as shown in the appendix Figure 1 and Figure 3 shown:

[0060] A data aggregation system for an industrial Internet of Things sub-service platform, including an object platform, a sensing network platform, a management platform, a service platform, and a user platform that interact in sequence.

[0061] The object platform interacts with the sensing network platform. The object platform includes multiple collection devices, and the multiple collection devices are used to collect corresponding data of industrial devices with multiple communication methods and multiple protocol specifications and transmit the data to the sensing network platform.

[0062] The sensing network platform adopts a front sub-platform layout. The front sub-platform layout means that the sensing network platform includes a sensing network total platform and multiple independent sensing network sub-platforms. The multiple sensing network sub-platforms are data collection modules in different communication networks, and the sensing network total platform is the total data collection module of all the communication networks. The sensing network sub-platforms collect industrial data of industrial devices through sensors, perform protocol conversion on the industrial data, and packet-transmit the industrial data to the sensing network total platform.

[0063] The sensing network total platform uploads the industrial data uploaded in packets to the data aggregation interface through the UDP network transmission protocol of the data collection interface; the data aggregation interface uploads the industrial data uploaded in packets as the first reported data to the management platform through the TCP network transmission protocol.

[0064] The format of the first reported data can be defined by the manufacturer of the Internet of Things terminal device, and the formats of the first reported data of Internet of Things terminal devices of different manufacturers can be different. The format of the second reported data meets the target format of the data aggregation platform. The product template includes the mapping relationship between the first reported data and the second reported data, so that the data aggregation platform can convert the reported data in different data formats into data that meets the target format of the data aggregation platform according to the product template, which is convenient for the subsequent maintenance of the data aggregation platform.

[0065] After receiving the first reported data, the product information of the Internet of Things terminal device can be obtained from the first reported data. For example, product industry / product type, manufacturer, and / or product model, etc.; the product information is matched with at least one product template pre-stored in the data aggregation platform to obtain the product template matched by the Internet of Things terminal device.

[0066] The first reported data includes the attribute value corresponding to the first attribute key, the second reported data includes the attribute value corresponding to the second attribute key, and the product template includes the mapping relationship between the first attribute key and the second attribute key. Among them, the conversion of the first reported data according to the product template pre-stored in the management network database and matching the product information of the industrial device to obtain the second reported data in the target format that meets the service platform includes: using the attribute value corresponding to the first attribute key of the first reported data as the attribute value corresponding to the second attribute key of the second reported data, so as to convert the first reported data into the second reported data.

[0067] Attribute keys defined by different merchants for the same attribute of the reported data are different, which is not convenient for subsequent storage, management, and application. A mapping relationship is established between the first attribute key of the first reported data and the second attribute key of the second reported data. For example, the first attribute key (key) is A, the corresponding attribute value (value) is a, the second attribute key (key) is B, and through data mapping, the attribute value a corresponding to the attribute key A can be used as the attribute value corresponding to the attribute key B to obtain the attribute value a corresponding to the attribute key B in the second reported data.

[0068] The management platform is arranged in a central split structure. The management platform includes a main management network database and multiple independent management sub-platforms. Each management sub-platform is provided with a corresponding sub-management database. The main management network database is used to uniformly manage the data of all the communication networks. After receiving the industrial data transmitted by the sensor network platform, the main management network database converts the first reported data according to the product template pre-stored in the main management network database and matching the product information of the Internet of Things terminal device to obtain the second reported data in the target format that meets the data aggregation platform, and then transmits the second reported data to the management sub-platform database for processing to form the third reported data and then transmits it to the main management network database. The main management network database transmits the third reported data to the service platform.

[0069] A service platform, including a sub-service database, a service sub-platform, and a main service database. The sub-service database is used to receive the third reported data sent by its corresponding management platform, and the sub-service database only interacts with its corresponding management platform for information; the service sub-platform is used to receive the third reported data sent by its corresponding sub-service database, and the service sub-platform only interacts with its corresponding sub-service database for information; all sub-service databases interact with the main service database. The main service database is used to receive the third reported data sent by all sub-service databases. The sub-service database generates a product template for the third reported data according to the reported template parameters defined by the user on the product template definition interface, and saves the product template to the data aggregation to the main service database to generate aggregated data information. There is also a product template in the sub-service database. One of the sensor network sub-platforms is a data monitoring and management sub-platform. The data monitoring module is used to obtain the industrial data collected by the data acquisition module and judge the status of industrial equipment through thresholds; the data monitoring module is also used to monitor the network communication status of the acquisition link between the data acquisition module and the industrial equipment and send an alarm message when an abnormality occurs. The user platform interacts with the service platform to facilitate the user to view various information aggregated on the service platform.

[0070] Also disclosed is a method for aggregating data of an industrial Internet of Things sub-service platform (as Figure 2 described), including the following steps:

[0071] Step 1: Multiple acquisition devices in the object platform collect corresponding data of multiple industrial devices with multiple communication methods and multiple protocol protocols and transmit them to the sensor network sub-platform;

[0072] Step 2: The sensor network sub-platform performs protocol conversion on the industrial data and packet-transmits it to the sensor network main platform;

[0073] Step 3: The sensor network main platform uploads the industrial data uploaded in packets to the data aggregation interface through the UDP network transmission protocol of the data acquisition interface; the data aggregation interface uploads the industrial data uploaded in packets as the first reported data to the main management network database through the TCP network transmission protocol;

[0074] Step 4: After receiving the industrial data transmitted by the sensor network main platform, the main management network database converts the first reported data according to the product template pre-stored in the main management network database and matching the product information of the Internet of Things terminal device to obtain the second reported data in the target format that meets the data aggregation platform, and then transmits the second reported data to the management sub-platform database for processing to form the third reported data and then transmits it to the main management network database and the sub-service database,

[0075] Step 5: The main service database is used to receive the third reported data sent by all sub-service databases. The sub-service databases generate product templates based on the reported template parameters defined by the user on the product template definition interface for the third reported data, and save the product templates to the data aggregation to the main service database to generate aggregated data information;

[0076] Step 6: The user views various device data aggregated on the service platform through the user platform.

[0077] Embodiment 2:

[0078] Compared with Embodiment 1, the difference is only that,

[0079] It further includes Step A: Before Step 1, configure the multiple collection devices to determine the corresponding communication methods and protocol agreements for each collection device, as well as the data collection frequency and priority.

[0080] It further includes Step B: In Step 2, the sensing network sub-platform further includes preprocessing the collected data, including data cleaning, data deduplication, and data compression, to optimize the data transmission efficiency.

[0081] It further includes Step C: In Step 3, the sensing network main platform further includes real-time monitoring of the industrial data to evaluate the bandwidth usage and latency of data transmission; it further includes Step D: After Step 3, according to the results of real-time monitoring, dynamically adjust the data aggregation strategy to optimize bandwidth usage and reduce latency.

[0082] Among them, Step D includes:

[0083] Step D1: When it is monitored that the bandwidth usage rate exceeds the preset threshold, reduce the collection frequency of non-critical data or suspend the collection of some non-critical data;

[0084] Step D2: When it is monitored that the latency exceeds the preset threshold, give priority to transmitting critical data and temporarily reduce the data collection accuracy to reduce the packet size;

[0085] Step D3: When it is monitored that both the bandwidth usage rate and the latency are within the acceptable range, keep the current data aggregation strategy unchanged;

[0086] Step D4: On the basis of Step D1 and Step D2, use machine learning algorithms to analyze the historical data transmission patterns to predict future bandwidth requirements and latency changes, so as to adjust the data aggregation strategy in advance;

[0087] Step D5: On the basis of Step D3, dynamically adjust the data aggregation strategy according to the classification of critical data and non-critical data defined by the user, as well as the specific requirements of the user for data real-time performance, to meet the needs of different users;

[0088] Step D6: Based on Steps D4 and D5, provide a user interface that allows the user to manually set the thresholds for bandwidth usage and latency, as well as the frequency and accuracy of data collection, according to the actual application scenario.

[0089] In Steps 1 and 2, data integrity detection is performed according to preset rules. When data loss is detected, the following steps are executed:

[0090] Step E1: Use preset data quality rules to monitor the collected data in real time to identify missing data events;

[0091] Step E2: Determine the recovery priority of the missing data according to the type and severity of the missing data;

[0092] Step E3: For different types of missing data, apply corresponding data recovery algorithms, including but not limited to interpolation algorithms, machine learning prediction models, or rule-based inference systems, to generate estimated values of the missing data;

[0093] Step E4: Perform protocol conversion and packet transmission on the estimated values together with the actually collected industrial data to ensure data integrity and consistency.

[0094] Step E3 includes:

[0095] Step E3a: For temporary missing data, use an interpolation algorithm based on time series analysis to predict the possible values of the missing data;

[0096] Step E3b: For long-term missing data, use a machine learning model to learn the data generation pattern based on historical data and relevant data to predict the estimated values of the missing data;

[0097] Step E3c: For the missing of critical data, use a rule-based inference system to infer the possible values of the missing data according to predefined business logics and rules.

[0098] In Step 4, when the data received by the main management network database contains missing values, the following steps are executed:

[0099] Step F1: Identify the missing part of the data and mark the location of the missing data;

[0100] Step F2: Classify the missing data according to the rules in the product template to determine which data recovery strategy to use;

[0101] Step F3: Apply corresponding data recovery strategies, including but not limited to using the mean, median, mode of historical data or inferring through data of similar devices, to fill in the missing data;

[0102] Step F4: Convert the filled complete data into the target format and upload it to the main service database.

[0103] In Steps Five and Six, when the data requested by the user contains missing values, perform the following steps:

[0104] Step G1: The user specifies the preference settings for data recovery through the user platform, including but not limited to the priority of data recovery, the algorithms or models used, and the accuracy requirements for data recovery;

[0105] Step G2: The sub-service database selects a data recovery strategy according to the user's preference settings;

[0106] Step G3: Generate the final aggregated data information from the recovered data and the actually collected industrial data and provide it to the user platform for viewing.

[0107] For specific use: Take a manufacturing factory that has a variety of industrial equipment, including sensors, controllers, robots, etc., and these devices exchange data through different communication methods (such as Wi-Fi, Bluetooth, wired Ethernet) and protocol specifications (such as Modbus, OPCUA, CANopen) as an example for further explanation.

[0108] Step One: The acquisition devices in the factory, such as PLCs and intelligent sensors, are responsible for collecting data on device status, temperature, pressure, production volume, etc. in real time. These data are sent to the sensing network sub-platform through their respective supported communication methods and protocol specifications. For example, the temperature sensor sends data through the Modbus protocol, while the robot controller uploads its operating status through the OPC UA protocol.

[0109] Step A: Before data acquisition, the IT department of the factory configures the acquisition devices, specifying the specific communication protocol and data reporting frequency used by each device. For example, the data of key devices is collected once a minute, while the data of non-critical devices is collected once every half hour.

[0110] Step Two: The sensing network sub-platform performs protocol conversion on the received data, unifies the data in different formats into an internally processable format, and performs packet processing. For example, convert the Modbus data packet into the same internal format as the OPC UA data packet.

[0111] Step B: Before data transmission, the sensing network sub-platform preprocesses the data, including cleaning outliers, removing duplicate data, and compressing data packets to improve the transmission efficiency.

[0112] Step 3: The total platform of the sensor network quickly uploads data to the data aggregation interface via the UDP protocol, and then the data aggregation interface stably uploads the data to the main management network database via the TCP protocol. For example, real-time production data is quickly sent via UDP to ensure real-time performance; historical data is sent via the TCP protocol to ensure data is not lost.

[0113] Step C: The total platform of the sensor network monitors the bandwidth usage and latency of data transmission in real time to ensure the stability of data transmission.

[0114] Step D: Dynamically adjust the data aggregation strategy according to the monitoring results. For example, during high bandwidth usage periods, reduce the collection frequency of non-critical data; when low latency is required, give priority to transmitting critical data. Specific steps: D1: Bandwidth usage rate monitoring and adjustment. Suppose the factory's sensors detect that the network bandwidth usage rate exceeds the preset 80% threshold during a certain period. To ensure that the transmission of critical production data is not affected, the system automatically reduces the collection frequency of non-critical data (such as ambient temperature data) or suspends the upload of some non-critical data. Step D2: Latency monitoring and optimization of critical data transmission. In another scenario, the system detects that the latency of data transmission exceeds the preset 200 ms threshold. To reduce latency, the system gives priority to transmitting critical data, such as the status information of the robot welding arm, and temporarily reduces the collection accuracy of non-critical data (such as equipment maintenance logs) to reduce the size of data packets. Step D3: Maintain the current data aggregation strategy. During normal factory operations, if both the bandwidth usage rate and latency are within an acceptable range, the system will maintain the current data aggregation strategy unchanged to ensure data continuity and consistency. Step D4: Predict future bandwidth requirements and latency changes. The factory's IT system uses machine learning algorithms to analyze the data transmission patterns of the past month and predicts that the data transmission demand will increase during the morning peak shift every day. Therefore, the system adjusts the strategy in advance to increase bandwidth resources to cope with the upcoming data transmission peak. Step D5: User-defined data aggregation strategy. The factory's quality control department requires real-time access to critical quality inspection data, while the R & D department needs detailed equipment operation logs. The system dynamically adjusts the data aggregation strategy according to these different requirements to ensure the real-time performance of critical data and the integrity of non-critical data. Step D6: Adjust the data aggregation strategy through the user interface. The factory operator manually sets the thresholds for bandwidth usage and latency, as well as the collection frequency and accuracy of data through the user interface. For example, the operator may increase the data collection frequency during planned maintenance to collect more equipment status information.

[0115] Step E: During data collection and transmission, monitor data integrity in real time. Once missing data is detected, immediately initiate a data recovery algorithm. For example, for temporarily missing temperature data, use an interpolation algorithm based on time series analysis to predict the missing values. Specifically, Step E1: Monitor missing data in real time. During the production process, if a sensor suddenly stops sending data, the system immediately identifies this missing data event through real-time monitoring. Step E2: Determine the data recovery priority. The system, according to preset rules, determines that this is a missing critical data because the sensor is responsible for monitoring the temperature of a critical production line. Step E3: Apply the data recovery algorithm. The system uses an interpolation algorithm based on time series analysis to predict and generate the missing temperature data according to the historical data pattern of the sensor. Step E3a: Handling temporarily missing data. If a non-critical sensor temporarily loses its signal, the system may use simple forward filling or backward filling methods, using the last valid data point or the next data point to fill in the missing value. Step E3b: Handling long-term missing data. For sensors with long-term failures, the system may use machine learning models to predict the missing values based on the data patterns of similar sensors. Step E3c: Handling critical missing data. For the missing values of critical production parameters, the system may initiate a rule-based inference system to infer the missing values according to production logic and rules, such as using the average value of other similar devices on the same production line.

[0116] Step 4: After the main management network database receives the industrial data, it performs format conversion on the data according to the pre-stored product template to generate the second reported data that meets the target format of the data convergence platform, and transmits it to the management sub-platform database for further processing.

[0117] Step F: When the main management network database detects missing data, according to the rules in the product template, adopt corresponding data recovery strategies, such as using the average value of historical data to fill in the missing values. Specifically, Step F1: Identify and mark the missing data. When the main management network database finds missing values in the received data, the system automatically marks the positions of the missing data. Step F2: Classify and determine the data recovery strategy. The system classifies the missing data according to the rules in the product template and determines the most appropriate data recovery strategy. Step F3: Fill in the missing data. The system uses the average value, median, mode of historical data or data of similar devices for inference to fill in the missing data. Step F4: Upload the complete data to the main service database. The filled complete data undergoes format conversion and is uploaded to the main service database for subsequent analysis and decision-making.

[0118] Step 5: The main service database receives the third reported data sent by all sub-service databases and generates the final aggregated data information according to the reporting template parameters defined by the user.

[0119] Step 6: The operator in the factory can view various device data after aggregation through the user platform for production monitoring and decision-making support.

[0120] Step G: The user can set data recovery preferences through the user platform, such as selecting a specific data recovery algorithm or adjusting the frequency and accuracy of data collection. Specifically, in Step G1, the user specifies the data recovery preferences. The quality control personnel in the factory set the data recovery preferences through the user platform and specify to preferentially use a machine learning model to predict missing values in case of missing data. In Step G2, the sub-service database selects a data recovery strategy. The sub-service database automatically selects a data recovery strategy according to the user's set preferences and applies it to the data aggregation process. In Step G3, the aggregated data information is generated and provided. The recovered data, together with the actually collected industrial data, generates the final aggregated data information and is provided through the user platform for the operators and managers in the factory to view and analyze.

[0121] Through the above embodiments, the data aggregation method of the industrial Internet of Things sub-service platform of the present invention can ensure that a large amount of industrial data in the factory is effectively and real-time aggregated and processed, thereby improving production efficiency and decision-making quality.

[0122] The above are only embodiments of the present invention. Common general knowledge such as specific structures and characteristics in the solutions is not described in detail here. It should be noted that for those skilled in the art, without departing from the structure of the present invention, several modifications and improvements can still be made, and these should also be regarded as the protection scope of the present invention, which will not affect the implementation effect of the present invention and the practicality of the patent. The protection scope required by this application should be subject to the content of its claims, and the specific implementation manners in the description can be used to interpret the content of the claims.

Claims

1. A data aggregation method for an industrial Internet of Things sub-service platform, characterized in that: The steps include: Step 1: Multiple acquisition devices in the object platform collect industrial data of multiple industrial devices with multiple communication methods and multiple protocols and transmit them to the sensor network sub-platform; Step 2: The sensor network sub-platform performs protocol conversion on the industrial data, and then transmits it to the sensor network main platform after packet processing; Step 3: The sensor network master platform uploads the industrial data uploaded in the packet to the data aggregation interface through the UDP network transmission protocol of the data acquisition interface; the data aggregation interface uploads the industrial data uploaded in the packet as the first reported data to the main management network database through the TCP network transmission protocol; Step 4: After receiving the industrial data transmitted by the sensor network platform, the main management network database converts the first reported data according to the product template pre-stored in the main management network database and matching the product information of the IoT terminal device to obtain the second reported data that meets the target format of the data aggregation platform, and then transmits the second reported data to the management sub-platform database for processing to form the third reported data, which is then transmitted to the main management network database and the sub-service database. Step 5: The main service database is used to receive the third reported data sent by all sub-service databases. The sub-service database generates a product template for the third reported data according to the reporting template parameters defined by the user on the product template definition interface, and saves the product template until the data is aggregated to the main service database to generate aggregated data information; Step 6: Users view various device data gathered on the service platform through the user platform; The method further includes step C: in step 3, the sensor network master platform also includes real-time monitoring of the industrial data to evaluate bandwidth usage and delay of data transmission; and step D: after step 3, dynamically adjusting the data aggregation strategy according to the result of real-time monitoring to optimize bandwidth usage and reduce delay; Wherein step D comprises: Step D1: when it is monitored that the bandwidth usage rate exceeds a preset threshold, the collection frequency of non-critical data is reduced or the collection of some non-critical data is suspended; Step D2: When the delay is detected to exceed the preset threshold, the key data is transmitted first, and the data collection accuracy is temporarily reduced to reduce the data packet size; Step D3: When it is monitored that the bandwidth usage and latency are both within an acceptable range, the current data aggregation strategy is maintained unchanged; Step D4: Based on step D1 and step D2, the historical data transmission pattern is analyzed using a machine learning algorithm to predict future bandwidth requirements and delay changes, thereby adjusting the data aggregation strategy in advance; Step D5: Based on step D3, the data aggregation strategy is dynamically adjusted according to the classification of key data and non-key data defined by the user, as well as the user's specific requirements for data real-time performance, to meet the needs of different users; Step D6: Based on step D4 and step D5, a user interface is provided to allow the user to manually set the bandwidth usage and delay thresholds, as well as the frequency and accuracy of data collection according to actual application scenarios.

2. The data aggregation method of the industrial Internet of Things sub-service platform according to claim 1 is characterized in that: The method further includes step A: before step 1, configuring the plurality of acquisition devices to determine the communication mode and protocol corresponding to each acquisition device, as well as the frequency and priority of data acquisition.

3. The data aggregation method of the industrial Internet of Things sub-service platform according to claim 1 is characterized in that: It also includes step B: In step 2, the sensor network sub-platform also includes pre-processing the collected data, including data cleaning, data deduplication and data compression, so as to optimize the efficiency of data transmission.

4. The data aggregation method of the industrial Internet of Things sub-service platform according to claim 3 is characterized in that: In both step 1 and step 2, data integrity checks are performed according to preset rules. When missing data is detected, missing values ​​are generated and the following steps are performed: Step E1: Use preset data quality rules to monitor the collected data in real time to identify missing data events; Step E2: Determine the priority of restoring the missing data according to the type and severity of the missing data; Step E3: for different types of missing data, applying corresponding data recovery algorithms, including but not limited to interpolation algorithms, machine learning prediction models, or rule-based reasoning systems, to generate estimates of the missing data; Step E4: Perform protocol conversion and packet transmission on the estimated value together with the actually collected industrial data to ensure the integrity and consistency of the data.

5. The data aggregation method of the industrial Internet of Things sub-service platform according to claim 4 is characterized in that: Step E3 includes: Step E3a: For temporary missing data, an interpolation algorithm based on time series analysis is used to predict the possible values ​​of the missing data; Step E3b: For long-term missing data, a machine learning model is used to learn the data generation pattern based on historical data and related data to predict the estimated value of the missing data; Step E3c: For missing key data, a rule-based reasoning system is used to infer the possible values ​​of the missing data according to predefined business logic and rules.

6. The data aggregation method of the industrial Internet of Things sub-service platform according to claim 3 is characterized in that: In step 4, when the data received by the main management network database contains missing values, the following steps are performed: Step F1: Identify the missing parts in the data and mark the location of the missing data; Step F2: Classify the missing data according to the rules in the product template to determine which data recovery strategy to use; Step F3: Apply the corresponding data recovery strategy, including but not limited to using the average, median, mode of historical data or inferring through data from similar devices to fill in the missing data; Step F4: Convert the filled complete data into the target format and upload it to the main service database.

7. The data aggregation method of the industrial Internet of Things sub-service platform according to claim 3 is characterized in that: In steps 5 and 6, when the data requested by the user contains missing values, perform the following steps: Step G1: The user specifies data recovery preferences through the user platform, including but not limited to data recovery priority, algorithms or models used, and data recovery accuracy requirements; Step G2: The sub-service database selects a data recovery strategy based on the user's preference settings; Step G3: The restored data is combined with the actually collected industrial data to generate the final aggregated data information, and the information is provided to the user platform for viewing.

8. A data aggregation system for an industrial Internet of Things sub-service platform, characterized by: It includes a sensor network platform, a management platform and a service platform that interact in sequence. The sensor network platform adopts a front-sub-platform layout, wherein the sensor network platform includes a sensor network master platform and a plurality of mutually independent sensor network sub-platforms, wherein the plurality of sensor network sub-platforms are data acquisition modules in different communication networks, and the sensor network master platform is the master data acquisition module of all the communication networks, and the sensor network sub-platforms collect industrial data of industrial equipment through sensors, and perform protocol conversion and packet transmission on the industrial data to the sensor network master platform; The sensor network master platform uploads the industrial data in the packet to the data aggregation interface through the UDP network transmission protocol of the data acquisition interface; the data aggregation interface uploads the industrial data in the packet as the first reported data to the management platform through the TCP network transmission protocol. The management platform adopts a central split structure, which includes a main management network database and multiple independent management sub-platforms. Each management sub-platform is provided with a corresponding sub-management database. The main management network database is used to uniformly manage the data of all the communication networks. After receiving the industrial data transmitted by the sensor network platform, the main management network database converts the first reported data according to the product template pre-stored in the main management network database and matching the product information of the Internet of Things terminal device to obtain the second reported data that meets the target format of the data aggregation platform, and then transmits the second reported data to the management sub-platform database for processing to form the third reported data and then transmits it to the main management network database. The main management network database transmits the third reported data to the service platform. The service platform includes a sub-service database, a service sub-platform and a main service database, wherein the sub-service database is used to receive the third reported data sent by the corresponding management platform, and the sub-service database only exchanges information with the corresponding management platform; the service sub-platform is used to receive the third reported data sent by the corresponding sub-service database, and the service sub-platform only exchanges information with the corresponding sub-service database; all sub-service databases exchange information with the main service database, and the main service database is used to receive the third reported data sent by all sub-service databases, and the sub-service database generates a product template for the third reported data according to the reporting template parameters defined by the user on the product template definition interface, and saves the product template until the data is aggregated to the main service database to generate aggregated data information.

Images