Industrial equipment data acquisition and forwarding system and method based on multi-protocol adaptation

Abstract

The invention discloses an industrial equipment data acquisition and forwarding system and method based on multi-protocol adaptation, and the system comprises a protocol adaptation layer which is used for interacting with various types of industrial equipment and is integrated with a plurality of protocol drive plug-ins; the data standardization processing layer is used for receiving and analyzing the original data message, mapping the data of the original data message according to a preset mapping rule and converting the data into a unified data model; the data routing forwarding layer is used for receiving the standardized data objects of the data standardization processing layer and forwarding the standardized data objects meeting matching conditions to corresponding output channels according to a rule engine with the pre-configured matching conditions; and the configuration management module is used for configuring and managing the protocol adaptation layer, the data standardization processing layer and the data routing forwarding layer. According to the invention, the target path of the data is changed from hard coding to configurability, and the forwarding and storage paths of the data are configured through the configuration rule, so that the method is more flexible.

Description

Technical Field

[0001] This invention relates to the technical field of heterogeneous data acquisition for industrial equipment, and in particular to a system and method for data acquisition and forwarding of industrial equipment based on multi-protocol adaptation. Background Technology

[0002] Modern industrial systems are rapidly evolving towards digitalization, networking, and intelligence. Against this backdrop, hundreds of millions of sensors, actuators, smart meters, and control units are deployed on production sites, generating massive amounts of real-time operational data. The efficient and reliable collection, aggregation, and transmission of this data are the cornerstones for achieving equipment status monitoring, production optimization, energy efficiency management, and predictive maintenance.

[0003] Currently, industrial sites commonly use SCADA systems or IoT edge gateways as the core for data aggregation. These systems are responsible for reading data from underlying devices and transmitting it to upper-level monitoring centers, real-time databases, or cloud platforms for analysis and processing. However, after long-term development, the industrial sector has developed numerous communication protocol standards with different technical systems and incompatibility, leading to a serious "data silo" phenomenon. Currently widely used systems and products face the problem of extremely high protocol heterogeneity when implementing data acquisition; different equipment manufacturers and devices used in different fields employ different communication protocols.

[0004] To address protocol diversity, existing common practices include: 1) Customized development: Writing dedicated data acquisition drivers or services for specific protocols in a specific project. This approach has a long development cycle, high cost, poor reusability, and is difficult to maintain. 2) Purchasing commercial kits: Buying commercial software or hardware gateways that include drivers for multiple protocols. However, these kits have poor scalability; adding a new protocol requires waiting for updates, making it difficult to quickly respond to on-site needs. Furthermore, their procurement and maintenance costs are high. 3) Multi-gateway stacking: Using multiple dedicated gateways supporting different protocols in parallel leads to complex system architecture, dispersed locations, increased cabling costs, and the introduction of more points of failure. 4) Challenges in data normalization and forwarding: Even if data is successfully acquired, the data frame formats, byte order, and register address mapping methods of different protocols vary greatly. Converting this heterogeneous data into a unified format and efficiently and accurately forwarding it to different destinations is a significant challenge. Existing systems typically lack flexible, configurable data processing pipelines, resulting in rigid data flow forwarding strategies that are difficult to adapt to diverse application scenarios.

[0005] In summary, industrial equipment in the field urgently needs a highly integrated, flexible, scalable, open, and efficient multi-protocol adaptation and data forwarding solution. This solution needs to shield the complexity of underlying protocols, providing unified and standardized data services for upper-layer applications, thereby breaking down data silos and providing a solid data foundation for industrial big data analysis and intelligent applications. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and propose an industrial equipment data acquisition and forwarding system and method based on multi-protocol adaptation to support industrial data integration and edge computing. Its core task is to solve the data access problem of massive heterogeneous industrial equipment.

[0007] This invention aims to overcome the shortcomings of existing technologies, such as excessive reliance on specific protocols and inability to flexibly adapt to multiple industrial communication protocols. Its primary objective is to provide a method that can uniformly access and support multiple mainstream and specialized industrial protocols, including ModbusTCP, ModbusRTU, OPCUA, OPCDA, IEC 104, IEC 61850-MMS, IEC 61850-GOOSE, ADS, and SNMP, thereby eliminating industrial data silos.

[0008] This invention aims to change the current situation where traditional data acquisition systems suffer from poor protocol scalability and require custom development or hardware replacement. The second objective is to design a modular, plug-in-the-loop protocol adaptation architecture, enabling rapid integration of new communication protocols simply by developing and configuring the corresponding protocol plug-in module. This significantly reduces the complexity and cost of system development and maintenance, ensuring the system's long-term evolution capabilities.

[0009] This invention aims to solve the problem of inconsistent formats and difficulties in unified utilization and forwarding of multi-source heterogeneous data. A third objective is to provide a universal data standardization processing method that parses and converts data from different protocols into a standardized unified data model, and designs a configurable, intelligent data forwarding and routing mechanism that can efficiently and accurately distribute processed data to different target storage locations or output it in various standard file formats based on data tags or business rules.

[0010] This invention aims to address the stringent requirements for data real-time performance and integrity in industrial settings. A fourth objective is to ensure the efficiency and stability of data acquisition under massive concurrent device access through optimized thread scheduling, connection management, and caching mechanisms, effectively reducing data loss rates and ensuring the continuity and reliability of critical data.

[0011] The objective of this invention is achieved through the following technical solution: a data acquisition and forwarding system for industrial equipment based on multi-protocol adaptation, employing an asynchronous network I / O model and thread pool technology to handle high-concurrency connections and data streams. This system includes: The protocol adaptation layer is used to interact with various industrial equipment. The protocol adaptation layer integrates multiple protocol driver plugins. Each protocol driver plugin implements a corresponding communication protocol. Each protocol driver plugin performs data acquisition and parsing of industrial equipment based on the adapted communication protocol to generate raw data packets. The data normalization processing layer is used to receive and parse the raw data packets. According to the preset mapping rules, it maps and converts the data in the raw data packets into a unified data model. The unified data model is defined as a structured intermediate representation format, which is a data object with the same fields. This unified data model is used as a normalized data object. The data routing and forwarding layer is used to receive normalized data objects from the data normalization processing layer, and forward normalized data objects that meet the matching conditions to the corresponding output channels according to the rule engine with pre-configured matching conditions. The output channels include databases, format files, message queues, and upper-level platforms. The configuration management module is used to configure and manage the protocol adaptation layer, data normalization processing layer, and data routing and forwarding layer. The configuration management module integrates a database for storing configuration information.

[0012] Furthermore, including: The memory cache queue is used to cache normalized data objects to prevent data loss. When the cache reaches a preset threshold, a backpressure mechanism is triggered to notify the protocol driver plugin to pause data acquisition and parsing, thereby achieving flow control. Conversely, the protocol driver plugin is not notified, allowing it to continue generating raw data packets.

[0013] Furthermore, the protocol adaptation layer includes: The protocol adaptation layer is configured with a driver interface for unifying various communication protocols. When the driver interface is connected to industrial equipment, the configured protocol driver plugin is automatically loaded through the driver interface. The protocol driver plugin is encapsulated and managed in a dynamic link library (DLL) or containerized manner and supports hot-swapping. Each protocol driver plugin encapsulates a connection management module, a data request scheduling module, and a raw data packet parsing module for the corresponding communication protocol.

[0014] Furthermore, the driver interface is developed based on the init function, connect function, readData function, and close function.

[0015] Furthermore, the communication protocols include ModbusTCP, ModbusRTU, OPCUA, OPCDA, IEC 104, IEC61850-MMS, IEC 61850-GOOSE, ADS, and SNMP.

[0016] Furthermore, the data routing and forwarding layer includes: A database adapter is used to forward received normalized data objects to a database for storage. The file output adapter is used to write the received normalized data objects into a specified format file according to preset rules; The message middleware adapter is used to forward received normalized data objects to the message queue for push subscription; The upper-level platform interface adapter is used to push the received standardized data objects to the upper-level platform.

[0017] The method for data acquisition and forwarding of industrial equipment based on multi-protocol adaptation, implemented according to the aforementioned system for data acquisition and forwarding of industrial equipment based on multi-protocol adaptation, includes the following steps: S1. Protocol driver loading and initialization: The configuration management module is invoked to read the configuration list of the industrial equipment, dynamically load the required protocol driver plugins, and initialize the connection of each protocol driver plugin according to the configuration parameters, including IP address, port, serial port number, and baud rate. S2. Multi-protocol data acquisition and parsing: Each protocol driver plugin actively requests data from industrial equipment or subscribes to industrial equipment data according to the preset scheduling strategy. After receiving the response message from the industrial equipment, the protocol driver plugin internally parses the response message according to the protocol specification and extracts the original value and status information of the data point, i.e. the original data message. S3. Data Normalization Transformation: Based on the predefined mapping configuration table, the data normalization processing layer is called to convert the raw data packets into a standard unified data model, completing the conversion from heterogeneous protocol data to internal unified format data. S4. Data Routing and Distribution: The unified data model is sent to the data routing and forwarding layer as a standardized data object. The rule engine matches the standardized data object according to the preset forwarding rules and decides to deliver it to at least one output channel. S5. Data Persistence and Output: After receiving the normalized data object, each output channel performs the output operation; S6. System Monitoring and Maintenance: During system operation, the connection status, data traffic, and forwarding latency performance metrics of each protocol driver plugin are continuously monitored. Once an anomaly occurs, the protocol driver plugin is driven to automatically attempt to reconnect and record the data quality status during the data interruption, further triggering the data re-collection mechanism.

[0018] Furthermore, step S5 includes: The database adapter in the data normalization processing layer is invoked to execute the SQL insert statement; the file adapter in the data normalization processing layer is invoked to append the normalized data object to the file buffer; and the message middleware adapter in the data normalization processing layer is invoked to serialize the normalized data object and publish it to the specified Topic.

[0019] A non-transitory computer-readable medium storing instructions, characterized in that, when the instructions are executed by a processor, the steps of the above-described industrial equipment data acquisition and forwarding method based on multi-protocol adaptation are performed.

[0020] A computing device includes a processor and a memory for storing processor-executable programs, characterized in that when the processor executes the program stored in the memory, it implements the above-described method for data acquisition and forwarding of industrial equipment based on multi-protocol adaptation.

[0021] Compared with the prior art, the present invention has the following advantages and beneficial effects: 1. Significantly improves system flexibility and scalability, and reduces development and maintenance costs. This invention adopts a hot-swappable protocol driver plug-in architecture and a unified driver interface standard, which is fundamentally different from the traditional approach that requires modifying core code or replacing hardware. When a new protocol needs to be integrated in the field, developers only need to develop a new driver plug-in following the standard interface. Tests show that the development cycle for integrating a new protocol can be shortened from the traditional 2-4 person-weeks to within 1 person-day, improving efficiency by over 80%.

[0022] 2. Effectively solves the data silo problem and achieves unified data governance and utilization. This invention designs a data normalization processing layer above the protocol layer, introducing a unified data model as an intermediate standard for the conversion of all heterogeneous data. Regardless of whether the source data comes from Modbus registers or IEC 61850 MMS reports, it is ultimately converted into data objects with the same fields. This allows all subsequent data processing, storage, and analysis modules to be developed only for this one standard format, greatly simplifying system complexity.

[0023] 3. It provides high data forwarding flexibility, meeting the needs of diverse application scenarios. This invention adopts a data routing and forwarding mechanism based on a rule engine, changing the data target path from hard-coded to configurable. Users can easily configure a rule through the interface to configure the data forwarding and storage path. This flexibility is unattainable by systems with traditional fixed forwarding logic. Attached Figure Description

[0024] Figure 1 This is an architecture diagram of an industrial equipment data acquisition and forwarding system. Detailed Implementation

[0025] The present invention will be further described below with reference to specific embodiments.

[0026] Example 1 See Figure 1 As shown, the industrial equipment data acquisition and forwarding system based on multi-protocol adaptation provided in this embodiment adopts the Reactor / Proactor asynchronous network I / O model and thread pool technology to handle high-concurrency connections and data streams. The system includes: 1) Protocol adaptation layer, used to interact with various industrial equipment. The protocol adaptation layer integrates multiple protocol driver plugins. Each protocol driver plugin implements a corresponding communication protocol. Each protocol driver plugin performs data acquisition and parsing of industrial equipment based on the adapted communication protocol to generate raw data packets.

[0027] The protocol adaptation layer is configured with a driver interface API for unifying various communication protocols. When this driver interface API is connected to industrial equipment, the configured protocol driver plugin is automatically loaded through the driver interface API. The driver interface API is developed based on the init function, connect function, readData function, and close function. Any new protocol only needs to implement this driver interface API and package it into a protocol driver plugin to be recognized and loaded by the system without modifying the core code. Protocol driver plugins are encapsulated and managed using dynamic link libraries (DLLs) or containerization and support hot-swapping. Each protocol driver plugin encapsulates a connection management module, a data request scheduling module, and a raw data packet parsing module for the corresponding communication protocol. The connection management module is responsible for establishing / disconnecting connections and maintaining heartbeats, the data request scheduling module is responsible for polling frequency and triggering conditions, and the raw data packet parsing module is responsible for parsing raw data packets. Communication protocols include ModbusTCP, ModbusRTU, OPCUA, OPCDA, IEC 104, IEC 61850-MMS, IEC 61850-GOOSE, ADS, and SNMP.

[0028] 2) Data normalization processing layer, used to receive and parse raw data packets, and map and convert the data of the raw data packets into a unified data model according to the preset mapping rules. The unified data model is defined as a structured intermediate representation format, which is a data object with the same fields. JSON or Protocol Buffers format is preferred. This unified data model is the normalized data object, such as pointld, value, and timestamp.

[0029] 3) Memory cache queue is used to cache normalized data objects to prevent data loss. When the cache reaches a preset threshold, a backpressure mechanism is triggered to notify the protocol driver plugin to pause data acquisition and parsing, thereby achieving flow control. Conversely, the protocol driver plugin is not notified, allowing it to continue generating raw data packets.

[0030] 4) Data routing and forwarding layer, which receives normalized data objects from the data normalization processing layer and forwards normalized data objects that meet the matching conditions to the corresponding output channels according to the rule engine with pre-configured matching conditions. The output channels include databases, format files, message queues and upper-level platforms. The data routing and forwarding layer includes a database adapter, a file output adapter, a message middleware adapter, and a higher-level platform interface adapter. The database adapter forwards the received normalized data objects to a database for storage, such as HighgoDB or TDEngine. The file output adapter writes the received normalized data objects to a specified format file, such as CSV, JSON, or XML, according to preset rules. The message middleware adapter forwards the received normalized data objects to Kafka or MQTT message queues for push, allowing other systems to subscribe. The higher-level platform interface adapter pushes the received normalized data objects to the higher-level platform via HTTP / RESTful API.

[0031] 5) Configuration management module, used to configure and manage the protocol adaptation layer, data normalization processing layer and data routing and forwarding layer. It realizes full lifecycle management of devices, drivers, data point mapping, forwarding rules, etc. through a web management interface or configuration center. The configuration management module integrates a database to store configuration information to ensure the persistence and consistency of configuration.

[0032] Example 2 This paper presents a method for data acquisition and forwarding of industrial equipment based on multi-protocol adaptation, using wind farm data as an application scenario. The method is implemented on an OpenKylin server using the industrial equipment data acquisition and forwarding system described in Example 1. The method includes the following steps: S1. Protocol driver loading and initialization: Log in to the system via a web browser, call the configuration management module to read the configuration list of industrial equipment. The main tasks are to configure the device table, point table, and forwarding rules, dynamically load the required protocol driver plugins, and initialize the connection of each protocol driver plugin according to the configuration parameters. The configuration parameters include IP address, port, serial port number, and baud rate. S2. Multi-protocol data acquisition and parsing: Each protocol driver plugin actively requests data from industrial equipment or subscribes to industrial equipment data according to the preset scheduling strategy. After receiving the response message from the industrial equipment, the protocol driver plugin internally parses the response message according to the protocol specification and extracts the original value and status information of the data point, i.e. the original data message. S3. Data Normalization Transformation: Based on the predefined mapping configuration table, the data normalization processing layer is called to convert the raw data packets into a standard unified data model, completing the conversion from heterogeneous protocol data to internal unified format data. S4. Data Routing and Distribution: The unified data model is sent to the data routing and forwarding layer as a standardized data object. The rule engine matches the standardized data object according to the preset forwarding rules and decides to deliver it to at least one output channel. S5. Data Persistence and Output: After receiving the normalized data object, each output channel performs output operations, including: calling the database adapter in the data normalization processing layer to execute SQL insert statements; calling the file adapter in the data normalization processing layer to append the normalized data object to the file buffer; and calling the message middleware adapter in the data normalization processing layer to serialize the normalized data object and publish it to the specified Topic.

[0033] S6. System Monitoring and Maintenance: During system operation, the connection status, data traffic, and forwarding latency performance metrics of each protocol driver plugin are continuously monitored. Once an anomaly occurs, the protocol driver plugin is driven to automatically attempt to reconnect and record the data quality status during the data interruption, further triggering the data re-collection mechanism.

[0034] Example 3 This embodiment discloses a non-transitory computer-readable medium storing instructions that, when executed by a processor, perform the steps of the industrial equipment data acquisition and forwarding method based on multi-protocol adaptation as described in Embodiment 2.

[0035] In this embodiment, the non-transitory computer-readable medium can be a disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), USB flash drive, portable hard drive, etc.

[0036] Example 4 This embodiment discloses a computing device, including a processor and a memory for storing processor-executable programs. When the processor executes the program stored in the memory, it implements the industrial equipment data acquisition and forwarding method based on multi-protocol adaptation described in Embodiment 2.

[0037] The computing device described in this embodiment may be a desktop computer, laptop computer, smartphone, PDA handheld terminal, tablet computer, programmable logic controller (PLC), or other terminal device with processor function.

[0038] The above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Therefore, any changes made in accordance with the shape and principle of the present invention should be covered within the protection scope of the present invention.

Claims

1. An industrial equipment data acquisition and forwarding system based on multi-protocol adaptation, characterized in that, The system employs an asynchronous network I / O model and thread pool technology to handle high-concurrency connections and data streams. It includes: The protocol adaptation layer is used to interact with various industrial equipment. The protocol adaptation layer integrates multiple protocol driver plugins. Each protocol driver plugin implements a corresponding communication protocol. Each protocol driver plugin performs data acquisition and parsing of industrial equipment based on the adapted communication protocol to generate raw data packets. The data normalization processing layer is used to receive and parse the raw data packets. According to the preset mapping rules, it maps and converts the data in the raw data packets into a unified data model. The unified data model is defined as a structured intermediate representation format, which is a data object with the same fields. This unified data model is used as a normalized data object. The data routing and forwarding layer is used to receive normalized data objects from the data normalization processing layer, and forward normalized data objects that meet the matching conditions to the corresponding output channels according to the rule engine with pre-configured matching conditions. The output channels include databases, format files, message queues, and upper-level platforms. The configuration management module is used to configure and manage the protocol adaptation layer, data normalization processing layer, and data routing and forwarding layer. The configuration management module integrates a database for storing configuration information.

2. The industrial equipment data acquisition and forwarding system based on multi-protocol adaptation according to claim 1, characterized in that, include: A memory-based cache queue is used to cache normalized data objects and prevent data loss. When the cache reaches a preset threshold, a backpressure mechanism is triggered, which notifies the protocol driver plugin to pause data collection and parsing to achieve flow control; otherwise, the protocol driver plugin is not notified, allowing it to continue generating raw data packets.

3. The industrial equipment data acquisition and forwarding system based on multi-protocol adaptation according to claim 1, characterized in that, The protocol adaptation layer includes: The protocol adaptation layer is configured with a driver interface for unifying various communication protocols. When the driver interface is connected to industrial equipment, the configured protocol driver plugin is automatically loaded through the driver interface. The protocol driver plugin is encapsulated and managed using dynamic link library (DLL) or containerization. Each protocol driver plugin encapsulates a connection management module, a data request scheduling module, and a raw data packet parsing module for the corresponding communication protocol.

4. The industrial equipment data acquisition and forwarding system based on multi-protocol adaptation according to claim 3, characterized in that, The driver interface is developed based on the init function, connect function, readData function, and close function.

5. The industrial equipment data acquisition and forwarding system based on multi-protocol adaptation according to claim 3, characterized in that, The communication protocols include ModbusTCP, ModbusRTU, OPCUA, OPCDA, IEC 104, IEC 61850-MMS, IEC61850-GOOSE, ADS, and SNMP.

6. The industrial equipment data acquisition and forwarding system based on multi-protocol adaptation according to claim 1, characterized in that, The data routing and forwarding layer includes: A database adapter is used to forward received normalized data objects to a database for storage. The file output adapter is used to write the received normalized data objects into a specified format file according to preset rules; The message middleware adapter is used to forward received normalized data objects to the message queue for push subscription; The upper-level platform interface adapter is used to push the received standardized data objects to the upper-level platform.

7. A method for data acquisition and forwarding of industrial equipment based on multi-protocol adaptation, characterized in that, The industrial equipment data acquisition and forwarding system based on multi-protocol adaptation as described in any one of claims 1-6 includes the following steps: S1. Protocol driver loading and initialization: The configuration management module is invoked to read the configuration list of the industrial equipment, dynamically load the required protocol driver plugins, and initialize the connection of each protocol driver plugin according to the configuration parameters, including IP address, port, serial port number, and baud rate. S2. Multi-protocol data acquisition and parsing: Each protocol driver plugin actively requests data from industrial equipment or subscribes to industrial equipment data according to the preset scheduling strategy. After receiving the response message from the industrial equipment, the protocol driver plugin internally parses the response message according to the protocol specification and extracts the original value and status information of the data point, i.e. the original data message. S3. Data Normalization Transformation: Based on the predefined mapping configuration table, the data normalization processing layer is called to convert the raw data packets into a standard unified data model, completing the conversion from heterogeneous protocol data to internal unified format data. S4. Data Routing and Distribution: The unified data model is sent to the data routing and forwarding layer as a standardized data object. The rule engine matches the standardized data object according to the preset forwarding rules and decides to deliver it to at least one output channel. S5. Data Persistence and Output: After receiving the normalized data object, each output channel performs the output operation; S6. System Monitoring and Maintenance: During system operation, the connection status, data traffic, and forwarding latency performance metrics of each protocol driver plugin are continuously monitored. Once an anomaly occurs, the protocol driver plugin is driven to automatically attempt to reconnect and record the data quality status during the data interruption, further triggering the data re-collection mechanism.

8. The method for data acquisition and forwarding of industrial equipment with multi-protocol adaptation according to claim 7, characterized in that, Step S5 includes: The database adapter in the data normalization processing layer is invoked to execute the SQL insert statement; the file adapter in the data normalization processing layer is invoked to append the normalized data object to the file buffer; and the message middleware adapter in the data normalization processing layer is invoked to serialize the normalized data object and publish it to the specified Topic.

9. A non-transitory computer-readable medium storing instructions, characterized in that, When the instruction is executed by the processor, the steps of the industrial equipment data acquisition and forwarding method based on multi-protocol adaptation as described in claim 7 or 8 are performed.

10. A computing device, comprising a processor and a memory for storing a processor-executable program, characterized in that, When the processor executes the program stored in the memory, it implements the industrial equipment data acquisition and forwarding method based on multi-protocol adaptation as described in claim 7 or 8.

Images