Data processing method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of computers, and provides a data processing method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring heterogeneous protocol data, and caching the acquired heterogeneous protocol data to a front-end sliding window cache region; continuously sliding from the front-end sliding window cache region based on a preset rear-end sliding window to obtain heterogeneous protocol data, and mapping the obtained heterogeneous protocol data to a preset address space of an industrial data unified architecture; and obtaining a data service demand of each client, and mapping data in the address space of the industrial data unified architecture to the address space of the takeout data service interface protocol according to the data service demand, so that the client accesses the data corresponding to the data service demand based on the address space of the takeout data service interface protocol. And the data acquisition gateway maps the data to an address space of a data delivery service interface protocol, so that data fusion, intercommunication and sharing among different protocols are realized.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data processing method, apparatus, electronic device and storage medium. Background Technology

[0002] With the rapid development of the integration of industrial automation and informatization (Operational Technology (OT) / Information Technology (IT)), the demand for data acquisition gateways in industrial communication is constantly increasing. Existing industrial communication protocols, such as Modbus, OPC UA, and MQTT, suffer from heterogeneity, leading to complexity and high costs in data interoperability, which severely restricts the large-scale application of the Industrial Internet of Things (IIoT).

[0003] In the pyramid model of the ISA95 standard, industrial data acquisition gateways are typically located at Layer 2 (monitoring layer). The communication protocols at this layer not only extensively use the Modbus protocol but also commonly employ the 104 protocol (such as the IEC 60870-5-104 protocol). With the advancement of the integration of informatization and industrialization, the number of devices connected to data acquisition gateways is constantly increasing, and the data scale is rapidly expanding, especially in complex industrial environments where the data volume grows exponentially.

[0004] Meanwhile, as a data service provider, the data acquisition gateway needs to meet the differentiated access requirements of multiple third-party systems. Different clients have significantly different requirements for data access, and their requirements for latency and reliability are stringent. This poses a significant challenge to existing data acquisition gateways in flexibly and efficiently handling the integration of multiple clients, multiple protocols, and heterogeneous data. For example, in the 104 protocol, multiple clients may simultaneously need to access the same or different data points, and existing solutions often cannot efficiently handle such complex data access needs. Therefore, how to efficiently integrate heterogeneous data and support multi-client access has become a key challenge for data acquisition gateways. Summary of the Invention

[0005] In view of this, embodiments of this application provide a data processing method, apparatus, electronic device, and storage medium to solve the problem in the prior art that data acquisition gateways cannot integrate heterogeneous protocol data.

[0006] A first aspect of this application provides a data processing method, comprising: acquiring heterogeneous protocol data and caching the acquired heterogeneous protocol data in a front-end sliding window cache area; continuously acquiring heterogeneous protocol data from the front-end sliding window cache area based on a pre-set back-end sliding window, and mapping the acquired heterogeneous protocol data to a pre-set address space of an industrial data unified architecture; acquiring the data service requirements of each client, and mapping the data in the address space of the industrial data unified architecture to the address space of an external data service interface protocol according to the data service requirements, so that the client accesses the data corresponding to the data service requirements based on the address space of the external data service interface protocol. A second aspect of this application provides a data processing apparatus, comprising: a data acquisition module for acquiring heterogeneous protocol data and caching the acquired heterogeneous protocol data in a front-end sliding window cache area; a back-end mapping module for continuously acquiring heterogeneous protocol data from the front-end sliding window cache area based on a pre-set back-end sliding window and mapping the acquired heterogeneous protocol data to a pre-set address space of an industrial data unified architecture; and a front-end mapping module for acquiring the data service requirements of each client and mapping the data in the address space of the industrial data unified architecture to the address space of an external data service interface protocol according to the data service requirements, so that the client accesses the data corresponding to the data service requirements based on the address space of the external data service interface protocol.

[0007] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method.

[0008] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.

[0009] The beneficial effects of this application embodiment compared with the prior art are as follows: The method of this application embodiment acquires heterogeneous protocol data and caches the acquired heterogeneous protocol data in the front-end sliding window cache area; based on the pre-set back-end sliding window, it continuously slides from the front-end sliding window cache area to acquire heterogeneous protocol data, and maps the acquired heterogeneous protocol data to the address space of the pre-set unified industrial data architecture; it acquires the data service requirements of each client, and maps the data in the address space of the unified industrial data architecture to the address space of the external data service interface protocol according to the data service requirements, so that the client accesses the data corresponding to the data service requirements based on the address space of the external data service interface protocol. The front-end receiving and back-end processing are decoupled, and the front-end blocks external surges, so the back-end has a natural risk isolation capability. The front end receives raw heterogeneous protocol data and caches it in a front-end sliding window buffer. The back end continuously retrieves cached heterogeneous protocol data from the front-end sliding window buffer and performs message protocol parsing and processing. Thus, external burst traffic is absorbed and rate-limited by the front end, while the back end gains natural risk isolation and stable input, thereby improving system throughput and reliability. Furthermore, the data acquisition gateway flexibly maps data to the address space of the external data service interface protocol according to each client's data service needs, achieving data fusion, interoperability, and sharing between different protocols. This solution not only improves data acquisition and processing efficiency and supports multi-protocol parallel operation but also provides customized data services based on client needs, enhancing system flexibility, scalability, and reliability. In addition, through standardized data mapping and precise data subset extraction, resource utilization is optimized, data management is simplified, and the system's real-time performance, reliability, and fault tolerance are improved, avoiding the problem in related technologies where data acquisition gateways cannot fuse heterogeneous protocol data. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart illustrating a data processing method provided in an embodiment of this application; Figure 2 This is a flowchart illustrating another data processing method provided in an embodiment of this application; Figure 3 This is a flowchart illustrating another data processing method provided in an embodiment of this application; Figure 4 This is a flowchart illustrating another data processing method provided in an embodiment of this application; Figure 5 This is a flowchart illustrating another data processing method provided in an embodiment of this application; Figure 6 This is a flowchart illustrating another optional data processing method provided in the embodiments of this application; Figure 7 This is a schematic diagram of a data processing apparatus provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0012] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, data processing apparatuses, circuits, and methods are omitted so as not to obscure the description of this application with unnecessary detail.

[0013] A data processing method and a data processing apparatus according to embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0014] Figure 1 This application provides a data processing method, such as... Figure 1 As shown, the method includes: S101. Obtain heterogeneous protocol data and cache the obtained heterogeneous protocol data in the front-end sliding window cache area; S102. Based on the pre-set backend sliding window, continuously slide to obtain heterogeneous protocol data from the frontend sliding window cache, and map the obtained heterogeneous protocol data to the address space of the pre-set unified industrial data architecture. S103. Obtain the data service requirements of each client, and map the data in the address space of the unified industrial data architecture to the address space of the external data service interface protocol according to the data service requirements, so that the client can access the data corresponding to the data service requirements based on the address space of the external data service interface protocol.

[0015] It is understood that the data processing method of this application is applied to a data acquisition gateway, which has multiple data acquisition interfaces. Each data acquisition interface corresponds to a different protocol driver, including at least one of the following: Modbus master driver, OPC data access client driver, OPC unified architecture client driver, and Ethernet industrial protocol client driver.

[0016] The data acquisition gateway is equipped with multiple data acquisition interfaces, each corresponding to a specific protocol driver. These drivers are responsible for communicating with external devices operating under a particular protocol (such as Modbus, OPC, or the Ethernet industrial protocol). For example, the Modbus master driver's data acquisition interface exchanges data with external devices supporting the Modbus protocol; the OPC data access client driver communicates with devices supporting the OPC DA protocol; and the OPC unified architecture client driver interacts with devices supporting the OPC UA protocol. Through these protocol drivers, the data acquisition gateway can efficiently acquire data from different protocols and map it uniformly into the address space of the unified industrial data architecture.

[0017] It is understandable that, to ensure the flexibility and stability of data acquisition, the data processing method provided in this application further includes: setting a corresponding operating mode for each protocol driver, including a simulation mode and a disabled mode; wherein, the simulation mode is used to instruct the protocol driver to acquire heterogeneous protocol data based on a preset simulation algorithm when the data acquisition gateway is not communicating with external devices; the disabled mode is used to temporarily isolate the communication between the external device and the data acquisition gateway when an abnormal communication of the external device is detected. In the simulation mode, the protocol driver does not communicate with the external device, and data updates are performed entirely according to the internally predetermined simulation algorithm (such as random, sine, step increment, step decrement, manual, etc.). The simulation mode is generally used for system debugging when the external device is unavailable; the disabled mode means that the protocol driver is in a disabled state, neither communicating with the outside nor performing simulation. The disabled state is generally used as an emergency temporary measure for external device abnormalities (such as data storms, data anomalies).

[0018] It is understandable that the operating mode also includes a normal mode. In normal mode, the protocol driver maintains normal communication with external devices, and data is exchanged according to the established protocol. The protocol driver can freely switch between three states online without reinstalling or reconfiguring data or restarting. It is understandable that if the protocol gateway corresponding to a certain data acquisition interface is set to normal mode, the data acquisition gateway can perform real-time data interaction with the corresponding external device based on the data acquisition interface to obtain heterogeneous protocol data. When the protocol gateway corresponding to a certain data acquisition interface is set to simulation mode, the protocol driver will generate heterogeneous protocol data based on a preset simulation algorithm when the external device is unavailable. When the protocol gateway corresponding to a certain data acquisition interface is set to disabled mode, the data acquisition gateway will temporarily disconnect the connection between the data acquisition interface and the corresponding external device to prevent abnormal data from affecting the stability of the system.

[0019] This application enables the data acquisition gateway to flexibly adjust its data acquisition methods according to different working environments and requirements by setting different operating modes (normal mode, simulation mode, and disabled mode) for the protocol driver. Especially when external devices are unavailable, the simulation mode generates simulated data, ensuring continuous system operation and preventing data loss. The disabled mode ensures rapid disconnection upon detecting communication anomalies in external devices, preventing the input of abnormal data from negatively impacting system stability. This mechanism effectively prevents system crashes or errors caused by device failures or abnormal conditions. In normal mode, the protocol driver maintains real-time data exchange with external devices, ensuring the data acquisition gateway can efficiently acquire data from different protocols, meeting the high requirements for data real-time performance and reliability in the Industrial Internet of Things (IIoT). By switching between different operating modes, the data acquisition gateway avoids invalid communication operations and reduces system resource waste when external devices are unavailable or malfunction, while ensuring adaptability and stability in different modes. This method supports multiple protocol drivers communicating with various external devices. Through intelligent switching between simulation and disabled modes, it ensures the data acquisition gateway is compatible with multiple devices and maintains good system performance when devices malfunction.

[0020] In some examples, the data processing method provided in this application further includes: configuring at least one protocol driver as multiple logical devices and communicating with the same external device through multiple logical devices. That is, each protocol driver can be configured as one or more logical devices, and one external device can be mapped to one logical device, or one external device can be mapped to multiple logical devices; when an external device is split and mapped to multiple logical devices, the splitting method is flexible and varied, and the splitting considerations can include efficiency improvement, functional isolation, etc.

[0021] Taking a Modbus peripheral as an example, assuming this peripheral has multiple input / output functions and can be split and mapped into multiple logical devices, let's take a Modbus peripheral split into three logical devices as an example. Logic device 1 is used to communicate with the digital input (DI) interface of the peripheral and read the digital input signals of the peripheral. For example, digital signal sources such as switches, buttons, or sensors on the peripheral will be mapped to logic device 1, which is responsible for reading these switch quantities (such as on / off states) and transmitting them to the data acquisition gateway. Logic device 2 is used to communicate with the analog input (AI) interface of the peripheral and read the analog input signals of the peripheral. Output signals from sensors on the peripheral, such as temperature, pressure, and flow, will be mapped to logic device 2, which is responsible for collecting these continuously changing analog data and sending them to the data acquisition gateway. Logic device 3 is used to communicate with the digital output (DO) and analog output (AO) interfaces of the peripheral and send control signals to the peripheral. For example, control modules such as relays, actuators, and valves on external devices can receive digital or analog signal commands through logic device 3 to realize operations such as switching and regulating the devices.

[0022] The splitting method can also be based on actual needs, either by groups or data sets. For example, an external device can be split into multiple groups or data sets, with each group or data set responsible for handling different functional modules of the external device, thereby achieving more granular management and control.

[0023] This application configures at least one protocol driver as multiple logical devices, and these multiple logical devices communicate with the same external device. By configuring the protocol driver as multiple logical devices, multiple logical devices can communicate with the same external device in parallel. Each logical device can handle different functional modules of the external device (such as different types of input and output signals), thereby processing more data in parallel within the same time, greatly improving the concurrency and efficiency of data acquisition. Especially when facing large-scale, complex industrial systems, this parallel processing significantly shortens the data acquisition cycle. By splitting and mapping the external device into multiple logical devices, it avoids a single logical device from bearing too many tasks. Each logical device focuses on the acquisition of specific functional modules (such as digital input, analog input, control output, etc.), thereby effectively balancing the load, avoiding overload of a single device, and improving resource utilization efficiency. Especially in high-frequency data acquisition scenarios, this configuration can reduce device latency and ensure the real-time performance of data acquisition. Each logical device can independently manage a specific functional module of the external device, achieving functional isolation. This independence means that when acquiring and controlling data from the external device, the operation of each module will not interfere with each other, and each module can be independently configured, debugged, and monitored as needed. For example, digital input modules, analog input modules, and control output modules can operate independently in different logic devices, ensuring the stability and accuracy of each function. By configuring multiple logic devices, the system can flexibly adapt to different functional requirements. For instance, if an external device needs to add a new functional module or undergo hardware expansion, only the new logic device needs to be added or reconfigured, without requiring large-scale modifications to the entire system. This flexible configuration greatly enhances the system's scalability, facilitating later maintenance and upgrades. By splitting the different functional modules of external devices into multiple logic devices, even if one logic device fails, it will not affect the normal operation of other modules. For example, if an analog input module fails, only that module's operation will be affected, not the normal operation of the digital input or output control module. This fault-tolerant design significantly improves system stability, ensuring system reliability under abnormal conditions. Since each logic device is associated with a specific functional module of the external device, device management and troubleshooting become simpler. If an external device malfunctions, the fault can be located to the specific logic device, allowing for rapid problem identification and corrective action. For example, if logic device 2 (analog input) malfunctions, that logic device can be checked individually without affecting the operation of the entire system. By breaking down the system into multiple logical devices, fine-grained control can be achieved for each functional module. For example, the data acquisition gateway can be configured with different acquisition cycles, sampling accuracies, and data processing strategies for different logical devices, enabling the system to achieve higher precision control and meet the high standards required in complex industrial environments.

[0024] In some examples, the data processing method provided in this application further includes: when the data acquisition gateway is connected to an external device via a serial bus, setting at least one protocol driver to communicate with multiple external devices; when the data point scale of any external device exceeds a preset data threshold and the external device supports concurrent communication, configuring multiple protocol driver instances of the same type for the external device based on the protocol driver corresponding to the external device. Protocol drivers can be categorized into different types such as Modbus master, OPC DA client, OPC UA client, and Ethernet / IP client. Each protocol driver can run multiple driver instances (e.g., it can run 100 Modbus master drivers simultaneously). Each protocol driver and external device (also called external system, external PLC, RTU, smart meter, etc., hereinafter collectively referred to as external device) can have a one-to-many relationship, a one-to-one relationship, or a many-to-one relationship. One-to-many typically refers to fieldbus protocols, such as Modbus. RTU refers to a protocol driver that can manage multiple external devices via an RS485 / RS422 serial bus. One-to-one means that one protocol driver corresponds to one external device, such as point-to-point communication. Many-to-one means that multiple protocol drivers of the same type correspond to one external device. This scenario is generally used when the data point scale of the external device is particularly large, or when the data performance requirements are high and the external device supports concurrent communication. Taking Modbus driver as an example, when the data point scale of the external device is 10,000 points (S), 4 (N) Modbus drivers can be configured, and each driver collects 2,500 points (S / N). The data acquisition performance can be improved by 3 times (N-1), and the data acquisition cycle is compressed to one-quarter (one-N) of that in one-to-one communication.

[0025] After acquiring heterogeneous protocol data, this application caches the acquired heterogeneous protocol data in a front-end sliding window buffer. Based on a pre-set back-end sliding window, it continuously retrieves heterogeneous protocol data from the front-end sliding window buffer and maps the acquired heterogeneous protocol data to a pre-set address space of an industrial data unified architecture. This application uses a front-end and back-end layered sliding window technology to decouple front-end reception and back-end processing. The front-end blocks external surges, thus providing the back-end with inherent risk isolation capabilities. The front-end is responsible for receiving raw heterogeneous protocol data and caching it in the front-end sliding window buffer; the back-end continuously retrieves the cached heterogeneous protocol data from the front-end sliding window buffer and performs message protocol parsing and processing. Therefore, external burst traffic is absorbed and rate-limited by the front-end, while the back-end obtains inherent risk isolation and stable input, thereby improving system throughput and reliability.

[0026] After mapping the acquired heterogeneous protocol data to the address space of the pre-set unified industrial data architecture, this application obtains the data service requirements of each client, and maps the data in the address space of the unified industrial data architecture to the address space of the external data service interface protocol according to the data service requirements, so that the client can access the data corresponding to the data service requirements based on the address space of the external data service interface protocol.

[0027] According to the above-mentioned scheme provided in this application, heterogeneous protocol data is acquired and cached in the front-end sliding window cache area; based on the pre-set back-end sliding window, heterogeneous protocol data is continuously acquired from the front-end sliding window cache area and mapped to the address space of the pre-set unified industrial data architecture; the data service requirements of each client are acquired, and the data in the address space of the unified industrial data architecture is mapped to the address space of the external data service interface protocol according to the data service requirements, so that the client accesses the data corresponding to the data service requirements based on the address space of the external data service interface protocol. The front-end receiving and back-end processing are decoupled, and the front-end blocks external surges, so the back-end has a natural risk isolation capability. The front end receives raw heterogeneous protocol data and caches it in a front-end sliding window buffer. The back end continuously retrieves cached heterogeneous protocol data from the front-end sliding window buffer and performs message protocol parsing and processing. Thus, external burst traffic is absorbed and rate-limited by the front end, while the back end gains natural risk isolation and stable input, thereby improving system throughput and reliability. Furthermore, the data acquisition gateway flexibly maps data to the address space of the external data service interface protocol according to each client's data service needs, achieving data fusion, interoperability, and sharing between different protocols. This solution not only improves data acquisition and processing efficiency and supports multi-protocol parallel operation but also provides customized data services based on client needs, enhancing system flexibility, scalability, and reliability. In addition, through standardized data mapping and precise data subset extraction, resource utilization is optimized, data management is simplified, and the system's real-time performance, reliability, and fault tolerance are improved, avoiding the problem in related technologies where data acquisition gateways cannot fuse heterogeneous protocol data.

[0028] Understandably, when the data point scale of external devices is large, configuring multiple protocol driver instances to work in parallel can significantly improve the concurrency and efficiency of data acquisition. For example, by configuring multiple Modbus driver instances for an external device with 10,000 data points, a large amount of data can be efficiently collected in a short time, significantly reducing the data acquisition cycle. By configuring multiple protocol driver instances, the data acquisition gateway can better distribute the load and avoid overloading a single protocol driver instance. Each driver instance is only responsible for collecting a portion of the data points, avoiding latency or performance degradation caused by a single protocol driver processing too many data points. This method is particularly suitable for external devices that need to process a large number of data points, such as industrial automation equipment and smart meters. This configuration effectively solves the challenges of high data volume and high concurrency data acquisition, meeting the needs of application scenarios with high data performance requirements. Since different protocol driver instances can be flexibly configured according to needs, the data acquisition gateway can easily adapt to future device expansion and changes in performance requirements. Whether increasing the number of external devices or dealing with a larger data point scale, the system can meet the needs through simple driver instance expansion. The configuration of multiple protocol driver instances improves the system's fault tolerance. Even if one protocol driver instance fails, other driver instances can continue to function normally, preventing system downtime or data loss and improving system reliability. Configuring multiple driver instances to work in parallel can significantly reduce the load and latency of a single driver instance, thereby greatly reducing data acquisition latency, especially when dealing with a large number of data points.

[0029] In some examples, such as Figure 2 As shown, the acquired heterogeneous protocol data is cached in the front-end sliding window cache area, including: S201. Based on the obtained heterogeneous protocol data and the heterogeneous protocol data stored in the front-end sliding window buffer, calculate the expected amount of data to be written to the front-end sliding window buffer. S202. If the expected amount of data written exceeds the target amount of data corresponding to the front-end sliding window buffer, delete the data at the head of the front-end sliding window buffer and cache the heterogeneous protocol data at the tail of the front-end sliding window buffer.

[0030] It is understandable that the front-end sliding window buffer uses a first-in-first-out (FIFO) circular buffer queue, arranging records in the order they are received. Each record contains a timestamp, source identifier, and original payload.

[0031] To ensure that the currently acquired heterogeneous protocol data can be accommodated and written completely to the front-end sliding window buffer at once, this application calculates the current data volume of the buffer based on the data volume of the heterogeneous protocol data and the heterogeneous protocol data in the buffer memory, and performs capacity verification. When the current data volume of the front-end sliding window buffer exceeds the target data volume after the heterogeneous protocol data is expected to be written, the earliest data entering the buffer is deleted from the front-end sliding window buffer in sequence according to the first-in-first-out strategy until the writing condition is met, and then the heterogeneous protocol data is written to the end of the buffer.

[0032] In other words, the expected amount of data written to the front-end sliding window buffer is equal to the sum of the amount of heterogeneous protocol data and the amount of heterogeneous protocol data stored in the front-end sliding window buffer. For example, if the amount of heterogeneous protocol data is set to X and the amount of heterogeneous protocol data stored in the front-end sliding window buffer is set to Y, then the expected amount of data written to the front-end sliding window buffer is Z = X + Y.

[0033] Then, the expected amount of data to be written is compared with the target amount of data corresponding to the front-end sliding window buffer (which is the maximum buffer size of the front-end sliding window buffer). If the expected amount of data to be written exceeds (is greater than or equal to) the target amount of data corresponding to the front-end sliding window buffer, the data at the head of the front-end sliding window buffer is deleted, and the heterogeneous protocol data is cached at the tail of the front-end sliding window buffer. If the expected amount of data to be written does not exceed the target amount of data corresponding to the front-end sliding window buffer, the heterogeneous protocol data is directly cached at the tail of the front-end sliding window buffer. For example, let the target amount of data be A, compare the expected amount of data to be written Z with the target amount of data A. If Z ≥ A, delete old data sequentially from the head of the front-end sliding window buffer until Z is less than A; if Z is less than A, directly cache the heterogeneous protocol data at the tail of the front-end sliding window buffer.

[0034] According to the scheme provided in the embodiments of this application, the "expected amount of data after writing" is calculated first and then the threshold is compared. If it exceeds the threshold, the old data is deleted sequentially from the head according to FIFO, avoiding cache overflow and process abnormality. The uncontrollable packet loss is transformed into a controllable orderly elimination, and the latest data and high-time information are retained first. After the capacity verification is completed (delete first and then write if necessary), the heterogeneous protocol data is written to the end of the cache in one go without splitting individual records, avoiding parsing errors caused by half-packet / packet merging, and improving data consistency. Pre-write rate limiting + FIFO elimination can absorb external traffic pulses, so that the backend can obtain stable input. Combined with the front-end and back-end layered sliding window, it forms a natural risk isolation, reducing back pressure transmission and jitter. The cache area is a FIFO circular buffer. Head deletion and tail writing are O(1) amortization operations, with small lock contention, which is suitable for high concurrency and high frequency sampling scenarios. The target data volume can be set according to the number of records / bytes / time window, and a safety margin can be left to adapt to different field loads. In conjunction with the "change update" and dynamic load balancing strategy, invalid data and resource occupation can be further reduced. FIFO semantics ensures data remains in chronological order, facilitating the backend sliding window's sequential parsing and mapping to the UDA address space, reducing rearrangement and compensation logic. Deletion counts / trigger counts can be statistically analyzed and alerted, facilitating threshold tuning and capacity planning, and improving system maintainability and reliability (MTBF).

[0035] In some examples, such as Figure 3 As shown, caching heterogeneous protocol data at the end of the front-end sliding window cache includes: S301. Perform edge preprocessing on heterogeneous protocol data, and filter the edge-preprocessed heterogeneous protocol data based on the change update strategy; S302. Cache the filtered heterogeneous protocol data at the end of the front-end sliding window cache area.

[0036] To ensure the validity, temporality, and parsability of heterogeneous protocol data cached in the front-end sliding window buffer, this application first performs edge preprocessing on the heterogeneous protocol data. Edge preprocessing includes, but is not limited to: (1) noise reduction / de-jittering (e.g., moving average or median filtering); (2) format normalization (unified units, precision, and field names); (3) timestamp alignment (aligned to fixed sampling boundaries); (4) outlier / duplicate packet removal (out-of-bounds values ​​and discarded duplicate frames); and (5) lightweight verification (sampling interval and source consistency verification). Preferably, only the above lightweight processing is implemented, without performing complete message protocol decoding. This achieves the technical effects of suppressing noise jitter, unifying data caliber, ensuring temporal order and recording atomicity, and reducing the back-end parsing burden.

[0037] Furthermore, to reduce invalid data volume and bandwidth / cache usage, this application filters edge-preprocessed heterogeneous protocol data based on a change update strategy. This filtering includes: acquiring at least one of the following: quality changes, timescale changes, and value changes (AI uses a threshold dead zone to determine value changes) of the edge-preprocessed heterogeneous protocol data. Then, based on these factors, if a quality change is acquired, and the quality identifier of the current data changes relative to the quality identifier of the most recently retained data (any transition between Good / Bad / Uncertain), it is considered a trigger, and the edge-preprocessed heterogeneous protocol data is retained. If a timescale change is acquired, and the current sampling timescale t is compared with the timescale of the most recently retained data... interval Greater than the preset threshold If the data crosses an alignment boundary (such as an integer second / minute), it is considered a trigger. The edge-preprocessed heterogeneous protocol data is then retained. If the measured value of the edge-preprocessed heterogeneous protocol data changes, and the difference between the measured value and the most recently retained value satisfies... (in, These are measurements of heterogeneous protocol data that have undergone edge preprocessing. The most recently retained measurement value. For absolute threshold, This is a relative threshold. To avoid zero values ​​causing relative computational instability (extremely small positive numbers), if a trigger condition is found, the edge-preprocessed heterogeneous protocol data is retained. In other words, if any of the above trigger conditions are met, the preprocessed heterogeneous protocol data is determined to be retained; if none of them are met, the heterogeneous protocol data is discarded.

[0038] According to the scheme provided in the embodiments of this application, edge preprocessing is performed on heterogeneous protocol data, and the preprocessed heterogeneous protocol data is filtered based on a change update strategy. The filtered heterogeneous protocol data is cached at the end of the front-end sliding window buffer. By performing edge preprocessing first and then filtering according to "quality / timescale / value change (dead zone / hysteresis)," the amount of duplicate and small jitter data can be significantly reduced, reducing link and cache usage. Under stable operating conditions, a high compression ratio (e.g., over 90%) can be achieved, for example. Only data with "meaningful changes" is written to the front-end sliding window buffer, reducing the amount of data that the back-end needs to parse and map, decreasing CPU / memory usage, and improving overall throughput and concurrency. The filtered records are enqueued in the order of reception using a FIFO tail write method, with each record written completely at once, avoiding partial packets / out-of-order delivery and facilitating back-end sequential decoding and UDA mapping. Change filtering + sliding window allows the front-end to absorb external pulse traffic first, decoupling the back-end from acquisition, reducing the risk of blocking and packet loss, and forming a natural risk isolation layer. Filtered data at the edge can be directly subscribed to / read via local data services (such as OPC UA, substation services), reducing link round trips and pressure on the central side. Parameters such as dead zone threshold, hysteresis, heartbeat guarantee, time-stamped threshold, and target data volume can be flexibly configured according to point / protocol, adapting to multiple scenarios of continuous, discrete, and event-driven data volumes. Metrics such as retain / discard count, threshold trigger count, and window utilization can be measured and alerted, facilitating threshold tuning, capacity planning, and fault location. The front end only performs lightweight preprocessing and change detection, without complete packet decoding, making it universal for heterogeneous protocols and easily expandable to more protocols / rules in the future. In conjunction with the target data volume threshold and FIFO head deletion strategy, it ensures that the "delete the head first, write the tail later" principle does not exceed the limit, guaranteeing stable cache operation.

[0039] In some examples, such as Figure 4 As shown, obtaining heterogeneous protocol data includes: S401. Determine the protocol processing complexity and current operating conditions for each protocol-driven protocol. S402. Determine the thread resources corresponding to each protocol driver based on the protocol processing complexity and the current working conditions, and obtain the heterogeneous protocol data corresponding to each protocol driver based on the thread resources.

[0040] In some examples, this application pre-sets the protocol processing complexity corresponding to each protocol driver. Subsequently, by determining the protocol processing complexity and current working conditions of each protocol driver, the application determines the thread resources corresponding to each protocol driver based on the protocol processing complexity and current working conditions, and obtains the heterogeneous protocol data corresponding to each protocol driver based on the thread resources.

[0041] In other words, this application introduces dynamic load balancing technology and an adaptive thread pool scheduling algorithm to dynamically adjust resources based on protocol processing complexity and current operating conditions. Protocols with higher processing complexity have higher γ values, where γ = number of threads / number of protocols, representing the thread occupancy rate of the protocols; 1 indicates one protocol has one dedicated thread, and 0.5 indicates two protocols share one thread. Current operating conditions refer to the allocation of thread resources to devices capable of normal communication.

[0042] According to the solution provided in the embodiments of this application, thread resources are generally prioritized for devices that can communicate normally, thereby improving system throughput and stability while ensuring the continuity of data collection.

[0043] In some examples, such as Figure 5 As shown, the acquired heterogeneous protocol data is mapped to the address space of a pre-set unified industrial data architecture, including: S501. Parse the heterogeneous protocol data to obtain heterogeneous parsed data; S502. Obtain the first mapping specification corresponding to each protocol driver instance, and map the heterogeneous parsing data to the address space of the unified industrial data architecture according to the first mapping specification.

[0044] First, after the data acquisition gateway obtains heterogeneous protocol data through the backend sliding window, it parses the heterogeneous protocol data corresponding to each data acquisition interface according to the protocol driver instance corresponding to the heterogeneous protocol data to obtain heterogeneous parsed data.

[0045] For example, suppose the data acquisition gateway acquires Modbus data frames through a backend sliding window. These Modbus data frames are acquired based on the Modbus protocol driver. The driver instance corresponding to the Modbus protocol driver parses the Modbus protocol data frames, extracting numerical values, status information, etc. This information constitutes heterogeneous parsed data. For external devices using the OPC UA protocol, the protocol driver instance will parse the OPC UA message format, extracting relevant data nodes and attributes to obtain the corresponding heterogeneous parsed data.

[0046] After parsing the heterogeneous protocol data, the data acquisition gateway will map the parsed data to the address space of the unified industrial data architecture according to the mapping specifications of each protocol driver instance.

[0047] Each protocol driver instance corresponds to a specific mapping specification, which defines how to convert parsed data into a format and location under a unified architecture. For example, parsed data from the Modbus protocol may need to be mapped to a specific node in the OPC UA address space, or mapped according to predefined address mapping rules in an industry-standard unified architecture.

[0048] Taking the Modbus protocol as an example, suppose the register data of this protocol needs to be mapped to a node of OPC UA. Through the first mapping specification driven by the protocol, the data acquisition gateway maps the data parsing results of Modbus to the node ID and attributes defined by the OPC UA protocol, forming a unified data structure, which is stored in the address space of the unified industrial data architecture.

[0049] For the OPC UA protocol, the protocol driver instance maps the parsed data points to the specified address nodes according to the OPC UA address space rules, enabling the data acquisition gateway to store and provide data access in a standardized manner.

[0050] The aforementioned approach in this application simplifies and standardizes access to heterogeneous protocol data through a unified industrial data architecture, eliminating the need for users to concern themselves with specific protocol details. The data acquisition gateway can flexibly adapt to external devices using different protocols and quickly adjust interfaces and data structures through mapping specifications, enhancing system compatibility and scalability. The standardized mapping process ensures that data generated by different protocols and devices is stored and managed consistently, avoiding inconsistencies in data format and location. Through protocol-driven instance mapping specifications, the data acquisition gateway can efficiently convert heterogeneous data from external devices into a unified format and store it in a designated address space, optimizing the data processing flow.

[0051] In some examples, such as Figure 6 As shown, data within the address space of the unified industrial data architecture is mapped to the address space of the external data service interface protocol according to data service requirements, including: S601. Based on data service requirements, determine the target data subset corresponding to each client from the address space of the unified industrial data architecture; S602. Obtain the second mapping specification corresponding to each external data service interface protocol, and map the target data subset to the address space of the corresponding external data service interface protocol according to the second mapping specification.

[0052] First, the data acquisition gateway receives data service requests from clients. These requests are based on the client's actual application needs and may include the need to acquire specific types of data (e.g., temperature, pressure, flow rate, etc.) or specific data fields (e.g., data from certain sensors, device status, etc.).

[0053] The data acquisition gateway analyzes and determines the data service requirements that need to be provided to a client based on the client's actual application needs. For example, a client may only need to obtain temperature and humidity data, or only need operational status data of certain specific devices.

[0054] After clarifying the client's data service requirements, the data acquisition gateway extracts the target data subset corresponding to those requirements from the address space of the unified industrial data architecture. This target data subset may include multiple data points and related equipment data. The data acquisition gateway will then search the unified architecture and return data that meets these requirements.

[0055] For example, assuming the client only cares about temperature and humidity data, the data acquisition gateway will extract a subset of temperature and humidity-related data from the address space of the unified industrial data architecture (e.g., obtain data from relevant nodes in sensor devices) according to the client's data service requirements.

[0056] Once the target data subset is determined, the data acquisition gateway will perform a data mapping process according to the selected outbound data service interface protocol. This process mainly involves converting the extracted data from the address space of the unified industrial data architecture to the format and address space required by the outbound protocol.

[0057] Specifically, based on the external data service interface protocol (such as OPC UA, Modbus, 104 protocol, etc.), the data acquisition gateway obtains the second mapping specification related to that external data service interface protocol. The mapping specification defines how to convert data in the unified architecture into the format required by the external protocol and map it into the protocol's address space. According to the second mapping specification, the data acquisition gateway maps a subset of the target data into the address space of the external protocol. Through this mapping, data can be interoperated and shared between different external data service interface protocols.

[0058] To better understand this application, this embodiment provides a more specific example for illustration. This application performs backend mapping and frontend mapping separately. Backend mapping: Model conversion on the data acquisition side. Backend mapping is the process of mapping data from different protocols to a unified industrial data architecture address space, which usually occurs on the data acquisition side. The specific steps are as follows: The data acquisition gateway communicates with external devices through multiple data acquisition interfaces (such as Modbus, OPC DA, Ethernet / IP, etc.) to obtain data from heterogeneous protocols. Due to the differences between these protocols, the data acquisition gateway needs to parse the data according to the standard of each protocol. During the parsing process, the data acquisition gateway extracts the data from each protocol (e.g., Modbus register data) and converts it into a unified format.

[0059] The parsed data is mapped into the unified address space of OPC UA. As an industry standard protocol, OPC UA provides data modeling specifications to ensure that all data can be stored and managed in a standardized format within a unified address space.

[0060] For example, certain register data of the Modbus protocol is mapped to specific node IDs in the OPC UA address space, ensuring that Modbus data can be accessed in OPC UA in a uniform format.

[0061] Front-end mapping: This involves mapping data to external data service protocols. Building upon the back-end mapping, front-end mapping maps data from the OPC UA address space to the address space of the external data service interface protocol (such as 104 protocol, Modbus TCP, MQTT, etc.). This process ensures that clients can access the required data via the external protocol. Based on the client's data service requirements, the data acquisition gateway first determines the target data subset to be provided to the client. These requirements are typically based on the client's application needs, such as needing only specific types of data (e.g., temperature, pressure, flow rate), or only data from certain sensors. The data acquisition gateway extracts and identifies these data subsets from the unified architecture's OPC UA address space. According to the requirements of the external data service protocol (e.g., OPC UA, Modbus, 104 protocol, etc.), the data acquisition gateway maps the extracted data subset to the address space of the external protocol. Each external protocol may have different formats and node requirements; the data acquisition gateway converts the data into the format required by the target protocol according to pre-defined mapping specifications.

[0062] For example, if the outbound protocol is OPC UA, the data acquisition gateway maps the target data subset to the OPC UA node ID, ensuring that the client can access this data through the OPC UA protocol. If the outbound protocol is 104 protocol, the data acquisition gateway maps the OPC UA node to a specific "InfoAddr" of the 104 substation, allowing the 104 master station to access this data through the 104 protocol.

[0063] For example, suppose the data acquisition gateway collects data from a Modbus external device. This external device contains multiple data points, such as temperature and humidity data. In the backend mapping, the data acquisition gateway parses and maps the Modbus protocol data (e.g., register data for temperature and humidity) to the nodes / Modbus / Temperature and / Modbus / Humidity in the unified address space of OPC UA. In the frontend mapping, if the outgoing protocol is OPC UA, the data acquisition gateway maps these nodes to the node addresses required by the OPC UA protocol, thus enabling clients to access this data via the OPC UA protocol.

[0064] If the outgoing protocol is the 104 protocol, the data acquisition gateway will map the node data of the OPC UA (such as / Modbus / Temperature) to the "information address" (such as InfoAddr = 0x1001) of the 104 substation, so that the 104 master station can obtain this data through the 104 protocol.

[0065] The solution provided in this application allows the data acquisition gateway to flexibly select and map different data subsets based on the data service needs of different clients, meeting the specific data requirements of different clients and thus achieving differentiated access between clients, avoiding redundant data transmission. By determining and mapping target data subsets based on service requirements, the data acquisition gateway can reduce unnecessary data transmission, providing only the data actually needed, optimizing network bandwidth utilization, and improving data access efficiency. Through mapping specifications, the data acquisition gateway can convert data in the unified architecture into data formats that conform to the requirements of different external data service interface protocols (such as OPC UA, Modbus, MQTT, etc.), ensuring compatibility between protocols and facilitating data sharing and exchange between different systems and devices. This method allows flexible support for multiple protocol types and different client access requirements, requiring only adjustments to the data mapping specifications without changing the underlying data architecture, enhancing the system's scalability and flexibility. By centrally managing a unified industrial data architecture, the data acquisition gateway can effectively control and monitor the access requirements of all clients, ensuring that data from different protocols can be processed and forwarded under a unified standard, simplifying the entire data processing process.

[0066] According to the technical solution provided in this application, heterogeneous protocol data is acquired through multiple data acquisition interfaces, and the heterogeneous protocol data is mapped to the address space of a pre-set unified industrial data architecture. The data service requirements of each client are acquired, and the data within the address space of the unified industrial data architecture is mapped to the address space of the external data service interface protocol based on these requirements. This allows the client to access the data corresponding to the data service requirements based on the address space of the external data service interface protocol. This application acquires heterogeneous protocol data through multiple data acquisition interfaces and maps it to a unified industrial data architecture address space. The data acquisition gateway flexibly maps data to the address space of the external data service interface protocol according to the data service requirements of each client, realizing data fusion, interoperability, and sharing between different protocols. This solution not only improves data acquisition and processing efficiency and supports parallel operation of multiple protocols, but also provides customized data services according to client needs, enhancing the system's flexibility, scalability, and reliability. Furthermore, through standardized data mapping and precise data subset extraction, resource utilization is optimized, data management is simplified, and the system's real-time performance, reliability, and fault tolerance are improved, avoiding the problem in related technologies where data acquisition gateways cannot fuse heterogeneous protocol data.

[0067] Taking Modbus substations as data acquisition gateways as an example, Modbus substation services support extremely flexible deployment methods of N multiplied by M, where N is the number of processes and M is the number of drivers that each process can contain. For example, if N is 1, it is the conventional single-process mode; if N≥2 and M is 1, and it is also a substation service, it is process-level substation service isolation, which provides the highest level of security.

[0068] The Modbus RTU protocol / Modbus RTU over TCP protocol / Modbus TCP protocol has the following characteristics: a maximum of 2000 DI points (function code 1 / 2) per frame; a maximum of 125 AI points (function code 3 / 4) per frame; and a maximum of 123 A0 points (function code 16) per frame.

[0069] The Modbus TCP Extend protocol has the following characteristics: A single frame can contain a maximum of 32766 * 16 (function code 3 / 4, one register contains 16 bits, each bit represents one DI) = 524,256, or 520,000. A single frame can contain a maximum of 32766 AI points (generally one AI point per register).

[0070] Taking function code 3 as an example, the Modbus TCP Extend request message structure is shown in Table 1 below: Table 1: Modbus TCP Extend Request Message Structure The Modbus TCP Extend response message structure is shown in Table 2 below: The substation services of Modbus RTU, Modbus TCP, Modbus RTU over TCP, and Modbus TCP Extend are collectively referred to as Modbus protocol family substation services (abbreviated as Modbus substation service, Modbus substation, substation).

[0071] When HSM-IO provides data access services as a 104 substation, front-end mapping is required. The external 104 master station accesses data based on the 104 protocol specification. Modbus substations support four modes: single-substation single-tenant, single-substation multi-tenant, multi-substation single-tenant, and multi-substation multi-tenant. Single-substation single-tenant is suitable for simple application scenarios; single-substation multi-tenant is suitable for scenarios with clients of the same level; multi-substation single-tenant is suitable for concurrent scenarios with a single client; and multi-substation multi-tenant is suitable for single-channel communication or multi-channel concurrent communication between the same client and clients of different levels. Each Modbus substation instance uses its own configuration file, which is in XML (or JSON) format and includes a whitelist policy.

[0072] The Modbus substation implements the following solution for client connections: ① Supports deploying multiple Modbus substation services within a single data acquisition service process. Each substation service must be configured with a different listening port, such as 502, 503, ... , denoted as Substation 1, Substation 2, ... respectively.

[0073] ② For any of the aforementioned listening ports, the Modbus substation service (let's call it substation 1) only supports one client connection for the same client IP address, and adopts the old-to-new principle: For example, suppose the Modbus master station with client IP1=192.168.0.1 already has a TCP connection with substation 1 (let's call this connection TCP1). When substation 1 receives a new client connection TCP2 (let's call its IP2), if IP2=IP1, then substation 1 will disconnect and delete TCP1, and switch to communicating with IP1 through TCP2. The implicit meaning of this mechanism is that when client IP1 initiates a new connection, it is implying that it no longer wants, cares about, or wants to manage the old connection, or has lost control over it. In this case, substation 1's old-to-new principle is consistent with its behavior and is a "reciprocal reward" for the client's behavior. In addition, the old-to-new principle allows clients to handle complex problems in a simple way.

[0074] ③ Substation 1 updates data periodically and responds to requests from the Modbus master station. Specifically, when the receiving substation identifies a Modbus TCP Extend message from the master station, the criteria for determining if it is a Modbus TCP Extend message are: the Byte Count field of the PDU is 0, and the Length field of the Header is valid, then a response is processed. A valid Length field in the Header means: 1) for function codes 1 and 2, the Length does not exceed 8199; 2) for function codes 3 and 4, the Length does not exceed 65529; 3) for function code 15, the Length does not exceed 8199; 4) for function code 16, the Length does not exceed 65525; 5) except for the restrictions of 1) to 4) above (the maximum Length obtained is denoted as LMax1), the maximum Length specified by the substation configuration file (the configured maximum Length is denoted as LMax2) is the smaller of LMax1 and LMax2. The reason is: considering factors such as system resource limitations and system stability.

[0075] Typical application scenarios supported by the above solution: 1. Single-device, single-vendor, single-client access: This means the vendor obtains all data from the sub-site through a single client. This is the simplest application scenario, with no deployment issues.

[0076] 2. Single-device, single-vendor client access: This implies that the data for each client's corresponding substation is different, employing ① adaptation. N substations are deployed with different listening ports, each accessing one of the N clients.

[0077] 3. Single-machine access for M*N clients from M vendors: The same client requires different substation data, but different clients may require the same or different substation data. The adaptation method (①) is still used. Deploy M*N substations with different listening ports, each connecting to one of the M*N clients. Advantages: Facilitates fault isolation and data isolation between clients.

[0078] 4. Other complex scenarios: Such as a pair of redundant dual machines, multiple pairs of redundant dual machines, independent multiple machines, etc., can all be decomposed into scenarios 1, 2, and 3 above. The redundant standby master station, according to Modbus industry practice, maintains the connection only with a TEST heartbeat (usually agreed to be any one of functions 1, 2, 3, 4, 5, 6, 16, using short messages; a simple message means not requesting or writing a large amount of data). The standby master station typically does not read data.

[0079] Specifically, assume that the external device has 100,000 data points, of which one external device provides 40,000 data points using the Modbus communication protocol, supporting concurrent communication.

[0080] Examples of dual remote points for external devices are shown in Table 3 below: Table 3, Examples of Dual Remote Points for External Devices Examples of dual remote points for external devices corresponding to the driver are shown in Table 4 below: Table 4: Example Table of Dual Remote Points for Drivers Corresponding to External Devices Table 5 below shows examples of Modbus substation operating modes: Table 5: Examples of Modbus Substation Working Modes The data acquisition gateway provided in this application is divided into front-end and back-end. The back-end uses various drivers of the data acquisition gateway to complete data acquisition from the field-side PLC and update it to the internal real-time database data model. The front-end provides data access services based on industrial standard protocols. This front-end / back-end separation model ensures that front-end data access and back-end data acquisition are independent and do not cause blockages. Each back-end driver operates independently, achieving fault isolation between drivers. A fault in one driver will not affect the operation of other drivers or the front-end data access service. Unified data modeling maps heterogeneous data from different southbound protocols, such as Modbus registers, 104 information addresses, MQTT payloads, Siemens S7 blocks, and ODBC tables, to OPC UA object nodes in a unified manner at the message parsing layer of each driver, achieving standardized storage and output. A single substation encoding can operate in four substation service modes: Modbus RTU, Modbus TCP, Modbus RTU over TCP, and Modbus TCP Extend. Among these, Modbus TCP Extend expands the single-frame data capacity to a maximum of 262 times that of Modbus TCP. Specifically, during runtime, the substation can dynamically switch between Modbus TCP and Modbus TCPExtend modes for any client. When the substation detects a Modbus TCP Extend message from the master station, it automatically switches to Modbus TCP Extend mode. The criteria for judging a Modbus TCP Extend message are: if the Byte Count field of the PDU is 0 and the Length field of the Header is valid, then an acknowledgment is processed. A valid Length field in the Header means: 1) for function codes 1 and 2, the Length does not exceed 8199; 2) for function codes 3 and 4, the Length does not exceed 65529; 3) for function code 15, the Length does not exceed 8199; 4) for function code 16, the Length does not exceed 65525; 5) except for the restrictions in 1) to 4) above (the maximum Length obtained is denoted as LMax1), the maximum Length specified in the substation configuration file (the configured maximum Length is denoted as LMax2) will ultimately be the smaller of LMax1 and LMax2. This is because of a comprehensive consideration of factors such as system resource limitations and system stability.

[0081] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.

[0082] Based on the same concept, this application also provides a data processing apparatus, such as... Figure 7 As shown, the data processing device includes: The data acquisition module 701 is used to acquire heterogeneous protocol data and cache the acquired heterogeneous protocol data in the front-end sliding window cache area; The backend mapping module 702 is used to continuously slide from the frontend sliding window cache to obtain heterogeneous protocol data based on a pre-set backend sliding window, and map the obtained heterogeneous protocol data to the address space of the pre-set industrial data unified architecture. The front-end mapping module 703 is used to obtain the data service requirements of each client and map the data in the address space of the unified industrial data architecture to the address space of the external data service interface protocol according to the data service requirements, so that the client can access the data corresponding to the data service requirements based on the address space of the external data service interface protocol.

[0083] In some examples, the data acquisition module 701 is also used to calculate the amount of data required to write to the front-end sliding window buffer based on the acquired heterogeneous protocol data and the heterogeneous protocol data stored in the front-end sliding window buffer; if the amount of data required to write exceeds the target data amount corresponding to the front-end sliding window buffer, the data at the head of the front-end sliding window buffer is deleted and the heterogeneous protocol data is cached at the tail of the front-end sliding window buffer.

[0084] In some examples, the data acquisition module 701 is also used to perform edge preprocessing on heterogeneous protocol data, filter the edge-preprocessed heterogeneous protocol data based on the change update strategy, and cache the filtered heterogeneous protocol data at the end of the front-end sliding window cache area.

[0085] In some examples, the data acquisition module 701 is also used to determine the protocol processing complexity and current operating condition of each protocol driver; determine the thread resources corresponding to each protocol driver based on the protocol processing complexity and current operating condition; and acquire the heterogeneous protocol data corresponding to each protocol driver based on the thread resources.

[0086] In some examples, the backend mapping module 702 is also used to parse the heterogeneous protocol data corresponding to each data acquisition interface to obtain heterogeneous parsed data; obtain the first mapping specification corresponding to each protocol driver instance, and map the heterogeneous parsed data to the address space of the unified industrial data architecture according to the first mapping specification.

[0087] In some examples, the front-end mapping module 703 is also used to determine the target data subset corresponding to each client from the address space of the unified industrial data architecture based on data service requirements; obtain the second mapping specification corresponding to each external data service interface protocol; and map the target data subset to the address space of the corresponding external data service interface protocol according to the second mapping specification.

[0088] In some examples, each data acquisition interface corresponds to a different protocol driver, which includes at least one of the following: Modbus master driver, OPC data access client driver, OPC unified architecture client driver, and Ethernet industrial protocol client driver.

[0089] According to the solution provided in the embodiments of this application, heterogeneous protocol data is obtained and cached in the front-end sliding window cache area. Based on the pre-set back-end sliding window, heterogeneous protocol data is continuously obtained from the front-end sliding window cache area and mapped to the address space of the pre-set unified industrial data architecture. The data service requirements of each client are obtained, and the data in the address space of the unified industrial data architecture is mapped to the address space of the external data service interface protocol according to the data service requirements. This allows the client to access the data corresponding to the data service requirements based on the address space of the external data service interface protocol. The front-end receiving and back-end processing are decoupled, and the front-end blocks external surges, thus giving the back-end a natural risk isolation capability. The front end receives raw heterogeneous protocol data and caches it in a front-end sliding window buffer. The back end continuously retrieves cached heterogeneous protocol data from the front-end sliding window buffer and performs message protocol parsing and processing. Thus, external burst traffic is absorbed and rate-limited by the front end, while the back end gains natural risk isolation and stable input, thereby improving system throughput and reliability. Furthermore, the data acquisition gateway flexibly maps data to the address space of the external data service interface protocol according to each client's data service needs, achieving data fusion, interoperability, and sharing between different protocols. This solution not only improves data acquisition and processing efficiency and supports multi-protocol parallel operation but also provides customized data services based on client needs, enhancing system flexibility, scalability, and reliability. In addition, through standardized data mapping and precise data subset extraction, resource utilization is optimized, data management is simplified, and the system's real-time performance, reliability, and fault tolerance are improved, avoiding the problem in related technologies where data acquisition gateways cannot fuse heterogeneous protocol data.

[0090] Figure 8 This is a schematic diagram of the electronic device 8 provided in an embodiment of this application. For example... Figure 8 As shown, the electronic device 8 of this embodiment includes a processor 801, a memory 802, and a computer program 803 stored in the memory 802 and executable on the processor 801. When the processor 801 executes the computer program 803, it implements the steps in the various method embodiments described above. Alternatively, when the processor 801 executes the computer program 803, it implements the functions of each module / unit in the various data processing device embodiments described above.

[0091] Electronic device 8 can be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 8 may include, but is not limited to, processor 801 and memory 802. Those skilled in the art will understand that... Figure 8 This is merely an example of electronic device 8 and does not constitute a limitation on electronic device 8. It may include more or fewer components than shown, or different components.

[0092] The processor 801 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0093] The memory 802 can be an internal storage unit of the electronic device 8, such as a hard disk or RAM of the electronic device 8. The memory 802 can also be an external storage device of the electronic device 8, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc., equipped on the electronic device 8. The memory 802 can also include both internal and external storage units of the electronic device 8. The memory 802 is used to store computer programs and other programs and data required by the electronic device.

[0094] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the data processing device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0095] If an integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium may include: any entity or data processing device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in a computer-readable medium can be appropriately added or removed according to regional requirements and patent practice requirements. For example, in some regions, according to regional requirements and patent practice, a computer-readable medium may not include electrical carrier signals and telecommunication signals.

[0096] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A data processing method, characterized in that, The data processing method is applied to a data acquisition gateway, and the method includes: Obtain heterogeneous protocol data and cache the obtained heterogeneous protocol data in the front-end sliding window cache area; Based on a pre-set backend sliding window, heterogeneous protocol data is continuously obtained from the frontend sliding window cache, and the obtained heterogeneous protocol data is mapped to the address space of a pre-set unified industrial data architecture. The system acquires the data service requirements of each client, and maps the data in the address space of the unified industrial data architecture to the address space of the external data service interface protocol based on the data service requirements, so that the client can access the data corresponding to the data service requirements based on the address space of the external data service interface protocol.

2. The method according to claim 1, characterized in that, The acquired heterogeneous protocol data is cached in the front-end sliding window cache area, including: Based on the obtained heterogeneous protocol data and the heterogeneous protocol data stored in the front-end sliding window buffer, calculate the amount of data required to write to the front-end sliding window buffer; If the amount of data required for writing exceeds the target amount of data corresponding to the front-end sliding window buffer, then delete the data at the beginning of the front-end sliding window buffer and cache the heterogeneous protocol data at the end of the front-end sliding window buffer.

3. The method according to claim 2, characterized in that, Cache the heterogeneous protocol data at the end of the front-end sliding window cache area, including: The heterogeneous protocol data is preprocessed at the edge, and the preprocessed heterogeneous protocol data is then filtered based on a change update strategy. The filtered heterogeneous protocol data is cached at the end of the front-end sliding window cache area.

4. The method according to claim 1, characterized in that, Acquiring heterogeneous protocol data includes: Determine the protocol processing complexity and current operating conditions for each protocol-driven protocol; Based on the protocol processing complexity and the current operating conditions, determine the thread resources corresponding to each protocol driver, and obtain the heterogeneous protocol data corresponding to each protocol driver based on the thread resources.

5. The method according to claim 4, characterized in that, The acquired heterogeneous protocol data is mapped to the address space of a pre-defined unified industrial data architecture, including: Parse the heterogeneous protocol data to obtain heterogeneous parsed data; Obtain the first mapping specification corresponding to each protocol driver instance, and map the heterogeneous parsing data to the address space of the unified industrial data architecture according to the first mapping specification.

6. The method according to claim 1, characterized in that, Mapping data within the address space of the unified industrial data architecture to the address space of the external data service interface protocol according to the data service requirements includes: Based on the data service requirements, the target data subset corresponding to each client is determined from the address space of the unified industrial data architecture; Obtain the second mapping specification corresponding to each of the aforementioned data transmission service interface protocols, and map the target data subset to the address space of the corresponding data transmission service interface protocol according to the second mapping specification.

7. The method according to claim 4, characterized in that, The data acquisition gateway is equipped with multiple data acquisition interfaces, each of which corresponds to a different protocol driver. The protocol driver includes at least one of the following: Modbus master driver, OPC data access client driver, OPC unified architecture client driver, and Ethernet industrial protocol client driver.

8. A data processing apparatus, characterized in that, The data processing device includes: The data acquisition module is used to acquire heterogeneous protocol data and cache the acquired heterogeneous protocol data in the front-end sliding window cache area; The backend mapping module is used to continuously slide and obtain heterogeneous protocol data from the frontend sliding window cache based on a pre-set backend sliding window, and map the obtained heterogeneous protocol data to the address space of a pre-set unified industrial data architecture. The front-end mapping module is used to obtain the data service requirements of each client, and map the data in the address space of the unified industrial data architecture to the address space of the external data service interface protocol according to the data service requirements, so that the client can access the data corresponding to the data service requirements based on the address space of the external data service interface protocol.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images