Device protocol and interpreter dynamic adaptation method and adaptation system based on DRL

Abstract

The invention aims to solve the problem that a large amount of resources of edge nodes are consumed when terminal equipment accesses due to the fact that an existing protocol combination mode of the edge nodes is not flexible enough. According to the DRL-based equipment protocol and interpreter dynamic adaptation method and system, dynamic protocol adaptation between cloud-side-end is achieved, data services needing to be provided by terminal equipment are negotiated in a dialogue mode, all protocol stacks do not need to be preloaded, protocol resident resource occupation of edge nodes is reduced, and the service performance of the edge nodes is improved. The communication protocol combination of the edge node and the terminal equipment can be optimized, and the resource consumption of the edge node is further reduced.

Description

A method and system for dynamic adaptation of device protocols and interpreters based on DRL Technical Field

[0001] This invention relates to the field of IoT device access and communication protocol management, and in particular to a dynamic adaptation method and system for device protocols and interpreters based on DRL. Background Technology

[0002] With the booming development of emerging fields such as the Industrial Internet, smart cities, and intelligent manufacturing, the application scenarios of the Internet of Things (IoT) continue to expand, and massive heterogeneous IoT devices are widely accessing networks through edge nodes. However, there are significant differences between different manufacturers and types of terminal devices in terms of communication paradigms, protocol types, data format architectures, and computing power. As the scale of device access increases daily, the difficulty of adapting device data parsing and the complexity of service configuration rise simultaneously. To address this issue, a cloud-edge-device collaborative access negotiation technology solution has been proposed. Among them, the patent with authorization announcement number CN118921276B discloses a negotiation method. This technology achieves a significant improvement in cloud center access efficiency and refined configuration of network resources by systematically planning the access timing and access strategy of terminal devices to the cloud center.

[0003] Despite the progress made in related technological explorations, current mainstream IoT device access methods still have many limitations. Whether it's standardized device description technology, plug-and-play technology, or service discovery technology, all rely on fixed communication protocols, standardized hardware interfaces, and lengthy and cumbersome information modeling systems, making it difficult to adapt to the flexible and ever-changing protocol types and dynamically iterating business requirements of IoT scenarios. Meanwhile, with the diversification of terminal device functions, a single device often supports multimodal communication methods to meet service delivery needs in different scenarios. In scenarios with large-scale IoT device cluster deployments, edge nodes typically adopt a "performance-first" protocol combination strategy to ensure stable operation and functional coverage of services across the entire domain. This includes strategies such as simultaneously deploying multiple protocol parsing libraries and integrating multi-format message decoders for full compatibility. While this strategy maximizes device compatibility, it leads to excessive consumption of core resources such as computing power, storage, and bandwidth at the edge nodes. This not only significantly increases the hardware deployment cost of edge nodes but also poses severe performance and operational challenges to already resource-constrained edge nodes when the scale of device access expands. Summary of the Invention

[0004] The purpose of this invention is to address the problem that existing IoT communication protocols between edge nodes and terminal devices are not flexible enough, leading to increased resource consumption at the edge nodes. This invention provides a dynamic adaptation method and system for device protocols and interpreters based on DRL, which dynamically selects the optimal protocol combination to reduce edge resource consumption while ensuring service quality.

[0005] To address the aforementioned problems, this invention employs the following technical solution: a dynamic adaptation method for device protocols and interpreters based on DRL, comprising the following steps: Step 1: The terminal device and the edge node determine the services required by the terminal device through dialogue; Step 2: The edge node uploads the dialogue content with the terminal device to the cloud, where the cloud constructs a DRL state vector based on the dialogue content, forming a DRL proxy; Step 3: An action space is defined, and different communication protocols and data interpreters are dynamically combined within the action space; Step 4: Based on a reward function set according to the associated service quality and resource consumption, the DRL proxy in the cloud selects the protocol combination that meets the service requirements and consumes the least resources; Step 5: The cloud's protocol library and interpreter library respectively load the selected communication protocol and interpreter and return them to the negotiation proxy of the edge node; Step 6: The negotiation proxy sends the selected communication protocol to the terminal device; Step 7: The terminal device confirms the communication protocol and configures the communication service.

[0006] Furthermore, the edge node is connected to the endpoint device via a hardware interface.

[0007] Furthermore, the dialogue between the terminal device and the edge node in step 1 also includes the basic device information of the terminal device, which is used to identify and distinguish different terminal devices.

[0008] Furthermore, the DRL state vector constructed in step 2 includes the edge node's CPU utilization, memory usage, service latency, service completion rate, and bandwidth usage.

[0009] Furthermore, the reward function in step 4 is defined as two parts: one part is a positive reward if the service demand is met; the other part is a negative reward if the resource consumption is higher.

[0010] Furthermore, in step 5, the edge node receives the communication protocol and interpreter sent by the cloud, loads the selected protocol or interpreter, and unloads unnecessary protocol modules.

[0011] Furthermore, in the initial stage of step 7, after completing the communication configuration of the terminal device, the terminal device first selects a high-performance OPC UA protocol to communicate with the edge node, requiring that the set service latency and service semantic requirements be met, and then gradually optimizes it.

[0012] Furthermore, after step 7 completes the communication configuration of the terminal device, the DRL agent in the cloud senses the resource consumption of the edge node and the service semantics and service latency fed back by the cloud in real time, and uses the DDQN model to optimize the combination of communication protocol and interpreter.

[0013] Furthermore, the communication protocols in step 3 include MQTT, CoAP, HTTP, and proprietary protocols; the decoding formats of the data interpreter include JSON, ProtoBuf, and Binary.

[0014] A dynamic device protocol and interpreter adaptation system based on DRL, based on the aforementioned adaptation method, comprises: a cloud: a central node responsible for global management and centralized data processing, communicating with all edge nodes; edge nodes: connected to on-site terminal devices via a digital network and connected to the cloud via a wireless or wired network; terminal devices: located at the network edge, used to collect data and perform preliminary processing; a protocol library: stored in the cloud, containing a large number of communication protocols used between edge nodes and terminal devices, forming the basis for establishing connections between edge nodes and devices; and an interpreter library: located in the cloud, storing a large number of interpretation rules for converting raw data from terminal devices into structured data. The system is used to convert raw data from terminal devices into structured data to provide data services for IoT applications; Communication service: connects the cloud and edge nodes to enable communication between edge nodes and the cloud, forwards the status of edge nodes to the DRL agent, and distributes the communication protocol and interpreter for DRL decision to the edge nodes; Negotiation agent: is used by the cloud to communicate with devices through edge nodes to determine the required data services; DRL agent: continuously interacts with the environment based on the status and service quality of edge nodes, dynamically selects communication protocols and interpreters from the protocol library and interpreter library, and minimizes the resource consumption of edge nodes while meeting the requirements of service semantics and service latency.

[0015] The beneficial effects of this invention are as follows: 1. It realizes dynamic protocol adaptation between cloud, edge, and terminal, negotiates the data services required by terminal devices through dialogue, eliminates the need to preload the entire protocol stack, reduces the resident protocol resource occupation of edge nodes, and optimizes the communication protocol combination between edge nodes and terminal devices, further reducing the resource consumption of edge nodes; 2. It reduces manual configuration costs, avoids manual setting of protocols and service items, and improves the efficiency of terminal devices accessing cloud-edge networks. Attached Figure Description

[0016] Figure 1 shows the cloud-edge-device protocol framework based on DRL proxy in Example 1; Figure 2 shows a schematic diagram of the terminal device negotiating messages with the cloud through the edge node in Example 1; Figure 3 shows a flowchart of the DRL proxy decision-making process in Example 1; Figure 4 shows an example diagram of the terminal device and supported protocols in Example 1; Figure 5 shows a schematic diagram of the device protocol adaptation method in Example 1. Detailed Implementation

[0017] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0018] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the figures only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0019] Example 1: As shown in Figures 1-5, a dynamic adaptation method for device protocols and interpreters based on DRL includes the following steps: Step 1: The terminal device and the edge node determine the service items required by the terminal device through dialogue; Step 2: The edge node uploads the dialogue content with the terminal device to the cloud, and the cloud constructs a DRL state vector based on the dialogue content to form a DRL agent; Step 3: Define an action space, and dynamically combine different communication protocols and data interpreters in the action space; Step 4: Based on the reward function set by the associated service quality and resource consumption, the DRL agent in the cloud selects the protocol combination that meets the service requirements and consumes the least resources; It should be noted that in Step 4, in order to save the matching time of communication protocols and interpreters, only a limited number of combination calculations are performed, and the protocol combination with the lowest resource consumption is selected from the limited number of calculations; Step 5: The cloud's protocol library and interpreter library respectively load the selected communication protocol and interpreter, and return them to the negotiation agent of the edge node; Step 6: The negotiation agent sends the selected communication protocol to the terminal device; Step 7: The terminal device confirms the communication protocol and configures the communication service.

[0020] The edge nodes are connected to the endpoint devices via hardware interfaces.

[0021] The dialogue between the terminal device and the edge node in step 1 also includes the terminal device's Basic Device Information (BDI), used to identify and distinguish different terminal devices. It should be noted that the dialogue between the terminal device and the edge node is in the form of lightweight information exchange, including the device's basic information and the Data Service List (ASL). After receiving the BDI, the edge node first confirms it and then sends a confirmation message to the terminal device. Upon receiving the confirmation message, the terminal device sends the ASL to the edge node, which then forwards both the BDI and ASL to the cloud. The edge node confirms the BDI through several methods, including validity checks, consistency checks, timeliness checks, and semantic logic checks. Validity checks verify whether the BDI fields conform to preset specifications, such as the device ID. The BDI process includes several key checks: format correctness, hardware capability parameters within a reasonable range, and service permissions within the edge node's support list; consistency verification, where the edge node compares the hardware information reported by the terminal device with the collected hardware fingerprint to verify consistency and prevent terminal devices from forging high-computing-power information to fraudulently obtain resources; timeliness verification, checking the timestamps in the BDI to determine if the information reported by the terminal device is within its validity period, requesting the edge node to resend if it has expired; and semantic logic verification, based on defined rules, to verify the logical consistency in the BDI, avoiding mismatches between the terminal device's defined functions and its configuration (e.g., an air conditioning device and an image acquisition function are incompatible). To prevent malicious terminal devices from accessing the network, a pre-authentication process is performed before the BDI acquisition process. In this example, a certificate-based two-way authentication is used, where the edge node and the terminal device exchange X.509 certificates or OPC UA application certificates, verifying certificate validity with the help of a certificate authority. In this example, the edge node is configured with an i.MX6ULL CPU, 256MB of memory, 256MB of ROM, and has 8 serial ports, 4 network ports, and 1 CAN interface; the connected terminal devices include PLCs, switching power supplies, smart air conditioners, and monitoring equipment.

[0022] The DRL state vector constructed in step 2 includes the CPU utilization, memory usage, service latency, service completion rate, and bandwidth usage of the edge node, represented as S = [edge node CPU utilization, edge node memory utilization, device service semantic performance, device service latency]. The output decision matrix is ​​represented as A = [(protocol, interpreter)]. k], where k is the encoding corresponding to the request from the terminal device. DRL agents are usually set up in the cloud to obtain greater computing power support and to quickly complete the selection process of communication protocols and interpreters; in some other implementations, DRL agents can also be deployed on edge nodes, which can more quickly complete the deployment of communication protocols and interpreters. After the edge node completes the selection, it will upload the corresponding interpreter and communication protocol to the cloud.

[0023] In step 3, the protocol library used to store the communication protocols and the interpreter library used to store the interpreters are both located in the cloud, which has a faster processing speed and larger capacity. The communication protocols include MQTT, CoAP, HTTP, and proprietary protocols; the decoding formats of the data interpreters include JSON, ProtoBuf, and Binary.

[0024] The reward function in step 4 is defined as two parts: one part is a positive reward if the service demand is met, and the other part is a negative reward if the resource consumption is higher. In this example, the reward function is designed as: R = 5 - CPU utilization - memory utilization - P, where P = max(0, (current latency - expected latency)) + max(0, (semantic expectation - current semantic value)), that is: the lower the CPU utilization and memory utilization, the higher the reward; if the performance of any one of the semantic or latency indicators is not met, the reward value is reduced.

[0025] In step 5, the edge node receives the communication protocol and interpreter sent by the cloud, loads the selected protocol or interpreter, and unloads unnecessary protocol modules.

[0026] In the initial stage of step 7, after completing the communication configuration of the terminal device, the terminal device selects the high-performance OPCUA protocol to communicate with the edge node, requiring that the set service latency and service semantic requirements be met, and the service semantics are verified according to the set rules.

[0027] As shown in Figure 3, after the communication configuration of the terminal device is completed in step 7, the DRL agent in the cloud senses the resource consumption of the edge node and the service semantics and service latency fed back by the cloud in real time. It uses the DDQN model to optimize the combination of communication protocol and interpreter. The goal of optimization is to obtain a higher reward function. This is because with the subsequent access of new terminal devices, the original optimal combination of communication protocol and interpreter may be changed, increasing the CPU and memory utilization of the edge node. Therefore, it is necessary to continuously optimize the combination of communication protocol and interpreter between the terminal devices of the edge node.

[0028] During implementation, the terminal device communicates with the edge node, and the communication and negotiation messages are shown in Figure 2. The complete negotiation process is shown in Figure 5. First, the terminal device establishes a physical connection with the negotiation agent deployed on the edge node through a physical interface. The terminal device then sends its basic device information to the negotiation agent. After receiving the BDI from the device, the negotiation agent sends a BDI confirmation message to the terminal device. Upon receiving the confirmation message, the terminal device sends a Data Service List (ASL) to the negotiation agent. The negotiation agent performs preliminary processing of the information and sends both the ASL and BDI to the cloud. The cloud application generates a Service Request List (RSL) based on the IoT application requirements and sends the BDI and RSL back to the negotiation agent. The negotiation agent parses the RSL, extracts the required set of communication protocols and data interpreters, and requests the communication protocols and data interpreters from the protocol library and interpreter library in the cloud, respectively. The protocol library and interpreter library return the requested protocols and data interpreters to the negotiation agent. The negotiation agent sends the RSL and communication protocols to the terminal device. The terminal device confirms the RSL and configures the communication services according to the RSL, completing the on-demand acquisition of data services and avoiding full loading.

[0029] A dynamic device protocol and interpreter adaptation system based on DRL, based on the aforementioned adaptation method, comprises: a cloud: a central node responsible for global management and centralized data processing, communicating with all edge nodes; edge nodes: connected to on-site terminal devices via a digital network and connected to the cloud via a wireless or wired network; terminal devices: located at the network edge, used to collect data and perform preliminary processing; a protocol library: stored in the cloud, containing a large number of communication protocols used between edge nodes and terminal devices, forming the basis for establishing connections between edge nodes and devices; and an interpreter library: located in the cloud, storing a large number of interpretation rules for converting raw data from terminal devices into structured data. The system is used to convert raw data from terminal devices into structured data to provide data services for IoT applications; Communication service: connects the cloud and edge nodes to enable communication between edge nodes and the cloud, forwards the status of edge nodes to the DRL agent, and distributes the communication protocol and interpreter for DRL decision to the edge nodes; Negotiation agent: is used by the cloud to communicate with devices through edge nodes to determine the required data services; DRL agent: continuously interacts with the environment based on the status and service quality of edge nodes, dynamically selects communication protocols and interpreters from the protocol library and interpreter library, and minimizes the resource consumption of edge nodes while meeting the requirements of service semantics and service latency.

[0030] The above description is merely a specific example of the present invention and does not constitute any limitation on the present invention. Obviously, those skilled in the art, after understanding the content and principles of the present invention, may make various modifications and changes in form and detail without departing from the principles and structure of the present invention; however, these modifications and changes based on the spirit of the present invention are still within the scope of protection of the claims of the present invention.

Claims

1. A method for dynamic adaptation of device protocols and interpreters based on DRL, characterized in that, The process includes the following steps: Step 1: The terminal device and the edge node determine the services required by the terminal device through a dialogue; Step 2: The edge node uploads the dialogue content with the terminal device to the cloud, where the cloud constructs a DRL state vector based on the dialogue content, forming a DRL agent; Step 3: Define an action space, dynamically combining different communication protocols and data interpreters within the action space; Step 4: Based on a reward function set according to associated service quality and resource consumption, the cloud's DRL agent selects the protocol combination that meets the service requirements and consumes the least resources; Step 5: The cloud's protocol library and interpreter library load the selected communication protocol and interpreter respectively, and return them to the edge node's negotiation agent; Step 6: The negotiation agent sends the selected communication protocol to the terminal device; Step 7: The terminal device confirms the communication protocol and configures the communication service.

2. The dynamic adaptation method according to claim 1, characterized in that, The edge nodes are connected to the terminal devices via hardware interfaces.

3. The dynamic adaptation method according to claim 1, characterized in that, The dialogue between the terminal device and the edge node in step 1 also includes the basic device information of the terminal device, which is used to identify and distinguish different terminal devices.

4. The dynamic adaptation method according to claim 1, characterized in that, The DRL state vector constructed in step 2 includes the edge node's CPU utilization, memory usage, service latency, service completion rate, and bandwidth usage.

5. The dynamic adaptation method according to claim 1, characterized in that, The reward function in step 4 is defined as two parts: one part is a positive reward if the service demand is met, and the other part is a negative reward if the resource consumption is higher.

6. The dynamic adaptation method according to claim 1, characterized in that, In step 5, the edge node receives the communication protocol and interpreter sent by the cloud, loads the selected protocol or interpreter, and unloads unnecessary protocol modules.

7. The dynamic adaptation method according to claim 1, characterized in that, In the initial stage of step 7, after completing the communication configuration of the terminal device, the terminal device first selects the high-performance OPC UA protocol to communicate with the edge node, requiring that the set service latency and service semantic requirements be met, and then gradually optimizes it.

8. The dynamic adaptation method according to claim 7, characterized in that, After step 7 completes the communication configuration of the terminal device, the DRL agent in the cloud senses the resource consumption of the edge node and the service semantics and service latency fed back by the cloud in real time, and uses the DDQN model to optimize the combination of communication protocol and interpreter.

9. The dynamic adaptation method according to claim 1, characterized in that, The communication protocols in step 3 include MQTT, CoAP, HTTP, and proprietary protocols; the data interpreter's decoding formats include JSON, ProtoBuf, and Binary.

10. A device protocol and interpreter dynamic adaptation system based on DRL, characterized in that, Based on the adaptation method according to any one of claims 1 to 9, the adaptation system includes: a cloud: a central node responsible for global control and centralized data processing, communicating with all edge nodes; edge nodes: connected to on-site terminal devices via a digital network and connected to the cloud via a wireless or wired network; terminal devices: located at the network edge, used to collect data and perform preliminary processing; a protocol library: stored in the cloud, storing a large number of communication protocols for edge nodes and terminal devices, serving as the basis for establishing connections between edge nodes and devices; and an interpreter library: located in the cloud, storing a large number of interpretation rules for converting raw data from terminal devices into structured data, used to interpret data from the terminal devices... The raw data from edge devices is converted into structured data to provide data services for IoT applications; Communication services connect the cloud and edge nodes, enabling communication between edge nodes and the cloud, forwarding the status of edge nodes to the DRL agent, and distributing the communication protocol and interpreter for DRL decisions to the edge nodes; Negotiation agent is used by the cloud to communicate with devices through edge nodes to determine the required data services; DRL agent continuously interacts with the environment based on the status and quality of service of edge nodes, dynamically selecting communication protocols and interpreters from the protocol library and interpreter library to meet service semantics and service latency requirements while minimizing the resource consumption of edge nodes.

Images