A multi-protocol access method for multiple internet of things devices

CN122554555APending Publication Date: 2026-08-11SHANGHAI JINJIANG FOREIGN SERVICE CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-05-19
Publication Date
2026-08-11

AI Technical Summary

Technical Problem

[0004]然而,在多物联网设备并发接入场景下,不同通信协议数据流在数据结构复杂度、报文长度以及解析处理耗时方面存在差异,使得工业智能网关在统一接收与解析过程中,各类数据流在共享处理资源下难以保持一致的处理进度,从而在持续数据输入条件下逐步产生局部处理等待与队列堆积现象,并进一步引起解析处理延迟持续累积,进而导致工业智能网关对多源异构数据的整体处理效率与响应能力下降

Benefits of technology

1.本申请通过根据通信数据流的协议类型标识和数据特征对协议数据单元(即数据段)进行细粒度分类,构建独立的处理队列,并基于历史执行负载信息和实时队列积压信息进行动态负载评估,自适应确定队列容量限制、处理速率限制及并发控制等资源配置参数,形成约束队列,从而有效隔离不同协议及同一协议下不同负载特性数据段之间的资源竞争,避免因长短报文混合处理导致的解析延迟累积,显著提升了多协议并发接入场景下的整体处理效率与响应能力。

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122554555A_ABST
    Figure CN122554555A_ABST
Patent Text Reader

Abstract

The application provides a multi-protocol access method of multiple Internet of Things devices, applied to the technical field of industrial Internet of Things data transmission and processing. An industrial intelligent gateway serves as an edge access node. When receiving multi-source heterogeneous device data, the protocol data unit (i.e., data segment) is classified in a fine-grained manner according to the protocol type identifier and data characteristics of the communication data stream, an independent processing queue is constructed, dynamic load evaluation is performed based on historical execution load information and real-time queue backlog information, resource configuration parameters such as queue capacity limit, processing rate limit and concurrent control are adaptively determined, and a constraint queue is formed, so as to effectively isolate the resource competition between different protocols and different load characteristic data segments under the same protocol, avoid the accumulation of analysis delay caused by mixed processing of long and short messages, significantly improve the overall processing efficiency and response capability in the multi-protocol concurrent access scene, and guarantee the real-time performance and system stability of industrial field data transmission.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of industrial Internet of Things (IoT) data transmission and processing technology, and in particular to a multi-protocol access method for multiple IoT devices. Background Technology

[0002] In existing industrial IoT scenarios, industrial smart gateways are typically used as multi-device access and data aggregation nodes to achieve unified access and data transmission processing for devices using various communication protocols. In practical applications, different IoT devices often use multiple communication methods such as Modbus, MQTT, CoAP, or custom proprietary protocols, requiring industrial smart gateways to simultaneously possess multi-protocol parsing and adaptation capabilities to complete the unified access and conversion processing of heterogeneous data.

[0003] In existing technologies, industrial smart gateways typically receive and process incoming communication data streams in a unified manner, identify the protocol of the data stream based on the protocol identifier in the communication message, and then parse and convert the data according to the corresponding protocol parsing rules, thereby realizing unified formatting processing of data from different communication protocols, and outputting the parsing results to the backend system for storage or further analysis and processing.

[0004] However, in scenarios where multiple IoT devices access concurrently, data streams from different communication protocols differ in terms of data structure complexity, message length, and parsing processing time. This makes it difficult for industrial smart gateways to maintain a consistent processing progress for various data streams under shared processing resources during the unified reception and parsing process. Consequently, under continuous data input conditions, local processing waits and queue accumulation gradually occur, further causing the parsing processing delay to accumulate continuously. This leads to a decrease in the overall processing efficiency and responsiveness of industrial smart gateways for multi-source heterogeneous data. Summary of the Invention

[0005] This application provides a multi-protocol access method for multiple IoT devices. The core of the method is as follows: by identifying the protocol and dividing the data stream into segments, a processing queue corresponding to the protocol data stream category is constructed. Based on historical execution load information and real-time queue backlog information, load assessment is performed to generate resource configuration parameters. The resource configuration parameters are bound to the processing queue to form a constraint queue. Thus, during the protocol parsing process, rate limiting control and rate limiting control are dynamically executed and de-limited according to the running data. Finally, the initial parsing results are summarized and reorganized based on the original data stream identifier and sequence identifier to obtain the final parsing result.

[0006] To achieve the above objectives, this application adopts the following technical solution: This application provides a method for multi-protocol access of multiple IoT devices, the method may include: Obtain the communication data streams and their original data stream identifiers sent by multiple IoT devices, and parse the communication data streams to obtain the protocol type identifier; The communication data stream is divided into data segments to obtain multiple data segments and their corresponding sequence identifiers, and the data characteristics of each data segment are obtained. The multiple data segments are classified based on the protocol type identifier and the data characteristics to obtain multiple protocol data stream categories, and corresponding processing queues are constructed based on the multiple protocol data stream categories; Obtain historical execution load information for each protocol data stream category within a preset time window, as well as real-time queue backlog information for the current processing queue; The historical execution load information and the real-time queue backlog information are subjected to load assessment to obtain assessment results. Based on the assessment results, resource configuration parameters are determined, and the resource configuration parameters are bound to the corresponding processing queues to obtain constraint queues. The data segments in each constraint queue are parsed according to the protocol to obtain the corresponding initial parsing results; During the protocol parsing process, the running data of each constraint queue is continuously acquired, and the running data is compared with a preset running data threshold. When the running data of any constraint queue is greater than or equal to the preset running data threshold, rate limiting control is applied to the corresponding constraint queue, and protocol parsing processing continues. When the running data of the corresponding constraint queue is less than the preset recovery threshold, the flow restriction control of the constraint queue is lifted; Based on the original data stream identifier and the sequence identifier, the initial parsing results are summarized and reorganized to obtain the final parsing result.

[0007] As can be seen from the above technical solution, this application has the following beneficial effects: 1. This application classifies protocol data units (i.e., data segments) in a fine-grained manner according to the protocol type identifier and data characteristics of the communication data stream, constructs independent processing queues, and performs dynamic load assessment based on historical execution load information and real-time queue backlog information. It adaptively determines resource configuration parameters such as queue capacity limits, processing rate limits, and concurrency control to form a constrained queue. This effectively isolates resource competition between different protocols and data segments with different load characteristics under the same protocol, avoids the accumulation of parsing delays caused by mixed processing of long and short messages, and significantly improves the overall processing efficiency and response capability in multi-protocol concurrent access scenarios.

[0008] 2. This application continuously monitors the running data (backlog length, waiting time) of each constraint queue during the protocol parsing process. When the running data exceeds a preset threshold, it automatically executes flow limiting control and lifts the flow limiting after the backlog is relieved, forming an adaptive flow regulation mechanism based on water level feedback to prevent system overload and crash. At the same time, it assigns, sorts, and reassembles the parsing results through the original data stream identifier and sequence identifier to ensure the correctness of the data order after parallel processing. This solves the problem of data disorder caused by parallel parsing of multiple queues and ensures the integrity and reliability of the output data of the industrial intelligent gateway. Attached Figure Description

[0009] The present application will be further described below with reference to the accompanying drawings.

[0010] Figure 1 A flowchart illustrating the first multi-protocol access method for multiple IoT devices provided in this application; Figure 2 A flowchart of the second multi-protocol access method for multiple IoT devices provided in this application. Detailed Implementation

[0011] The terms "first," "second," and "third," etc., used in this application specification, claims, and drawings are used to distinguish different objects, not to limit a specific order.

[0012] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0013] In industrial IoT field, multiple IoT devices (such as Modbus RTU temperature and humidity sensors, MQTT protocol PLC controllers, and proprietary protocol vibration detectors) are connected to the industrial smart gateway through interfaces such as Ethernet and RS485.

[0014] Research has revealed that in scenarios with multiple IoT devices accessing concurrently, data streams from different communication protocols differ in terms of data structure complexity, message length, and parsing processing time. This makes it difficult for industrial smart gateways to maintain a consistent processing progress across various data streams under shared processing resources during unified reception and parsing. Consequently, under continuous data input conditions, localized processing waits and queue accumulation gradually occur, further leading to a continuous accumulation of parsing processing delays. Ultimately, this results in a decrease in the overall processing efficiency and responsiveness of industrial smart gateways for multi-source heterogeneous data.

[0015] Example 1

[0016] To address the aforementioned issues, this application provides a multi-protocol access method for multiple IoT devices. Please refer to [link to relevant documentation]. Figure 1 .

[0017] S1, obtain the communication data streams and original data stream identifiers of the communication data streams sent by multiple IoT devices, parse the communication data streams, and obtain the protocol type identifier.

[0018] In this step, communication data streams sent by multiple IoT devices are received through an industrial smart gateway. For each communication data stream, source address information (such as source MAC address or source IP address) is extracted from the communication data stream as the original data stream identifier, used to distinguish the data source of different devices.

[0019] Extract the first few bytes of the application layer message in the communication data stream as protocol feature fields (e.g., the protocol identifier of Modbus TCP, the message type bit of the MQTT fixed header), match the feature fields with the feature values ​​in the preset protocol feature information, and obtain the protocol type identifier of the communication data stream (e.g., "Modbus_TCP", "MQTT_3.1.1" or "private protocol") after a successful match.

[0020] This step provides tag information for subsequent data segmentation, protocol data stream classification, and processing queue construction.

[0021] To ensure clarity and conciseness in the description of the following embodiments, a brief introduction to the relevant terms is given first: An industrial smart gateway refers to a hardware or software device deployed at the edge of the industrial Internet of Things (IoT) to connect to various IoT devices and perform data collection, protocol conversion, and forwarding.

[0022] A communication data stream refers to a sequence of data continuously sent by IoT devices. Each communication data stream contains multiple complete application layer protocol messages and their network layer and link layer encapsulation information.

[0023] The original data stream identifier is an identifier used to uniquely distinguish different communication data streams. In this embodiment, it is extracted from the source MAC address or source IP address of the communication data stream.

[0024] The protocol type identifier refers to the type of communication protocol used by the identified communication data stream, such as Modbus_TCP, MQTT_3.1.1, CoAP, or a proprietary protocol.

[0025] Parsing the communication data stream means parsing only a few characteristic fields in the message header of the communication data stream to identify the protocol type, without performing complete application layer data decoding.

[0026] Protocol characteristic fields refer to fixed bytes or fields in the protocol header used to distinguish the protocol type, such as the protocol identifier (0x0000) in the Modbus TCP MBAP header and the message type bit in the MQTT fixed header.

[0027] Preset protocol feature information refers to the pre-defined mapping relationship between each protocol type and its feature fields, which is used to quickly match and identify protocol types.

[0028] In some possible implementations, in multi-protocol concurrent access scenarios of the Industrial Internet of Things (IIoT), multiple IoT devices simultaneously send communication data streams to the industrial smart gateway. These data streams from different devices arrive physically mixed. If the source of each data stream cannot be distinguished, subsequent processing will be unable to correctly assign the parsing results. Furthermore, if the protocol type of each data stream cannot be identified in advance, data streams with different protocols cannot be allocated to independent processing queues, leading to contention for shared processing resources and accumulating parsing delays.

[0029] This step is designed to solve this problem. Its core principle is to use the source address information (source MAC address or source IP address) carried by the communication data stream itself as the original data stream identifier. In the IEEE 802.3 Ethernet frame, the source MAC address field is 6 bytes long, assigned by the device's network card manufacturer, and is globally unique. The source IP address in the IP packet is assigned by the network administrator and is unique within the same subnet. When receiving the communication data stream, the industrial intelligent gateway directly reads this address field from the corresponding offset position by parsing the link layer or network layer header, without the need for additional generation or allocation, thus ensuring the uniqueness and traceability of the identifier.

[0030] To obtain the protocol type identifier, the communication data stream is parsed, specifically as follows: This approach extracts fixed-length features from the application layer message headers in the communication data stream, without performing complete protocol decoding. This is because most Industrial IoT protocols have fixed identifier fields in the message header to distinguish protocol types. For example, Modbus TCP uses 0x0000 at offset 2-3 bytes in the MBAP header; MQTT defines 14 message types in the high 4 bits of the first byte of its fixed header. These feature fields have a deterministic mapping relationship with the protocol type. Using pre-defined protocol feature information (stored as key-value pairs between each protocol type and its feature fields), the gateway, upon receiving the communication data stream, extracts the first 4-8 bytes of the application layer message as a key and performs a fast hash lookup in the feature database, determining the protocol type identifier within microseconds. This shallow parsing process avoids recursive decoding of nested TLV structures or processing of variable-length fields, resulting in extremely low computational overhead and no significant increase in system load.

[0031] This step efficiently and accurately generates the original data stream identifier and protocol type identifier for each communication data stream at the data entry point, decoupling the mixed data stream into independent units with clear sources and protocol labels, providing reliable basic data for subsequent independent queue construction, dynamic load balancing, and rate limiting control.

[0032] S2, the communication data stream is divided into data segments to obtain multiple data segments and their corresponding sequence identifiers, and the data characteristics of each data segment are obtained.

[0033] In this step, the communication data stream is segmented according to the protocol type identifier corresponding to that protocol, using data frame boundary identification rules that match the protocol. For example, for the Modbus TCP protocol, the message boundary is determined by parsing the length field in the MBAP header; for the Modbus RTU protocol, the idle time interval between messages (usually 3.5 character times) is used as the frame delimiter; for the MQTT protocol, the message boundary is parsed based on the remaining length field in the fixed header; for proprietary protocols, segmentation can be based on a fixed start byte or a custom delimiter. After segmentation, the same communication data stream is divided into multiple data segments, each corresponding to a complete protocol message.

[0034] During the partitioning process, each data segment is assigned a sequence identifier within the communication data stream. The sequence identifier can be an incrementing sequence number (such as 1, 2, 3...) or a timestamp to represent the order of the data segments in the original communication data stream.

[0035] Obtain the data characteristics of each data segment, which include at least the data segment length, protocol type identifier, important fields in the data segment (such as Modbus address, MQTT topic), and the timestamp of the data segment arrival.

[0036] Through the above processing, the continuous communication data stream is decomposed into discrete data segments with sequential identifiers, and the key features of each data segment are extracted, providing a basic unit for subsequent classification, processing queue construction and load balancing based on protocol type identifiers and data features.

[0037] To ensure clarity and conciseness in the description of the following embodiments, a brief introduction to the relevant terms is given first: A data segment is an independent data unit obtained by dividing a continuous communication data stream according to the protocol boundary. Each data segment corresponds to a complete application layer protocol message.

[0038] Sequence identifiers are used to indicate the sequential position of data segments in the original communication data stream, such as incrementing sequence numbers or timestamps.

[0039] Data characteristics refer to the key attributes of each data segment, including at least the data segment length, protocol type identifier, key field values, and arrival timestamp. Frame boundary identification rules refer to methods for determining the start and end positions of a message based on the characteristics of different protocols (such as length fields, time intervals, or delimiters).

[0040] In some possible implementations, different protocols use different methods to identify message boundaries, and data segments within the same communication data stream have inherent temporal dependencies. If the data stream is not correctly segmented, it is impossible to parse continuous bit streams into meaningful independent messages; if sequence identifiers are missing, the data cannot be reassembled in its original order after subsequent parallel processing by multiple queues, resulting in data corruption.

[0041] This step is designed to solve this problem. Its core principle is to use the frame boundary detection logic in the corresponding protocol parser to perform deterministic segmentation of the data stream based on the protocol type identifier identified by S1.

[0042] For length-based protocols (such as Modbus TCP and MQTT), the end of the current message is calculated by reading the length field at a fixed position, thus locating the start of the next message. For time-based protocols (such as Modbus RTU), the end of the frame is determined by detecting the time interval between characters. For delimiter-based protocols, segmentation is performed by matching specific byte sequences. This adaptive segmentation strategy ensures the accuracy of segmentation.

[0043] A sequence identifier (e.g., a monotonically increasing sequence number) is generated for each data segment. This identifier is subsequently used to reassemble data segments scattered across different processing queues in their original order. The extraction of data features (e.g., length, key field values) provides a basis for classification based on data features in S2. For example, data stream types can be further subdivided based on data segment length thresholds or specific field values.

[0044] This step transforms a mixed, continuous data stream into labeled, independently schedulable, and traceable discrete processing units, laying the necessary foundation for subsequent classification queue construction, parallel processing, and sequential reordering.

[0045] S3, classify the multiple data segments based on the protocol type identifier and the data characteristics to obtain multiple protocol data stream categories, and construct corresponding processing queues based on the multiple protocol data stream categories.

[0046] In this step, the protocol type identifier is matched against a preset protocol parsing rule base to obtain the target protocol type. The preset protocol parsing rule base stores complete parsing rules (including message structure, field definitions, parsing functions, etc.) corresponding to different protocol type identifiers. For example, a data segment with the protocol type identifier "Modbus_TCP" matches the corresponding Modbus TCP parsing rule; a data segment with the identifier "MQTT_3.1.1" matches the MQTT parsing rule.

[0047] For each data segment, feature analysis is performed to obtain the data feature analysis results. The data feature analysis results include a quantitative description of the data segment's length range, key field value distribution, payload pattern, and other characteristics.

[0048] The target protocol type and the data feature parsing results are combined and mapped to obtain a classification mapping relationship. For example, even if the protocol types are the same (both are Modbus TCP), if the data segment lengths differ significantly (e.g., long messages versus short messages), or if key fields (e.g., function codes) are different, they can be further subdivided into different subcategories. Through this combined mapping, data segments with similar processing characteristics are grouped into the same protocol data stream category.

[0049] Based on the classification mapping relationship, the multiple data segments are classified to obtain multiple protocol data stream categories. Each protocol data stream category corresponds to a group of data segments with the same protocol type and similar data characteristics.

[0050] Multiple initial queues are constructed, and these initial queues are associated with the protocol data stream categories to obtain associated queues that correspond one-to-one with the initial queues for each protocol data stream category. Based on the characteristics of the protocol data stream categories (such as expected processing load, real-time requirements, etc.), the associated queues are configured with queue capacity (e.g., setting the maximum queue length) and processing priority (e.g., assigning higher priority to high real-time categories) to obtain processing queues corresponding to the protocol data stream categories.

[0051] Through the above processing, discrete data segments are classified into different processing queues according to protocol type and data characteristics, providing a queue infrastructure for subsequent independent and differentiated parsing processing and load balancing.

[0052] To ensure clarity and conciseness in the description of the following embodiments, a brief introduction to the relevant terms is given first: A pre-built protocol parsing rule base is a database that stores complete parsing rules (including message structure, field definitions, parsing functions, etc.) corresponding to different protocol type identifiers. It is used to map protocol type identifiers to target protocol types.

[0053] The target protocol type refers to the specific protocol specification that corresponds to the protocol type identifier of the data segment, which is matched from the preset protocol parsing rule base and is used to guide the subsequent complete protocol parsing.

[0054] Data feature parsing results refer to the descriptive information obtained after quantitative analysis of the data features of a data segment, such as length range, key field value distribution, and payload pattern.

[0055] Classification mapping relationship refers to the mapping rule formed by combining the target protocol type with the data feature parsing results, which is used to classify data segments into specific protocol data stream categories.

[0056] Protocol data stream category refers to the category obtained by grouping data segments according to protocol type identifier and data characteristics. Data segments in the same category have similar protocol types and processing characteristics.

[0057] The initial queue is an empty queue that is pre-created for each protocol data stream category to temporarily store data segments belonging to that category.

[0058] An associated queue is a queue with a category identifier obtained by binding an initial queue to a protocol data stream category.

[0059] Queue capacity configuration refers to the maximum number of data segments or maximum memory usage set for each processing queue, used to prevent queues from growing indefinitely.

[0060] Processing priority configuration refers to the scheduling priority assigned to each processing queue. Data segments in high-priority queues will be processed first.

[0061] A processing queue is a queue structure that, after being configured in terms of capacity and priority, corresponds one-to-one with the protocol data stream category and is used to carry data segments to be parsed.

[0062] In some possible implementations, classifying data segments solely based on protocol type identifiers may not effectively distinguish data streams with different processing characteristics under the same protocol. For example, Modbus TCP packets may contain long packets (batch reads) and short packets (single register reads), with significant differences in parsing time. If they are placed in the same queue without distinction, resource contention and latency accumulation between long and short packets will still occur. Furthermore, different protocol data streams have different real-time requirements (e.g., alarm messages require high-priority processing).

[0063] This step is designed to solve this problem. Its core principle is to combine protocol type identifiers with data characteristics to form fine-grained protocol data stream categories, and to independently configure processing queues and priorities for each category. Specifically: A pre-defined protocol parsing rule base is used to match protocol type identifiers to the target protocol type. This rule base is typically stored in a hash table or trie, with the key being the protocol type identifier string and the value being the corresponding parsing rule object. The matching process is completed in constant time with minimal overhead. Data features are analyzed. These features include the data segment length and key field values ​​obtained in S2. For example, for a Modbus TCP data segment, function code fields can be extracted (such as 0x03 for reading holding registers, 0x10 for writing multiple registers). Different function codes correspond to different message lengths and parsing complexities. Through feature analysis, feature tags such as "long message write operation" or "short message read operation" can be obtained.

[0064] The target protocol type is combined with the data feature parsing results for mapping. For example, the combination key could be "Modbus_TCP + function code 03" or "Modbus_TCP + data segment length greater than 100 bytes". This combined mapping method can separate data segments with the same protocol but different payload characteristics into different protocol data stream categories. The classification mapping relationship can be implemented through predefined rule tables or decision trees.

[0065] After obtaining multiple protocol data stream categories, an independent processing queue is created for each category. To accommodate the processing needs of different categories, the queues need to be configured with capacity (e.g., a smaller capacity can be set for short message queues with strong bursts but fast processing, and a larger capacity can be set for long message queues with slow processing to prevent overflow) and priority (e.g., alarm message queues are set to high priority, and ordinary data queues are set to low priority). After configuration, the final processing queues are obtained.

[0066] This step distributes data segments to different queues based on finer-grained characteristics, achieving the aggregation of similar data segments and the isolation of dissimilar data segments. This provides clear queue boundaries for independent load assessment, resource allocation, and rate limiting control in subsequent steps, effectively avoiding resource contention and latency accumulation caused by varying packet lengths or processing characteristics, and significantly improving the overall efficiency and predictability of multi-protocol concurrent processing.

[0067] S4, obtain historical execution load information of each protocol data stream category within a preset time window and real-time queue backlog information of the current processing queue.

[0068] In this step, for each protocol data stream category obtained in S3 (i.e., the data stream category corresponding to each processing queue), its historical execution load information within a preset time window is obtained, and the real-time queue backlog information of the current processing queue of that category is also obtained.

[0069] The preset time window can be flexibly set according to system configuration, for example, taking a time period of 5 seconds, 10 seconds, or 30 seconds in the past. For each protocol data stream category, the historical execution load information of protocol parsing processing of data segments of that category within the time window is statistically analyzed. The historical execution load information includes, but is not limited to: historical processing latency of that category (such as the average parsing time per data segment), historical processing throughput (such as the number of data segments processed per unit time), and historical queue length change trends. This information can be obtained by recording the timestamp and resource consumption after each parsing, and then aggregating and statistically analyzing them within the sliding window.

[0070] Real-time monitoring of the backlog in each processing queue. Specifically, for each protocol data stream category, the system obtains in real-time the number of data segments that have not yet been processed in the queue (i.e., the queue backlog length) and the waiting time of the earliest data segment in the queue (i.e., the queue waiting time). This real-time queue backlog information can be obtained directly through the queue management interface (e.g., reading the queue length field or recording the difference between the enqueue timestamp and the current time).

[0071] Through the above processing, historical load characteristics and current backlog status were collected for each protocol data stream category, providing a quantitative basis for subsequent load assessment and resource allocation.

[0072] To ensure clarity and conciseness in the description of the following embodiments, a brief introduction to the relevant terms is given first: The preset time window refers to the pre-set time length for statistical analysis of historical execution load information, such as 5 seconds, 10 seconds, or 30 seconds.

[0073] Historical execution load information refers to the load characteristics obtained by statistically analyzing the processing of a certain protocol data stream category within a preset time window, including at least historical processing latency, historical processing throughput, and historical queue length change trends.

[0074] Historical processing latency refers to the time consumed in parsing the protocol for each data segment in that category, usually taken as the average or quantile.

[0075] Historical processing throughput refers to the number of data segments successfully processed per unit of time for this category.

[0076] Real-time queue backlog information refers to the backlog status of each processing queue at the current moment, including at least the queue backlog length and queue waiting time.

[0077] Queue backlog length refers to the number of data segments waiting to be processed in the current processing queue.

[0078] Queue waiting time refers to the time that the first data segment enqueued in the current processing queue has been waiting in the queue.

[0079] In some possible implementations, the processing load characteristics of different protocol data stream categories vary, and the load changes dynamically over time (e.g., a sudden surge in data reports from a certain type of device). If the historical load trends and current backlog status of each category cannot be perceived in real time, subsequent resource allocation will lack a basis, potentially leading to persistent backlogs in categories with insufficient processing capacity or wasted resources in categories with excessive processing capacity. Furthermore, relying solely on static classification while ignoring dynamic load changes cannot achieve adaptive load balancing.

[0080] This step is designed to address this issue. Its core principle lies in periodically or event-triggeredly collecting historical execution load (such as average latency and throughput) for each protocol data stream category using a sliding time window, while simultaneously monitoring the real-time backlog length and waiting time of each processing queue. This data forms the input characteristics for subsequent load assessment and resource adjustment, specifically: A circular buffer or sliding window array is maintained for each protocol data stream category to store the timestamps and resource consumption of the N most recently completed parsing operations. When historical execution load information is needed, statistics are performed on the data within the window (such as calculating average latency, P95 latency, throughput, etc.). Simultaneously, through atomic operations provided by the queue management module, the current length of each processing queue and the waiting time of the head element (the difference between the enqueue timestamp and the current time) are read in real time.

[0081] The two types of information obtained (historical load and real-time backlog) will be encapsulated into a structured load report and passed to S5 for load assessment and resource allocation.

[0082] Through this step, the system obtains the "past performance" and "current pressure" of each protocol data stream category, enabling subsequent dynamic adjustments (such as increasing processing threads, adjusting queue capacity, and rate limiting) to be targeted and effective, thereby avoiding continuous backlog in individual queues and a decline in overall processing efficiency, and achieving adaptive load management for multi-protocol concurrent processing.

[0083] S5, perform load assessment on the historical execution load information and the real-time queue backlog information to obtain the assessment result, determine the resource configuration parameters based on the assessment result, and bind the resource configuration parameters with the corresponding processing queue to obtain the constraint queue.

[0084] In this step, for each protocol data stream category, its historical execution load information and real-time queue backlog information are obtained from S4. Feature extraction is performed on the historical execution load information to obtain historical processing latency information and historical processing throughput information. For example, average parsing latency, latency variance, and the number of data segments processed per unit time are extracted from the historical execution load information as latency and throughput features.

[0085] Feature extraction is performed on the real-time queue backlog information to obtain queue backlog length information and queue waiting time information. For example, the number of data segments to be processed in the current queue is extracted as the backlog length, and the waiting time of the data segment at the head of the queue is extracted as the waiting time.

[0086] The historical processing latency information, historical processing throughput information, queue backlog length information, and queue waiting time information are comprehensively processed to obtain an evaluation result. This comprehensive processing can employ methods such as weighted summation or fuzzy logic. For example, latency and throughput are combined to calculate processing efficiency, and backlog length and waiting time are combined to calculate the current load pressure, ultimately outputting a quantified evaluation value or level.

[0087] The evaluation results are matched against a preset load level table to obtain the target load level. The preset load level table defines load levels (such as "low load", "medium load", and "high load") corresponding to different evaluation value ranges. After matching, the target load level to which the current protocol data stream category belongs is determined.

[0088] Based on the target load level, the target resource configuration requirement type is determined using a preset resource configuration library. This library stores resource configuration requirement types corresponding to different load levels; for example, "high load" corresponds to "increase processing threads" and "expand queue capacity." After determining the target resource configuration requirement type, resource configuration parameters are generated accordingly. These parameters include queue capacity limit parameters (such as maximum queue length), processing rate limit parameters (such as the maximum number of data segments processed per second), and processing concurrency control parameters (such as the number of parallel processing threads).

[0089] Based on the queue capacity limit parameters, the processing rate limit parameters, and the processing concurrency control parameters, capacity constraints, rate constraints, and concurrency constraints are applied to the corresponding processing queues to obtain constrained processing queues. In other words, these parameters are bound to the corresponding processing queues so that when processing data segments in the queue, the above constraints are followed (e.g., queue length cannot exceed capacity limits, processing rate cannot exceed rate limits, and concurrency cannot exceed concurrency control parameters), thus obtaining constrained queues.

[0090] Through the above processing, dynamically adapted resource configuration parameters are generated for each protocol data stream category and bound to the corresponding processing queue, forming a constrained queue, which provides a basis for subsequent protocol parsing and rate limiting control.

[0091] To ensure clarity and conciseness in the description of the following embodiments, a brief introduction to the relevant terms is given first: Historical processing latency information refers to the characteristics extracted from historical execution load information that reflect the time consumed in parsing and processing such data segments, such as average latency, maximum latency, or latency variance.

[0092] Historical processing throughput information refers to the characteristics extracted from historical execution load information that reflect the processing rate of this type of data segment, such as the number of data segments processed per unit time.

[0093] Queue backlog length information refers to the number of data segments waiting to be processed in the current processing queue, extracted from the real-time queue backlog information.

[0094] Queue waiting time information refers to the time that the queue header data segment extracted from the real-time queue backlog information has been waiting.

[0095] The evaluation result refers to the numerical value or level obtained after comprehensively processing the above characteristics, which is used to quantify the load status of the current protocol data stream category.

[0096] The preset load level table refers to a pre-defined table that maps the numerical range of evaluation results to load levels (such as low load, medium load, and high load).

[0097] The target load level refers to the load level of the current protocol data stream category, which is obtained by matching a preset load level table.

[0098] A pre-built resource configuration library is a database that stores resource configuration requirements for different load levels.

[0099] Resource configuration requirement type refers to the type of resources that need to be adjusted based on the load level, such as adding processing threads, expanding queue capacity, or increasing processing speed.

[0100] Resource configuration parameters refer to specific control parameters generated based on the type of resource configuration requirements, including queue capacity limit parameters, processing rate limit parameters, and processing concurrency control parameters.

[0101] Queue capacity limit parameters are parameters used to limit the maximum length of the processing queue, such as the maximum number of data segments allowed to back up.

[0102] Processing rate limit parameters are parameters used to limit the number of data segments processed per unit of time, such as the maximum number of data segments processed per second.

[0103] Concurrency control parameters are parameters used to limit the number of threads or tasks that can process data segments simultaneously.

[0104] Capacity constraints refer to applying queue capacity limit parameters to the processing queue so that its actual length does not exceed the limit.

[0105] Rate constraint refers to applying a processing rate limit parameter to a processing queue to ensure that its processing rate does not exceed the limit.

[0106] Concurrency constraints refer to applying concurrency control parameters to a processing queue to ensure that the number of threads processing simultaneously does not exceed the limit.

[0107] A constrained queue is a processing queue that has been constrained by capacity, rate, and concurrency constraints and is bound to resource configuration parameters.

[0108] In some possible implementations, the load on different protocol data stream categories changes dynamically. If resource configuration remains fixed, high-load queues may suffer from insufficient processing capacity and continuous backlog, while low-load queues may be idle and wasteful. Traditional methods lack adaptive resource adjustment mechanisms based on historical load and real-time backlog, making it difficult to cope with load fluctuations.

[0109] This step is designed to solve this problem. Its core principle is to extract latency and throughput characteristics from historical execution loads, as well as length and waiting time characteristics from real-time queue backlogs, and quantify the current load pressure through comprehensive evaluation; match corresponding resource configuration requirements from a preset resource configuration library according to the load level to generate specific capacity, rate, and concurrency constraint parameters; finally, bind these parameters to the corresponding processing queues to form constrained queues, specifically: Historical processing latency information can be calculated using the average parsing time within a sliding window, while historical processing throughput information can be calculated using the number of data segments processed per unit time. Real-time queue backlog length can be directly read from the number of queue elements, and queue waiting time can be obtained from the difference between the enqueue time of the first element and the current time. During comprehensive evaluation, these multi-dimensional features are fused to output a quantitative evaluation result, which is then compared with a preset load level table to obtain the target load level. The preset load level table can be divided into low load, medium load, and high load ranges.

[0110] In the preset resource configuration library, different load levels correspond to different resource configuration requirement types: low load corresponds to reducing resources (such as reducing concurrency and shrinking queue capacity), medium load corresponds to maintaining the status quo, and high load corresponds to increasing resources (such as increasing concurrency, expanding queue capacity, and increasing processing speed). The generated resource configuration parameters directly affect the operation of the processing queue. For example, queue capacity limit parameters are used to prevent the queue from growing indefinitely; processing speed limit parameters are used to avoid downstream congestion caused by processing too quickly; and concurrency control parameters are used to control the number of parallel processing tasks.

[0111] This step enables dynamic adjustment of resource quotas for each processing queue based on real-time load, achieving adaptive load balancing, effectively alleviating queue backlog and latency accumulation issues, and improving overall processing efficiency; the formation of constraint queues provides a stable operating boundary for subsequent protocol parsing and rate limiting control.

[0112] S6 performs protocol parsing on the data segments in each constraint queue to obtain the corresponding initial parsing results. Please refer to [link / reference]. Figure 2 .

[0113] In this step, for each constraint queue, based on the target protocol type corresponding to the protocol data stream category bound to that queue, the corresponding protocol parser is invoked to completely parse each data segment in the queue. The protocol parser decodes the binary message of the data segment into structured data according to the message structure definition of the target protocol type, extracts the protocol header fields and payload content, and generates an initial parsing result. Each initial parsing result is associated and stored with the original data stream identifier and sequence identifier of that data segment to ensure that it can be correctly reassembled according to its source and order in subsequent steps.

[0114] During the protocol parsing process, operational data for each constraint queue is continuously acquired. This operational data includes at least one of queue backlog length information and queue waiting time information. For example, the number of data segments awaiting processing in the current queue (queue backlog length) and the waiting time of the data segment at the head of the queue (queue waiting time) can be read in real time through the queue management interface. Continuous acquisition can be achieved through timed sampling (e.g., every 100 milliseconds) or event triggering.

[0115] The acquired operational data is compared with preset operational data thresholds. The preset operational data thresholds include at least one of a backlog length threshold and a waiting time threshold.

[0116] S601: When the running data of any constraint queue is greater than or equal to a preset running data threshold, the target constraint queue whose running data exceeds the threshold is identified, and rate limiting control is immediately applied to the target constraint queue. Rate limiting control is used to reduce the rate at which data segments enter the processing queue. Specifically, this includes limiting the number of data segments entering the processing queue from the target constraint queue (e.g., using a token bucket algorithm to limit the number of segments enqueued per unit time), or reducing the rate at which data segments enter the target constraint queue (e.g., lengthening the enqueue interval). After rate limiting control, protocol parsing processing continues for the data segments already existing in the queue, without rate limiting the parsing process itself.

[0117] The system continuously monitors the operational data of the target constraint queue. When the operational data of the target constraint queue falls below a preset recovery threshold, the rate limiting control for the constraint queue is lifted. Lifting the rate limiting control includes lifting the limit on the number of data segments entering the queue, or restoring the data segment entry rate to a preset base entry rate (i.e., the normal queuing rate). After lifting the rate limiting control, the queue resumes its normal data segment reception and processing flow.

[0118] S602, if the running data of all constraint queues is less than the preset running data threshold, then each constraint queue is processed for stable operation, that is, the current data segment entry rate of each constraint queue is maintained, as well as the current queue capacity configuration and processing concurrency configuration are maintained, without any rate limiting intervention.

[0119] By using S6, the status of each queue is dynamically detected during the protocol parsing process, and rate limiting and recovery are performed adaptively to ensure the overall processing stability of the system.

[0120] To ensure clarity and conciseness in the description of the following embodiments, a brief introduction to the relevant terms is given first: Operational data refers to queue operation status parameters detected in real time during protocol parsing, including at least one of queue backlog length information and queue waiting time information.

[0121] The preset running data threshold refers to the pre-set critical value used to trigger flow limiting control, including at least one of the backlog length threshold and the waiting time threshold.

[0122] The target constraint queue refers to the constraint queue where the running data is greater than or equal to a preset running data threshold, and rate limiting control needs to be performed.

[0123] Rate limiting control refers to control operations that limit the rate or number of data segments entering the processing queue in order to alleviate queue backlog and latency accumulation.

[0124] The preset recovery threshold refers to the low water level threshold that is set in advance to release the flow restriction control. When the operating data is lower than this threshold, the flow restriction is released.

[0125] Stable operation processing means maintaining the current configuration and not implementing rate limiting intervention when all constraint queues' running data are normal.

[0126] The initial parsing result refers to the structured data obtained after parsing the data segment according to the protocol, including protocol header field values, payload content, and key information extracted during the parsing process.

[0127] A protocol parser is a program module implemented for a specific protocol type, used to decode binary data segments into structured data.

[0128] In some possible implementations, even with prior adaptive resource configuration, sudden traffic spikes can still cause queue backlogs. Without intervention, this backlog will continue to worsen, leading to a significant increase in parsing latency. Therefore, it is essential to monitor the queue status in real-time during the parsing process and implement dynamic rate limiting to prevent system overload.

[0129] This step is designed to solve this problem. Its core principle is to collect the running data (backlog length and waiting time) of each constraint queue in parallel and continuously during protocol parsing, and compare it with preset thresholds. When the upper limit threshold is exceeded, rate limiting control (voltage reduction) is triggered to actively limit the incoming rate of upstream data segments; when the data rate falls below the recovery threshold, rate limiting is lifted, and normal processing resumes. This level-based feedback control mechanism ensures that the system will not crash due to backlog overload and can quickly restore throughput capacity when the load decreases.

[0130] It should be noted that the initial parsing result obtained from each data segment should be stored in association with the original data stream identifier and sequence identifier of that data segment. The original data stream identifier is used to distinguish the data sources of different IoT devices, and the sequence identifier is used to ensure the sequential order of data segments within the same data stream. In this way, during the subsequent S7 aggregation and reorganization, the scattered parsing results can be categorized by source according to the original data stream identifier, and then sorted in the correct order according to the sequence identifier, ultimately restoring the complete and correctly ordered original data stream.

[0131] "Continuously acquiring runtime data for each constraint queue" can be achieved by sampling the queue status at fixed time intervals (e.g., every 100 milliseconds) using a separate detection thread, or by checking queue metrics after each parsing task is completed, to ensure real-time performance and low overhead.

[0132] Specifically, preset data thresholds are typically set to two values: a high threshold (triggering rate limiting) and a low threshold (restoring). For example, the high threshold for the queue length might be set to 1000 data segments, and the low threshold to 200 data segments. When the backlog length reaches 1000, rate limiting is initiated; when the backlog length decreases to 200, rate limiting is lifted. Rate limiting can be achieved simply by pausing the reception of new data from the network buffer to the processing queue, or by using a token bucket to add data at a lower rate.

[0133] This step enables adaptive flow control during protocol parsing, significantly improving the system's resilience to sudden traffic surges, preventing the infinite accumulation of parsing latency, and ensuring the reliability and response speed of industrial IoT edge processing. Simultaneously, it provides the necessary identifier associations for S7 aggregation and reorganization.

[0134] S7. Based on the original data stream identifier and the sequence identifier, the initial parsing results are summarized and reorganized to obtain the final parsing result.

[0135] In this step, all initial parsing results are grouped according to the original data stream identifier they carry. Initial parsing results with the same original data stream identifier are grouped together, and all results in this group belong to the original communication data stream sent by the same IoT device. Through attribution processing, parsing results scattered across different constraint queues and completed at different times are re-aggregated according to data source.

[0136] For each group of initial parsing results with the same original data stream identifier, obtain the sequence identifier (e.g., an incrementing sequence number or timestamp) corresponding to each result, and arrange the results within the group in ascending order of the sequence identifiers, so that each result restores its chronological order in the original communication data stream.

[0137] The payloads or structured data from the initial parsing results, which are arranged in the correct order under the same original data stream identifier, are sequentially concatenated or combined to form a complete and correctly ordered final parsing result. The final parsing result can be output to the backend business system for storage, analysis, or further processing.

[0138] Through the above processing, the discrete results after multi-queue parallel parsing are reassembled according to data source and original order, restoring the complete and correct original data stream content. Simultaneously, through the above attribution, sorting, and summarization reorganization, the dispersed parsed protocol data units are restored into a complete original communication data stream, achieving orderly reorganization of protocol data units in multi-protocol access scenarios and ensuring the correctness of protocol processing.

[0139] To ensure clarity and conciseness in the description of the following embodiments, a brief introduction to the relevant terms is given first: Attribution refers to classifying the initial parsing results according to their data source based on the original data stream identifier, with parsing results of the same original data stream identifier grouped together.

[0140] Sequential sorting refers to arranging the results within the parsing result group of the same original data stream identifier in the order in which the original data segments were sent, according to the sequence identifier.

[0141] Summarizing and reorganizing refers to sequentially combining the payloads or structured data from the sorted initial parsing results to form a complete original data stream.

[0142] The final parsing result refers to the complete and correctly ordered original communication data stream content obtained after attribution, sorting, summarization, and reorganization.

[0143] In some possible implementations, steps S1-S6 split the original communication data stream into data segments and distribute them to constraint queues of different protocol data stream categories for independent parsing. Due to differences in processing speed and resource allocation among different queues, different data segments of the same original communication data stream may be parsed out of order. Directly outputting the parsing results without summarizing and reassembling them will result in corrupted final data, making it impossible to reconstruct the original information.

[0144] This step is designed to solve this problem. Its core principle is as follows: using the original data stream identifier allocated in S1, the scattered parsing results are grouped according to the data source; then, using the sequence identifier allocated in S2, the grouped results are arranged in the correct order; finally, the sorted payloads are concatenated in sequence to recover the complete and correct original data stream. Specifically: The attribution operation can be completed using a hash table: Using the original data stream identifier as the key, the corresponding initial parsing result is added to the list associated with that key. After attribution, the results in each list are sorted according to their sequential identifiers (e.g., using a comparison sorting algorithm). Once all data segments of a data stream have been parsed (e.g., by checking the continuity of sequential identifiers or using a timeout mechanism), the sorted results are sequentially extracted for their payloads and concatenated to form the final parsed result. The reassembled data is completely identical in content and order to the original communication data stream.

[0145] This step enables the restoration of a complete and ordered original data stream from discrete and disordered analytical results, solving the problem of data reorganization after multi-queue parallel processing and ensuring the correctness and reliability of the output data of the industrial intelligent gateway.

[0146] It should be noted that the various preset parameters involved in the above embodiments (including but not limited to preset time windows, preset running data thresholds (backlog length thresholds, waiting time thresholds), preset recovery thresholds, preset load level tables, preset resource configuration libraries, preset protocol parsing rule libraries, preset protocol feature information, etc.) are all pre-calibrated by those skilled in the art based on the actual needs of the industrial IoT scenario, equipment performance, historical running data, or experimental testing. Under different application scenarios, the specific values ​​of the above parameters can be obtained through offline testing, simulation, or online adaptive adjustment. The specific values ​​given in this embodiment (such as a preset time window of 5 seconds, a high threshold of 1000 data segments for the queue length, and a low threshold of 200 data segments, etc.) are only for illustrative purposes and do not constitute a limitation on the scope of protection of this application. Those skilled in the art can flexibly configure or dynamically adjust the above parameters according to the actual system load characteristics, processing resource capabilities, and real-time requirements, which still falls within the scope of protection of this application.

[0147] Through the above embodiments, this application classifies protocol data units (i.e., data segments) in a fine-grained manner according to the protocol type identifier and data characteristics of the communication data stream, constructs independent processing queues, and performs dynamic load assessment based on historical execution load information and real-time queue backlog information. It adaptively determines resource configuration parameters such as queue capacity limits, processing rate limits, and concurrency control to form a constrained queue. This effectively isolates resource competition between different protocols and data segments with different load characteristics under the same protocol, avoids the accumulation of parsing delays caused by mixed processing of long and short messages, and significantly improves the overall processing efficiency and response capability in multi-protocol concurrent access scenarios.

[0148] This application continuously monitors the operational data (backlog length, waiting time) of each constraint queue during the protocol parsing process. When the operational data exceeds a preset threshold, it automatically executes flow limiting control and lifts the flow limiting after the backlog is relieved, forming an adaptive flow regulation mechanism based on water level feedback to prevent system overload and crash. At the same time, it assigns, sorts, and reassembles the parsing results through the original data stream identifier and sequence identifier, ensuring the correctness of the data order after parallel processing. This solves the problem of data disorder caused by parallel parsing of multiple queues and ensures the integrity and reliability of the output data of the industrial intelligent gateway.

[0149] The foregoing has shown and described the basic principles, main features, and advantages of this application. Those skilled in the art should understand that this application is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of this application. Various changes and modifications can be made to this application without departing from the spirit and scope thereof, and all such changes and modifications fall within the scope of this application as claimed. The scope of protection of this application is defined by the appended claims and their equivalents.

Claims

1. A method for multi-protocol access of multiple IoT devices, characterized in that, The method includes: Obtain the communication data streams and their original data stream identifiers sent by multiple IoT devices, and parse the communication data streams to obtain the protocol type identifier; The communication data stream is divided into data segments to obtain multiple data segments and their corresponding sequence identifiers, and the data characteristics of each data segment are obtained. The multiple data segments are classified based on the protocol type identifier and the data characteristics to obtain multiple protocol data stream categories, and corresponding processing queues are constructed based on the multiple protocol data stream categories; Obtain historical execution load information for each protocol data stream category within a preset time window, as well as real-time queue backlog information for the current processing queue; The historical execution load information and the real-time queue backlog information are subjected to load assessment to obtain assessment results. Based on the assessment results, resource configuration parameters are determined, and the resource configuration parameters are bound to the corresponding processing queues to obtain constraint queues. The data segments in each constraint queue are parsed according to the protocol to obtain the corresponding initial parsing results; During the protocol parsing process, the running data of each constraint queue is continuously acquired, and the running data is compared with a preset running data threshold. When the running data of any constraint queue is greater than or equal to the preset running data threshold, rate limiting control is applied to the corresponding constraint queue, and protocol parsing processing continues. When the running data of the corresponding constraint queue is less than the preset recovery threshold, the flow restriction control of the constraint queue is lifted; Based on the original data stream identifier and the sequence identifier, the initial parsing results are summarized and reorganized to obtain the final parsing result.

2. The method of claim 1, wherein, The classification of the multiple data segments based on the protocol type identifier and the data characteristics yields multiple protocol data stream categories, including: The protocol type identifier is matched with a preset protocol parsing rule base to obtain the target protocol type; The data features are analyzed to obtain the data feature analysis results; The target protocol type and the data feature parsing results are combined and mapped to obtain a classification mapping relationship; Based on the classification mapping relationship, the multiple data segments are classified to obtain multiple protocol data stream categories.

3. The method of claim 2, wherein, The construction of corresponding processing queues based on the multiple protocol data stream categories includes: Multiple initial queues are constructed, and the initial queues are associated with the protocol data stream categories to obtain associated queues that correspond one-to-one with the initial queues of the protocol data stream categories; Based on the protocol data stream category, the associated queue is configured with queue capacity and processing priority to obtain the processing queue corresponding to the protocol data stream category.

4. The method of claim 1, wherein, The process of performing load assessment on the historical execution load information and the real-time queue backlog information to obtain assessment results includes: Feature extraction is performed on the historical execution load information to obtain historical processing latency information and historical processing throughput information; Feature extraction is performed on the real-time queue backlog information to obtain queue backlog length information and queue waiting time information; The historical processing latency information, the historical processing throughput information, the queue backlog length information, and the queue waiting time information are comprehensively processed to obtain the evaluation result.

5. The method of claim 4, wherein, The process of determining resource allocation parameters based on the evaluation results includes: The evaluation results are matched with a preset load level table to obtain the target load level; Based on the target load level, the target resource configuration requirement type is determined in conjunction with the preset resource configuration library; Resource configuration parameters are generated based on the resource configuration requirement type. These parameters include queue capacity limit parameters, processing rate limit parameters, and processing concurrency control parameters.

6. The method of claim 5, wherein, The step of binding the resource configuration parameters with the corresponding processing queue to obtain a constraint queue includes: Based on the queue capacity limit parameter, the processing rate limit parameter, and the processing concurrency control parameter, capacity constraints, rate constraints, and concurrency constraints are applied to the corresponding processing queue to obtain the constrained processing queue.

7. The method of claim 6, wherein, The operational data includes at least one of queue backlog length information and queue waiting time information. When the operational data of any constrained queue is greater than or equal to a preset operational data threshold, rate limiting control is applied to the corresponding constrained queue, including: When the running data of any constraint queue is greater than or equal to the preset running data threshold, the target constraint queue whose running data is greater than or equal to the preset running data threshold is determined. Rate limiting control is applied to the target constraint queue to reduce the rate at which data segments enter the processing queue. The rate limiting control includes limiting the number of data segments from the target constraint queue entering the processing queue, or reducing the rate at which data segments enter the target constraint queue.

8. The method of claim 7, wherein, The step of releasing the flow control of the constraint queue when the running data of the corresponding constraint queue is less than the preset recovery threshold includes: When the running data of the target constraint queue is lower than the preset recovery threshold, the flow restriction control of the target constraint queue is lifted. The lifting of the flow restriction control includes lifting the limit on the number of data segments entering the target constraint queue or restoring the data segment entry rate of the target constraint queue to the preset base entry rate.

9. The method of claim 8, wherein, The process of summarizing and reorganizing the initial parsing results based on the original data stream identifier and the sequence identifier to obtain the final parsing result includes: Based on the original data stream identifier, each initial parsing result is assigned to a specific group to obtain the assigned initial parsing result. Based on the sequence identifier, the initial parsing results after attribution are sorted sequentially to obtain the sorted initial parsing results; The sorted initial parsing results are summarized and reorganized to obtain the final parsing results.

10. The method of claim 1, wherein, After comparing the running data with a preset running data threshold, the method further includes: When the running data of each constraint queue is less than the preset running data threshold, stable operation processing is performed on each constraint queue. The stable operation processing includes maintaining the current data segment entry rate of each constraint queue or maintaining the current queue capacity configuration and processing concurrency configuration of each constraint queue.