Internet of Things equipment long protocol text sequence modeling method and device, equipment and medium

Abstract

The invention discloses an Internet of Things equipment long protocol text sequence modeling method and device, equipment and a medium, and relates to the cross technical field of the Internet of Things communication technology, deep learning model optimization and the natural language processing technology, and the method comprises the steps: collecting Internet of Things equipment protocol texts of a plurality of manufacturers; performing data cleaning, word segmentation and semantic marking operation on the collected protocol text to generate a structured sequence; designing a hierarchical residual position correction module based on dynamic residual position coding, constructing a protocol model, and training the protocol model through the structured sequence; and analyzing the to-be-analyzed protocol text based on the trained protocol model, and generating a mapping rule and type conversion logic between the to-be-analyzed protocol text and the target protocol. The problems of position information attenuation, gradient instability, poor dynamic adaptability and the like in long protocol analysis can be effectively solved.

Description

Technical Field

[0001] The present application relates to the interdisciplinary technical field of Internet of Things communication technology, deep learning model optimization, and natural language processing technology, and specifically to a method, apparatus, device, and medium for modeling long protocol text sequences of Internet of Things devices. Background Art

[0002] The rapid development of IoT technology has driven the large-scale deployment of devices in scenarios such as smart campuses and the Industrial Internet of Things. However, differences in communication protocols between vendors' devices have become a core barrier to data interoperability. Specifically:

[0003] (1) Protocol heterogeneity complicates things: IoT devices (such as sensors and controllers) from different manufacturers use private communication protocols (such as Modbus, MQTT, and CoAP). Their protocol formats, field definitions (such as data types and validation rules), and data encoding methods (binary, JSON, and XML) differ significantly. For example, in the temperature sensor protocol of manufacturer A, the temperature field is defined as a 16-bit unsigned integer, while the same type of field from manufacturer B may use a 32-bit floating point number with a unit identifier (such as "°C"). This heterogeneity requires that protocol parsing be customized for each type of device, forming "protocol islands."

[0004] (2) Inefficient manual configuration: Traditional protocol adaptation relies on manual rule configuration (such as regular expression matching and XPath parsing), requiring the pre-compilation of a large number of rule bases to cover different protocol formats. However, as protocol versions iterate (such as HTTP / 1.1 upgraded to HTTP / 2) or vendors extend fields (such as the addition of an energy efficiency identification field), the rule base needs to be frequently updated, resulting in high maintenance costs.

[0005] (3) Insufficient parsing capabilities for long protocol texts: Sequence models based on deep learning provide a possibility for intelligent parsing of IoT protocols. However, complex IoT protocols often contain nested fields, dynamic loads, or verification rules, and the length of the protocol text exceeds 1,000 characters (for example, an industrial PLC control protocol contains 128 nested fields with a total length of 1,500 characters). Existing sequence models (such as LSTM and GRU) are difficult to capture long-distance dependencies due to the gradient vanishing problem, resulting in a high parsing error rate for key fields (such as checksums and version numbers). Although the standard Transformer model supports parallel computing, its fixed sinusoidal position encoding significantly attenuates position information in long sequences, often causing the position encoding weight of the protocol tail field to be reduced to less than 15% of the head field, affecting the accuracy of field mapping.

[0006] (4) Lack of dynamic protocol adaptability: Existing tools (such as the open source library Eclipse Paho and industrial protocol converters) rely on static rule configuration and cannot dynamically adapt to changes in protocol formats. For example, after a manufacturer added an "energy efficiency level" field to the protocol, traditional parsing tools discarded data because the rules were not updated, affecting the real-time performance of the campus energy efficiency analysis system.

[0007] It can be seen that the existing sequence models for parsing IoT device protocols have the following main problems: 1. Insufficient traditional positional encoding: The standard sinusoidal positional encoding of the Transformer model generates a position vector through a predefined function. Although it can distinguish the sequence order, it cannot adapt to the dynamic characteristics of long protocol texts. For example, when parsing nested fields, the positional relationship between parent and child fields requires higher-precision position sensitivity, and the periodic characteristics of sinusoidal encoding lead to confusion in the position information of deeply nested fields, which in turn causes semantic parsing deviations; 2. Residual connections are not deeply integrated with positional encoding: Existing residual connection technologies (such as ResNet) are mainly used to alleviate the gradient vanishing problem in deep network training. However, when it is directly applied to the Transformer model, it is not optimized for position encoding. After the 6-layer encoder stack of the Transformer model, the position information of the protocol tail field is attenuated by more than 60% due to the multi-layer nonlinear transformation, resulting in the model being unable to accurately identify field boundaries; Limitations of block processing and sliding window technology: For long protocol texts, existing solutions often use block processing (such as splitting the protocol into multiple 512-character segments) or sliding windows (local context modeling). However, such methods destroy the overall structure of the protocol and cannot model global dependencies across blocks (such as the version number of the protocol header affecting the data format definition of the tail), resulting in errors in the generation of field mapping rules. Summary of the Invention

[0008] The present application provides a method, apparatus, device and medium for modeling long protocol text sequences of IoT devices, which can effectively solve problems such as position information attenuation, gradient instability and poor dynamic adaptability in long protocol parsing.

[0009] In a first aspect, an embodiment of the present application provides a method for modeling a long protocol text sequence of an IoT device, the method comprising:

[0010] Collect IoT device protocol texts from multiple manufacturers, perform data cleaning, word segmentation, and semantic tagging on the collected protocol texts to generate structured sequences;

[0011] Based on the implementation of dynamic residual position coding and the design of a layered residual position correction module, a protocol model is constructed, and the protocol model is trained through the structured sequence;

[0012] The protocol text to be parsed is parsed based on the trained protocol model, and mapping rules and type conversion logic between the protocol text to be parsed and the target protocol are generated.

[0013] In conjunction with the first aspect, in one embodiment, collecting IoT device protocol texts from multiple manufacturers, and performing data cleaning, word segmentation, and semantic tagging operations on the collected protocol texts to generate a structured sequence specifically includes:

[0014] Collect IoT device protocol texts from multiple manufacturers in smart parks and industrial IoT scenarios;

[0015] Use regular expressions to match invalid fields in the agreement text to clean the agreement text and unify the structure of the agreement text;

[0016] The BERT model pre-trained on the IoT protocol corpus is used to perform fine-grained word segmentation on the protocol text, and semantic tags are annotated to generate structured sequences.

[0017] In conjunction with the first aspect, in one implementation, the dynamic residual position coding is implemented as follows:

[0018] Based on the sinusoidal position encoding and combined with the learnable residual vector, the final position encoding is generated by superposition. The specific method is:

[0019] PE final =PE sniusoidal +α·W res ·PE learned

[0020] Among them, PE sniusoidal represents the basic position code generated based on the sine function or cosine function, α represents the protocol length adaptation coefficient, which is dynamically calculated based on the protocol length, α=log(length / 500), length represents the protocol length, W res Represents the trainable weight matrix, PE learned Represents the adaptive residual vector generated by a two-layer fully connected network, with activation functions GELU and PE final Represents the final position encoding.

[0021] In conjunction with the first aspect, in one embodiment,

[0022] The structure of the layered residual position correction module includes input, processing and output;

[0023] The input of the layered residual position correction module is the output code of the current Transformer layer;

[0024] The processing of the layered residual position correction module is to input it into the fully connected network after layer normalization;

[0025] The output of the layered residual position correction module is the residual increment of the position code, which is added to the basic position code to update it. The position update mechanism is specifically as follows:

[0026] PE updated =PE final +Dropout(GELU(W c LayerNorm(PE final )))

[0027] Among them, PE updated represents the updated position encoding, Dropout represents random inactivation, LayerNorm represents layer normalization, W c represents the trainable parameters of the layered residual position correction module, and GELU represents the activation function.

[0028] In combination with the first aspect, in one embodiment, the protocol model includes 6 layers of Transformer encoders, and a layered residual position correction module is inserted after each layer of Transformer encoder, and the weight matrix W is dynamically adjusted according to the length of the input protocol text. res , to achieve adaptation of the protocol text length, the input of the layered residual position correction module is the output code of the current Transformer layer, and the output is the residual increment of the position code.

[0029] In conjunction with the first aspect, in one embodiment, the protocol text to be parsed is parsed based on the trained protocol model, and mapping rules and type conversion logic between the protocol text to be parsed and the target protocol are generated, specifically including:

[0030] After word segmentation and semantic tagging, the protocol text to be parsed is converted into a high-dimensional vector sequence through the Embedding layer and added to the dynamic residual position code;

[0031] A multi-layer encoding method is used, where the first layer is used to initially capture the local dependencies between fields, the third layer is used to enhance the position weights of nested fields through a hierarchical residual position correction module, and the sixth layer is used to output the global semantic code.

[0032] The output global semantic encoding is input into the fully connected layer and Softmax classifier to output the semantic label of the protocol field. At the same time, the attention weight between the protocol text to be parsed and the target protocol is calculated to generate mapping rules and type conversion logic.

[0033] In combination with the first aspect, in one implementation, the global semantic encoding includes field types, rules, and cross-protocol mapping relationships.

[0034] In a second aspect, an embodiment of the present application provides a device for modeling a long protocol text sequence of an Internet of Things device, the device comprising:

[0035] The acquisition module is used to collect IoT device protocol texts from multiple manufacturers, and perform data cleaning, word segmentation, and semantic tagging operations on the collected protocol texts to generate structured sequences;

[0036] A construction module, which is used to construct a protocol model based on dynamic residual position coding implementation and layered residual position correction module design, and train the protocol model through the structured sequence;

[0037] The execution module is used to parse the protocol text to be parsed based on the trained protocol model, and generate mapping rules and type conversion logic between the protocol text to be parsed and the target protocol.

[0038] In a third aspect, an embodiment of the present application provides a long protocol text sequence modeling device for an Internet of Things device, wherein the long protocol text sequence modeling device for an Internet of Things device comprises a processor, a memory, and an Internet of Things device long protocol text sequence modeling program stored on the memory and executable by the processor, wherein when the Internet of Things device long protocol text sequence modeling program is executed by the processor, the steps of the above-mentioned Internet of Things device long protocol text sequence modeling method are implemented.

[0039] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a long protocol text sequence modeling program for an Internet of Things device is stored. When the long protocol text sequence modeling program for an Internet of Things device is executed by a processor, the steps of the above-mentioned method for modeling a long protocol text sequence for an Internet of Things device are implemented.

[0040] The beneficial effects of the technical solutions provided in the embodiments of the present application include:

[0041] It deeply integrates the IoT communication protocol parsing needs and deep learning model optimization technology, and effectively solves the semantic modeling problem of long protocol texts through dynamic residual position encoding and hierarchical correction mechanism. At the same time, it achieves breakthroughs in long sequence parsing capabilities, significantly optimizes training stability, and enhances dynamic protocol adaptability, providing efficient and adaptive protocol compatibility solutions for scenarios such as smart parks and industrial IoT, while promoting the innovative application of artificial intelligence technology in vertical fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a flowchart of the method for modeling long protocol text sequences for IoT devices in this application;

[0043] Figure 2This is a schematic diagram of the structure of dynamic residual position coding;

[0044] Figure 3 Schematic diagram of the structure of the layered residual position correction module;

[0045] Figure 4 This is a schematic diagram of the functional modules of the device for modeling long protocol text sequences of IoT devices in this application;

[0046] Figure 5 Schematic diagram of the hardware structure of the device for modeling long protocol text sequences of IoT devices in this application. DETAILED DESCRIPTION

[0047] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0048] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0049] First, the present invention provides a method for modeling long protocol text sequences for IoT devices. This method combines residual connections with dynamic position encoding to perform sequence modeling, thereby parsing long IoT protocol texts. By improving the position encoding mechanism of the Transformer model and integrating residual connection technology, it effectively solves problems such as position information attenuation, gradient instability, and poor dynamic adaptability in long protocol parsing. Specifically, the following effects can be achieved:

[0050] (1) Intelligent parsing of IoT communication protocols: This technology addresses heterogeneous communication protocols (such as MQTT, CoAP, Modbus-TCP, and HTTP) for devices from multiple vendors (such as sensors, controllers, and energy monitoring devices) in scenarios such as smart parks and industrial IoT. It effectively addresses the "protocol island" problem caused by differences in protocol field definitions, data formats, and encoding rules. Furthermore, it supports efficient parsing and semantic mapping of long protocol texts (such as communication protocols containing nested fields, validation rules, and dynamic payloads), meeting the data interoperability requirements of complex IoT devices.

[0051] (2) Deep learning and sequence modeling optimization: Based on the improved technology of the Transformer model, we focus on solving the problem of position information modeling for long sequence data (length exceeding 1000 characters), breaking through the insufficient position sensitivity and gradient attenuation bottleneck of traditional position encoding (such as sine / cosine encoding) in long texts. At the same time, we combine the residual network (ResNet) with the self-attention mechanism to optimize the stability of deep model training and enhance the ability to capture long-distance dependencies between protocol fields;

[0052] (3) Application of natural language processing in IoT protocols: Adapt natural language processing technologies (such as word segmentation, semantic tagging, and sequence annotation) to the field of IoT protocol text analysis to achieve structured extraction and semantic understanding of key protocol fields (such as data types, communication rules, and check bits). At the same time, based on the domain specificity of protocol texts (high-density professional terms and fixed grammatical structures), design lightweight model architectures and adaptive training strategies to achieve "plug-and-play" access to devices from multiple manufacturers, thereby reducing the development and operation and maintenance costs of protocol adaptation.

[0053] In one embodiment, referring to Figure 1 , Figure 1 This is a flow chart of the method for modeling long protocol text sequences of IoT devices in this application. Figure 1 As shown in Figure 2, the long protocol text sequence modeling method for IoT devices includes:

[0054] S1: Collect IoT device protocol texts from multiple manufacturers, perform data cleaning, word segmentation, and semantic tagging on the collected protocol texts to generate structured sequences;

[0055] S2: Based on the dynamic residual position coding implementation and the layered residual position correction module design, a protocol model is constructed, and the protocol model is trained through the structured sequence;

[0056] S3: Parse the protocol text to be parsed based on the trained protocol model, and generate mapping rules and type conversion logic between the protocol text to be parsed and the target protocol.

[0057] Furthermore, in one embodiment, protocol texts of IoT devices from multiple manufacturers are collected, and data cleaning, word segmentation, and semantic tagging operations are performed on the collected protocol texts to generate structured sequences, specifically including:

[0058] S101: Collect IoT device protocol texts from multiple manufacturers in smart parks and industrial IoT scenarios;

[0059] Specifically, the collected protocol texts cover mainstream protocols such as MQTT, Modbus-TCP, CoAP, and HTTP / 2, including device control instructions, sensor data reporting, and device status queries. The length of the protocol text ranges from 200 to 2000 characters.

[0060] S102: using regular expressions to match invalid fields in the agreement text, so as to clean the agreement text and unify the structure of the agreement text;

[0061] Specifically, regular expressions are used to match invalid fields in the protocol text, such as debug information and version comments, to ensure the integrity of core fields (such as data_type and checksum), and to unify the timestamp format (UTC), numeric types (such as converting uint16 to standard integer representation), and unit identifiers (such as "℃" to "C").

[0062] S103: Use the BERT model pre-trained on the IoT protocol corpus to perform fine-grained word segmentation on the protocol text, and annotate the semantic tags to generate a structured sequence.

[0063] Specifically, we use the domain-optimized BERT model to perform fine-grained word segmentation on the agreement text and annotate semantic tags. The following example shows:

[0064] Field type: such as<data_type:int> 、 <payload:binary>;

[0065] Protocol rules: such as <checksum:crc32>、<version:1.1> ;

[0066] At the same time, it generates and outputs structured sequences, such as:

[0067] [ <header>,<data_type:float>, <length:uint16> , <payload:binary>,<checksum:he x> ].

[0068] Furthermore, after the above data preparation and preprocessing are completed, the protocol model is constructed and trained.

[0069] For the dynamic residual position coding implementation, specifically:

[0070] Based on the sinusoidal position encoding and combined with the learnable residual vector, the final position encoding is generated by superposition. The specific method is:

[0071] PE final =PE sniusoidal +α·W res ·PE learned

[0072] Among them, PE sniusoidal represents the basic position code generated based on the sine function or cosine function, α represents the protocol length adaptation coefficient, which is dynamically calculated based on the protocol length, α=log(length / 500), length represents the protocol length, W res Represents the trainable weight matrix, PE learned Represents the adaptive residual vector generated by a two-layer fully connected network, with activation functions GELU and PE final Represents the final position encoding.

[0073] Specifically, based on the traditional sinusoidal position encoding, a learnable residual vector is introduced to generate the final position encoding through superposition. Furthermore, for residual connection fusion, in each layer of Transformer encoder, the position encoding and self-attention output are fused through residual connections to enhance the stable transmission of position information.

[0074] See also Figure 2 As shown in the figure, it is a structural diagram of dynamic residual position coding, which shows the generation process of dynamic residual position coding, including sinusoidal coding, learnable residual vector and final position coding after superposition. The residual weight is dynamically adjusted by the protocol length adaptive coefficient α to enhance the position sensitivity of long protocols.

[0075] The design of the layered residual position correction module includes input, processing, and output. The input is the output code of the current Transformer layer. The processing is normalized through the layer and then input to the fully connected network. The output is the residual increment of the position code, which is added to the basic position code to update. The position update mechanism is as follows:

[0076] PE updated =PE final +Dropout(GELU(W c LayerNorm(PE final )))

[0077] Among them, PE updated represents the updated position encoding, Dropout represents random inactivation, LayerNorm represents layer normalization, W c Represents the trainable parameters of the layered residual position correction module, and GELU represents the activation function. Figure 3 The figure shows the structure of the layered residual position correction module. After the input code is normalized by the layer, the residual increment is generated through the fully connected network, and the position code is dynamically updated to alleviate the information attenuation in the deep network.

[0078] Specifically, a lightweight fully connected network (hierarchical residual position correction module) is added after each layer of the Transformer encoder to dynamically adjust the weight distribution of the position encoding. The input of the hierarchical residual position correction module is the output encoding of the current Transformer layer, and the output is the residual increment of the position encoding.

[0079] Furthermore, a protocol length adaptive mechanism is added to dynamically adjust the weight matrix W according to the length of the input protocol text. res For example, for very long protocols (>1500 characters), sparse initialization is used to enhance position sensitivity; for short protocols (<500 characters), dense initialization is used to preserve global semantics. Furthermore, segmented position encoding is introduced to divide the protocol text into logical segments (such as the protocol header, payload, and checksum), and each segment is assigned an independent position encoding coefficient to improve the parsing accuracy of nested fields.

[0080] For the architectural configuration of the protocol model constructed in this application:

[0081] Number of encoder layers: 6-layer Transformer encoder, each hidden layer dimension 512, number of attention heads 8;

[0082] Layered Residual Position Correction Module: A layered residual position correction module is inserted after each layer of the Transformer encoder, with an input dimension of 512 and an output dimension of 512;

[0083] Protocol length adaptation: Dynamically initialize the weight matrix W according to the length of the input protocol text res ,The short protocol uses the identity matrix and the long protocol uses the sparse matrix.

[0084] Next, the protocol model is trained, and the loss function includes:

[0085] Cross entropy loss: used for protocol field classification;

[0086] Contrastive loss: Optimize the similarity of cross-protocol field mappings (Margin = 0.2);

[0087] Optimizer: AdamW, learning rate 2e-5, weight decay 0.01;

[0088] The training configuration includes:

[0089] Batch size: 64 (NVIDIA RTX4090 GPU, 4 cards in parallel);

[0090] Training epochs: 100, early stopping strategy (patience = 10);

[0091] Gradient clipping: threshold 1.0 to prevent gradient explosion.

[0092] Furthermore, in one embodiment, the protocol text to be parsed is parsed based on the trained protocol model, and mapping rules and type conversion logic between the protocol text to be parsed and the target protocol are generated, specifically including:

[0093] S301: After word segmentation and semantic tagging, the protocol text to be parsed is converted into a high-dimensional vector sequence through the Embedding layer and added to the dynamic residual position code. In practical applications, it is converted into a 512-dimensional vector.

[0094] S302: A multi-layer encoding method is used, where the first layer is used to initially capture local dependencies between fields, the third layer is used to enhance the position weights of nested fields through a hierarchical residual position correction module, and the sixth layer is used to output a global semantic code;

[0095] Specifically, a multi-layer encoding method is adopted. The first layer is used to initially capture local dependencies between fields (such as the association between data_type and length). The third layer is used to enhance the position weight of nested fields (such as substructures in payload) through a hierarchical residual position correction module. The sixth layer is used to output a global semantic code. The global semantic code includes field types, rules, and cross-protocol mapping relationships.

[0096] S303: Input the output global semantic code into the fully connected layer and Softmax classifier, and output the semantic label of the protocol field (such as <temperature:float>), and at the same time calculate the attention weight between the protocol text to be parsed and the target protocol, generate mapping rules (such as manufacturer A.temp → target Schema.temperature) and type conversion logic (such as uint16 → float needs to be divided by 100).

[0097] The following is a detailed description of the method for modeling long protocol text sequences of IoT devices in this application with reference to multiple examples.

[0098] Example 1

[0099] Scenario: Analysis of the "PLC Control Protocol" (1800 characters long) in an industrial IoT scenario;

[0100] Input protocol: contains nested fields {control:{cmd:uint8,params:[float,float]},checksum:crc32};

[0101] Dynamic encoding: Generates segment position codes based on the protocol length to distinguish the logical boundaries between control and checksum;

[0102] Layered correction: In the 4th layer encoder, the correction module increases the position weight of the params array to avoid type misjudgment (such as identifying float as int);

[0103] Output mapping: Automatically generates the rule control.params[0]→targetSchema.parameter1 and verifies the integrity of the crc32 field.

[0104] Example 2

[0105] Scenario: Protocol matching between an "intelligent lighting controller" (Manufacturer X protocol) and an "energy management system" (Target Schema) in a smart park.

[0106] 1) Input protocol

[0107] {

[0108] "cmd":"light_control",

[0109] "params":{"brightness":80,"mode":"auto"},

[0110] "timestamp":1672531200,

[0111] "crc":"0xA3F2"

[0112] };

[0113] 2) Parsing and matching process

[0114] Word segmentation and tagging:

[0115] [ <cmd:string> , <params:object> , <brightness:uint8> , <mode:string> , <timestamp:int64> , <crc:hex>];

[0116] Dynamic position coding: protocol length 620 characters, α = 1.1, enhance the position weight of the CRC field;

[0117] Residual correction: The second-layer encoder corrects the positional association between brightness and mode to avoid type confusion;

[0118] Output mapping rules:

[0119] cmd→command,

[0120] params.brightness→brightness_level(uint8→0-100%),

[0121] crc→checksum(hex→base64).

[0122] Example 3

[0123] Scenario: In an industrial IoT system at an automobile manufacturer, multiple PLCs (programmable logic controllers) use communication protocols from different manufacturers. These protocols contain nested fields and dynamic payloads, and are often longer than 1,500 characters. Traditional parsing tools are unable to process long protocols, resulting in delays in device control commands and impacting production line efficiency.

[0124] Implementation steps:

[0125] 1) Input protocol example

[0126] {

[0127] "header":{"version":"1.2","device_id":"PLC_001"},

[0128] "command":{

[0129] "type":"motor_control",

[0130] "params":{"speed":1500,"direction":"forward","acceleration":0.5}

[0131] },

[0132] "checksum":"0x8F3A",

[0133] "timestamp":1672531200

[0134] }

[0135] Protocol length: 1620 characters;

[0136] 2) Data preprocessing

[0137] Word segmentation and tagging: Use the domain-optimized BERT model to generate structured sequences:

[0138] [ <header:object> , <version:string>,<device_id:string>, <command:object> , <type:string> , <params:object> , <speed:uint16> , <direction:string> , <acceleration:float> , <checksum:hex> , <timestamp:int64>]

[0139] Dynamic residual coding: Based on the protocol length (1620 characters), α=1.25 is calculated to enhance the position sensitivity of nested fields (such as params);

[0140] 3) Model analysis and output

[0141] Multi-layer coding: The third-layer encoder uses a layered residual position correction module to adjust the associated weights between speed and acceleration to avoid misjudgment of numerical types. The sixth-layer encoder outputs a global semantic code and accurately identifies the checksum field's CRC-16 verification algorithm.

[0142] Mapping rule generation:

[0143] command.params.speed→targetSchema.motor_speed(uint16→rpm),

[0144] command.params.acceleration→targetSchema.acceleration_rate(float→0.0-1.0).

[0145] Example 4

[0146] Scenario: A smart home platform needs to integrate temperature and humidity sensors from multiple manufacturers (Manufacturer A, B, and C). The protocol formats differ significantly (for example, temperature units and data encoding methods), making it difficult to manage data in a unified manner.

[0147] Implementation steps:

[0148] 1) Input protocol comparison

[0149] Manufacturer A's protocol: {temp:25,unit:"C",hum:60} (JSON format, 400 characters long);

[0150] Manufacturer B Agreement: <sensor> <t> 25< / t> C <h> 60< / h> < / sensor> (XML format, 600 characters long);

[0151] Manufacturer C protocol: 0x19 0x43 0x3C (binary format, temperature = 25, unit "℃", humidity = 60);

[0152] 2) Preprocessing and parsing

[0153] Convert XML and binary protocols into an intermediate JSON format. Short protocols (400 characters) use dense residual weights to preserve global semantics, while binary protocols use segmented position encoding to enhance field boundary recognition.

[0154] 3) Cross-protocol matching output

[0155] The semantic mapping rules are:

[0156] Manufacturer A.temp→Unified Schema.temperature(int→float),

[0157] Manufacturer Bt→Unified Schema.temperature(string→float),

[0158] Manufacturer C.0x19 → Unified Schema.temperature (hex → float)

[0159] At the same time, the conversion code is automatically generated (for example, hexadecimal to floating point conversion requires dividing by 10).

[0160] The method for modeling long protocol text sequences of IoT devices in this application is specifically: (1) dynamic position-sensitive modeling: enhancing the adaptability of position coding to long protocol texts, accurately distinguishing nested fields and long-distance dependencies; (2) gradient-stable transmission: ensuring the complete transmission of position information in deep networks through the deep fusion of residual connections and position coding; (3) dynamic protocol adaptation: supporting automatic perception of protocol format changes and mapping rule generation, reducing manual intervention.

[0161] The long protocol text sequence modeling method for IoT devices in this application can achieve:

[0162] (1) Intelligent parsing of IoT communication protocols: This solution addresses heterogeneous communication protocols (such as MQTT, CoAP, Modbus-TCP, and HTTP) for devices from multiple vendors (such as sensors, controllers, and energy monitoring devices) in scenarios such as smart parks and industrial IoT, solving the "protocol island" problem caused by differences in protocol field definitions, data formats, and encoding rules. It supports efficient parsing and semantic mapping of long protocol texts (such as communication protocols containing nested fields, validation rules, and dynamic loads), meeting the data interoperability needs of complex IoT devices.

[0163] (2) Deep learning and sequence modeling optimization: Based on the improved technology of the Transformer model, we focus on solving the problem of position information modeling for long sequence data (length exceeding 1000 characters), breaking through the insufficient position sensitivity and gradient attenuation bottleneck of traditional position encoding (such as sine / cosine encoding) in long texts, combining the residual network (ResNet) with the self-attention mechanism to optimize the stability of deep model training and enhance the ability to capture long-distance dependencies between protocol fields;

[0164] (3) Application of natural language processing in IoT protocols: Adapt natural language processing technologies (such as word segmentation, semantic tagging, and sequence annotation) to the field of IoT protocol text analysis to achieve structured extraction and semantic understanding of key protocol fields (such as data types, communication rules, and check bits). Based on the domain specificity of protocol texts (high-density professional terms and fixed grammatical structures), design lightweight model architectures and adaptive training strategies to achieve "plug-and-play" access to devices from multiple manufacturers and reduce the development and operation and maintenance costs of protocol adaptation.

[0165] The innovation of this application lies in:

[0166] (1) Dynamic residual position coding: Through learnable residual vectors and protocol length adaptive coefficients, dynamic enhancement of position information in long sequences is achieved, solving the position confusion problem of traditional sinusoidal coding in nested fields (e.g., the position distinction accuracy of a 1500-character protocol is improved by 40%).

[0167] (2) Layered residual correction technology: A lightweight correction module is added after each encoder layer to prevent the layer-by-layer attenuation of position information in deep networks. Experiments show that after stacking 6 layers of encoders, the position information retention rate is improved from 35% in traditional models to 85%;

[0168] (3) Protocol length adaptive mechanism: Dynamically adjust the residual weight and segmented encoding strategy according to the protocol length, supporting efficient parsing of the full range from short instructions (such as 200-character MQTT heartbeat packets) to very long protocols (such as 2000-character industrial control protocols);

[0169] (4) End-to-end semantic mapping: Combined with the cross-protocol attention mechanism, field mapping rules are automatically generated (such as manufacturer A.temp → target Schema.temperature), reducing the workload of manual configuration.

[0170] The resulting benefits include:

[0171] (1) Breakthrough in long sequence parsing capabilities: It can process complex protocol texts up to 2,000 characters in length, and the parsing accuracy of key fields (such as checksums and version numbers) is increased to 92% (compared to 67% for traditional Transformers). The semantic matching F1 value of nested fields (such as multi-layer JSON / XML structures) reaches 0.89, which is 50% higher than the LSTM model.

[0172] (2) Significantly improved training stability: The residual connection and layer-by-layer residual position correction modules reduce the gradient vanishing rate of a 50-layer deep network to below 3%, and increase the model convergence speed by 30%;

[0173] (3) Enhanced dynamic protocol adaptability: Supports real-time update and adaptation of protocol formats. For example, after a manufacturer adds an "energy efficiency identification" field, the protocol model's mapping accuracy remains above 85% without retraining.

[0174] In a second aspect, an embodiment of the present application also provides a device for modeling a long protocol text sequence of an Internet of Things device.

[0175] In one embodiment, referring to Figure 4 , Figure 4 This is a functional module diagram of the long protocol text sequence modeling device for IoT devices in this application. Figure 4 As shown, the long protocol text sequence modeling device for IoT devices includes: an acquisition module, a construction module, and an execution module.

[0176] The acquisition module is used to collect IoT device protocol texts from multiple manufacturers, and perform data cleaning, word segmentation and semantic tagging operations on the collected protocol texts to generate structured sequences; the construction module is used to build a protocol model based on dynamic residual position coding implementation and layered residual position correction module design, and train the protocol model through the structured sequence; the execution module is used to parse the protocol text to be parsed based on the trained protocol model, and generate mapping rules and type conversion logic between the protocol text to be parsed and the target protocol.

[0177] In a third aspect, an embodiment of the present application provides a long protocol text sequence modeling device for an IoT device. The long protocol text sequence modeling device for an IoT device can be a personal computer (PC), a laptop computer, a server, or other device with data processing capabilities.

[0178] Reference Figure 5 , Figure 5 Schematic diagram of the hardware structure of the long protocol text sequence modeling device for IoT devices involved in the embodiment of the present application. In the embodiment of the present application, the long protocol text sequence modeling device for IoT devices may include a processor, a memory, a communication interface, and a communication bus.

[0179] The communication bus may be of any type and is used to interconnect the processor, memory, and communication interface.

[0180] Communication interfaces include input / output (I / O) interfaces, physical interfaces, and logical interfaces, which are used to interconnect components within the IoT device's long protocol text sequence modeling device, as well as interfaces used to interconnect the IoT device's long protocol text sequence modeling device with other devices (such as other computing devices or user devices). Physical interfaces can include Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user devices can include displays, keyboards, etc.

[0181] The memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0182] The processor may be a general-purpose processor, which may call the IoT device long protocol text sequence modeling program stored in the memory and execute the IoT device long protocol text sequence modeling method provided in the embodiment of the present application. For example, the general-purpose processor may be a central processing unit (CPU). The method executed when the IoT device long protocol text sequence modeling program is called may refer to the various embodiments of the IoT device long protocol text sequence modeling method of the present application, and will not be repeated here.

[0183] Those skilled in the art will understand that Figure 5 The hardware structure shown in the figure does not constitute a limitation to the present application and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0184] In a fourth aspect, an embodiment of the present application also provides a computer-readable storage medium.

[0185] The computer-readable storage medium of the present application stores a long protocol text sequence modeling program for Internet of Things devices, wherein when the long protocol text sequence modeling program for Internet of Things devices is executed by a processor, the steps of the long protocol text sequence modeling method for Internet of Things devices as described above are implemented.

[0186] Among them, the method implemented when the long protocol text sequence modeling program of the Internet of Things device is executed can refer to the various embodiments of the long protocol text sequence modeling method of the Internet of Things device of this application, and will not be repeated here.

[0187] The terms "including" and "having," and any variations thereof, in the specification and claims of this application and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to the process, method, product, or apparatus. The terms "first," "second," and "third" are used to distinguish between different objects, etc., and do not indicate a sequential order, nor do they limit the "first," "second," and "third" to different types.

[0188] In the description of the embodiments of this application, words such as "exemplary," "for example," or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary," "for example," or "for example" in the embodiments of this application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary," "for example," or "for example" is intended to present the relevant concepts in a concrete manner.

[0189] In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" refers to two or more than two.

[0190] In some processes described in the embodiments of the present application, multiple operations or steps are included that appear in a specific order. However, it should be understood that these operations or steps may not be performed in the order in which they appear in the embodiments of the present application or may be performed in parallel. The sequence numbers of the operations are only used to distinguish between different operations, and the sequence numbers themselves do not represent any order of execution. In addition, these processes may include more or fewer operations, and these operations or steps may be performed in sequence or in parallel, and these operations or steps may be combined.

[0191] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device to execute the methods described in each embodiment of the present application.

[0192] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application. < / checksum:hex> < / acceleration:float> < / direction:string> < / speed:uint16> < / params:object> < / type:string> < / command:object> < / version:string> < / header:object> < / crc:hex> < / timestamp:int64> < / mode:string> < / brightness:uint8> < / params:object> < / cmd:string> < / temperature:float> < / payload:binary> < / length:uint16> < / header> < / payload:binary>

Claims

1. A method for modeling long protocol text sequences of IoT devices, characterized by: The method for modeling a long protocol text sequence of an IoT device includes: Collect IoT device protocol texts from multiple manufacturers, perform data cleaning, word segmentation, and semantic tagging on the collected protocol texts to generate structured sequences; Based on the implementation of dynamic residual position coding and the design of a layered residual position correction module, a protocol model is constructed, and the protocol model is trained through the structured sequence; The protocol text to be parsed is parsed based on the trained protocol model, and mapping rules and type conversion logic between the protocol text to be parsed and the target protocol are generated.

2. The method for modeling a long protocol text sequence of an Internet of Things device according to claim 1, characterized in that: The method collects IoT device protocol texts from multiple manufacturers, performs data cleaning, word segmentation, and semantic tagging on the collected protocol texts, and generates structured sequences, specifically including: Collect IoT device protocol texts from multiple manufacturers in smart parks and industrial IoT scenarios; Use regular expressions to match invalid fields in the agreement text to clean the agreement text and unify the structure of the agreement text; The BERT model pre-trained on the IoT protocol corpus is used to perform fine-grained word segmentation on the protocol text, and semantic tags are annotated to generate structured sequences.

3. The method for modeling a long protocol text sequence of an Internet of Things device according to claim 1, characterized in that: For the dynamic residual position coding implementation, specifically: Based on the sinusoidal position encoding and combined with the learnable residual vector, the final position encoding is generated by superposition. The specific method is: INSTEAD final =PE sniusoidal +α·W res ·INSTEAD learned Among them, PE sniusoidal represents the basic position code generated based on the sine function or cosine function, α represents the protocol length adaptation coefficient, which is dynamically calculated based on the protocol length, α=log(length / 500), length represents the protocol length, W res Represents the trainable weight matrix, PE learned Represents the adaptive residual vector generated by a two-layer fully connected network, with activation functions GELU and PE final Represents the final position encoding.

4. The method for modeling a long protocol text sequence of an Internet of Things device according to claim 3, wherein: The structure of the layered residual position correction module includes input, processing and output; The input of the layered residual position correction module is the output code of the current Transformer layer; The processing of the layered residual position correction module is to input it into the fully connected network after layer normalization; The output of the layered residual position correction module is the residual increment of the position code, which is added to the basic position code to update it. The position update mechanism is specifically as follows: INSTEAD updated =PE final +Dropout(GREEN(W c ·LayerNorm(PE final ))) Among them, PE updated represents the updated position encoding, Dropout represents random inactivation, LayerNorm represents layer normalization, W c represents the trainable parameters of the layered residual position correction module, and GELU represents the activation function.

5. The method for modeling a long protocol text sequence of an Internet of Things device according to claim 4, characterized in that: The protocol model includes 6 layers of Transformer encoders, and a layered residual position correction module is inserted after each layer of Transformer encoder, and the weight matrix W is dynamically adjusted according to the length of the input protocol text. res , to achieve adaptation of the protocol text length, the input of the layered residual position correction module is the output code of the current Transformer layer, and the output is the residual increment of the position code.

6. The method for modeling a long protocol text sequence of an Internet of Things device according to claim 5, characterized in that: The protocol model based on the training is used to parse the protocol text to be parsed, and generate mapping rules and type conversion logic between the protocol text to be parsed and the target protocol, specifically including: After word segmentation and semantic tagging, the protocol text to be parsed is converted into a high-dimensional vector sequence through the Embedding layer and added to the dynamic residual position code; A multi-layer encoding method is used, where the first layer is used to initially capture the local dependencies between fields, the third layer is used to enhance the position weights of nested fields through a hierarchical residual position correction module, and the sixth layer is used to output the global semantic code. The output global semantic encoding is input into the fully connected layer and Softmax classifier to output the semantic label of the protocol field. At the same time, the attention weight between the protocol text to be parsed and the target protocol is calculated to generate mapping rules and type conversion logic.

7. The method for modeling a long protocol text sequence of an Internet of Things device according to claim 6, characterized in that: The global semantic encoding includes field types, rules and cross-protocol mapping relationships.

8. A device for modeling long protocol text sequences of IoT devices, characterized in that: The IoT device long protocol text sequence modeling device comprises: The acquisition module is used to collect IoT device protocol texts from multiple manufacturers, and perform data cleaning, word segmentation, and semantic tagging operations on the collected protocol texts to generate structured sequences; A construction module, which is used to construct a protocol model based on dynamic residual position coding implementation and layered residual position correction module design, and train the protocol model through the structured sequence; The execution module is used to parse the protocol text to be parsed based on the trained protocol model, and generate mapping rules and type conversion logic between the protocol text to be parsed and the target protocol.

9. A device for modeling long protocol text sequences of IoT devices, characterized in that: The IoT device long protocol text sequence modeling device includes a processor, a memory, and an IoT device long protocol text sequence modeling program stored on the memory and executable by the processor, wherein when the IoT device long protocol text sequence modeling program is executed by the processor, the steps of the IoT device long protocol text sequence modeling method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a long protocol text sequence modeling program for an Internet of Things device, wherein when the long protocol text sequence modeling program for an Internet of Things device is executed by a processor, the steps of the long protocol text sequence modeling method for an Internet of Things device as described in any one of claims 1 to 7 are implemented.

Images