Method and system for supporting multi-protocol custom splicing internet of things device data analysis

Abstract

The application discloses a kind of support multi-protocol custom splicing Internet of Things equipment data analysis method and system, it is related to communication protocol technical field, its method includes: constructing multi-protocol equipment access link;In response to the original data message that Internet of Things equipment is reported on multi-protocol equipment access link, determine the product model of Internet of Things equipment based on original data message;Based on the attribute of product model, original data message is split into multiple function paragraphs;Parse each function paragraph, obtain function paragraph data, and map function paragraph data to the corresponding attribute of product model;The function paragraph data after mapping is stored in time series database and relational database.The application can be customized to create external data interface, facilitate business system to conveniently obtain equipment data, operating data, frozen data, and has real-time copy control and other functions, and can guarantee data transmission security, effectively deal with a large number of protocol concurrent scene under equipment access, improve access efficiency.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of communication protocols, and in particular to a method and system for analyzing data of Internet of Things devices supporting multiple protocols and custom splicing. BACKGROUND

[0002] The core of Internet of Things technology is to realize interconnection and intercommunication between devices through various sensors, actuators and communication modules, and then to perform data collection, information exchange, remote control and intelligent decision-making. With the in-depth development of Internet of Things applications, the number of connected devices is growing exponentially, and the types and functions of devices are becoming increasingly diverse, which puts higher requirements on Internet of Things device access platforms. Internet of Things device access platforms not only need to handle massive concurrent connections, but also need to be compatible with a wide variety of communication protocols and data formats.

[0003] Due to the wide variety of communication protocols used by Internet of Things devices, there are not only standard network protocols such as TCP / IP, UDP, MQTT, CoAP, HTTP, etc., but also fieldbus protocols commonly used in the industrial field such as Modbus, Profibus, OPC UA, etc., and even a large number of private protocols defined by manufacturers. Traditional access solutions often use a point-to-point customized development mode, i.e., for each new device type or protocol, a large amount of manpower and resources need to be invested to develop exclusive protocol analysis and data processing logic, and whenever a new device model or protocol version appears, the adaptation code needs to be redeveloped or modified, resulting in low access efficiency.

[0004] In summary, existing Internet of Things device access technologies are difficult to cope with device access in a large number of concurrent protocol scenarios, and have low access efficiency. SUMMARY

[0005] Embodiments of the present application provide a method and system for analyzing data of Internet of Things devices supporting multiple protocols and custom splicing, which can effectively cope with device access in a large number of concurrent protocol scenarios and improve access efficiency.

[0006] To achieve the above-mentioned purpose, embodiments of the present application adopt the following technical solutions:

[0007] In a first aspect, a method for analyzing data of Internet of Things devices supporting multiple protocols and custom splicing is provided, which is applied to an Internet of Things device access platform deployed in an electronic device, and the method comprises:

[0008] constructing a multi-protocol device access link;

[0009] determining a product model of the Internet of Things device based on the original data message in response to the original data message reported by the Internet of Things device through the multi-protocol device access link, wherein the attributes of the product model include fixed attributes and variable attributes;

[0010] Based on the attributes of the product model, the original data message is split into multiple functional segments, wherein each functional segment corresponds to a data unit in the target protocol, and the target protocol is determined by the attributes of the product model;

[0011] Each functional segment is parsed to obtain functional segment data, and the functional segment data is mapped to the corresponding attribute of the product model;

[0012] The mapped functional segment data is stored in a time-series database and a relational database, wherein the time-series database is used to store device operation data and the relational database is used to store device configuration information.

[0013] In one possible implementation of the first aspect, the step of splitting the original data message into multiple functional segments based on the attributes of the product model includes:

[0014] The start and end positions of the paragraphs are determined based on the fixed attributes of the product model.

[0015] The paragraph length is dynamically adjusted based on the variable attributes of the product model to obtain the dynamically adjusted paragraph.

[0016] Determine whether the protocol type of the original data packet is a numeric protocol or a character protocol;

[0017] When the protocol type is the digital protocol, the dynamically adjusted segment is split into multiple functional segments according to the byte boundary rules and data length rules.

[0018] When the protocol type is a character-based protocol, the dynamically adjusted paragraphs are split according to the delimiter and preset keyword rules to obtain multiple functional paragraphs.

[0019] In another possible implementation of the first aspect, the dynamically adjusted paragraph is split according to byte boundary rules and data length rules to obtain multiple functional paragraphs, including:

[0020] The split point position is determined according to the byte boundary rules defined in the product model, wherein the byte boundary rules include alignment and padding rules;

[0021] The dynamically adjusted paragraph is divided into multiple sub-paragraphs according to the stated splitting point positions;

[0022] Each sub-paragraph is subjected to length validation using data length rules, which include fixed length and variable length types.

[0023] If the length verification fails, the length of the sub-segment is adjusted according to a preset repair strategy until the length verification passes.

[0024] Assign a unique functional paragraph identifier to each validated sub-paragraph;

[0025] The functional paragraph identifier and its corresponding sub-paragraph are associated and stored to obtain a functional paragraph set, which includes multiple functional paragraphs.

[0026] In another possible implementation of the first aspect, the method for determining the target protocol includes:

[0027] Obtain the protocol type identifier defined in the product model;

[0028] The protocol configuration library is queried based on the protocol type identifier, and the format characteristics of the original data packet are matched with the protocol templates in the protocol configuration library to obtain the matching result;

[0029] The target protocol is determined based on the matching results;

[0030] The step of determining the target protocol based on the matching result includes:

[0031] The parameters of the target protocol are loaded into the protocol parsing engine.

[0032] In another possible implementation of the first aspect, parsing each of the functional segments to obtain functional segment data includes:

[0033] The data type and encoding format of each functional segment are determined according to the parsing rules of the target protocol;

[0034] The raw byte data in the functional section is converted according to the data type to obtain structured data;

[0035] The structured data is decoded according to the encoding format to obtain readable functional paragraph data.

[0036] In another possible implementation of the first aspect, mapping the functional segment data to the corresponding attributes of the product model includes:

[0037] Feature extraction is performed on the functional paragraph data to obtain the features of the functional paragraph data;

[0038] The corresponding instruction template in the protocol instruction library is matched according to the features of the functional segment data. The protocol instruction library stores instruction templates and mapping rules for different device types.

[0039] The target attribute is determined based on the mapping relationship defined in the instruction template, and it is determined whether the data type of the functional paragraph data is compatible with that of the target attribute.

[0040] If the data type of the functional paragraph is compatible with that of the target attribute, the functional paragraph data is written into the corresponding attribute field of the product model.

[0041] In another possible implementation of the first aspect, the method further includes:

[0042] In response to the access signal from the IoT device, determine the data type and business requirements;

[0043] When the data type is real-time monitoring data, the real-time monitoring data is stored in a time-series database;

[0044] When the data type is configuration parameter data, the configuration parameter data is stored in a relational database;

[0045] Data synchronization is performed between the time-series database and the relational database.

[0046] In another possible implementation of the first aspect, the data synchronization between the time-series database and the relational database includes:

[0047] Establish a data synchronization channel between the time-series database and the relational database, and periodically identify data items to be synchronized;

[0048] The direction and frequency of data synchronization are determined according to a synchronization strategy, which includes one-way synchronization and two-way synchronization.

[0049] In another possible implementation of the first aspect, the data synchronization between the time-series database and the relational database includes:

[0050] Establish a data synchronization channel between the time-series database and the relational database, and periodically identify data items to be synchronized;

[0051] The direction and frequency of data synchronization are determined according to a synchronization strategy, which includes one-way synchronization and two-way synchronization.

[0052] Secondly, this application provides a data parsing system for IoT devices that supports multi-protocol custom splicing, including:

[0053] Electronic devices, equipped with an IoT device access platform;

[0054] Multiple Internet of Things (IoT) devices, each of which is connected to the electronic device.

[0055] Thirdly, this application provides a machine-readable storage medium storing instructions that cause a machine to execute the above-described method for parsing data from multi-protocol custom-connected IoT devices.

[0056] The above technical solution effectively solves the problem of low efficiency in traditional IoT device access technologies under scenarios with a large number of concurrent protocols by constructing a multi-protocol device access link and a custom splitting mechanism based on product models. First, the construction of the multi-protocol device access link provides a unified access channel for different types of IoT devices. Regardless of whether the device uses standard network protocols such as TCP / IP, UDP, MQTT, CoAP, and HTTP, or industrial fieldbus protocols such as Modbus, Profibus, and OPC UA, or even a manufacturer's custom proprietary protocol, they can all be processed through the same access architecture, fundamentally breaking free from the constraints of the traditional point-to-point customized development model. Secondly, the raw data message processing mechanism based on the product model achieves standardization and automation of protocol parsing. Through the definition of fixed and variable attributes in the product model, it can automatically identify device types and determine the corresponding target protocols, thereby intelligently splitting complex raw data messages into multiple functional segments. Each functional segment precisely corresponds to a specific data unit in the target protocol. This model-driven splitting method not only improves the accuracy of data parsing but, more importantly, enables configurable protocol adaptation. When faced with new device types or protocol versions, adaptation can be completed simply by adjusting the attribute configuration of the product model, without the need for redeveloping code. Furthermore, the mapping mechanism from functional segment data to product model attributes ensures unified processing of data from different protocols. The hierarchical storage strategy of time-series databases and relational databases further optimizes data management efficiency. The time-series database specifically handles the high-frequency writing and querying needs of device operation data, while the relational database is responsible for the structured storage of device configuration information. This storage architecture not only improves concurrent processing capabilities but also provides a solid foundation for subsequent data analysis and device management, thereby achieving efficient, standardized, and scalable IoT device access.

[0057] Other features and advantages of the embodiments of this application will be described in detail in the following detailed description section. Attached Figure Description

[0058] Figure 1 A flowchart illustrating a method for parsing data from IoT devices that supports custom splicing of multiple protocols, provided as an embodiment of this application;

[0059] Figure 2 A system architecture diagram for supporting multi-protocol custom splicing IoT device data parsing is provided in an embodiment of this application;

[0060] Figure 3 This is a schematic diagram of the attribute structure of a product model provided in an embodiment of this application. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for illustration and explanation of the embodiments of this application and are not intended to limit the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0062] It should be noted that if the embodiments of this application involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.

[0063] Furthermore, if the embodiments of this application involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0064] Figure 1 The illustration shows a flowchart of a data parsing method for supporting multi-protocol custom splicing IoT devices according to an embodiment of this application. Figure 1 As shown in the figure, this application provides a method for parsing data from IoT devices that supports multi-protocol custom splicing, applied to an IoT device access platform. The method may include the following steps.

[0065] S110, Construct multi-protocol device access links;

[0066] S120. In response to the raw data packet reported by the IoT device through the multi-protocol device access link, determine the product model of the IoT device based on the raw data packet, wherein the attributes of the product model include fixed attributes and variable attributes.

[0067] S130. Based on the attributes of the product model, the original data message is split into multiple functional segments, where each functional segment corresponds to a data unit in the target protocol, and the target protocol is determined by the attributes of the product model.

[0068] S140. Parse each functional paragraph to obtain functional paragraph data, and map the functional paragraph data to the corresponding attributes of the product model;

[0069] S150. The mapped functional segment data is stored in a time-series database and a relational database, wherein the time-series database is used to store device operation data and the relational database is used to store device configuration information.

[0070] Among them, determining the product model of IoT devices based on raw data packets includes:

[0071] Extract the device identification information and protocol header information from the raw data packets;

[0072] Search the preset equipment product model library based on the equipment identification information;

[0073] Match protocol header information with protocol features in the product model library;

[0074] The corresponding product model is determined based on the matching results.

[0075] Figure 2 This application provides a system architecture diagram for supporting multi-protocol custom splicing IoT device data parsing, as illustrated in an embodiment of this application. Figure 2 As shown, Class 1, Class 2, and Class 3 devices are all IoT devices. IoT devices transmit data with terminals through link channels (i.e., multi-protocol device access links). This embodiment applies to terminals, and the IoT center is an IoT device access platform deployed in electronic devices. In this embodiment, constructing a multi-protocol device access link establishes a unified access channel supporting multiple communication protocols, enabling compatible connections for heterogeneous IoT devices. In specific implementation, various communication components need to be initialized first, including TCP client / server, UDP server, HTTP server, MQTT server, and dedicated IoT platform interfaces. Each communication method corresponds to different device access scenarios. For example, TCP connections are suitable for industrial equipment requiring reliable transmission, UDP connections are suitable for sensor devices with high real-time requirements, HTTP interfaces are suitable for smart devices with network capabilities, and the MQTT protocol is suitable for resource-constrained IoT terminals.

[0076] In another embodiment of this application, the data interface of the IoT center can be customized to facilitate business systems to obtain device data, operation data, frozen data, etc., and has specific functions such as real-time control. The interface adopts 3DES encryption to ensure data transmission security.

[0077] By reading pre-configured communication link parameters, including port number, authentication information, and encryption method, a corresponding network listening service is dynamically created. For each active communication link, an independent thread pool is allocated for message processing, ensuring that data processing between different protocols does not interfere with each other. Simultaneously, a connection status monitoring mechanism is established to detect the health status of each communication link in real time, automatically reconnecting when a connection anomaly is detected. This multi-protocol parallel support architecture allows a single platform to simultaneously handle connection requests from IoT devices from different manufacturers and using different technical standards, significantly improving system device compatibility and access efficiency.

[0078] In response to raw data packets reported by IoT devices through multi-protocol device access links, the product model of the IoT devices is determined based on these raw data packets, enabling automatic device identification and classification management. When a device sends a data packet through an established access link, the received raw data is first preprocessed, including data integrity verification and format standardization. Next, device identification information is extracted by analyzing specific fields in the data packet to obtain the device's unique identifier, such as the device serial number, MAC address, and device code. Simultaneously, the protocol header information of the data packet is parsed to extract protocol characteristic parameters such as protocol type identifier, version number, and function code.

[0079] Based on the extracted device identification information, a pre-defined device product model library is queried. This library stores product definitions for various types of equipment, including detailed parameters such as device type, manufacturer information, supported protocol versions, and data format specifications. A matching algorithm between the device identification and the model library allows for rapid location of the corresponding product model.

[0080] To improve matching accuracy, the protocol header information is verified against the protocol features in the product model to ensure protocol compatibility. When multiple matching product models exist, a similarity scoring mechanism is used to comprehensively consider factors such as device identifier matching degree and protocol feature similarity to select the best matching result. This enables automatic device identification and classification, providing an accurate model foundation for subsequent data analysis and processing.

[0081] Based on the attributes of the product model, the raw data message is split into multiple functional segments to achieve protocol parsing. The attributes defined in the product model are divided into two main categories: fixed attributes and variable attributes. Fixed attributes include data fields that remain unchanged in the protocol, such as protocol header identifiers, function codes, and check bits, while variable attributes include data fields that change according to actual conditions, such as device address, data length, and sensor values.

[0082] Based on the predefined protocol structure specifications in the product model, the original data packets are deconstructed segment by segment using a "train carriage" model. Each "carriage" represents a functional data unit in the protocol, with a clearly defined starting position, data length, and data type. For numeric protocols, segments are divided according to hexadecimal byte sequences, with each functional segment corresponding to a specific length of byte sequence. For example, in the Modbus-RTU protocol, the communication address occupies 1 byte, the function code occupies 1 byte, the register address occupies 2 bytes, the register count occupies 2 bytes, and the CRC checksum occupies 2 bytes. For character-based protocols, segments are divided according to the protocol's hierarchical structure or delimiters; for example, JSON format data packets are parsed according to a key-value pair structure.

[0083] During the splitting process, the integrity and format correctness of the data in each functional segment are verified. Data segments that do not conform to the protocol specifications are marked as abnormal and error information is recorded, thereby ensuring that data packets of different protocol types can be accurately parsed, laying a solid foundation for subsequent data processing.

[0084] Each functional segment is parsed to obtain functional segment data, and this data is mapped to the corresponding attributes of the product model to achieve data standardization. For each segment, based on its definition in the product model, corresponding data processing rules are applied for parsing and transformation. These processing rules cover seven categories: encryption / decryption rules for handling data segments requiring secure transmission, such as 3DES encrypted device authentication information; verification rules for verifying data integrity, such as CRC16 and CRC32 verification algorithms; encoding rules for data format conversion, such as hexadecimal to decimal and ASCII code conversion; arithmetic rules for numerical calculations, such as data scaling and unit conversion; positional rules for string operations, such as character reversal, truncation, and insertion; custom rules for handling special business logic; and object rules for handling complex data structure transformations.

[0085] In practice, appropriate rule combinations are selected based on the data type and processing requirements of the functional segments. For example, for a functional segment containing temperature data, a hexadecimal-to-decimal conversion may be performed first, followed by applying a scaling factor to obtain the actual temperature value. After processing, the converted functional segment data is mapped to the corresponding attributes of the product model, establishing a relationship between the data and model attributes. This mapping relationship includes not only direct numerical correspondence but also semantic identification of the data, such as mapping the original hexadecimal data 0x01F4 to the "register address" attribute and identifying its decimal value as 500. Through this refined data parsing and mapping mechanism, the original binary data message is converted into structured data with clear business meaning.

[0086] By storing the mapped functional segment data in both a time-series database and a relational database, data persistence and efficient management can be achieved. Specifically, a dual-engine database strategy is adopted to optimize the storage performance of different types of data, with the time-series database specifically used to store device operation data and the relational database used to store device configuration information.

[0087] For equipment operation data, such as sensor readings, equipment status, and operation logs—data with time-series characteristics—it is stored in a time-series database. The time-series database is specifically optimized for time-series data, supporting high-frequency writes, compressed storage, and fast time-range queries. During storage, different storage strategies are configured based on the importance and access frequency of the data; for example, real-time data uses high-performance storage, while historical data uses compressed storage.

[0088] Simultaneously, it supports tiered data storage, storing frequently accessed data in high-speed storage media and migrating less accessed data to lower-cost storage media. For relatively stable data such as device configuration information, including device files, product model definitions, and protocol rules, it stores them in a relational database. Relational databases provide robust transaction support and data consistency guarantees, ensuring the accuracy and reliability of configuration information. During data storage, frozen records and historical snapshots can be automatically generated based on preset business rules. For example, for electricity meter data, daily frozen records can be automatically generated at midnight for use in electricity billing and other business scenarios. This dual-engine database architecture not only ensures high performance and high reliability of data storage but also provides flexible and diverse data access interfaces for upper-layer applications.

[0089] This embodiment supports multiple communication protocols such as TCP, UDP, HTTP, and MQTT through a unified access architecture, achieving compatible access for heterogeneous IoT devices and effectively solving the problems of single protocol support and poor device compatibility in traditional IoT platforms. The intelligent identification mechanism based on product models can automatically identify and classify different types of IoT devices, greatly reducing the configuration complexity of device access. The "train carriage" protocol parsing model decomposes the complex protocol structure into visual functional segments, allowing users to customize and debug protocols through a graphical interface, significantly shortening the device access cycle. The dual-engine database architecture ensures efficient storage and fast access to massive amounts of device data; the time-series database is specifically optimized for storing device operation data, while the relational database ensures the consistency and reliability of device configuration information. Simultaneously, it supports rapid access to new protocols and convenient expansion for new device types, providing a solid technical foundation for the rapid deployment and continuous evolution of IoT applications.

[0090] In one embodiment of this example, based on the attributes of the product model, the original data message is split into multiple functional segments, including the following steps:

[0091] S210. Determine the start and end positions of the paragraphs based on the fixed attributes of the product model;

[0092] S220. Based on the variable attributes of the product model, dynamically adjust the paragraph length to obtain the dynamically adjusted paragraph;

[0093] S230. Determine whether the protocol type of the original data message is a numeric protocol or a character protocol;

[0094] S240. When the protocol type is a digital protocol, the dynamically adjusted segment is split into multiple functional segments according to the byte boundary rules and data length rules.

[0095] S250. When the protocol type is a character-based protocol, the dynamically adjusted paragraphs are split according to the delimiter and preset keyword rules to obtain multiple functional paragraphs.

[0096] When the protocol type is a character-based protocol, the dynamically adjusted paragraphs are split according to delimiters and preset keyword rules to obtain multiple functional paragraphs, including:

[0097] Identify preset delimiters in the raw data message. Preset delimiters include commas, semicolons, and newlines.

[0098] The dynamically adjusted paragraphs are divided into an initial paragraph set based on preset separators;

[0099] Search for target keywords in the initial paragraph set based on keyword rules. Target keywords are used to identify different types of data fields.

[0100] The initial paragraph set is reorganized based on the location information of the target keywords to obtain multiple functional paragraphs with semantic meaning.

[0101] Determine the start and end positions of paragraphs based on the fixed attributes of the product model, including:

[0102] Extract the location configuration information of fixed attributes from the product model. The location configuration information includes byte offset and field length.

[0103] Locate the start position of the segment in the original data packet based on the byte offset;

[0104] Calculate the end position of the paragraph based on the field length;

[0105] After calculating the end position of the paragraph based on the field length, it also includes:

[0106] Create a paragraph position index table to record the paragraph boundary information corresponding to each fixed attribute;

[0107] The paragraph position index table is cached in memory to improve the efficiency of subsequent accesses.

[0108] The paragraph length is dynamically adjusted based on the variable attributes of the product model, resulting in dynamically adjusted paragraphs, including:

[0109] Obtain the length calculation rules for variable attributes from the product model. The length calculation rules include dependent fields and calculation formulas.

[0110] Parse the current value of the dependent field and substitute it into the calculation formula to calculate the actual paragraph length;

[0111] Adjust the end position of the paragraph based on the calculated actual paragraph length;

[0112] Determine whether the adjusted paragraph length meets the maximum and minimum limits specified in the protocol.

[0113] If the adjusted paragraph length exceeds the limit, a preset exception handling strategy is used to correct the length, resulting in a dynamically adjusted paragraph.

[0114] Figure 3 This application provides a schematic diagram of a product model attribute structure, as illustrated in an embodiment. Figure 3As shown, in this embodiment, determining the start and end positions of paragraphs based on the fixed attributes of the product model enables precise data parsing. Specifically, position configuration information for fixed attributes is extracted from the product model. This configuration information defines the precise position of each fixed attribute within the data packet. The position configuration information mainly includes two key parameters: byte offset and field length. The byte offset represents the byte distance of the attribute field relative to the start position of the data packet, while the field length represents the number of bytes occupied by the attribute field. For example, in the Modbus-RTU protocol, the byte offset of the communication address field is 0, and the field length is 1 byte; the byte offset of the function code field is 1, and the field length is 1 byte; the byte offset of the register address field is 2, and the field length is 2 bytes.

[0115] Based on this location configuration information, the start position of each segment is precisely located in the original data packet using byte offsets. The location process employs pointer arithmetic, adding the byte offset to the start address of the data packet to obtain the start address of the target segment. Next, the end position of the segment is calculated based on the field length using the formula: End Position = Start Position + Field Length - 1.

[0116] To improve the efficiency of subsequent data access, a paragraph position index table is created to record the paragraph boundary information corresponding to each fixed attribute. This index table is stored in key-value pair format, where the key is the attribute name and the value is a structure containing boundary information including the start and end positions. After the index table is created, it is cached in memory to avoid repeatedly calculating paragraph positions, thereby significantly improving data parsing performance.

[0117] Paragraph lengths are dynamically adjusted based on variable attributes of the product model to handle variable-length data fields. The length of variable attributes typically depends on the values ​​of other fields, requiring dynamic calculation to determine the actual paragraph length.

[0118] The length calculation rules for variable attributes are derived from the product model. These rules contain two core elements: dependent fields and calculation formulas. Dependent fields refer to other data fields that affect the length of the current variable attribute, such as the data length field and the record count field. The calculation formula defines how to calculate the actual paragraph length based on the values ​​of the dependent fields. For example, for a data field paragraph, the length calculation rule could be: Actual length = Data length field value × 2 (because each byte is represented by two hexadecimal characters).

[0119] When performing dynamic adjustments, the current values ​​of the dependent fields are first parsed. Then, these parsed values ​​are substituted into the calculation formula to determine the actual paragraph length of the current variable attribute. Based on the calculation results, the end position of the paragraph is readjusted to ensure the accuracy of the paragraph boundaries.

[0120] To ensure the reliability of data parsing, the adjusted paragraph length is verified to meet the maximum and minimum limits specified in the protocol. When the adjusted paragraph length exceeds the limits specified in the protocol, a preset exception handling strategy is used to correct the length, such as setting it to the maximum allowed length or triggering an error handling process, ultimately resulting in a dynamically adjusted paragraph that conforms to the specifications.

[0121] Determining whether the original data packet's protocol type is numeric or character-based is crucial for selecting the correct parsing strategy. Accurate protocol type identification directly impacts the choice of subsequent segmentation methods and the correctness of data parsing. Multiple feature analyses are used to determine the protocol type. First, data format feature analysis is performed. Numeric protocols typically use binary encoding, containing a large number of hexadecimal characters (0-9, AF) with a relatively even character distribution. Character protocols, on the other hand, typically use ASCII encoding, containing readable text characters, punctuation marks, and control characters.

[0122] Next, protocol header feature recognition is performed. Different protocol types have specific header identification patterns, such as the function code field in the Modbus-RTU protocol and the curly braces at the beginning of JSON. Then, data structure feature detection is performed. Numeric protocols usually have fixed field lengths and strict byte alignment, while character protocols have hierarchical structures or delimiter-based features.

[0123] Finally, statistical feature analysis is performed to calculate the proportional distribution of numeric characters, alphanumeric characters, and special symbols in the data message. Threshold comparisons are used to assist in determining the protocol type. Based on the combined results of these feature analyses, a weighted scoring mechanism is employed to arrive at the final protocol type determination. When the scoring result leans towards numeric characteristics, it is determined to be a numeric protocol; when the scoring result leans towards character characteristics, it is determined to be a character protocol.

[0124] When the protocol type is numeric, the dynamically adjusted segments are split according to byte boundary rules and data length rules to achieve accurate data parsing. Numeric protocols have strict binary structure characteristics, and each data field has a clearly defined byte boundary and length.

[0125] Byte boundary rules ensure that each functional segment is divided into bytes, avoiding cross-byte data splits. For hexadecimal data messages, every two characters represent one byte, therefore the start and end positions of a segment must be even-numbered. Data length rules determine the specific number of bytes in each segment based on the field lengths defined in the protocol specification.

[0126] For example, when processing the Modbus-RTU protocol, the communication address segment is 1 byte long (2 hexadecimal characters), the function code segment is 1 byte long, the register address segment is 2 bytes long (4 hexadecimal characters), the register quantity segment is 2 bytes long, and the CRC check segment is 2 bytes long.

[0127] The splitting process proceeds sequentially according to the order in which the fields appear in the protocol. After each segment is split, the current parsing position pointer is updated to prepare for the next segment. During the splitting process, some special cases need to be handled, such as variable-length data field segments, whose length needs to be dynamically determined based on the previously parsed length field. For complex segments containing substructures, recursive splitting is required, further subdividing large segments into smaller functional units.

[0128] When the protocol type is character-based, the dynamically adjusted paragraphs are split according to delimiters and preset keyword rules. Character-based protocols typically use text format and have hierarchical structure or delimiter-based segmentation features, requiring semantic analysis to achieve accurate paragraph splitting.

[0129] First, identify the default delimiters in the raw data message. These delimiters are special characters defined in the protocol specification and are used to separate different data fields or data blocks. Common default delimiters include commas (,) for separating data items at the same level, semicolons (;) for separating data blocks of different types, newlines (\n) for separating different data records, and other specific delimiters such as colons (:) and vertical bars (|).

[0130] The recognition process employs a character matching algorithm, scanning the data packet character by character and recording the position information when a preset delimiter is encountered. Next, based on the recognized preset delimiters, the dynamically adjusted paragraphs are divided into an initial paragraph set, with the content between every two adjacent delimiters constituting one initial paragraph.

[0131] Then, based on keyword rules, the system searches the initial paragraph set for target keywords. These target keywords are special identifiers used in the protocol to identify different types of data fields, such as "temperature" for temperature data, "pressure" for pressure data, and "timestamp" for timestamps. The search process uses a string matching algorithm and supports both exact matching and fuzzy matching modes.

[0132] Finally, the initial paragraph set is reorganized based on the location information of the target keywords, and the relevant data fields are combined into functional paragraphs with complete semantic meaning, such as combining the temperature keyword and its corresponding value into a temperature data paragraph.

[0133] This embodiment, based on a paragraph positioning mechanism with fixed attributes of the product model, ensures that each data field can be accurately identified and extracted, effectively avoiding positional offset errors that are prone to occur in traditional parsing methods. The dynamic paragraph length adjustment function effectively solves the parsing challenge of variable-length data fields, achieving accurate parsing of complex data structures through dynamic calculation of dependent field values. The intelligent protocol type identification mechanism, through multi-dimensional feature analysis, ensures the correct selection of parsing strategies, avoiding parsing errors caused by misjudgment of protocol types. Byte boundary rules and data length rules for numeric protocols guarantee precise segmentation and parsing of binary data, making it particularly suitable for scenarios such as industrial control and sensor data. Separator and keyword rules for character protocols enable semantic parsing of text-formatted data, particularly suitable for structured text protocols such as JSON and XML. Simultaneously, it supports rapid access to new protocol types and convenient expansion of new data formats, providing comprehensive technical support for the diverse access needs of IoT devices.

[0134] In one embodiment of this invention, the dynamically adjusted paragraph is split into multiple functional paragraphs according to byte boundary rules and data length rules, including the following steps:

[0135] S310. Determine the split point location based on the byte boundary rules defined in the product model. The byte boundary rules include alignment and padding rules.

[0136] S320. Divide the dynamically adjusted paragraph into multiple sub-paragraphs according to the splitting point position;

[0137] S330. Apply data length rules to each sub-paragraph for length validation. Data length rules include fixed length and variable length types.

[0138] S340. In the event that the length verification fails, adjust the length of the sub-paragraphs according to the preset repair strategy until the length verification passes.

[0139] S350. Assign a unique functional paragraph identifier to each validated sub-paragraph;

[0140] S360. Data association storage is performed between the functional paragraph identifier and the corresponding sub-paragraph to obtain a functional paragraph set, which includes multiple functional paragraphs.

[0141] In this embodiment, the splitting point is determined according to the byte boundary rules defined in the product model, enabling precise data segmentation. Byte boundary rules define how data is stored and accessed in memory, ensuring data integrity and consistency. Alignment specifies how data fields are arranged on byte boundaries; common alignment methods include byte alignment, word alignment, and double-word alignment. Byte alignment requires each data field to start from a byte boundary, meaning the starting position must be a multiple of 8; word alignment requires data fields to start from a 16-bit boundary, meaning the starting position must be a multiple of 16; double-word alignment requires data fields to start from a 32-bit boundary, meaning the starting position must be a multiple of 32.

[0142] Padding rules define how to handle situations where the length of a data field does not meet alignment requirements. Typically, zero padding or padding with a specific value is used to fill in byte boundaries. For example, if a data field is 3 bytes long but requires 4-byte alignment, one padding byte needs to be added to the end.

[0143] When determining the split point location, first analyze the alignment requirements and length definition of each data field in the product model, and then calculate the actual starting position of each field according to the alignment method. The calculation process needs to consider the end position of the previous field and the influence of padding bytes to ensure that each split point strictly conforms to the byte boundary rules. For complex nested structures, it is necessary to recursively calculate the alignment position of the internal fields to ensure the alignment consistency of the entire data structure.

[0144] The dynamically adjusted paragraphs are divided into multiple sub-paragraphs based on the splitting point positions to achieve structured data decomposition. Based on the splitting point positions determined in the previous step, pointer arithmetic and memory copy techniques are used to divide consecutive data paragraphs into independent sub-paragraphs.

[0145] In this embodiment, there are 42 tools in 7 categories for processing paragraphs, which can perform encryption, decryption, verification, calculation and other processing on paragraphs, and can be expanded as needed.

[0146] The segmentation process proceeds sequentially according to the splitting points. Each splitting point marks the end position of a sub-segment and the start position of the next sub-segment. For each sub-segment, its starting offset, data length, and end position in the original data need to be recorded to form complete segment description information.

[0147] During the segmentation process, padding bytes generated by byte boundary alignment also need to be handled. These padding bytes do not contain valid data and need to be marked or filtered in the sub-segments. For variable-length sub-segments, the segmentation process needs to dynamically calculate their actual length to avoid data truncation or redundancy caused by fixed-length assumptions.

[0148] After segmentation, each sub-segment becomes an independent data unit, containing clear boundary information and data content, providing a structured foundation for subsequent data verification and processing. Simultaneously, a mapping relationship is established between the sub-segments and the original data locations, facilitating quick location and access to the original data when needed.

[0149] Each sub-parameter is validated using data length rules to ensure data integrity and protocol compliance. Data length rules define length constraints for each type of data field, including both fixed-length and variable-length types.

[0150] Fixed-length types require data fields to have strict length limits; for example, the function code field in the Modbus protocol must be 1 byte, and the register address field must be 2 bytes. Variable-length types allow data fields to vary within a specified range; for example, the length of a data field can vary between 1 and 125 bytes depending on the actual amount of data.

[0151] In the specific implementation and verification process, the data length rule definition for each sub-segment is first obtained, including the minimum length, maximum length, and length calculation formula. Then, the actual data length of the sub-segment is measured. For data represented in hexadecimal, the length calculation formula is: actual byte length = number of hexadecimal characters ÷ 2.

[0152] Next, the actual length is compared and verified with the rule definition to check whether the length constraint is met. For fixed-length types, the verification condition is: actual length = specified length; for variable-length types, the verification condition is: minimum length ≤ actual length ≤ maximum length. The verification process also needs to consider special cases, such as dynamic length calculations that depend on the values ​​of other fields. In such cases, the values ​​of the dependent fields need to be parsed first, and then the expected length needs to be determined according to the calculation formula.

[0153] In the event of a length verification failure, the length of the sub-segments is adjusted according to a preset repair strategy until the length verification passes, thereby achieving automatic data repair and fault tolerance. Length verification failures are usually caused by data transmission errors, differences in protocol implementation, or deviations in parsing logic, requiring the use of intelligent repair strategies to restore data availability.

[0154] The preset repair strategies include several processing methods: the truncation repair strategy is suitable for cases where the data length exceeds the specified range, by truncating the first N bytes to meet the length requirement, but it is necessary to ensure that the truncation does not destroy the semantic integrity of the data; the padding repair strategy is suitable for cases where the data length is insufficient, by adding zero padding at the end or repeating the last byte to reach the specified length; the interpolation repair strategy is suitable for repairing variable-length data, by analyzing data patterns and context information to infer the missing data content; and the rollback repair strategy, when direct repair is not possible, rolls back to the previous level paragraph for re-parse and splitting.

[0155] The repair process employs a priority mechanism, first attempting the least impactful repair method, such as simple padding or truncation. If the initial repair fails, more complex repair algorithms are tried, such as pattern matching repair based on historical data. Each repair operation is logged, including information such as the original length, repair method, and length after repair, facilitating subsequent quality analysis and issue tracking. After repair is complete, length verification is re-executed to ensure that the repaired sub-parameters comply with the protocol specifications.

[0156] Each validated sub-paragraph is assigned a unique functional paragraph identifier to enable data identification and management. This functional paragraph identifier serves as a unique identifier for each sub-paragraph, facilitating data indexing and retrieval, and providing a unified referencing mechanism for subsequent data processing, storage, and distribution.

[0157] The identifier generation employs a multi-layered encoding strategy to ensure global uniqueness and semantic readability. The first layer of encoding represents the protocol type, such as "MB" for Modbus protocol and "JSON" for JSON format protocol; the second layer of encoding represents the device identifier, generated from the device ID or the hash value of the device address; the third layer of encoding represents the segment type, such as "ADDR" for address segment, "DATA" for data segment, and "CRC" for check segment; the fourth layer of encoding represents the segment sequence number, numbered according to the order of appearance in the protocol.

[0158] The complete identifier format is: Protocol Type_Device Identifier_Segment Type_Segment Number, for example, "MB_A1B2C3_ADDR_001" represents the first address segment of device A1B2C3 in the Modbus protocol. The identifier generation process also considers timestamp information, embedding a hash value of the generation time into the identifier to ensure that identifiers generated under the same conditions are unique.

[0159] To improve the efficiency of identifier generation, a memory caching mechanism is used to store generated identifiers, avoiding redundant calculations. Simultaneously, a bidirectional index is established between identifiers and paragraph content, allowing for quick location of paragraph content using identifiers and reverse lookup of corresponding identifiers using paragraph content.

[0160] Functional paragraph identifiers are associated with their corresponding sub-paragraphs and stored as a data set to achieve data organization and management. This functional paragraph set, as a structured data container, not only contains all functional paragraphs but also maintains the relationships and hierarchical structure between paragraphs.

[0161] The data association storage employs multiple data structures to optimize different types of access needs. The main index structure is implemented using a hash table, with functional paragraph identifiers as keys and paragraph content as values, supporting fast lookups with O(1) time complexity. The auxiliary index structure is implemented using a B+ tree, supporting range queries and sorting operations by paragraph type, device identifier, timestamp, and other dimensions. The association relationship storage is implemented using a graph structure, recording the dependencies, references, and hierarchical relationships between paragraphs, supporting complex association queries and traversal operations.

[0162] Each functional segment in the collection contains not only the raw data content but also rich metadata information, such as segment type, data format, parsing rules, quality identifiers, and timestamps. To support efficient processing of large-scale data, the functional segment collection also implements a sharded storage and parallel access mechanism, dividing the large collection into multiple smaller subsets distributed across different storage nodes. Simultaneously, it implements collection serialization and deserialization functions, supporting persistent data storage and network transmission.

[0163] This embodiment, based on a byte boundary rule-based splitting point determination mechanism, ensures the accuracy of data segmentation and the consistency of memory alignment, avoiding boundary errors and performance issues prone to occur in traditional methods. Dynamic segmentation technology effectively handles complex data structures and variable-length fields, achieving efficient data decomposition through pointer operations and memory management. A strict length verification mechanism provides crucial assurance for data quality, enabling timely detection and handling of various data anomalies. Intelligent repair strategies significantly improve the system's fault tolerance and robustness, maintaining good data processing capabilities even in the event of data transmission errors or protocol implementation differences. A unique identifier allocation mechanism provides a unified management foundation for large-scale data processing, supporting efficient data indexing, retrieval, and association operations. The data association storage of functional segment sets achieves structured data organization and high-performance access, meeting access needs in different scenarios through the combined use of various data structures. Simultaneously, it can handle various complex digital protocols, providing comprehensive technical support for the diverse data processing needs of IoT devices.

[0164] In one embodiment of this invention, the method for determining the target protocol includes the following steps:

[0165] S410. Obtain the protocol type identifier defined in the product model;

[0166] S420. Query the protocol configuration library according to the protocol type identifier, and match the format characteristics of the original data packet with the protocol template in the protocol configuration library to obtain the matching result;

[0167] S430. Determine the target protocol based on the matching results;

[0168] This includes, after determining the target protocol based on the matching results:

[0169] S440. Load the parameters of the target protocol into the protocol parsing engine.

[0170] In this embodiment, obtaining the protocol type identifier defined in the product model can determine the target protocol. This identifier serves as a fundamental clue for protocol identification, providing important guidance for subsequent protocol matching and parsing. The protocol type identifier is a standardized code predefined during the product model creation phase, used to identify the types of communication protocols supported by the product. These identifiers adopt a hierarchical encoding structure. The first layer represents the protocol family, such as "MODBUS" for the Modbus protocol family, "JSON" for the JSON format protocol family, and "XML" for the XML format protocol family; the second layer represents the protocol subtype, such as "MODBUS_RTU" for the Modbus RTU protocol and "MODBUS_TCP" for the Modbus TCP protocol; the third layer represents the protocol version, such as "V1.0", "V2.1", etc.

[0171] During the acquisition process, the corresponding product model record is first located using the device identification information. Then, the protocol type identifier is extracted from the protocol configuration field of the product model. To improve acquisition efficiency, a caching mechanism is used to preload frequently used product model information into memory, avoiding frequent database query operations. Simultaneously, an index structure for protocol type identifiers is established to support fast identifier lookup and matching.

[0172] For product models that support multiple protocols, the protocol type identifier can contain multiple values, requiring further filtering and determination based on the characteristics of the data packets. The acquisition process also includes identifier validity verification to ensure that the acquired protocol type identifier conforms to predefined encoding specifications, avoiding protocol matching failures due to incorrect identifiers.

[0173] The protocol configuration library is queried based on the protocol type identifier, and the format characteristics of the original data packet are matched with the protocol templates in the library to achieve accurate protocol identification. The protocol configuration library is a knowledge base that centrally stores various protocol definitions and configuration information, including detailed information such as protocol structure specifications, field definitions, data formats, and verification rules.

[0174] Based on the obtained protocol type identifier, preliminary filtering is performed in the protocol configuration library to narrow down the matching range and improve matching efficiency. The query process adopts a multi-level index structure: the first level is classified by protocol family, the second level by protocol subtype, and the third level by protocol version, enabling fast location and search.

[0175] Next, format feature matching is performed, which is a complex pattern recognition process that requires analyzing multiple dimensions of the original data packet's features. First, structural feature matching is performed, analyzing the overall structural pattern of the data packet, such as fixed headers, variable data fields, and checksum trailers, and comparing it with the structural pattern defined in the protocol template.

[0176] Then, field feature matching is performed to extract key fields from the data packet, such as protocol identifier, version number, and function code, and accurately match them with the corresponding fields in the protocol template. Next, data format feature matching is performed to analyze the data's encoding method, byte order, data type, and other format characteristics to ensure consistency with the format definition of the protocol template.

[0177] Finally, a verification feature matching process is performed to verify whether the checksum in the data packet conforms to the verification algorithm and rules defined in the protocol template. The matching process adopts a weighted scoring mechanism, assigning different weights to different types of features, and comprehensively calculating the matching degree score to obtain a quantitative matching result.

[0178] The target protocol is determined based on the matching results to complete the decision-making process for protocol identification. The matching results include multiple candidate protocols and their corresponding matching scores, confidence assessments, feature matching details, and other information.

[0179] The target protocol is determined using a multi-judgment mechanism. First, a threshold judgment is performed, setting a minimum matching degree threshold. Only candidate protocols exceeding this threshold are considered valid matches. Then, the optimal selection is performed, choosing the protocol with the highest matching degree score from the valid matching candidate protocols as the target protocol.

[0180] When multiple candidate protocols have similar matching scores, a secondary judgment mechanism is employed. This mechanism analyzes detailed feature matching data, such as the accuracy of key field matching and the completeness of structural matching, to make more precise distinctions. For cases with low matching scores or ambiguities, a manual intervention mechanism is activated, submitting the matching results to professionals for manual judgment and confirmation.

[0181] The process also includes protocol compatibility verification, checking whether the target protocol is compatible with the current system environment and whether it has the corresponding parsing capabilities and processing resources. Simultaneously, the decision-making process and basis for protocol selection are recorded, including matching scores, selection rationale, and verification results, to facilitate subsequent auditing and optimization.

[0182] The parameters of the target protocol are loaded into the protocol parsing engine to enable protocol parsing. This process transforms static protocol definitions into dynamic parsing capabilities, giving the system the ability to process specific protocol data.

[0183] The protocol parsing engine is a configurable data processing framework that can dynamically adjust parsing strategies and processing logic based on the loaded protocol parameters. The loading process first extracts the complete parameter set of the target protocol from the protocol configuration library, including protocol structure definitions, field mapping rules, data conversion rules, verification algorithms, exception handling strategies, etc.

[0184] The protocol structure definition describes the protocol's hierarchical structure and field organization, providing a blueprint for data splitting for the parsing engine. Field mapping rules define the correspondence between protocol fields and product model attributes, guiding the semantic transformation of data. Data transformation rules include various data processing algorithms, such as encoding conversion, numerical calculation, and format standardization. Verification algorithms define methods for verifying data integrity and correctness, such as CRC checksum, parity check, and hash checksum. Anomaly handling strategies specify the handling methods and recovery mechanisms when encountering data anomalies.

[0185] The loading process employs a modular approach, loading different types of parameters into the corresponding modules of the parsing engine, such as the structure parsing module, data transformation module, and validation module. After loading is complete, the parsing engine performs parameter validity verification and compatibility checks to ensure all parameters are correctly loaded and work together. Simultaneously, a fast access index for parameters is established to optimize parameter lookup efficiency during the parsing process.

[0186] This embodiment utilizes a protocol type identification mechanism based on the product model, providing accurate foundational information for protocol identification and effectively avoiding the inefficiency and misidentification caused by blind matching. The protocol configuration library query and format feature matching technology, through multi-dimensional feature analysis and a weighted scoring mechanism, achieves accurate identification of complex protocols, significantly improving the accuracy and reliability of protocol matching. A scientific target protocol determination method, through multiple judgment mechanisms and compatibility verification, ensures the correctness of protocol selection and the stability of the system. A dynamic protocol parameter loading mechanism gives the protocol parsing engine strong adaptability and scalability, enabling rapid support for the access and processing of new protocols. Through standardized protocol management and intelligent identification algorithms, it can automatically adapt to various protocol changes and device differences, greatly reducing the complexity of protocol processing and maintenance costs, providing strong technical support for the large-scale access and management of IoT devices.

[0187] In one embodiment of this invention, parsing each functional segment to obtain functional segment data includes the following steps:

[0188] S510. Determine the data type and encoding format of each functional segment according to the parsing rules of the target protocol;

[0189] S520. Convert the raw byte data in the functional section according to the data type to obtain structured data;

[0190] S530. Decode the structured data according to the encoding format to obtain readable functional paragraph data.

[0191] In this embodiment, the data type and encoding format of each functional segment are determined according to the parsing rules of the target protocol, enabling accurate data parsing. This step establishes accurate data description information for each functional segment by analyzing the field definitions in the protocol specification. The parsing rules are an important component of the target protocol parameters, containing detailed definition information for each field in the protocol. The data type definition describes the basic attributes of the field data. Common data types include integers (such as 8-bit unsigned integers, 16-bit signed integers, and 32-bit long integers), floating-point numbers (such as single-precision floating-point numbers and double-precision floating-point numbers), character types (such as ASCII strings and Unicode strings), boolean types (such as switch status and flag bits), enumeration types (such as device status codes and error codes), and time types (such as timestamps and date / time).

[0192] Encoding format definitions describe how data is encoded during transmission and storage, including numerical encoding (such as big-endian, little-endian, BCD encoding, and two's complement representation), character encoding (such as UTF-8, GBK, and ASCII), and compression encoding (such as LZ77 and Huffman encoding).

[0193] The determination process begins by searching for the corresponding field definition in the parsing rules based on the functional segment's identifier, thus obtaining the data type and encoding format information for that segment. For complex nested structures, it is necessary to recursively analyze the type definitions of internal fields to establish a complete data type hierarchy. Simultaneously, considering protocol version differences and vendor-customized implementations, version adaptation and compatibility handling are employed to ensure the accuracy of type determination.

[0194] The determination process also includes type verification, which checks whether the data type definition matches the actual data length and format, and promptly identifies and handles type definition errors, thereby providing accurate type guidance for subsequent data conversion and decoding processes.

[0195] The raw byte data in the functional section is converted according to its data type to obtain structured data, thereby achieving data standardization. Raw byte data usually exists in the form of hexadecimal strings or binary byte sequences, and needs to be converted according to the determined data type to become a structured data object that the program can directly process.

[0196] The type conversion process employs a factory pattern design, implementing a dedicated converter for each data type. For integer data conversion, byte order must be considered. Big-Endian stores the most significant byte at the lowest address, while little-Endian stores the least significant byte at the lowest address. During conversion, the bytes must be correctly rearranged according to the byte order specified by the protocol. For example, converting the hexadecimal string "01F4" to a 16-bit unsigned integer results in 500 in big-endian and 62721 in little-endian.

[0197] For floating-point data conversion, it is necessary to parse according to the IEEE 754 standard, reorganizing the byte data into sign, exponent, and mantissa bits, and then calculating the actual floating-point value. For character data conversion, it is necessary to convert the byte data into a string according to the character encoding format. For example, ASCII encoding directly maps bytes to characters, while UTF-8 encoding requires handling combinations of multi-byte characters.

[0198] For Boolean data conversion, non-zero values ​​are typically converted to true, and zero values ​​are converted to false. For enumerated data conversion, the values ​​need to be converted to the corresponding enumerated constants according to a predefined enumeration mapping table. The conversion process also includes data validity verification, checking whether the converted data is within a reasonable range of values, and marking or correcting data that exceeds the range.

[0199] The structured data is decoded according to the encoding format to obtain readable functional paragraph data, thereby achieving data semantics. This process further processes the type-converted structured data into readable data with business meaning.

[0200] Decoding involves processing algorithms for various encoding formats, requiring the adoption of corresponding decoding strategies based on different encoding methods. For the decoding of numerical encoding, BCD encoding requires interpreting each half-byte (4 bits) as a decimal number, such as hexadecimal "1234" representing the decimal number 1234 in BCD encoding; two's complement encoding requires conversion between positive and negative numbers based on the sign bit, such as 8-bit two's complement "FF" representing the decimal number -1.

[0201] For character encoding decoding, conversion is required according to the specific character set standard. UTF-8 encoding supports multi-byte characters and requires correct handling of character boundaries and encoding sequences; GBK encoding is mainly used for Chinese characters and requires handling combinations of double-byte characters. For compressed encoding decoding, corresponding decompression algorithms need to be implemented. For example, the LZ77 algorithm restores data through dictionary matching, and Huffman coding decodes data through frequency statistics trees.

[0202] The decoding process also includes data formatting, such as converting timestamps to readable date and time formats, adding unit identifiers to numerical values, and converting status codes to descriptive text. For complex data structures, recursive decoding is required to ensure that data at all levels is correctly decoded.

[0203] The decoding process also needs to handle various abnormal situations, such as encoding format mismatch, data corruption, and unsupported character sets. Fault tolerance mechanisms are employed to ensure the stability of the decoding process. The final functional segment data has good readability and business semantics, and can be directly used for subsequent business processing and user display.

[0204] This embodiment, based on the target protocol parsing rules' data type and encoding format determination mechanism, provides accurate type guidance for each functional segment, ensuring the correctness and consistency of subsequent conversion processing. Systematic type conversion processing, through a dedicated converter, achieves precise conversion of various data types, effectively handling complex issues such as byte order, data format, and value range, converting raw byte data into standard structured data objects. Diverse encoding format decoding processing supports decoding of various encoding methods such as numerical encoding, character encoding, and compression encoding. Through corresponding decoding algorithms and fault-tolerance mechanisms, it achieves semantic data conversion and formatted output. Simultaneously, it possesses high modularity and scalability, supporting rapid access to new data types and encoding formats, and adapting to constantly evolving protocol standards and business needs. Through precise type conversion and intelligent decoding processing, complex binary protocol data can be converted into intuitive and readable business data, greatly reducing data processing complexity, improving data usability and value, and laying a solid technical foundation for the in-depth application and analysis of IoT device data.

[0205] In one embodiment of this invention, mapping functional segment data to corresponding attributes of the product model includes the following steps:

[0206] S610. Extract features from the functional paragraph data to obtain the features of the functional paragraph data;

[0207] S620. Match the corresponding instruction template in the protocol instruction library according to the characteristics of the functional segment data. The protocol instruction library stores instruction templates and mapping rules for different device types.

[0208] S630. Determine the target attribute based on the mapping relationship defined in the instruction template, and determine whether the data types of the functional paragraph data and the target attribute are compatible.

[0209] S640. If the data types of the functional paragraph data and the target attribute are compatible, write the functional paragraph data into the corresponding attribute field of the product model.

[0210] Feature extraction is performed on functional paragraph data to obtain its features, enabling accurate data mapping. The feature extraction process analyzes the multi-dimensional attributes of the functional paragraph data to generate feature descriptors for subsequent matching and mapping, providing fundamental information for accurate data classification and processing.

[0211] Feature extraction encompasses data feature analysis at multiple levels. First, data content features are extracted, analyzing attributes such as numerical range, precision, and sign. For example, temperature data typically ranges from -50 to 100 degrees Celsius, while voltage data may contain two decimal places. Next, data format features are extracted, identifying format attributes such as data representation, encoding type, and length specifications, such as hexadecimal representation, BCD encoding, and 4-byte length.

[0212] Next, semantic feature extraction is performed on the data. This involves analyzing the business meaning of the data using methods such as keyword recognition and pattern matching. For example, the keyword "temp" indicates temperature data, and the keyword "volt" indicates voltage data. Data location feature extraction is also required, recording the positional information of functional paragraphs within the protocol, their sequential relationships, and other positional attributes, such as being the third paragraph or immediately following an address paragraph. Simultaneously, statistical feature extraction is performed, calculating statistical attributes such as data distribution characteristics, trends, and anomalies.

[0213] The feature extraction process employs a combination of algorithms, including statistical analysis, pattern recognition, and machine learning. The extracted features are stored in structured feature vector form for easy matching and comparison operations.

[0214] Intelligent data mapping is achieved by matching corresponding instruction templates from the protocol instruction library based on the characteristics of functional segment data. The protocol instruction library, serving as the system's knowledge base, stores instruction templates and mapping rules for different device types and protocol versions, providing standardized reference templates for data mapping.

[0215] The instruction template contains complete definition information for the instruction, such as instruction name, function description, paragraph structure, data field definitions, and mapping relationships. Each instruction template is associated with corresponding feature descriptions for matching and comparison with the features of the functional paragraph data.

[0216] The matching process employs a multi-level similarity calculation algorithm. First, coarse-grained matching is performed, quickly filtering candidate templates based on basic features such as data type, length, and position to narrow down the matching range. Then, fine-grained matching is performed, using similarity calculations of feature vectors, such as Euclidean distance, cosine similarity, and Jaccard coefficient, to quantitatively evaluate the degree of matching between the functional paragraph data features and the instruction template features.

[0217] The matching process also considers contextual information such as device type, protocol version, and manufacturer customization, improving matching accuracy through weighted calculations. For complex multi-segment instructions, a sequence matching algorithm is used, comprehensively considering the order relationship and combination pattern between segments. Matching results are output in the form of matching score and confidence assessment, supporting the sorting and selection of multiple candidate templates.

[0218] At the same time, a matching history record and learning mechanism are established. By analyzing the accuracy of historical matching results, the matching algorithm and feature weights are continuously optimized to improve the intelligence level of matching.

[0219] The target attribute is determined based on the mapping relationship defined in the instruction template, and the data type compatibility between the functional section data and the target attribute is verified to ensure correct data mapping. The mapping relationship in the instruction template defines the correspondence between protocol fields and product model attributes, providing clear guidance for accurate data attribution.

[0220] Mapping relationships include various types of mapping definitions, such as one-to-one mapping (a single protocol field corresponds to a single model attribute), one-to-many mapping (a single protocol field corresponds to multiple model attributes), many-to-one mapping (a combination of multiple protocol fields corresponds to a single model attribute), and computational mapping (the target attribute value is calculated using a formula).

[0221] The process of determining the target attribute first parses the mapping rules in the instruction template, and then finds the corresponding mapping definition based on the identifier and location information of the functional paragraph. For complex mapping relationships, dependencies and constraints need to be considered, such as the mapping of certain attributes depending on the values ​​or states of other fields.

[0222] After determining the target attribute, perform data type compatibility verification to check whether the data type of the functional paragraph data is compatible with the data type of the target attribute. Compatibility verification includes basic type matching (such as integer to integer, character to character), precision matching (such as numeric precision, string length), value range matching (such as maximum and minimum value limits for numeric values), and format matching (such as date and time formats, enumeration value ranges), etc.

[0223] For cases where the data types are not perfectly matched but can be converted, assess the feasibility of the conversion and the risk of data loss. Compatibility verification also considers the rationality of the business logic, such as temperature data not being mapped to device address attributes, and voltage data not being mapped to timestamp attributes.

[0224] Provided the data types of the functional paragraph data and the target attribute are compatible, the functional paragraph data is written to the corresponding attribute field of the product model to complete the data mapping. This step, through a secure and reliable data writing mechanism, accurately stores the verified functional paragraph data into the corresponding attribute of the product model, achieving persistent data storage and subsequent access.

[0225] The data writing process employs transactional operations to ensure data consistency and integrity. First, a final verification is performed before writing, including data integrity checks, format correctness verification, and business rule compliance checks, to ensure the quality of the written data. Then, data format conversion is performed, making necessary format adjustments to the functional segment data according to the storage format requirements of the target attributes, such as adjusting numerical precision, converting string encoding, and standardizing time formats.

[0226] Next, the actual data write operation is performed, employing atomic operations to ensure the uninterrupted nature of the write process and avoid data inconsistency due to system anomalies. The write process also includes data version management, generating a version identifier for each write operation to support historical data tracing and rollback operations.

[0227] Simultaneously, a data change log is established to record detailed information about write operations, including write time, data source, and changed content, facilitating auditing and troubleshooting. After the write operation is complete, a data synchronization operation is performed to synchronize the updated attribute data to the relevant cache, index, and backup systems, ensuring data consistency. It is also necessary to trigger relevant business events and notification mechanisms, such as data change notifications, threshold alarms, and status updates.

[0228] This embodiment utilizes a multi-dimensional feature extraction-based data analysis mechanism to establish a complete feature profile for each functional segment of data. This profile includes feature information across multiple levels, such as content, format, semantics, location, and statistics, providing rich judgment criteria for subsequent intelligent matching. The feature-matching-based instruction template recognition technology, through multi-level similarity calculation and contextual information weighting, automatically associates functional segment data with standard instruction templates, significantly improving the accuracy and efficiency of mapping. A rigorous mapping relationship determination and compatibility verification mechanism ensures the correctness and rationality of data mapping, avoiding data anomalies caused by type mismatches or business logic errors. A secure and reliable data writing mechanism, through transactional operations, version management, and change logs, guarantees data consistency, integrity, and traceability. Simultaneously, it possesses a high level of automation and intelligence, capable of handling complex mapping relationships and diverse data types, supporting efficient processing of large-scale data. Through precise feature matching and intelligent mapping decisions, it can automatically identify and process various protocol data, accurately mapping them to corresponding product model attributes, providing strong technical support for the standardized management and in-depth application of IoT device data.

[0229] In one embodiment of this invention, the following steps are also included:

[0230] S710: In response to access signals from IoT devices, determine the data type and business requirements;

[0231] S720. When the data type is real-time monitoring data, the real-time monitoring data will be stored in the time series database.

[0232] S730. When the data type is configuration parameter data, the configuration parameter data is stored in a relational database.

[0233] S740. Perform data synchronization between time-series databases and relational databases.

[0234] In this embodiment, in response to the access signal from an IoT device, the data type and business requirements are determined to implement an intelligent data storage strategy. When an IoT device sends an access signal through an established multi-protocol access link, it is necessary to quickly and accurately identify the nature and purpose of the data to provide a basis for subsequent storage strategy selection.

[0235] Access signals contain device and data information, such as key elements like device identifiers, data packets, timestamps, and protocol types. Data type determination employs a multi-dimensional analysis method. First, data content analysis is performed, identifying the basic attributes of the data by parsing the field content and numerical characteristics of the data packets.

[0236] Real-time monitoring data typically has time-series characteristics, with a high data update frequency, and includes dynamic information such as sensor readings, equipment status, and operating parameters; configuration parameter data, on the other hand, is relatively stable, with a low update frequency, and includes static information such as equipment files, protocol configurations, and user settings.

[0237] Next, data frequency analysis is performed to determine the update cycle and frequency of changes in statistical data. High-frequency updated data is typically real-time monitoring data, while low-frequency updated data is typically configuration parameter data. Then, business semantic analysis is conducted to determine the business type and storage requirements of the data based on the business meaning and application scenario of the data fields.

[0238] Determining business requirements considers multiple factors, including data access patterns (such as real-time queries, historical analysis, and batch processing), storage periods (such as short-term caching and long-term archiving), query frequency (such as high-frequency access and low-frequency access), and data importance (such as critical business data and auxiliary reference data). By comprehensively analyzing these dimensions of information, and using decision trees or rule engines, the data types and business requirements are automatically determined, providing accurate guidance for subsequent storage strategies.

[0239] When the data type is real-time monitoring data, the real-time monitoring data is stored in a time-series database for specialized storage of high-frequency dynamic data. Time-series databases are specifically optimized for time-series data, featuring high-throughput writing, efficient compressed storage, and fast time-range query capabilities, making them ideal for handling the large volumes of real-time monitoring data generated by IoT devices.

[0240] The storage process for real-time monitoring data employs batch writing and asynchronous processing mechanisms to improve storage efficiency and system response performance. First, the real-time monitoring data undergoes preprocessing, including data format standardization, timestamp calibration, and outlier detection, to ensure data quality and consistency. Timestamp calibration is particularly important, requiring the unification of time bases across different devices and addressing time zone differences and clock skew issues.

[0241] Then, based on the business attributes and query requirements of the data, a reasonable storage model is designed, including table structure design, partitioning strategy, and index configuration. The table structure typically adopts a wide table design, storing multiple monitoring metrics for the same device in the same record, reducing the complexity of join queries. The partitioning strategy is based on a time dimension, such as by day, week, or month, facilitating the management and query optimization of historical data.

[0242] Next, data write operations are performed, using batch insertion to improve write efficiency. Appropriate write buffers and flushing strategies are configured to balance performance and data security. The storage process also includes data compression. Time-series databases typically support multiple compression algorithms, such as Delta compression, dictionary compression, and LZ4 compression. The optimal compression method is selected based on data characteristics, significantly reducing storage space usage.

[0243] When the data type is configuration parameter data, the configuration parameter data is stored in a relational database for standardized storage of structured static data. Relational databases provide comprehensive ACID transaction support, rich query functions, and strong data consistency guarantees, making them very suitable for storing relatively stable structured data such as device configurations, user settings, and system parameters.

[0244] The storage process for configuration parameter data emphasizes data standardization and integrity constraints. First, a data model is designed, establishing a reasonable table structure and relationships based on the business relationships and access patterns of the configuration parameters. The table structure adheres to the three normalization principles of database design to avoid data redundancy and update anomalies. The main table structures include equipment file tables, product model tables, protocol configuration tables, and user settings tables, with foreign key relationships establishing connections between them.

[0245] Next, data integrity constraints are designed, including primary key constraints, foreign key constraints, uniqueness constraints, and check constraints, to ensure data correctness and consistency. Then, data write operations are performed, using transactional operations to ensure data atomicity. When configuration parameters involve multiple related tables, a transaction mechanism ensures the synchronous updating of all related data.

[0246] The write process also includes data validation, checking the correctness of the configuration parameter format, the rationality of the values, and the consistency of business logic. Data that does not meet the requirements is rejected or corrected. After storage, the corresponding index structure is established, including primary key indexes, foreign key indexes, and business query indexes, to optimize data query performance. Simultaneously, backup and recovery mechanisms are configured to regularly back up the configuration parameter data, ensuring data security and recoverability.

[0247] Data synchronization between time-series databases and relational databases is necessary to maintain data consistency and support cross-database queries. Since real-time monitoring data and configuration parameter data are stored in different types of databases, an effective data synchronization mechanism is required to ensure the consistency and integrity of the relevant data.

[0248] Data synchronization employs a combination of strategies, selecting the appropriate synchronization method based on data characteristics and business needs. For the synchronization of basic equipment information, a master-slave synchronization model is used, with equipment files in a relational database as the primary data source. Equipment information is periodically synchronized to the metadata table in the time-series database to ensure the accuracy of equipment information in the time-series data records.

[0249] The synchronization process employs an incremental synchronization mechanism, synchronizing only changed data records. Timestamp comparison and version number control are used to identify data changes requiring synchronization. For real-time status information synchronization, a bidirectional synchronization mode is used, synchronizing the latest device status information from the time-series database to the device status table in the relational database, facilitating configuration management and status queries.

[0250] The synchronization process also includes data conversion. Because the two databases have different data formats and storage structures, corresponding data format conversions and field mappings are required. A synchronization monitoring and anomaly handling mechanism is established to monitor the execution status of the synchronization process in real time. When synchronization anomalies or data inconsistencies are detected, timely alarms and repairs are initiated. Simultaneously, manual synchronization and full synchronization functions are provided to support complete data synchronization operations during system maintenance or data repair.

[0251] This embodiment utilizes an intelligent judgment mechanism based on data type and business requirements. Through multi-dimensional analysis and a decision engine, it automatically identifies the nature and purpose of data, providing a scientific basis for selecting storage strategies. For the time-series database storage strategy for real-time monitoring data, techniques such as batch writing, data compression, and partition management achieve efficient storage and rapid querying of high-frequency dynamic data, meeting the processing needs of massive time-series data in IoT scenarios. For the relational database storage strategy for configuration parameter data, standardized design, integrity constraints, and transaction guarantees ensure accurate storage and reliable management of structured static data. A robust data synchronization mechanism, through incremental synchronization, bidirectional synchronization, and anomaly monitoring, ensures the consistency of data between the two databases and the continuity of business processing. Simultaneously, it fully leverages the technical advantages of different database types, guaranteeing both high-performance processing of real-time data and highly reliable management of configuration data. This provides strong technical support for the full lifecycle management of IoT device data, effectively supporting the data storage and processing needs of large-scale IoT applications.

[0252] In one embodiment of this invention, data synchronization between a time-series database and a relational database includes the following steps:

[0253] S810. Establish a data synchronization channel between the time-series database and the relational database, and periodically identify the data items to be synchronized;

[0254] S820. Determine the direction and frequency of data synchronization according to the synchronization strategy. The synchronization strategy includes one-way synchronization and two-way synchronization.

[0255] In this embodiment, a data synchronization channel is established between a time-series database and a relational database, and data items to be synchronized are identified periodically to achieve cross-database data consistency. As a bridge connecting two different database systems, the data synchronization channel needs to handle complex technical issues such as data format differences, connection management, and transmission reliability.

[0256] The synchronization channel is established using a layered architecture, including a connection layer, a protocol layer, a data layer, and an application layer. The connection layer is responsible for establishing and maintaining physical connections with the two databases, employing connection pooling technology to manage database connections, improving connection reuse efficiency and system performance. The connection pool configuration includes parameters such as minimum number of connections, maximum number of connections, connection timeout, and idle connection recycling time, dynamically adjusting the connection pool size according to system load.

[0257] The protocol layer handles the differences in communication protocols between different databases. Time-series databases typically use proprietary query languages ​​and API interfaces, while relational databases use standard SQL protocols, requiring protocol conversion and adaptation. The data layer handles data format conversion and field mapping, including data type conversion, character encoding conversion, and time format standardization. The application layer provides a unified data synchronization interface and management functions, including synchronization task scheduling, status monitoring, and exception handling.

[0258] The identification of data items to be synchronized employs a combination of strategies, including timestamp comparison, version number checking, hash value comparison, and incremental log analysis. The periodic identification process is implemented through a scheduled task scheduler, supporting flexible scheduling strategy configurations such as fixed-interval scheduling, Cron expression scheduling, and event-triggered scheduling.

[0259] The identification process also includes a data change detection algorithm, which accurately identifies changed data records by comparing information such as the last modification time, version identifier, and content hash, thus avoiding unnecessary data transmission and processing overhead.

[0260] The direction and frequency of data synchronization are determined based on the synchronization strategy to achieve precise data synchronization control. As a guiding principle for data synchronization, the synchronization strategy needs to comprehensively consider multiple factors such as data characteristics, business requirements, system performance, and consistency requirements to formulate the optimal synchronization plan.

[0261] Synchronization strategies encompass two key dimensions: synchronization direction and synchronization frequency. The synchronization direction determines the data flow path. One-way synchronization is suitable for scenarios with a clear master-slave data relationship, such as unidirectionally synchronizing device configuration information from a relational database to the metadata table of a time-series database, ensuring that the time-series data records contain accurate device information. One-way synchronization has advantages such as logical simplicity, fewer conflicts, and high performance, but its flexibility is relatively low.

[0262] Two-way synchronization is suitable for scenarios where both databases can generate data updates. For example, device status information can be updated in real-time in a time-series database or through a management interface in a relational database. Two-way synchronization requires handling data conflict detection and resolution mechanisms. When the same data item is updated in both databases, it needs to be handled according to a preset conflict resolution strategy, such as timestamp priority, business priority, or manual intervention.

[0263] Determining the synchronization frequency requires balancing data consistency requirements and system performance overhead. High-frequency synchronization can ensure better data consistency, but it will increase system load and network overhead; low-frequency synchronization can reduce system overhead, but it can lead to a longer time window for data inconsistency.

[0264] Frequency settings support multiple modes, including real-time synchronization (synchronization immediately after data changes), timed synchronization (synchronization at fixed time intervals), batch synchronization (synchronization after accumulating a certain amount of data), and conditional synchronization (synchronization triggered when specific conditions are met). Synchronization strategies also need to consider data priority, using high-frequency synchronization for critical business data and low-frequency synchronization for auxiliary reference data, thus optimizing overall performance through differentiated synchronization strategies.

[0265] This embodiment's layered architecture synchronization channel design, through connection pool management, protocol adaptation, and data conversion, achieves reliable connection and efficient data transmission between two different database systems. An intelligent data item identification mechanism, using multiple detection strategies and timed scheduling, accurately identifies data changes and triggers synchronization operations promptly, avoiding the performance overhead of full synchronization. Flexible synchronization strategy configuration supports both one-way and two-way synchronization modes, adapting to different business scenarios and data flow requirements. Scientific synchronization frequency control, through multiple synchronization modes and priority mechanisms, optimizes system performance while ensuring data consistency. A robust conflict detection and resolution mechanism ensures data consistency and business logic correctness in two-way synchronization scenarios. Furthermore, it possesses high configurability and scalability, supporting flexible adjustments to synchronization strategies and parameter configurations according to business needs, providing reliable technical support for cross-database management and application of IoT device data, and effectively supporting data consistency requirements in complex business scenarios.

[0266] This application also provides a data parsing system for IoT devices that supports multi-protocol custom splicing, including:

[0267] Electronic devices, equipped with an IoT device access platform;

[0268] Multiple IoT devices, each connected to an electronic device.

[0269] In this embodiment, the electronic device can be a tablet computer, desktop computer, laptop computer, handheld computer, wearable device, laptop computer, ultra-mobile personal computer (UMPC), netbook, or other device with a processor. Of course, the electronic device can also be a server. This application embodiment does not impose any special limitations on the specific form of the electronic device.

[0270] This application also provides a machine-readable storage medium storing instructions that cause a machine to execute the above-described method for parsing data from a multi-protocol custom-connected IoT device.

[0271] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0272] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0273] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0274] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0275] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0276] Memory can include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0277] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0278] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0279] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for parsing data from multi-protocol custom-stitched IoT devices, characterized in that, This method, applied to an IoT device access platform, includes: Construct access links for multi-protocol devices; In response to the raw data packets reported by the IoT device through the multi-protocol device access link, the product model of the IoT device is determined based on the raw data packets, wherein the attributes of the product model include fixed attributes and variable attributes; The start and end positions of the paragraphs are determined based on the fixed attributes of the product model. The paragraph length is dynamically adjusted based on the variable attributes of the product model to obtain the dynamically adjusted paragraph. Determine whether the protocol type of the original data packet is a numeric protocol or a character protocol; When the protocol type is the digital protocol, the split point position is determined according to the byte boundary rules defined in the product model, and the byte boundary rules include alignment and padding rules. The dynamically adjusted paragraph is divided into multiple sub-paragraphs according to the stated splitting point positions; Each sub-paragraph is subjected to length validation using data length rules, which include fixed length and variable length types. If the length verification fails, the length of the sub-segment is adjusted according to a preset repair strategy until the length verification passes. Assign a unique functional paragraph identifier to each validated sub-paragraph; The functional paragraph identifier and its corresponding sub-paragraph are associated and stored to obtain a functional paragraph set, which includes multiple functional paragraphs. When the protocol type is a character-based protocol, the dynamically adjusted paragraphs are split according to the delimiter and preset keyword rules to obtain multiple functional paragraphs. Each functional paragraph corresponds to a data unit in the target protocol, and the target protocol is determined by the attributes of the product model. Each functional segment is parsed to obtain functional segment data, and the functional segment data is mapped to the corresponding attribute of the product model; The mapped functional segment data is stored in a time-series database and a relational database, wherein the time-series database is used to store device operation data and the relational database is used to store device configuration information.

2. The method according to claim 1, characterized in that, The method for determining the target protocol includes: Obtain the protocol type identifier defined in the product model; The protocol configuration library is queried based on the protocol type identifier, and the format characteristics of the original data packet are matched with the protocol templates in the protocol configuration library to obtain the matching result; The target protocol is determined based on the matching results; The step of determining the target protocol based on the matching result includes: The parameters of the target protocol are loaded into the protocol parsing engine.

3. The method according to claim 1, characterized in that, The parsing of each functional segment to obtain functional segment data includes: The data type and encoding format of each functional segment are determined according to the parsing rules of the target protocol; The raw byte data in the functional section is converted according to the data type to obtain structured data; The structured data is decoded according to the encoding format to obtain readable functional paragraph data.

4. The method according to claim 1, characterized in that, The step of mapping the functional segment data to the corresponding attributes of the product model includes: Feature extraction is performed on the functional paragraph data to obtain the features of the functional paragraph data; The corresponding instruction template in the protocol instruction library is matched according to the features of the functional segment data. The protocol instruction library stores instruction templates and mapping rules for different device types. The target attribute is determined based on the mapping relationship defined in the instruction template, and it is determined whether the data type of the functional paragraph data is compatible with that of the target attribute. If the data type of the functional paragraph is compatible with that of the target attribute, the functional paragraph data is written into the corresponding attribute field of the product model.

5. The method according to claim 1, characterized in that, The method also includes: In response to the access signal from the IoT device, determine the data type and business requirements; When the data type is real-time monitoring data, the real-time monitoring data is stored in a time-series database; When the data type is configuration parameter data, the configuration parameter data is stored in a relational database; Data synchronization is performed between the time-series database and the relational database.

6. The method according to claim 5, characterized in that, The data synchronization between the time-series database and the relational database includes: Establish a data synchronization channel between the time-series database and the relational database, and periodically identify data items to be synchronized; The direction and frequency of data synchronization are determined according to a synchronization strategy, which includes one-way synchronization and two-way synchronization.

7. A data parsing system for IoT devices supporting multi-protocol custom splicing, applied to the data parsing method for IoT devices supporting multi-protocol custom splicing as described in any one of claims 1-6, characterized in that, include: Electronic devices, equipped with an IoT device access platform; Multiple Internet of Things (IoT) devices, each of which is connected to the electronic device.

8. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores instructions for causing the machine to execute the data parsing method for supporting multi-protocol custom splicing IoT devices as described in any one of claims 1 to 6.

Images