Compatible data transmission method and system based on dynamic text structure definition

Abstract

The invention provides a compatible data transmission method and system based on dynamic text structure definition, and the method comprises the steps: S1, a transmitting end creates and maintains a preset dynamic text structure definition file, and carries out the network real-time publishing of the preset dynamic text structure definition file, so that a receiving end carries out the real-time subscription or query; s2, the sending end obtains target data, organizes the target data into memory blocks based on a preset dynamic text structure definition file, and transmits the memory blocks to the receiving end; and S3, receiving the memory block by the receiving end, and dynamically analyzing the received memory block based on a preset dynamic text structure definition file subscribed or queried by the receiving end in real time to obtain structured data.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and more specifically, to a compatible data transmission method and system based on a dynamic text structure definition. Background Technology

[0002] With the booming development of the Internet of Things (IoT), the data interaction needs between different devices and systems are becoming increasingly complex and diverse. Various IoT communication protocols are emerging to achieve efficient data transmission.

[0003] Patent document CN119814773A (application number: 202510286452.9) discloses a general data transmission method, device and storage medium for the Internet of Things. This method decouples the transmission protocol and the transmission content, enabling different Internet of Things communication protocols to work together efficiently on the same data across different physical layer media and MCUs.

[0004] However, with the expansion of IoT application scenarios, such as the need for sensors in smart homes to accurately upload dynamically changing environmental data in real time, and the continuous changes in business objects as production processes evolve in industrial production, more stringent requirements are being placed on data transmission. In these scenarios, existing technologies reveal significant shortcomings in areas such as data transmission granularity, data structure definition mechanisms, receiver data parsing and compatibility, and the utilization of the advantages of dynamic structure parsing. These shortcomings make it difficult to meet the needs of the deep development of the IoT, and a more advanced, universal, and flexible data transmission solution is urgently needed to overcome these bottlenecks. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the purpose of this invention is to provide a compatible data transmission method and system based on a dynamic text structure definition.

[0006] A compatible data transmission method based on dynamic text structure definition provided by the present invention includes: Step S1: The sending end creates and maintains a preset dynamic text structure definition file, and publishes the preset dynamic text structure definition file to the network in real time so that the receiving end can subscribe or query in real time; Step S2: The sending end obtains the target data, organizes the target data into memory blocks based on the preset dynamic text structure definition file, and transmits the memory blocks to the receiving end; Step S3: The receiving end receives the memory block and dynamically parses the received memory block based on the preset dynamic text structure definition file that the receiving end subscribes to or queries in real time to obtain structured data.

[0007] Preferably, the preset dynamic text structure definition file includes memory block layout information; the memory block layout information includes: the starting offset, data type, data length, byte order, and nesting structure of each field.

[0008] Preferably, step S3 includes: Step S3.1: The receiving end reads the preset dynamic text structure definition file and interprets the memory blocks one by one according to the layout information of the memory blocks in the preset dynamic text structure definition file; Step S3.2: Convert the dynamically parsed data into key-value pairs to obtain structured data.

[0009] Preferably, the method further includes: when the receiving end has a custom dynamic text structure definition file, converting the data parsed from the memory block into a data format that conforms to the data format in the custom dynamic text structure definition file of the receiving end.

[0010] Preferably, all fields in the receiver's custom dynamic text structure definition file are traversed. For each field of the receiver, its field identifier is used as the key to search in the set of key-value pairs parsed from the sender's memory block. If a field with the same name is found, data assignment is performed, and the value from the sender is assigned to the corresponding field of the receiver. If no field with the same name is found, the field of the receiver will be assigned a default value.

[0011] A compatible data transmission system based on dynamic text structure definition, provided by the present invention, includes: Module M1: The sending end creates and maintains a preset dynamic text structure definition file, and publishes the preset dynamic text structure definition file to the network in real time so that the receiving end can subscribe or query in real time; Module M2: The sending end acquires the target data, organizes the target data into memory blocks based on a preset dynamic text structure definition file, and transmits the memory blocks to the receiving end; Module M3: The receiving end receives memory blocks and dynamically parses the received memory blocks based on the preset dynamic text structure definition file that the receiving end subscribes to or queries in real time to obtain structured data.

[0012] Preferably, the preset dynamic text structure definition file includes memory block layout information; the memory block layout information includes: the starting offset, data type, data length, byte order, and nesting structure of each field.

[0013] Preferably, the module M3 includes: Module M3.1: The receiving end reads the preset dynamic text structure definition file and interprets the memory blocks one by one according to the layout information of the memory blocks in the preset dynamic text structure definition file; Module M3.2: Converts dynamically parsed data into key-value pairs to obtain structured data.

[0014] Preferably, the system further includes: when the receiving end has a custom dynamic text structure definition file, converting the data parsed from the memory block into a data format that conforms to the data format in the custom dynamic text structure definition file of the receiving end.

[0015] Preferably, all fields in the receiver's custom dynamic text structure definition file are traversed. For each field of the receiver, its field identifier is used as the key to search in the set of key-value pairs parsed from the sender's memory block. If a field with the same name is found, data assignment is performed, and the value from the sender is assigned to the corresponding field of the receiver. If no field with the same name is found, the field of the receiver will be assigned a default value.

[0016] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention achieves a high degree of decoupling between the data structure definition of the sender and receiver, allowing both parties to independently update and evolve their respective data structures without the need for close negotiation and frequent synchronization, which significantly improves the flexibility and maintainability of the system. 2. This invention fundamentally ensures that the system can be compatible with different versions of data structures to the greatest extent. Even if the sender and receiver use different versions or slightly different data definitions, as long as there is overlap in the data fields, effective communication can be achieved through runtime dynamic parsing and intelligent conversion, which greatly reduces communication failures and system upgrade costs caused by version inconsistency. 3. This invention cleverly combines the flexibility of dynamic text structure definition with the extreme efficiency of memory block transmission, avoiding the performance overhead caused by traditional transmission of complete XML documents or rigid fixed formats, while maintaining the high self-descriptiveness and dynamic adaptability of the data. Attached Figure Description

[0017] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a schematic diagram of a compatibility data transmission system based on a dynamic text structure definition.

[0018] Figure 2 Flowchart for a compatibility data transmission method defined based on a dynamic text structure.

[0019] Figure 3 Define an example diagram for dynamic text structures.

[0020] Figure 4 This is for parsing and transforming the logic diagram. Detailed Implementation

[0021] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0022] Example 1 According to the present invention, a compatible data transmission method and system based on dynamic text structure definition includes: dynamically publishing dynamic text structure definition files, such as XML, YAML, and JSON, on a network to achieve real-time, on-demand sharing of data structures. The sender efficiently transmits memory block data based on this dynamic definition, while the receiver can flexibly parse and reconstruct the memory block based on the sender's latest published or its own custom dynamic text structure definition file. This achieves a high degree of decoupling between the sender and receiver, and ensures maximum data version compatibility and smooth evolution.

[0023] The compatibility data transmission system based on dynamic text structure definition, such as Figures 1 to 4 As shown, it includes: The sender data definition and publishing module is used by the sender to create and maintain a dynamic text structure definition file. This file defines in real-time details the fields, types, order, and other information of the structured data to be sent. It acquires data generated by users or the system and organizes it into a compact memory block according to the preset dynamic text structure definition. The generated text structure definition is then dynamically published to specific topics or services on the network, such as message queues or configuration centers, for receivers to subscribe to or query in real time.

[0024] The receiver's structure definition acquisition module is used to subscribe to or query topics / services on the network in real time to dynamically acquire dynamic text structure definition files published by the sender; it receives and stores the dynamic text structure definitions published by the sender. This dynamically acquired text structure definition will serve as the latest blueprint for subsequent parsing of memory blocks. The receiver can also choose to load its own defined text structure definitions to achieve personalized parsing or data reconstruction.

[0025] Data memory block transmission module: The sender sends the actual data, organized according to its current dynamic text structure definition, to the network in an efficient memory block form, such as serialized binary data; the receiver receives the raw memory block from the network.

[0026] Receiver Dynamic Parsing and Conversion Module: After receiving a memory block, the receiver uses the previously obtained dynamic text structure definition defined by the sender or the receiver's own dynamic text structure definition to dynamically parse the memory block.

[0027] Specifically, the receiver dynamic parsing and conversion module includes: The parser submodule: The core of the decoupler submodule lies in its dynamic parsing capability. Instead of relying on pre-compiled, fixed data structures, such as structs in C / C++, it loads a dynamically provided "structure definition file" at runtime, such as an XML, JSON, or custom description file. This definition file specifies in detail the layout of memory blocks, including but not limited to: the starting offset of each field, data type (int, float, double, string, etc.), data length, byte order (big-endian / little-endian), nested structures, etc. The parser reads this definition at runtime and, like an interpreter, interprets memory block by block according to the rules in the definition.

[0028] Key-value pair extraction submodule: Converts dynamically parsed data into key-value pair format, such as ABc=1, for easier internal processing.

[0029] The struct reconstruction submodule: If the receiver has a custom dynamic text struct definition, this submodule will attempt to convert the data parsed from the memory block into a data format that conforms to the receiver's custom dynamic text struct definition. For example, it will generate a compatible memory block or directly populate a new struct object. The system will iterate through all fields in the receiver's struct definition. For each field in the receiver, it will search for its field identifier in the set of key-value pairs parsed from the sender's memory block. If a field with the same name is found, data assignment will be performed: the sender's value will be assigned to the corresponding field in the receiver. If no field with the same name is found, the receiver's field will be assigned a default value (such as 0, null, or an empty string).

[0030] Finally, the reconstructed and compatible structured data is provided to the recipient for subsequent business logic or application processing.

[0031] Example 2 Example 2 is a preferred example of Example 1. A compatible data transmission method based on dynamic text structure definition provided by the present invention includes: Step S1: The sending end creates and maintains a preset dynamic text structure definition file, and publishes the preset dynamic text structure definition file to the network in real time so that the receiving end can subscribe or query in real time; Specifically, the preset dynamic text structure definition file includes the layout information of memory blocks; the layout information of the memory blocks includes: the starting offset, data type, data length, byte order and nesting structure of each field.

[0032] Step S2: The sending end obtains the target data, organizes the target data into memory blocks based on the preset dynamic text structure definition file, and transmits the memory blocks to the receiving end; Step S3: The receiving end receives the memory block and dynamically parses the received memory block based on the preset dynamic text structure definition file that the receiving end subscribes to or queries in real time to obtain structured data.

[0033] Specifically, step S3 includes: Step S3.1: The receiving end reads the preset dynamic text structure definition file and interprets the memory blocks one by one according to the layout information of the memory blocks in the preset dynamic text structure definition file; Step S3.2: Convert the dynamically parsed data into key-value pairs to obtain structured data.

[0034] Specifically, the method further includes: when the receiving end has a custom dynamic text structure definition file, converting the data parsed from the memory block into a data format that conforms to the custom dynamic text structure definition file of the receiving end.

[0035] Iterate through all fields in the receiver's custom dynamic text structure definition file. For each field on the receiver, use its field identifier as the key to search for it in the set of key-value pairs parsed from the sender's memory block. If a field with the same name is found, perform data assignment and assign the sender's value to the corresponding field on the receiver. If no field with the same name is found, the receiver's field will be assigned a default value.

[0036] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0037] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A compatible data transmission method based on dynamic text structure definition, characterized in that, include: Step S1: The sending end creates and maintains a preset dynamic text structure definition file, and publishes the preset dynamic text structure definition file to the network in real time so that the receiving end can subscribe or query in real time; Step S2: The sending end obtains the target data, organizes the target data into memory blocks based on the preset dynamic text structure definition file, and transmits the memory blocks to the receiving end; Step S3: The receiving end receives the memory block and dynamically parses the received memory block based on the preset dynamic text structure definition file that the receiving end subscribes to or queries in real time to obtain structured data.

2. The compatible data transmission method based on dynamic text structure definition according to claim 1, characterized in that, The preset dynamic text structure definition file includes the layout information of memory blocks; the layout information of memory blocks includes: the starting offset, data type, data length, byte order and nesting structure of each field.

3. The compatible data transmission method based on dynamic text structure definition according to claim 1, characterized in that, Step S3 includes: Step S3.1: The receiving end reads the preset dynamic text structure definition file and interprets the memory blocks one by one according to the layout information of the memory blocks in the preset dynamic text structure definition file; Step S3.2: Convert the dynamically parsed data into key-value pairs to obtain structured data.

4. The compatible data transmission method based on dynamic text structure definition according to claim 1, characterized in that, The method further includes: when the receiving end has a custom dynamic text structure definition file, converting the data parsed from the memory block into a data format that conforms to the data format in the custom dynamic text structure definition file of the receiving end.

5. The compatible data transmission method based on dynamic text structure definition according to claim 4, characterized in that, Iterate through all fields in the receiver's custom dynamic text structure definition file. For each field on the receiver, use its field identifier as the key to search for it in the set of key-value pairs parsed from the sender's memory block. If a field with the same name is found, perform data assignment and assign the sender's value to the corresponding field on the receiver. If no field with the same name is found, the receiver's field will be assigned a default value.

6. A compatible data transmission system based on a dynamic text structure definition, characterized in that, include: Module M1: The sending end creates and maintains a preset dynamic text structure definition file, and publishes the preset dynamic text structure definition file to the network in real time so that the receiving end can subscribe or query in real time; Module M2: The sending end acquires the target data, organizes the target data into memory blocks based on a preset dynamic text structure definition file, and transmits the memory blocks to the receiving end; Module M3: The receiving end receives memory blocks and dynamically parses the received memory blocks based on the preset dynamic text structure definition file that the receiving end subscribes to or queries in real time to obtain structured data.

7. The compatible data transmission system based on dynamic text structure definition according to claim 6, characterized in that, The preset dynamic text structure definition file includes the layout information of memory blocks; the layout information of memory blocks includes: the starting offset, data type, data length, byte order and nesting structure of each field.

8. The compatible data transmission system based on dynamic text structure definition according to claim 6, characterized in that, The module M3 includes: Module M3.1: The receiving end reads the preset dynamic text structure definition file and interprets the memory blocks one by one according to the layout information of the memory blocks in the preset dynamic text structure definition file; Module M3.2: Converts dynamically parsed data into key-value pairs to obtain structured data.

9. The compatible data transmission system based on dynamic text structure definition according to claim 6, characterized in that, The system further includes: when the receiving end has a custom dynamic text structure definition file, the data parsed from the memory block is converted into a data format that conforms to the data format in the custom dynamic text structure definition file of the receiving end.

10. The compatible data transmission system based on dynamic text structure definition according to claim 9, characterized in that, Iterate through all fields in the receiver's custom dynamic text structure definition file. For each field on the receiver, use its field identifier as the key to search for it in the set of key-value pairs parsed from the sender's memory block. If a field with the same name is found, perform data assignment and assign the sender's value to the corresponding field on the receiver. If no field with the same name is found, the receiver's field will be assigned a default value.

Images