Data transmission method and device based on JSON self-description, equipment and medium

Abstract

The invention relates to the technical field of data exchange and transmission, in particular to a data transmission method, device and equipment based on JSON self-description and a medium, the method comprises the following steps: generating a message header based on a JSON format, the message header being used for describing a data structure; to-be-transmitted data is organized into a message body in a plain text format, fields and rows are segmented according to separators defined in a message header, and a nested structure is processed according to a nested structure identifier defined in the message header; splicing the message header in the JSON format and the message body in the text format into a complete message; the spliced message is encrypted and / or compressed and then transmitted; and after the message is received, analyzing the message header, analyzing the message body according to the separator and the field sequence described by the message header, and restoring the message body into the structured data. The data is accurately segmented and processed according to the description of the message header, so that the accuracy and efficiency of data transmission are effectively improved, and the error rate in the data transmission process is reduced.

Description

Technical Field

[0001] The present application relates to the field of data exchange and transmission technology, and specifically to a data transmission method, apparatus, device, and medium based on JSON self-description. Background Art

[0002] Currently, XML (Extensible Markup Language) and JSON (JavaScript Object Notation) are two widely adopted standard formats for data exchange and transmission. XML, with its powerful functionality and flexibility, has long held a prominent position in data representation and transmission. However, with the advent of the big data era, XML has gradually exposed numerous drawbacks when handling large-scale data transmission. Its significant redundancy leads to extremely large data volumes, which not only increases network bandwidth usage, significantly prolongs data transmission time, reduces data transmission efficiency, and places a heavy burden on data transmission systems.

[0003] JSON, with its relatively concise syntax, has alleviated data redundancy to a certain extent and has been widely used in many scenarios. However, JSON's performance remains unsatisfactory when processing large amounts of data in repetitive formats. When faced with large-scale, highly repetitive data transmission needs, JSON-formatted data still occupies a large amount of storage space. Furthermore, due to the lack of more efficient data organization and transmission mechanisms during data parsing and processing, data processing efficiency is unable to meet the growing demand for large-scale data processing.

[0004] Furthermore, there are some data protocols on the market that offer high compression ratios. While these protocols excel at data compression and can effectively reduce data transmission volume, they generally lack the ability to self-describe the message data structure. This means that after receiving a message, the receiver cannot directly obtain detailed information about the data structure from the message, resulting in poor message readability. To correctly parse the message data, the receiver often needs to write additional parsing logic or adopt specialized serialization methods, which undoubtedly increases the complexity of data processing and development costs. Summary of the Invention

[0005] In view of the problem that in existing transmission methods, it is difficult for the receiver to directly obtain detailed information of the data structure from the message after receiving the message, resulting in poor message readability, the present invention provides a data transmission method, device, equipment and medium based on JSON self-description.

[0006] In a first aspect, the technical solution of the present invention provides a data transmission method based on JSON self-description, comprising the following steps: S1. Generate a message header based on the JSON format, where the message header is used to describe the data structure, including field name, data type, separator, and nested structure identifier; S2. Organize the data to be transmitted into a message body in plain text format, wherein fields and rows are separated according to the delimiters defined in the message header, and nested structures are processed according to the nested structure identifiers defined in the message header; S3. Concatenate the message header in JSON format and the message body in text format into a complete message; S4. Encrypting and / or compressing the spliced ​​message before transmitting; S5. After receiving the message, the receiver parses the message header, and parses the message body according to the delimiter and field order described in the message header to restore it to structured data.

[0007] The JSON-formatted message header and text-formatted message body are concatenated into a complete message, which is then encrypted and / or compressed before transmission. Encryption effectively protects data security, preventing theft or tampering during transmission. Compression further reduces message size, lowering network bandwidth usage and improving data transmission speed and efficiency. These measures ensure the security, integrity, and efficiency of data transmission.

[0008] As a further limitation of the technical solution of the present invention, step S1 specifically includes: S11: extract all field names and nested relationships in the data to be transferred; S12: Specify the data type for each field, and set the order and data constraints of the fields in the message body; S13: Define field separators and row separators; S14: If there is a nested relationship, define the nested structure and nested delimiters and hierarchical relationship; S15: Integrate the field name, data type, field order, data constraint, row separator, field separator, and nested structure definition into a JSON object to form a message header.

[0009] It allows fields and rows to be segmented based on the delimiters defined in the message header, and handles nested structures according to nested structure identifiers. This flexible data format processing method allows it to adapt to a variety of complex data structures, accurately describing and transmitting both simple flat data and complex nested data, greatly enhancing the versatility and applicability of the data transmission method.

[0010] As a further limitation of the technical solution of the present invention, in step S2, when organizing the message body, if there is a nested structure, the nested data is merged into a row with the upper-level data as a column of data according to the nested delimiter and field order defined in the message header.

[0011] As a further limitation of the technical solution of the present invention, in step S2, if there is a nested structure, the specific steps of organizing the message body are as follows: S21b: Parse the nested structure definition in the message header and extract the description information of the nested structure from the JSON format message header, including the nested field name, nested data type, nested delimiter, and nested hierarchical relationship; S22b: Extract nested data and connect fields with nested delimiters to organize text data hierarchically. S23b: Fill the column where the nested data is located in the field of the upper-level message directly with the text data organized in step S22b, so as to embed the nested data as a text column into the upper-level structure; S24b: If the nested structure still contains nesting, recursively execute steps S21b-S23b, organize the text data level by level using the defined delimiters to form a message body in plain text format.

[0012] As a further limitation of the technical solution of the present invention, in step S2, the step of organizing the data to be transmitted into a message body in plain text format includes: S20: Parse the data to be transmitted to determine the data structure; If the data is a nested structure, execute step S21b; If the data is a tile structure, execute step S21a; S21a: Extract the values ​​of the corresponding fields in sequence according to the order of the fields specified in the message header; S22a: Use the field delimiter defined in the message header to connect all field values ​​in the current row; S23a: Repeat steps S21a-S22a to generate multiple lines of text; S24a: Use the line separator defined in the message header to connect all lines of text to form a plain text message body.

[0013] As a further limitation of the technical solution of the present invention, step S5 specifically includes: S51: After receiving the message, the message header is decomposed from the message and deserialized into a data structure including field description information, separator information, and nested structure; S52: Parse the row data using the row delimiter described in the message header, parse the field data based on the field delimiter, match the fields based on the order of the fields in the message header, and organize the fields according to the data type and data constraints; S53: Determine whether there is a nested structure; If yes, go to step S54; if no, go to step S55; S54: Continue parsing the lower-level message according to the nested definition and recursively restore the nested data; execute step S55; S55: Forming list data, describing each row of data in a Map format.

[0014] After receiving a message, the receiver only needs to parse the message header to accurately parse the message body according to the delimiters and field order described in the message header, restoring it to structured data. This simple and efficient data parsing method eliminates the need for the receiver to write additional parsing logic or adopt special serialization methods, reducing data processing complexity and development costs. Furthermore, because the message header clearly describes the data structure, the message is highly readable, allowing the receiver to intuitively understand the data structure within the message and manually modify the message information as needed, improving the flexibility and convenience of data processing.

[0015] As a further limitation of the technical solution of the present invention, the format of the message includes the following parts: JSON format, used for describing the message header and plain text format of the data structure, and the message body organized according to the delimiters and nested structure defined in the message header.

[0016] In a second aspect, the technical solution of the present invention also provides a data transmission method based on JSON self-description, including a message header generation module, a message body organization module, a message generation module, a transmission processing module and a parsing processing module; A message header generation module, configured to generate a message header based on the JSON format, wherein the message header is used to describe a data structure including a field name, a data type, a separator, and a nested structure identifier; A message body organization module is used to organize the data to be transmitted into a message body in plain text format, in which fields and rows are separated according to the delimiters defined in the message header, and nested structures are processed according to the nested structure identifiers defined in the message header; The message generation module is used to combine the JSON format message header and the text format message body into a complete message; A transmission processing module, used for encrypting and / or compressing the spliced ​​messages before transmission; The parsing processing module is used for parsing the message header after the receiver receives the message, parsing the message body according to the delimiter and field sequence described in the message header, and restoring it into structured data.

[0017] The data transmission process is divided into modules such as message header generation, message body organization, message generation, transmission processing, and parsing processing. Each module has independent functions, facilitating system maintenance and expansion. Modules interact through clear interfaces, reducing system coupling and improving system stability and reliability.

[0018] As a further limitation of the technical solution of the present invention, the message header generation module includes an extraction unit, a setting unit, a delimiter definition unit, a nested structure definition unit and a message header generation unit; An extraction unit, used to extract all field names and nested relationships in the data to be transmitted; The setting unit is used to specify the data type for each field and set the order and data constraints of the fields in the message body; The delimiter definition unit is used to define field delimiters and row delimiters; A nested structure definition unit is used to define the nested structure, nested delimiters, and hierarchical relationships if a nested relationship exists; The message header generation unit integrates field names, data types, field order, data constraints, row delimiters, field delimiters, and nested structure definitions into a JSON object to form a message header. Through the extraction unit, setting unit, delimiter definition unit, nested structure definition unit, and message header generation unit, message header generation is automated and standardized, improving the accuracy and efficiency of message header generation.

[0019] As a further limitation of the technical solution of the present invention, the message body organization module includes a nested structure organization unit, which is used to organize the message body. If there is a nested structure, the nested data is merged into a row with the upper-level data as a column of data according to the nested delimiter and field order defined in the message header.

[0020] The nested structure organization unit combines the nested data and the parent data into one row according to the nested delimiter and field order defined in the message header, ensuring the integrity and accuracy of the nested structure during transmission.

[0021] As a further limitation of the technical solution of the present invention, the nested structure organization unit includes a first parsing submodule, a text organization submodule and a nesting processing submodule; The first parsing submodule is used to parse the nested structure definition in the message header and extract the description information of the nested structure from the message header in JSON format; including the nested field name, nested data type, nested delimiter and nested hierarchical relationship; The text organization submodule is used to extract nested data and connect fields with nested delimiters to organize text data hierarchically; The nested processing submodule is used to directly fill the column where the nested data is located in the field of the parent message with organized text data, so as to embed the nested data as a text column into the parent structure; if the nested structure still contains nesting, the processing process is recursively executed, and the text data is organized level by level with the defined separator to form a message body in plain text format.

[0022] Through the first parsing submodule, the text organization submodule and the nesting processing submodule, accurate parsing, text organization and processing of the nested structure are achieved, thereby improving the accuracy and efficiency of nested data transmission.

[0023] As a further limitation of the technical solution of the present invention, the message body organization module includes a data structure judgment unit and a tile structure organization unit; A data structure determination unit, configured to parse the data to be transmitted and determine the data structure; If the data is a nested structure, the nested structure organization unit is triggered; if the data is a flat structure, the flat structure organization unit is triggered; The flat structure organization unit includes a field value extraction submodule, a connection submodule, a line text generation submodule and a message body generation submodule; The field value extraction submodule is used to extract the values ​​of the corresponding fields in sequence according to the field order specified in the message header; The connection submodule is used to connect all field values ​​of the current row using the field separator defined in the message header; Line text generation submodule, used to generate multiple lines of text; The message body generation submodule is used to connect all line texts using the line separator defined in the message header to form a message body in plain text format.

[0024] Through data structure judgment units and flat structure organization units, different organization methods are adopted according to whether the data structure is a nested structure or a flat structure, ensuring that various data structures can be accurately and efficiently organized into message bodies.

[0025] As a further limitation of the technical solution of the present invention, the parsing and processing module is specifically used to decompose the message header from the message after receiving the message, and deserialize it into a data structure containing field description information, separator information, and nested structure; parse the row data through the row separator described in the message header, parse the field data according to the field separator, match the fields according to the order of the fields in the message header, and organize the fields according to the data type and data constraints; if there is a nested structure, continue to parse the lower-level message according to the nested definition, and finally form list data, describing each row of data in Map format.

[0026] The parsing and processing module parses the message header and message body, and accurately restores structured data based on information such as field order, data type, and data constraints to ensure data integrity and accuracy.

[0027] As a further limitation of the technical solution of the present invention, the format of the message includes the following parts: JSON format, used for describing the message header and plain text format of the data structure, and the message body organized according to the delimiters and nested structure defined in the message header.

[0028] It is clear that the message consists of a message header in JSON format and a message body in plain text format. The message body is organized according to the delimiters and nested structures defined in the message header, so that the message format is unified and standardized, which facilitates data transmission and parsing.

[0029] In a third aspect, the technical solution of the present invention also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; the memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the JSON self-description-based data transmission method as described in the first aspect.

[0030] In a fourth aspect, the technical solution of the present invention also provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions enable the computer to execute the JSON self-description-based data transmission method as described in the first aspect.

[0031] As can be seen from the above technical solution, this application has the following advantages: by generating a message header based on the JSON format and describing the data structure in detail, including information such as field names, data types, delimiters, and nested structure identifiers, the data to be transmitted can be organized into a plain text message body according to predefined rules. This structured data organization method not only facilitates data storage and management, but also enables accurate segmentation and processing of data according to the description of the message header during data transmission, effectively improving the accuracy and efficiency of data transmission and reducing the error rate during data transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solution of the present application, the following is a brief introduction to the drawings required for the description. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0033] Figure 1 A schematic flow chart of a method provided in an embodiment of the present invention.

[0034] Figure 2 A block diagram of a device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to make the application objectives, features, and advantages of this application more obvious and easy to understand, the technical solutions protected by this application will be clearly and completely described below using specific embodiments and drawings. Obviously, the embodiments described below are only part of the embodiments of this application, not all of them. Based on the embodiments in this patent, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this patent.

[0036] like Figure 1 As shown, an embodiment of the present invention provides a data transmission method based on JSON self-description, comprising the following steps: S1. Generate a message header based on the JSON format, where the message header is used to describe the data structure, including field name, data type, separator, and nested structure identifier; In the embodiment of the present invention, generating a message header requires defining a basic data structure, including: Field name: List all field names that need to be transferred (such as timestamp, sensor_id, value).

[0037] Data type: Specify the type for each field (such as string, number, boolean).

[0038] Field order: specifies the order in which the fields are arranged in the message body (e.g. [0:timestamp, 1:sensor_id, 2:value]).

[0039] The rules for setting delimiters include: Row delimiter: defines the symbol that separates different data rows (such as \n).

[0040] Field separator: defines the symbol that separates fields in the same line (such as,).

[0041] Nested delimiter: If there is a nested structure (such as an array or object), define the delimiter for its internal fields and rows (such as | to separate fields, # to separate array elements).

[0042] When a nested structure exists, it is necessary to describe the nested structure, mainly including: Nested field name: A name that identifies a nested structure (such as sub_data).

[0043] Nested Types: Declare nested types (array or object).

[0044] Nested field definition: Define the field name, type, and order within a nested structure in the same way as you define the underlying data structure.

[0045] Nested delimiter: Specify a separate delimiter for nested structures (e.g. | for fields and # for array elements).

[0046] Meta information needs to be added when necessary, including: Compression flag: declares whether to compress and the compression algorithm (such as "compression": "gzip").

[0047] Encryption flag: declares whether encryption is used and the encryption algorithm (e.g. "encryption": "aes-256-gcm").

[0048] Verification information: Add data verification method (such as "checksum":"crc32").

[0049] Integrate all the above information into a JSON object to form the final TSON message header.

[0050] It should be noted here that the format of this application is defined as the TSON protocol, which consists of two parts: header and body. The header part uses the JSON format and describes in detail the data structure to be transmitted, including but not limited to data type, field name, order, and separator information. The body part uses plain text format to transmit actual data. Predefined separators are used to distinguish between each field and data at different levels. The message body can be compressed if necessary. In specific implementation, TSON defines a set of standardized rule sets to guide how to parse the data in the body part, ensuring that even in the face of complex nested structures, the transmission can be completed accurately. The highlight is its high self-descriptiveness and flexibility, which enables TSON to not only significantly reduce the data volume, but also adapt to the needs of various application scenarios.

[0051] S2. Organize the data to be transmitted into a message body in plain text format, wherein fields and rows are separated according to the delimiters defined in the message header, and nested structures are processed according to the nested structure identifiers defined in the message header; In this step, the general process of organizing the message body includes: parsing the data into a list, organizing the row data according to the column sorting information, data type and separator defined in the message header; parsing multiple lines to form a string list; then merging them into a row of data according to the row separator; the nested structure is also organized as a column of data according to the nested information defined in the message header and merged with the parent list into a row of data; data compression can also be performed as needed to further reduce the message size.

[0052] S3. Concatenate the message header in JSON format and the message body in text format into a complete message; S4. Encrypting and / or compressing the spliced ​​message before transmitting; S5. After receiving the message, the receiver parses the message header, and parses the message body according to the delimiter and field order described in the message header to restore it to structured data.

[0053] The JSON-formatted message header and text-formatted message body are concatenated into a complete message, which is then encrypted and / or compressed before transmission. Encryption effectively protects data security, preventing theft or tampering during transmission. Compression further reduces message size, lowering network bandwidth usage and improving data transmission speed and efficiency. These measures ensure the security, integrity, and efficiency of data transmission.

[0054] In some embodiments, step S1 specifically includes: S11: extract all field names and nested relationships in the data to be transferred; S12: Specify the data type for each field, and set the order and data constraints of the fields in the message body; S13: Define field separators and row separators; S14: If there is a nested relationship, define the nested structure and nested delimiters and hierarchical relationship; S15: Integrate the field name, data type, field order, data constraint, row separator, field separator, and nested structure definition into a JSON object to form a message header.

[0055] In some embodiments, in step S2, when organizing the message body, if there is a nested structure, the nested data is combined into a row with the parent data as a column of data according to the nested delimiter and field order defined in the message header. Specifically, if there is a nested structure, the specific steps for organizing the message body are as follows: S21b: Parse the nested structure definition in the message header and extract the description information of the nested structure from the JSON format message header, including the nested field name, nested data type, nested delimiter, and nested hierarchical relationship; S22b: Extract nested data and connect fields with nested delimiters to organize text data hierarchically. S23b: Fill the column where the nested data is located in the field of the upper-level message directly with the text data organized in step S22b, so as to embed the nested data as a text column into the upper-level structure; S24b: If the nested structure still contains nesting, recursively execute steps S21b-S23b, organize the text data level by level using the defined delimiters to form a message body in plain text format.

[0056] In some embodiments, in step S2, the step of organizing the data to be transmitted into a message body in plain text format includes: S20: Parse the data to be transmitted to determine the data structure; If the data is a nested structure, execute step S21b; If the data is a tile structure, execute step S21a; S21a: Extract the values ​​of the corresponding fields in sequence according to the order of the fields specified in the message header; S22a: Use the field delimiter defined in the message header to connect all field values ​​in the current row; S23a: Repeat steps S21a-S22a to generate multiple lines of text; S24a: Use the line separator defined in the message header to connect all lines of text to form a plain text message body.

[0057] In an embodiment of the present invention, when generating a message body, the input is the original data to be transmitted (such as a JSON object, a database record, or an in-memory data structure). If the data is a nested structure (such as a JSON object), it is expanded according to the nesting rules defined in the message header. If the data is a flat structure (such as a CSV row), the field value is directly extracted. According to the field order specified by the fields array in the message header (such as ["timestamp", "sensor_id", "value"]), the values ​​of the corresponding fields are extracted in sequence. If a field has no value, it is filled with an empty string or a placeholder agreed upon by the protocol (such as NULL). According to the field type defined in the message header (such as string, number, boolean), it is uniformly converted into a string: Numeric type: directly converted to a string (e.g. 25.5 → "25.5").

[0058] Boolean type: converts to "true" or "false".

[0059] Date type: serialized according to the agreed format (such as ISO 8601: "2025-01-01T00:00:00Z").

[0060] Concatenate all field values ​​in the current row using the field delimiter defined in the message header. If a field is a nested object or array, first serialize it into sub-text using the nesting delimiters (such as | and #), then embed the entire text into the main row. Repeat this process for multiple data records (such as multi-line database query results) to generate multiple lines of text. Concatenate all lines using the line delimiter defined in the message header (such as \n) to form the final message body.

[0061] In some embodiments, step S5 specifically includes: S51: After receiving the message, the message header is decomposed from the message and deserialized into a data structure including field description information, separator information, and nested structure; S52: Parse the row data using the row delimiter described in the message header, parse the field data based on the field delimiter, match the fields based on the order of the fields in the message header, and organize the fields according to the data type and data constraints; S53: Determine whether there is a nested structure; If yes, go to step S54; if no, go to step S55; S54: Continue parsing the lower-level message according to the nested definition and recursively restore the nested data; execute step S55; S55: Forming list data, describing each row of data in a Map format.

[0062] After receiving a message, the receiver only needs to parse the message header to accurately parse the message body according to the delimiters and field order described in the message header, restoring it to structured data. This simple and efficient data parsing method eliminates the need for the receiver to write additional parsing logic or adopt special serialization methods, reducing data processing complexity and development costs. Furthermore, because the message header clearly describes the data structure, the message is highly readable, allowing the receiver to intuitively understand the data structure within the message and manually modify the message information as needed, improving the flexibility and convenience of data processing.

[0063] In fact, from the above content, it can be seen that the data transmission of this application is equivalent to using the TSON protocol for data transmission. Here, the specific application steps for IoT data transmission include: Construct TSON message header: Use JSON format to describe the data structure, including field name, data type, field order, row separator, field separator, and nested structure definition; Organizing the TSON message body: organizing the IoT device data in a plain text format according to the delimiters and field order defined in the TSON message header to form a TSON message body; Transmitting data: combining the TSON message header and the TSON message body into a complete TSON message, and transmitting the data through the Internet of Things communication protocol; Deserialization: The receiving end parses the TSON message header, parses the TSON message body according to the delimiter and field order described therein, and restores it to structured IoT device data.

[0064] like Figure 2As shown, an embodiment of the present invention further provides a data transmission method based on JSON self-description, including a message header generation module, a message body organization module, a message generation module, a transmission processing module and a parsing processing module; A message header generation module, configured to generate a message header based on the JSON format, wherein the message header is used to describe a data structure including a field name, a data type, a separator, and a nested structure identifier; A message body organization module is used to organize the data to be transmitted into a message body in plain text format, in which fields and rows are separated according to the delimiters defined in the message header, and nested structures are processed according to the nested structure identifiers defined in the message header; The message generation module is used to combine the JSON format message header and the text format message body into a complete message; A transmission processing module, used for encrypting and / or compressing the spliced ​​messages before transmission; The parsing processing module is used for parsing the message header after the receiver receives the message, parsing the message body according to the delimiter and field sequence described in the message header, and restoring it into structured data.

[0065] The data transmission process is divided into modules such as message header generation, message body organization, message generation, transmission processing, and parsing processing. Each module has independent functions, facilitating system maintenance and expansion. Modules interact through clear interfaces, reducing system coupling and improving system stability and reliability.

[0066] In some embodiments, the message header generation module includes an extraction unit, a setting unit, a delimiter definition unit, a nested structure definition unit, and a message header generation unit; An extraction unit, used to extract all field names and nested relationships in the data to be transmitted; The setting unit is used to specify the data type for each field and set the order and data constraints of the fields in the message body; The delimiter definition unit is used to define field delimiters and row delimiters; A nested structure definition unit is used to define the nested structure, nested delimiters, and hierarchical relationships if a nested relationship exists; The message header generation unit integrates field names, data types, field order, data constraints, row delimiters, field delimiters, and nested structure definitions into a JSON object to form a message header. Through the extraction unit, setting unit, delimiter definition unit, nested structure definition unit, and message header generation unit, message header generation is automated and standardized, improving the accuracy and efficiency of message header generation.

[0067] In some embodiments, the message body organization module includes a nested structure organization unit, which is used to organize the message body. If there is a nested structure, the nested data is merged into a row with the upper-level data as a column of data according to the nested delimiter and field order defined in the message header.

[0068] The nested structure organization unit combines the nested data and the parent data into one row according to the nested delimiter and field order defined in the message header, ensuring the integrity and accuracy of the nested structure during transmission.

[0069] In some embodiments, the nested structure organization unit includes a first parsing submodule, a text organization submodule, and a nesting processing submodule; The first parsing submodule is used to parse the nested structure definition in the message header and extract the description information of the nested structure from the message header in JSON format; including the nested field name, nested data type, nested delimiter and nested hierarchical relationship; The text organization submodule is used to extract nested data and connect fields with nested delimiters to organize text data hierarchically; The nested processing submodule is used to directly fill the column where the nested data is located in the field of the parent message with organized text data, so as to embed the nested data as a text column into the parent structure; if the nested structure still contains nesting, the processing process is recursively executed, and the text data is organized level by level with the defined separator to form a message body in plain text format.

[0070] Through the first parsing submodule, the text organization submodule and the nesting processing submodule, accurate parsing, text organization and processing of the nested structure are achieved, thereby improving the accuracy and efficiency of nested data transmission.

[0071] In some embodiments, the message body organization module includes a data structure determination unit and a tile structure organization unit; A data structure determination unit, configured to parse the data to be transmitted and determine the data structure; If the data is a nested structure, the nested structure organization unit is triggered; if the data is a flat structure, the flat structure organization unit is triggered; The flat structure organization unit includes a field value extraction submodule, a connection submodule, a line text generation submodule and a message body generation submodule; The field value extraction submodule is used to extract the values ​​of the corresponding fields in sequence according to the field order specified in the message header; The connection submodule is used to connect all field values ​​of the current row using the field separator defined in the message header; Line text generation submodule, used to generate multiple lines of text; The message body generation submodule is used to connect all line texts using the line separator defined in the message header to form a message body in plain text format.

[0072] Through data structure judgment units and flat structure organization units, different organization methods are adopted according to whether the data structure is a nested structure or a flat structure, ensuring that various data structures can be accurately and efficiently organized into message bodies.

[0073] In some embodiments, the parsing and processing module is specifically used to decompose the message header from the message after receiving the message, and deserialize it into a data structure containing field description information, separator information, and a nested structure; parse the row data through the row separator described in the message header, parse the field data according to the field separator, match the fields according to the order of the fields in the message header, and organize the fields according to the data type and data constraints; if there is a nested structure, continue to parse the lower-level message according to the nested definition, and finally form list data, describing each row of data in Map format.

[0074] The parsing and processing module parses the message header and message body, and accurately restores structured data based on information such as field order, data type, and data constraints to ensure data integrity and accuracy.

[0075] An embodiment of the present invention further provides an electronic device comprising: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus. The communication bus can be used to transmit information between the electronic device and a sensor. The processor can invoke logic instructions in the memory to execute the following method: S1. Generate a message header based on the JSON format, wherein the message header is used to describe a data structure, wherein the data structure includes field names, data types, delimiters, and nested structure identifiers; S2. Organize the data to be transmitted into a plain text message body, wherein fields and rows are separated according to delimiters defined in the message header, and nested structures are processed according to nested structure identifiers defined in the message header; S3. Concatenate the JSON message header and the text message body into a complete message; S4. Encrypt and / or compress the concatenated message before transmission; S5. Upon receiving the message, the recipient parses the message header and parses the message body according to the delimiters and field order described in the message header to restore it to structured data.

[0076] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0077] An embodiment of the present invention provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions enable a computer to execute the method provided by the above-mentioned method embodiment, for example, including: S1, generating a message header based on the JSON format, wherein the message header is used to describe the data structure, and the data structure includes a field name, a data type, a separator, and a nested structure identifier; S2, organizing the data to be transmitted into a message body in plain text format, wherein the fields and rows are divided according to the separators defined in the message header, and the nested structure is processed according to the nested structure identifier defined in the message header; S3, splicing the message header in JSON format and the message body in text format into a complete message; S4, encrypting and / or compressing the spliced ​​message and transmitting it; S5, after receiving the message, the recipient parses the message header, parses the message body according to the separator and field order described in the message header, and restores it to structured data.

[0078] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data transmission method based on JSON self-description, characterized in that: The following steps are involved: S1. Generate a message header based on the JSON format, where the message header is used to describe the data structure, including field name, data type, separator, and nested structure identifier; S2. Organize the data to be transmitted into a message body in plain text format, wherein fields and rows are separated according to the delimiters defined in the message header, and nested structures are processed according to the nested structure identifiers defined in the message header; S3. Concatenate the message header in JSON format and the message body in text format into a complete message; S4. Encrypting and / or compressing the spliced ​​message before transmitting; S5. After receiving the message, the receiver parses the message header, and parses the message body according to the delimiter and field order described in the message header to restore it to structured data.

2. The data transmission method based on JSON self-description according to claim 1, characterized in that Step S1 specifically includes: S11: extract all field names and nested relationships in the data to be transferred; S12: Specify the data type for each field, and set the order and data constraints of the fields in the message body; S13: Define field separators and row separators; S14: If there is a nested relationship, define the nested structure and nested delimiters and hierarchical relationship; S15: Integrate the field name, data type, field order, data constraint, row separator, field separator, and nested structure definition into a JSON object to form a message header.

3. The data transmission method based on JSON self-description according to claim 2, characterized in that In step S2, when organizing the message body, if there is a nested structure, the nested data is combined into a row with the upper-level data as a column of data according to the nested delimiter and field order defined in the message header.

4. The data transmission method based on JSON self-description according to claim 3, characterized in that In step S2, if there is a nested structure, the specific steps for organizing the message body are as follows: S21b: Parse the nested structure definition in the message header and extract the description information of the nested structure from the message header in JSON format; Including nested field names, nested data types, nested delimiters, and nested hierarchical relationships; S22b: Extract nested data and connect fields with nested delimiters to organize text data hierarchically. S23b: Fill the column where the nested data is located in the field of the upper-level message directly with the text data organized in step S22b, so as to embed the nested data as a text column into the upper-level structure; S24b: If the nested structure still contains nesting, recursively execute steps S21b-S23b, organize the text data level by level using the defined delimiters to form a message body in plain text format.

5. The data transmission method based on JSON self-description according to claim 4, characterized in that: In step S2, the step of organizing the data to be transmitted into a message body in plain text format includes: S20: Parse the data to be transmitted to determine the data structure; If the data is a nested structure, execute step S21b; If the data is a tile structure, execute step S21a; S21a: Extract the values ​​of the corresponding fields in sequence according to the order of the fields specified in the message header; S22a: Use the field delimiter defined in the message header to connect all field values ​​in the current row; S23a: Repeat steps S21a-S22a to generate multiple lines of text; S24a: Use the line separator defined in the message header to connect all lines of text to form a plain text message body.

6. The data transmission method based on JSON self-description according to claim 5, characterized in that: Step S5 specifically includes: S51: After receiving the message, the message header is decomposed from the message and deserialized into a data structure including field description information, separator information, and nested structure; S52: Parse the row data using the row delimiter described in the message header, parse the field data based on the field delimiter, match the fields based on the order of the fields in the message header, and organize the fields according to the data type and data constraints; S53: Determine whether there is a nested structure; If yes, go to step S54; if no, go to step S55; S54: Continue parsing the lower-level message according to the nested definition and recursively restore the nested data; execute step S55; S55: Forming list data, describing each row of data in a Map format.

7. The data transmission method based on JSON self-description according to claim 6, characterized in that: The message format consists of the following parts: JSON format, used for describing the message header and plain text format of the data structure, and the message body organized according to the delimiters and nested structure defined in the message header.

8. A data transmission method based on JSON self-description, characterized in that: It includes a message header generation module, a message body organization module, a message generation module, a transmission processing module and a parsing processing module; A message header generation module, configured to generate a message header based on the JSON format, wherein the message header is used to describe a data structure including a field name, a data type, a separator, and a nested structure identifier; A message body organization module is used to organize the data to be transmitted into a message body in plain text format, in which fields and rows are separated according to the delimiters defined in the message header, and nested structures are processed according to the nested structure identifiers defined in the message header; The message generation module is used to combine the JSON format message header and the text format message body into a complete message; A transmission processing module, used for encrypting and / or compressing the spliced ​​messages before transmission; The parsing processing module is used for parsing the message header after the receiver receives the message, parsing the message body according to the delimiter and field sequence described in the message header, and restoring it into structured data.

9. An electronic device, characterized in that: The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; the memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor to enable the at least one processor to execute the JSON self-description-based data transmission method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, characterized in that The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to execute the JSON self-description-based data transmission method according to any one of claims 1 to 7.

Images