Method and system for simplifying real-time pushed data in weak network environment

Abstract

The invention discloses a method and system for simplifying real-time pushed data in a weak network environment, and the method comprises the steps: a server side executes data cache increment extraction to generate entity change information, and dynamically adjusts a cache and a push window according to the network quality; the front end and the rear end negotiate to complete data field on-demand filtering based on the white list; and carrying out data mapping compression operation on the filtered data packet, mapping a key value into a short identifier, and finishing code pushing. According to the method, through three-level collaborative simplification of entity increment-field filtering-dictionary compression, the problems of low data pushing efficiency and large redundancy in a weak network environment are solved, and the network environment adaptability and pushing efficiency are improved.

Description

Technical Field

[0001] This invention belongs to the field of computer network communication technology, and in particular relates to a simplified method and system for real-time data push in a weak network environment. Background Technology

[0002] In scenarios such as mobile internet, IoT, and edge computing, clients (e.g., mobile devices, embedded sites) are often in poor network conditions (limited bandwidth, high latency, unstable connections). Servers need to push data updates (such as status changes, message notifications, configuration updates, etc.) to these clients in real time. Traditional real-time push technologies simply and directly push updated data to the client, but this faces significant challenges in weak network environments.

[0003] 1. Redundant Data Transmission: Traditional methods often employ full push or simple incremental push based on timestamps. Full push sends complete data with every update, even if only a small number of fields change, resulting in significant redundancy. Incremental push based on timestamps may cause clients to miss updates or receive duplicate data due to network latency or out-of-order delivery, and it cannot accurately identify specific changes at the entity level (such as additions, updates, and deletions).

[0004] 2. Field Redundancy: The data structure pushed to the client often contains fields that the client does not currently need. For example, an IoT terminal that only cares about "temperature" receives a complete data packet containing irrelevant fields such as "humidity" and "air pressure," wasting valuable bandwidth.

[0005] 3. Low transmission efficiency: Lengthy field names and repetitive string values ​​(such as status descriptions "running" and "stopped") generated by data serialization (e.g., JSON, XML) significantly increase transmission load and latency in weak network environments. Although general compression algorithms (such as gzip) have some effect, their compression ratio depends on the degree of data redundancy, and compression / decompression itself consumes computational resources, which may be counterproductive under weak network and high latency conditions.

[0006] 4. Poor adaptability: Fixed push strategies (such as fixed intervals and fixed cache sizes) are difficult to adapt to dynamic changes in weak network environments (such as sudden drops in bandwidth and latency jitter), which can easily lead to data backlog, loss, or connection interruption.

[0007] These shortcomings result in low data transmission efficiency, increased latency, and decreased reliability for real-time push in weak network environments, leading to a poorer user experience and increased resource consumption (bandwidth, power, and computing power) for both the server and client. Summary of the Invention

[0008] Purpose of the Invention: The purpose of this invention is to provide a method and system for effectively simplifying real-time push data in weak network environments. Through incremental data caching, on-demand field filtering, and data mapping compression, the amount of data pushed is minimized while ensuring data real-time performance and integrity.

[0009] Technical solution: The simplified method for real-time data push in weak network environments includes:

[0010] On the server side, perform the following operations:

[0011] Perform incremental data cache extraction to generate entity change information to be pushed; wherein, the cache time window and / or push time window in the incremental data cache extraction are dynamically adjusted according to the monitored network quality of the client;

[0012] Based on the whitelist of field requirements declared by the client, the data fields of the entity change information are filtered as needed;

[0013] After data mapping and compression of the field key-value pairs in the filtered push data, it is pushed to the client.

[0014] Optionally, the incremental extraction of the data cache is based on the operation type and operation time in the entity update event.

[0015] Preferably, the incremental data cache extraction specifically includes:

[0016] Maintain a sliding cache for the primary key of entity data, storing the latest data and operation record list for each entity, wherein the operation record includes the operation type and the time of occurrence;

[0017] In response to an entity update event, update the entity data and operation record sequence of the corresponding entity;

[0018] Based on the cache time window, clean up the entities in the operation record list whose latest record's occurrence time is more than the cache time window from the current time;

[0019] When pushing data to a client, based on the client's last push time for a certain type of data, entities whose operation occurred later than that time are extracted from the cache, and the operation type finally pushed to the client is calculated based on the logical relationship of their operation record list.

[0020] Optionally, calculating the operation type finally pushed to the client based on the logical relationship of its operation record list includes:

[0021] If the operation type of the latest operation record is ADD / UPDATE, and there was a DELETE operation before it, then it will be pushed as UPDATE type;

[0022] If the operation type of the latest operation record is ADD / UPDATE, and there was an ADD operation before, then it will be pushed according to the ADD type;

[0023] If the operation type of the latest operation record is DELETE and there was an ADD operation before, then the entity data will not be pushed; if there was no ADD operation before, then it will be pushed as a DELETE.

[0024] Optionally, the dynamic adjustment based on the monitored network quality of the client includes:

[0025] A network quality score is calculated based on the monitored network latency, packet loss rate, and available bandwidth.

[0026] Based on the network quality score, the duration of the cache time window and / or the trigger interval for pushing to the client are dynamically adjusted.

[0027] Optionally, the on-demand filtering of data fields includes: before pushing, the server iterates through and matches data fields according to the whitelist of each client to generate a data object containing only the fields required by the client; wherein, the whitelist is sent to the server by the client through a control channel and supports dynamic updates.

[0028] Optionally, the data mapping compression includes:

[0029] The server maintains a shared dictionary independently for each client. The shared dictionary contains a mapping of field names and / or field values ​​to short identifiers. When pushing data, the field names and / or field values ​​are replaced with the corresponding short identifiers and serialized.

[0030] Preferably, the serialization method uses the CBOR encoding rule, which, as an efficient binary encoding, has the advantages of high compression ratio, strong self-description and high parsing efficiency, and can work with the field name / value replacement mechanism to achieve the goal of data simplification.

[0031] Optionally, the shared dictionary is dynamically updated based on sampling statistics of the pushed content, including:

[0032] During the data push process, the push content is sampled at random intervals, and the frequency of occurrence of each field name / value is recorded in each sample and stored in a loop counting window of length N;

[0033] When an existing field in the shared dictionary appears less frequently than the first threshold within the last K window periods, incremental synchronization is immediately performed with the client. After receiving confirmation from the client, the existing field is removed from the shared dictionary.

[0034] When a new field appears more frequently than the second threshold within M consecutive window periods, incremental synchronization is immediately performed with the client. After receiving confirmation from the client, the new identifier is added to the shared dictionary.

[0035] The parameters N, K, M, the first threshold, and the second threshold mentioned above are all configuration parameters and can be dynamically set according to the number of fields, network quality, and service performance in different business scenarios. This invention does not impose any rigid requirements on them.

[0036] The simplified system for real-time data push in a weak network environment includes a server, which includes:

[0037] The data cache incremental extraction module is used to generate entity change information through data cache incremental extraction; wherein, the cache time window and / or push time window in the data cache incremental extraction are dynamically adjusted according to the monitored network quality of the client;

[0038] The data field on-demand filtering module is used to filter the entity change information based on the whitelist of field requirements declared by the client.

[0039] The data mapping and compression module is used to perform data mapping and compression on the field key-value pairs in the filtered push data before pushing it to the client.

[0040] Optionally, the system also includes a client, which includes a whitelist declaration module. This module is used to declare a whitelist of field requirements and communicates with the server through a front-end and back-end negotiation method. It works with the data field on-demand filtering module to complete the field filtering of the entity change information.

[0041] Optionally, the client further includes a data mapping conversion module, which maintains a dictionary for key-value mapping shared with the server, the key-value mapping including mappings of field names and / or field values ​​to short identifiers; the data mapping conversion module is used to perform the following process after receiving a data packet sent by the data mapping compression module:

[0042] First, check the dictionary version number; if they are inconsistent, request a dictionary update.

[0043] The data packet is deserialized into a binary data stream according to the CBOR rule;

[0044] By reverse-lookup the dictionary, the identifier is restored to its original field name / value and reconstructed into a standard JSON object for use by the business layer.

[0045] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0046] 1. Data simplification, accuracy, and efficiency in weak network environments, balancing real-time performance and integrity: This invention constructs a three-level optimization mechanism of "incremental extraction + on-demand filtering + mapping compression," achieving data simplification across the entire transmission chain from source to delivery. Incremental extraction merges intermediate states based on the logical relationships of entity operation records, avoiding invalid transmission; on-demand filtering relies on a dynamically declared whitelist on the client side to accurately retain the required fields; and mapping compression, through a combination of a shared dictionary and CBOR encoding, significantly reduces the volume of key-value transmissions. Therefore, this invention's three-level collaborative mechanism minimizes the amount of data pushed while ensuring that data real-time performance and integrity remain unaffected.

[0047] 2. Outstanding network adaptive adjustment capabilities and significantly improved push stability: An innovative dynamic window adjustment strategy based on network quality quantification scoring is introduced. By monitoring network latency, packet loss rate, and available bandwidth to generate a comprehensive score, the cache time window and push trigger interval are flexibly adjusted. In poor network conditions, requests are merged to ensure data delivery; in good network conditions, rapid response improves timeliness, forming an adaptive closed loop. This effectively addresses bandwidth fluctuations and latency jitter in weak network environments, significantly improving the reliability and data delivery rate of real-time pushes.

[0048] 3. High flexibility in scenario adaptation, capable of dynamically responding to changes in business needs: The whitelist of client field requirements supports dynamic updates, and the shared dictionary achieves incremental synchronization based on push content sampling statistics, which can flexibly adapt to the dynamic adjustment of client business needs; the operation type merging rules for incremental extraction, network quality scoring dimensions, etc. all support parameter configuration, and can be personalized according to the characteristics of different weak network scenarios (such as mobile networks, remote area networks, etc.), with strong scenario scalability and business adaptability.

[0049] 4. Effectively reduces front-end and back-end resource consumption: On the one hand, the amount of data received by the client is greatly reduced, which reduces the processing load and power consumption of the network module; on-demand filtering eliminates the need for the client to process irrelevant data, saving memory and computing resources; the overhead of deserialization and dictionary lookup is far lower than parsing complete JSON / XML text; another invention is that the server's precise incremental push and field filtering reduce the occupation of network I / O and computing resources, and data mapping compression can ensure that a high compression ratio is achieved while avoiding the high CPU overhead brought by general compression algorithms. Attached Figure Description

[0050] Figure 1 This is a schematic diagram of incremental data cache extraction in one embodiment of the present invention;

[0051] Figure 2 This is a schematic diagram illustrating on-demand filtering of data fields in one embodiment of the present invention;

[0052] Figure 3This is a schematic diagram of data mapping compression in one embodiment of the present invention. Detailed Implementation

[0053] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0054] A simplified method for real-time data push in weak network environments includes performing the following operations on the server side:

[0055] Perform incremental data cache extraction to generate entity change information to be pushed; wherein, the cache time window and / or push time window in the incremental data cache extraction are dynamically adjusted according to the monitored network quality of the client;

[0056] Based on the whitelist of field requirements declared by the client, the data fields of the entity change information are filtered as needed;

[0057] After data mapping and compression of the field key-value pairs in the filtered push data, it is pushed to the client.

[0058] A streamlined system for real-time data push in a weak network environment includes a server, the server comprising:

[0059] The data cache incremental extraction module is used to generate entity change information through data cache incremental extraction; wherein, the cache time window and / or push time window in the data cache incremental extraction are dynamically adjusted according to the monitored network quality of the client;

[0060] The data field on-demand filtering module is used to filter the entity change information based on the whitelist of field requirements declared by the client.

[0061] The data mapping and compression module is used to perform data mapping and compression on the field key-value pairs in the filtered push data before pushing it to the client.

[0062] In some implementations, the system also includes a client, which includes a whitelist declaration module for declaring a whitelist of field requirements. This module negotiates with the server using a front-end / back-end negotiation approach and works with the data field on-demand filtering module to filter the fields of the entity change information.

[0063] Furthermore, the client also includes a data mapping and conversion module, which maintains a dictionary for key-value mapping shared with the server, the key-value mapping including mappings of field names and / or field values ​​to short identifiers.

[0064] The principle of this invention is as follows: precise incremental calculation based on primary keys and operation records, client-driven field-level on-demand filtering, and semantic compression based on a client-specific dynamic dictionary. These three simplification strategies work synergistically to achieve data compression and transmission optimization effects far exceeding those of a single technology in weak network environments.

[0065] Example 1

[0066] like Figure 1 As shown, the incremental data cache extraction includes:

[0067] (11) Entity data sliding cache

[0068] a. In the server-side program, allocate a buffer and record the updates received from the data provider based on the primary key of each data type.

[0069] In this embodiment, a hash table is used as the main storage data structure, with the entity ID (primary key) as the key and the value as a structure containing entity data and a list of operation records. Each operation record contains the operation type (ADD / UPDATE / DELETE) and the time when the operation occurred.

[0070] b. When an entity update event is received from the data provider (the message should specify the current operation type and the entity data containing the entity ID), the operation record list is automatically merged and the hash table is updated, according to the following rules:

[0071] The system searches the hash table based on the entity ID. If the entity does not exist, a new key-value pair is created. If the entity already exists, all previous consecutive UPDATE records, as well as all consecutive records of the same type as the current operation, are deleted.

[0072] For newly created or existing values, update their entity data and insert an operation record consisting of the current operation type and the current time into their operation record list.

[0073] c. Periodically clean up expired entity cache within the cache time window (default 5 minutes), that is, delete key-value pairs in the entity operation record list whose latest occurrence time is more than the window time from the current time.

[0074] (12) Incremental push to the client

[0075] a. Record the last push time of each client for each type of data in the server-side cache;

[0076] b. Each time a push is sent to a client, retrieve the last push time of a certain type of data, and retrieve only the values ​​of the latest operation record whose occurrence time is later than the last push time from the entity cache. After automatically calculating the operation type, push the entity update information to the client.

[0077] The operation type is calculated based on a list of operation records that are later than that time in the value:

[0078] When the operation type of the latest record is ADD / UPDATE, if there was a DELETE operation before, it will be pushed as UPDATE; otherwise, if there was an ADD operation before, it will be pushed as ADD; otherwise, it will be pushed as the latest operation type.

[0079] When the operation type of the latest record is DELETE, if there was an ADD operation before, the entity data will not be pushed; otherwise, it will be pushed as a DELETE.

[0080] c. After a successful push to the client, overwrite the client's last push time for this type of data with the push time, which is the time the value is retrieved from the entity cache.

[0081] (13) Dynamic window time adjustment

[0082] (131) Collect client network metrics every 30 seconds via heartbeat packets, including:

[0083] Latency: UDP probe packets are sent every 5 seconds, and the average RTT of the sliding window is calculated;

[0084] Packet loss rate: The rate of loss of 100 consecutive data packets is calculated using ICMP or QUIC protocols.

[0085] Bandwidth: Use TCP BBR or iperf for short-term testing, calibrating once per hour.

[0086] (132) Calculate the weighted comprehensive score of network quality:

[0087]

[0088] in, This represents the average RTT (ms) from the client to the server. This indicates the packet loss rate (%). Indicates available bandwidth (Mbps); It is a value between 0 and 1, representing the overall score, where 1 is the best.

[0089] (133) Introduce a network quality assessment system to conduct a comprehensive assessment of the network quality between different clients and servers in real time.

[0090] In this embodiment, the comprehensive evaluation is performed by dynamically adjusting the cache time window and push time window based on the network quality weighted comprehensive score, as shown in Table 1 below:

[0091] Table 1 Dynamic Window Time Adjustment Strategy

[0092] Network quality evaluation range Cache time window (s) Push time window (s) illustrate 0-0.3 300 60 Reduce the number of requests 0.3-0.6 200 30 Balanced mode 0.6-0.8 100 15 Optimize real-time performance 0.8-1 30 5 Pursuing low latency

[0093] Example 2

[0094] like Figure 2 As shown, the on-demand filtering of the data fields includes:

[0095] (21) Client whitelist declaration

[0096] a. For a type of real-time data, the client maintains a field whitelist, using regular expressions to describe the list of required fields, and sends the whitelist to the server when the connection is established.

[0097] The front-end (client-side) defines a whitelist of required data fields for each type, formatted as an array, where each element supports regular expressions, for example:

[0098] ["temperature", "operator.leader"," / (pressure|sensor_[0-9]+_value) / "]

[0099] b. The statement is sent to the server in real time via a lightweight control channel (such as a WebSocket control frame, an HTTP2 HEADERS frame, or a specially designed signaling message).

[0100] (22) Server-side parsing and filtering

[0101] a. The server maintains a whitelist declaration for each type of data corresponding to each client connection. When it receives an update to the client's declaration, the server parses and verifies its validity and updates the corresponding declaration status.

[0102] b. Before each data push, the server retrieves the whitelist declaration of the client for that type of data, iterates through the data fields, filters them using regular expression matching, and generates a data object that exactly meets the client's needs.

[0103] (23) Dynamic updates of the whitelist

[0104] a. When the current view changes or a user action occurs (such as switching views or expanding / collapsed panels), the client updates its maintained whitelist and immediately sends an incremental notification to the server. This ensures that the server subsequently pushes data that the client truly needs at the current stage, including data for deletion / addition of lists, for example:

[0105] {

[0106] CATEGORY: "XXX",

[0107] REMOVE:[

[0108] "operator.leader",

[0109] ],

[0110] INSERT:[

[0111] "volume", ]

[0113] }

[0114] b. When the server receives a whitelist update notification, it immediately parses and verifies its validity, updates the declaration status of the corresponding connection, and then matches and filters as needed in subsequent pushes.

[0115] Example 3

[0116] like Figure 3 As shown, the data mapping compression includes:

[0117] (31) Establishment of server-side shared dictionary

[0118] a. For all real-time data, the server maintains a shared dictionary of key-value mappings independently for each client and synchronizes it with the corresponding client periodically.

[0119] b. When establishing a connection, the server transmits the initial dictionary (containing predefined high-frequency fields) to the client via a lightweight binary protocol;

[0120] c. When updating the dictionary, only the changed entries (add / delete instructions + identifier - original value pair) are transmitted.

[0121] The shared dictionary is used to define short integer identifiers for frequently occurring field names and optional field values, including:

[0122] Field name mapping table: High-frequency field names (such as "temperature") are mapped to 2-byte short integer identifiers (such as 0x1001);

[0123] Field value mapping table (optional): Specific high-frequency values ​​(such as the "ACTIVE" status) are mapped to 1-byte identifiers (such as 0x1A).

[0124] (32) Server-side sampling update

[0125] a. During the process of pushing data from the server to a client, at random time intervals within a certain range [ , (e.g., 5-30 seconds) Perform multiple sampling statistics on the pushed content, record the frequency of occurrence of each field name / value in each sampling, and store it in a loop counting window of length N (N≥50).

[0126] b. When the frequency of an existing field (key / value) in the dictionary is lower than the first threshold (e.g., ≤3 times) within the last K window periods (K≥20), immediately perform incremental synchronization with the client, and remove the existing field (key / value) from the dictionary after receiving the client's confirmation receipt;

[0127] c. When the frequency of a new field in a consecutive M window period (M≥5) exceeds the second threshold (e.g., ≥15 times), immediately perform incremental synchronization with the client, and add the new identifier to the dictionary after receiving the client's confirmation receipt.

[0128] (33) Front-end and back-end mapping conversion

[0129] When a server pushes data to a client, for each key or optional value, it looks up the corresponding identifier in the client's dictionary. If found, it replaces the original identifier and encodes it using a binary serialization format. Upon receiving the pushed binary data, the client deserializes it according to the same format and, using the same dictionary as the server, converts the identifiers in the data back to the original content. Specifically, this includes:

[0130] a. Each time data is pushed, the server iterates through each key-value pair of the data to be pushed, queries the client's dictionary, and replaces the field name or value (optional) with an identifier;

[0131] b. The server uses CBOR (Concise Binary Object Representation) encoding rules to serialize the replaced data. Identifiers are encoded in uint16 type (fixed 2 bytes), and the original values ​​retain the original CBOR encoding rules (such as strings encoded in UTF-8), and a dictionary version number check header is added.

[0132] c. After receiving the data packet, the client first checks the dictionary version number. If they are inconsistent, it requests a dictionary update.

[0133] d. The client deserializes the data into a binary data stream according to the CBOR rule;

[0134] e. The client reverse-lookup of the local dictionary restores the identifier to its original field name / value, and reconstructs it into a standard JSON object for use by the business layer.

Claims

1. A method for reducing real-time push data in a weak network environment, characterized in that, This method includes performing the following operations on the server side: Perform incremental data cache extraction to generate entity change information to be pushed; wherein, the cache time window and / or push time window in the incremental data cache extraction are dynamically adjusted according to the monitored network quality of the client; Based on the whitelist of field requirements declared by the client, the data fields of the entity change information are filtered as needed; After data mapping and compression of the field key-value pairs in the filtered push data, it is pushed to the client.

2. The method for simplifying real-time data push in a weak network environment according to claim 1, characterized in that, The incremental data cache extraction includes: Maintain a sliding cache for the primary key of entity data, storing the latest data and operation record list for each entity, wherein the operation record includes the operation type and the time of occurrence; In response to an entity update event, update the entity data and operation record sequence of the corresponding entity; Based on the cache time window, clean up the entities in the operation record list whose latest record's occurrence time is more than the cache time window from the current time; When pushing data to a client, based on the client's last push time for a certain type of data, entities whose operations occurred later than that time are extracted from the cache, and the operation type is finally pushed to the client based on the logical relationship of their operation record list.

3. The method for simplifying real-time data push in a weak network environment according to claim 2, characterized in that, The calculation of the operation type ultimately pushed to the client based on the logical relationship of its operation record list includes: If the operation type of the latest operation record is ADD / UPDATE, and there was a DELETE operation before it, then it will be pushed as UPDATE type; If the operation type of the latest operation record is ADD / UPDATE, and there was an ADD operation before, then it will be pushed according to the ADD type; If the operation type of the latest operation record is DELETE and there was an ADD operation before, then the entity data will not be pushed; if there was no ADD operation before, then it will be pushed as a DELETE.

4. The method for simplifying real-time data push in a weak network environment according to claim 1, characterized in that, The dynamic adjustment based on the monitored network quality of the client includes: A network quality score is calculated based on the monitored network latency, packet loss rate, and available bandwidth. Based on the network quality score, the duration of the cache time window and / or the trigger interval for pushing to the client are dynamically adjusted.

5. The method for simplifying real-time data push in a weak network environment according to claim 1, characterized in that, The on-demand filtering of data fields includes: before pushing data, the server iterates through and matches data fields according to the whitelist of each client to generate a data object containing only the fields required by the client; wherein, the whitelist is sent to the server by the client through a control channel and supports dynamic updates.

6. The method for simplifying real-time data push in a weak network environment according to claim 1, characterized in that, The data mapping compression includes: The server maintains a shared dictionary independently for each client. The shared dictionary contains a mapping of field names and / or field values ​​to short identifiers. When pushing data, the field names and / or field values ​​are replaced with the corresponding short identifiers and serialized.

7. The method for simplifying real-time data push in a weak network environment according to claim 6, characterized in that, The shared dictionary is dynamically updated based on sampling statistics of the pushed content, including: During the data push process, the push content is sampled at random intervals, and the frequency of occurrence of each field name / value is recorded in each sample and stored in a loop counting window of length N; When an existing field in the shared dictionary appears less frequently than the first threshold within the last K window periods, incremental synchronization is immediately performed with the client. After receiving confirmation from the client, the existing field is removed from the shared dictionary. When a new field appears more frequently than the second threshold within M consecutive window periods, incremental synchronization is immediately performed with the client. After receiving confirmation from the client, the new identifier is added to the shared dictionary.

8. A simplified system for real-time data push in a weak network environment, characterized in that, The system includes a server, and the server includes: The data cache incremental extraction module is used to generate entity change information through data cache incremental extraction; wherein, the cache time window and / or push time window in the data cache incremental extraction are dynamically adjusted according to the monitored network quality of the client; The data field on-demand filtering module is used to filter the entity change information based on the whitelist of field requirements declared by the client. The data mapping and compression module is used to perform data mapping and compression on the field key-value pairs in the filtered push data before pushing it to the client.

9. The simplified system for real-time data push in a weak network environment according to claim 8, characterized in that, The system also includes a client, which includes a whitelist declaration module. This module is used to declare a whitelist of field requirements and communicates with the server through a front-end and back-end negotiation method. It works with the data field on-demand filtering module to filter the fields of the entity change information.

10. The simplified system for real-time data push in a weak network environment according to claim 8, characterized in that, The client further includes a data mapping conversion module, which maintains a dictionary for key-value mapping shared with the server, the key-value mapping including mappings of field names and / or field values ​​to short identifiers; the data mapping conversion module is used to perform the following process after receiving a data packet sent by the data mapping compression module: First, check the dictionary version number; if they are inconsistent, request a dictionary update. The data packet is deserialized into a binary data stream according to the CBOR rule; By reverse-lookup the dictionary, the identifier is restored to its original field name / value and reconstructed into a standard JSON object for use by the business layer.

Images