Real-time data pushing method for binary format transmission data

Abstract

The invention relates to the technical field of network communication, and provides a real-time data pushing method for binary format transmission data, and the method comprises the steps: S1, judging whether to start server pushing or not through a pushing decision engine; s2, when the server receives sensor data, converting the structured data into HTTP / 2 frames with priority marks through a binary encoder; s3, scheduling an HTTP / 2 frame transmission sequence through a flow controller according to a current network state, wherein the current network state at least comprises a data volume which can be currently received by the client; and S4, after receiving the HTTP / 2 frame, the client sends a preset confirmation frame to confirm receiving according to the HTTP / 2 frame serial number, and if the server does not receive the preset confirmation frame within the preset timeout time, the server marks that the HTTP / 2 frame is lost and re-uploads the HTTP / 2 frame marked as lost. Compared with WebSocket, the method has the advantages that the protocol overhead is reduced, the transmission efficiency is improved, the average delay of key data is reduced, and the method is compatible with the existing HTTP / 2 client.

Description

Technical Field

[0001] This application relates to the field of network communication technologies, and particularly relates to a real-time data push method for transmitting data in binary format. Background Art

[0002] The traditional HTTP / 2 protocol relies on plain text format transmission in data interaction (such as the redundant key-value pair structure of JSON and the nested tag system of XML). Its lack of binary encoding feature leads to the need to check syntax rules character by character during parsing, which easily causes wasted CPU computing power in high-frequency data streams. At the same time, the explicit field names and formatting symbols of the text significantly increase the data volume, having the defects of low parsing efficiency and large data redundancy. Real-time push usually relies on WebSocket or long polling, but it requires an additional connection to be established and has poor compatibility (such as being restricted by firewalls). Summary of the Invention

[0003] To help solve the above technical problems, this application provides a real-time data push method for transmitting data in HTTP / 2 binary format, adopting the following technical solutions: A real-time data push method for transmitting data in HTTP / 2 binary format, wherein the method adopts a real-time data push system for transmitting data in HTTP / 2 binary format. The system includes a binary encoder, a flow controller, a push decision engine, and a congestion awareness module. The method includes: Step S1: Determine whether to enable server push through the push decision engine; Step S2: When the server receives sensor data, convert JSON / XML structured data into an HTTP / 2 frame with a priority tag through the binary encoder; Step S3: Schedule the frame transmission order according to the current network status through the flow controller, where the current network status at least includes the amount of data that the client can currently receive; Step S4: After the client receives the frame, send an ACK frame for confirmation according to the frame sequence number. If the server does not receive the confirmation within the preset timeout period, mark the frame as lost and re-upload the frame marked as lost.

[0004] Preferably, step S2 includes: compressing the request header through the HPACK algorithm, and if the data exceeds the preset data size, transmitting it in fragments through the MTU.

[0005] Preferably, step S2 includes: maintaining a stream dependency tree subscribed by the client on the server side. In the stream dependency tree, the child node stream depends on the parent node stream. Set the priority weight through a preset priority field and associate the priority weight with the corresponding parent node of the stream dependency tree.

[0006] Preferably, step S3 includes: allocating data to an emergency queue and a normal queue according to the priority weights in step S2, obtaining the current network status through a preset flow control frame, and dynamically adjusting the frame size through a congestion awareness module.

[0007] Preferably, step S3 includes: if the client subscribes to relevant resources, before the server pushes data corresponding to the relevant resources, it sends a push promise frame to declare the resource identifier, and the client rejects redundant pushes through a stream abort frame.

[0008] In summary, the present application has the following beneficial effects: 1. Improved transmission efficiency: reducing protocol overhead compared to WebSocket; 2. Guaranteed real-time performance: reducing the average latency of critical data; 3. Compatibility: compatible with existing HTTP / 2 clients without modifying the browser / mobile SDK. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 It is a schematic block diagram of an embodiment of the real-time data push system for transmitting data based on the HTTP / 2 binary format of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0010] The present application will be further described below with reference to the accompanying drawings. The structure and principle of the present application are very clear to those skilled in the art. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0011] Figure 1 It is a schematic block diagram of an embodiment of the real-time data push system for transmitting data based on the HTTP / 2 binary format of the present application. When the client pushes data to the server, it involves the format of the data to be transmitted and the data transmission method. The real-time data push system for transmitting data based on the HTTP / 2 binary format of the present application includes a binary encoder, a flow controller, a push decision engine, and a congestion awareness module. The real-time data push method for transmitting data based on the HTTP / 2 binary format of the present application includes: Step S1: Determine whether to enable server push through the push decision engine. The push decision engine may make a judgment by considering multiple factors, such as the client's request history, user behavior patterns, current network environment, and business requirements. By analyzing and evaluating these factors, the engine can intelligently make a decision on whether to push data. This avoids data pushing in unnecessary situations and reduces resource waste on the server side, such as computing resources and bandwidth resources.

[0012] Step S2: When the server receives sensor data, it converts the structured data into an HTTP / 2 frame with a priority tag through a binary encoder. Step S2 includes: compressing the request headers through the HPACK algorithm; if the data exceeds the preset data size, transmitting it in fragments through the MTU. The server maintains a stream dependency tree subscribed by the client. In the stream dependency tree, the child node stream depends on the parent node stream. The priority weight is set through a preset priority field, and the priority weight is associated with the corresponding parent node of the stream dependency tree. Here, it should be noted that the sensor refers to a software device used to collect financial industry data. The collected data will be transmitted to the server and then pushed to the client.

[0013] Binary encoding converts structured data into a binary format, which can more efficiently utilize network bandwidth, reduce data redundancy, and facilitate subsequent transmission and processing. The HPACK algorithm reduces the amount of data transmitted and improves transmission efficiency by establishing and maintaining a dynamic dictionary of request headers and performing compressed encoding on repeatedly occurring request header fields. Network transmission is limited by the maximum transmission unit. When the data exceeds this limit, it is split into multiple small data packets for transmission to ensure that the data can be successfully transmitted over the network. Stream dependency tree and priority: By constructing a stream dependency tree and setting priority weights, the order of data transmission can be reasonably arranged according to the importance and urgency of the data, ensuring that critical data is transmitted first.

[0014] Binary encoding and HPACK compression reduce the amount of data transmitted, and MTU fragmentation ensures the smooth transmission of data over the network, overall improving the efficiency of data transmission. Through the stream dependency tree and priority mechanism, important and urgent data can be transmitted first, ensuring that critical financial industry data reaches the client in a timely manner to meet business requirements. MTU fragmentation transmission can adapt to the MTU limitations of different network environments and ensure reliable data transmission under different network conditions.

[0015] Step S3: The stream controller schedules the frame transmission order according to the current network state, where the current network state at least includes the amount of data that the client can currently receive. Step S3 includes: allocating the data to the emergency queue and the normal queue according to the priority weight in Step S2, obtaining the current network state through a preset flow control frame, and dynamically adjusting the frame size through a congestion awareness module. If the client subscribes to relevant resources, before the server pushes the data corresponding to the relevant resources, it sends a push promise frame to declare the resource identifier, and the client rejects redundant pushes through a stream abort frame.

[0016] This step can dynamically adjust the frame transmission order and frame size according to the current network status, effectively coping with network congestion and bandwidth changes, and ensuring the stability and reliability of data transmission. Through reasonable queue management and traffic control, the network bandwidth resources are fully utilized to improve the data transmission efficiency. The push promise frame and stream abort frame mechanisms avoid pushing redundant data to the client, reducing the waste of network bandwidth and also reducing the processing burden on the client.

[0017] Step S4: After the client receives the HTTP / 2 frame, it sends a preset acknowledgment frame to confirm the reception according to the frame sequence number. If the server does not receive the preset acknowledgment frame within the preset timeout period, it marks the HTTP / 2 frame as lost and re-uploads the frame marked as lost. The ACK frame (Acknowledgment Frame) is the core mechanism in the network communication protocol for confirming the successful reception of data. Its core role is to achieve reliable transmission through explicit feedback and avoid data loss or duplication.

[0018] The client feeds back the data reception situation to the server by sending an acknowledgment frame. The server determines whether the data is successfully transmitted based on whether it receives the acknowledgment frame. The server sets a preset timeout period. If it does not receive the acknowledgment frame within the timeout period, it considers the data lost and triggers the retransmission mechanism to re-upload the lost data frame.

[0019] Next, each step will be described in detail: 1. Client capability negotiation and push initialization When the client establishes an HTTP / 2 connection, it declares the server push capabilities it supports by sending a SETTINGS frame. For example, setting PUSH_ENABLED = 1 indicates that pushing is allowed, and the frame types to be received can be specified. This step corresponds to the push decision engine and realizes the negotiation between the client and the server through the CAPABILITY_NEGOTIATE frame extension. The client carries the Subscribe pseudo-header field (such as Subscribe: / market / stock / *) in the HEADERS frame to declare the subscription demand for financial market data. The server records the client's subscription topic, constructs a Stream Dependency Tree, marks the high-priority stream (such as real-time market quotes) as the parent node, and the low-priority streams such as logs as the child nodes.

[0020] 2. Data encoding and priority marking When the server receives sensor data, the binary encoder converts JSON / XML structured data into HTTP / 2 frames. The HPACK algorithm is used to compress the request headers, and fields such as Content-Type: application / json are compressed into static table indexes to reduce the frame size. If the data exceeds 16KB (the preset data size), it is fragmented according to the MTU, and each fragment is set with END_STREAM=0, and the last fragment is set with END_STREAM=1 to identify the message boundary. The priority weight is set through the Priority field (preset priority field) of the HEADERS frame (such as financial market weight 255, log weight 1), and is associated with the corresponding parent node of the StreamDependency Tree.

[0021] 3. Stream Scheduling and Congestion Control The server stream controller schedules the frame transmission order according to the current network status. Priority queue: High-priority streams (such as market data) enter the emergency queue, and low-priority streams (such as logs) enter the normal queue. The TCP window size and RTT are monitored through the WINDOW_UPDATE frame (preset flow control frame), and the congestion awareness module is called to dynamically adjust the frame size (such as from 16KB to 8KB) to avoid network congestion. If the client subscribes to relevant resources, before pushing the sub-stream, the server first sends a PUSH_PROMISE (push promise frame) to declare the resource identifier (such as Promise-ID: 1001), and the client can reject redundant pushes through the RST_STREAM (stream abort frame).

[0022] 4. Client Acknowledgment and Fault Tolerant Retransmission After receiving the frame, the client sends an ACK frame for confirmation according to the frame sequence number. If the server does not receive the confirmation within the preset timeout (such as 500ms), the frame is marked as lost. Loss detection: The server detects lost frames through the sliding window protocol. The original frame data is directly retransmitted through the memory mapping technology to avoid the overhead of data copying. Emergency data (such as market snapshots) preempts bandwidth through frame-level QoS to ensure that the latency of low-priority streams (such as logs) does not affect critical services.

Claims

1. A real-time data push method for transmitting data based on the HTTP / 2 binary format, characterized in that, The method uses a real-time data push system that transmits data based on the HTTP / 2 binary format. The system includes a binary encoder, a stream controller, a push decision engine, and a congestion awareness module. The method includes: Step S1: Determine whether to enable server push through the push decision engine; Step S2: When the server receives sensor data, convert the structured data into an HTTP / 2 frame with a priority tag through the binary encoder; Step S3: Schedule the transmission order of the HTTP / 2 frames according to the current network status through the stream controller. The current network status at least includes the amount of data that the client can currently receive; Step S4: After the client receives the HTTP / 2 frame, send a preset acknowledgment frame to confirm the reception based on the HTTP / 2 frame sequence number. If the server does not receive the preset acknowledgment frame within the preset timeout period, mark the HTTP / 2 frame as lost and re-upload the HTTP / 2 frame marked as lost.

2. The real-time data push method for transmitting data based on the HTTP / 2 binary format according to claim 1, wherein, Step S2 includes: Compressing the request header through the HPACK algorithm. If the data exceeds the preset data size, transmit it in fragments through the MTU.

3. The real-time data push method for transmitting data based on the HTTP / 2 binary format according to claim 1, characterized in that Step S2 includes: Maintaining a stream dependency tree subscribed by the client on the server side. In the stream dependency tree, the child node stream depends on the parent node stream. Set the priority weight through a preset priority field and associate the priority weight with the corresponding parent node of the stream dependency tree.

4. The real-time data push method for transmitting data based on the HTTP / 2 binary format according to claim 1, characterized in that Step S3 includes: Allocating data to the emergency queue and the normal queue according to the priority weight in Step S2, obtaining the current network status through a preset flow control frame, and dynamically adjusting the size of the HTTP / 2 frame through the congestion awareness module.

5. The real-time data push method for transmitting data based on the HTTP / 2 binary format according to claim 1, characterized in that Step S3 includes: If the client subscribes to relevant resources, the server sends a push promise frame to declare the resource identifier before pushing the data corresponding to the relevant resources. The client rejects redundant pushes through a stream abort frame.

Images