Data transmission method and apparatus, data storage method and apparatus, server and storage medium

A data transmission method and data transmission request technology, applied in the field of big data communication, can solve the problems of repeated data transmission, data loss, affecting the accuracy of data transmission, etc.

Active Publication Date: 2019-04-16
BEIJING KINGSOFT CLOUD NETWORK TECH CO LTD +1
View PDF8 Cites 9 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] In the existing data transmission method, the proxy server usually sends data one by one, and the server server also receives data one by one. When performing data transmission, especially

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data transmission method and apparatus, data storage method and apparatus, server and storage medium
  • Data transmission method and apparatus, data storage method and apparatus, server and storage medium
  • Data transmission method and apparatus, data storage method and apparatus, server and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0123] In view of this, if figure 1 As shown, the embodiment of the present invention provides a data transmission method applied to a proxy server, and the method may include the following steps:

[0124] S101. Obtain data to be transmitted.

[0125] In the embodiment of the present invention, the proxy server can obtain different data through different business services, that is, it can obtain data to be transmitted, for example, obtain log data through a log business service, or obtain user query data through a user interaction service. The specific format of the acquired data may be a log format, a file format and other data formats.

[0126] S102. If the sending condition of the data to be transmitted is satisfied, determine the acquired data to be transmitted as batch data, and set a unique batch identifier for the batch data.

[0127] It is easy to understand that during the operation of the proxy server, its data acquisition process is continuous. Therefore, a sendi...

Embodiment 2

[0140] Such as figure 2 As shown, the embodiment of the present invention also provides a data transmission method applied to a proxy server, and the method may include the following steps:

[0141] S201. Determine whether the total amount of currently sent data reaches a first preset threshold.

[0142] In order to prevent the avalanche of the server server (causing a crash due to too much received data), when the proxy server meets the preset conditions, the amount of data sent by the proxy server can be limited. For example, a first preset threshold is set for the total amount of data sent by the proxy server, so that the judgment is made based on the size relationship between the total amount of currently sent batch data and the first preset threshold. The above-mentioned first preset threshold can be set by the server operation and maintenance personnel according to the actual business situation, for example, set to 500GB, 1TB, 10TB, which will not be repeated in this e...

Embodiment 3

[0173] Such as Figure 4 As shown, the embodiment of the present invention provides a data storage method, which is applied to the server server, and the method may include the following steps:

[0174] S301. Receive a data transmission request from an agent server.

[0175] In the embodiment of the present invention, the server server can receive the data transmission request sent by the proxy server, and the data transmission request can carry a batch identifier and batch data, wherein the batch identifier is used to indicate the batch of the batch data , the batch data is formed by compressing at least one piece of data, and it can be 1 piece or 1000 pieces, which can be set by the server operation and maintenance personnel according to the actual business situation.

[0176] The received batch data has a unique batch identifier, so that after receiving a batch of data, the server server can identify the batch data through the batch identifier.

[0177] S302, judging whet...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiment of the invention provides a data transmission method and apparatus, a data storage method and apparatus, a server and a storage medium. The transmission method comprises the following steps: acquiring data to be transmitted, if sending conditions of the data to be transmitted are satisfied, determining the acquired data to be transmitted as batch data, and setting a unique batch identifier for the batch data; and sending a data transmission request. The storage method comprises the following steps: receiving a data transmission request; judging whether a stored identifier of a batch identifier exists locally; when the stored identifier of the batch identifier exists, determining that the batch data have been, wherein the data transmission request is a repeated transmission request; and when the stored identifier of the batch identifier does not exist, determining that the batch data are not stored; and storing the batch data, and adding the stored identifier for the batch identifier. By adoption of the data transmission method and the data storage method provided by the embodiment of the invention, the data transmission accuracy during large data size transmission can be improved.

Description

technical field [0001] The present invention relates to the technical field of big data communication, in particular to a data transmission method, a data storage method, a device, a server and a storage medium. Background technique [0002] The service nodes of large Internet service providers are usually distributed all over the country to provide business services. In order to optimize, guarantee or assist business services, service nodes distributed across the country need to communicate with back-end nodes and send business-related data to the back-end nodes, so that the back-end nodes can perform calculations on business-related data to achieve real-time Or offline analysis and statistics, service billing, etc. Specifically, the agent service Agent (agent server) deployed in each service node establishes a connection with the service server (server server) of the backend node through the network, so as to realize data transmission between the agent server and the serv...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): H04L29/08H04L29/06
CPCH04L63/0428H04L67/56H04L67/62H04L67/63H04L67/60
Inventor 关海南邹静
Owner BEIJING KINGSOFT CLOUD NETWORK TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap