A file upload method and device

A file upload and file technology, applied in the field of communication, can solve problems such as slow speed, memory explosion, failure and so on

Active Publication Date: 2020-03-17
TENCENT TECH (SHENZHEN) CO LTD
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0002] Object storage service usually means that the client uploads files to the storage server through the push request of the Hypertext transfer protocol (Hypertext transfer protocol, HTTP), and the client sends a push request to the Common Gateway Interface (CGI), and the body of the push request The CGI program caches the file content and then writes it to the storage. The design of the existing CGI framework usually needs to cache the complete file content in the push request body before forwarding the content to the CGI process. Processing, when the file is very large, the framework needs to cache the huge file content, which can easily lead to memory explosion; in addition, a CGI process can only process one request at a time, which can easily cause input and output (Input Output, IO) interface blocking , when the IO interface is blocked, the existing technology often solves it by brutally increasing the number of CGI processes, and the simple and crude solution of one CGI process after another will bring huge pressure to the server, and it is very difficult when the concurrency is high. It is easy to cause abnormal phenomena such as server stuck, slow speed or even failure, which seriously affects upload performance

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A file upload method and device
  • A file upload method and device
  • A file upload method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0023] This embodiment will describe the file upload method provided by the present invention from the perspective of the storage server, such as figure 2 As shown, the method of the present embodiment includes the following steps:

[0024] Step 201, receiving a file upload request sent by the client, where the file upload request includes complete file content;

[0025] When the client needs to upload a file, it sends a file upload request. The file upload request can be an HTTP push request, and the complete file content can be carried in the main body of the push request. The file upload device receives the file upload request sent by the client.

[0026] Step 202, parsing the file upload request, and storing the parsed data stream segment into the cache;

[0027] Step 203, when the size of the data stream segment stored in the cache reaches the preset threshold, the micro-thread in the scheduling process writes the data stream segment in the cache into the storage module...

Embodiment 2

[0057] The method described in Embodiment 1, this embodiment will give an example for further detailed description, such as image 3 As shown, the method of the present embodiment includes:

[0058] Step 301, receiving a file upload request sent by the client, where the file upload request includes complete file content;

[0059] When the client needs to upload a file, it sends a file upload request. The file upload request can be an HTTP push request, and the complete file content can be carried in the main body of the push request. The file upload device receives the file upload request sent by the client.

[0060] Step 302, parsing the file upload request and storing the parsed data stream segment into the cache;

[0061] Step 303, when the size of the data stream segment stored in the cache reaches a preset threshold, add a serial number to the data stream segment in the cache, and create a storage index;

[0062] The specific value of the preset threshold can be customi...

Embodiment 3

[0100] In order to better implement the above method, the embodiment of the present invention also provides a storage server, such as Figure 4 As shown, the storage server in this embodiment includes: a receiving unit 401, an analyzing unit 402 and a scheduling unit 403, as follows:

[0101] (1) receiving unit 401;

[0102] The receiving unit 401 is configured to receive a file upload request sent by a client, and the file upload request includes complete file content.

[0103] Specifically, the file upload request may be an HTTP push request, and the complete file content may be carried in the body of the push request.

[0104] (2) analysis unit 402;

[0105] A parsing unit 402, configured to parse the file upload request, and store the parsed data stream segments into the cache;

[0106] (3) scheduling unit 403;

[0107]The dispatching unit 403 is configured to, when the size of the data stream fragments stored in the cache reaches a preset threshold, the micro-thread i...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiment of the present invention discloses a file uploading method and device. The file uploading method includes: receiving a file uploading request sent by a client, the file uploading request includes complete file content; parsing the file uploading request, and analyzing to obtain Store the data flow fragments in the cache; when the size of the data flow fragments stored in the cache reaches the preset threshold, the micro-thread in the scheduling process writes the data flow fragments in the cache into the storage module, and repeat this step until All data stream segments of the complete file content are written into the storage module. The embodiments of the present invention can reduce cache pressure and improve upload performance.

Description

technical field [0001] Embodiments of the present invention relate to the field of communication technologies, and in particular to a file upload method and device. Background technique [0002] Object storage service usually means that the client uploads files to the storage server through the push request of the Hypertext transfer protocol (Hypertext transfer protocol, HTTP), and the client sends a push request to the Common Gateway Interface (CGI), and the body of the push request The CGI program caches the file content and then writes it to the storage. The design of the existing CGI framework usually needs to cache the complete file content in the push request body before forwarding the content to the CGI process. Processing, when the file is very large, the framework needs to cache the huge file content, which can easily lead to memory explosion; in addition, a CGI process can only process one request at a time, which can easily cause input and output (Input Output, IO...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): H04L29/08
CPCH04L67/06H04L67/568
Inventor 王海龙陈峥张力姜凤波王楠刘德刚
Owner TENCENT TECH (SHENZHEN) CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap