Data caching and sequencing on-line processing method based on cache pool
A data caching and processing method technology, applied in the Internet field, can solve the problems of out-of-order batch concurrent downloading of downloaded data, inability to realize caching, sorting and deduplication of downloaded data, inability to support real-time online identification, retrieval and other processing, etc., to achieve real-time good sex effect
Inactive Publication Date: 2012-11-21
HARBIN INST OF TECH +1
View PDF0 Cites 1 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0011] In order to solve the problem of downloading a large amount of multimedia program data from the Internet in a multi-user and multi-thread manner, the current common multi-thread download software cannot realize online caching of downloaded data based on memory, Sorting and deduplication, thus unable to support real-time online identification, retrieval and other processing problems, the present invention provides an online processing method for caching and sorting data based on a buffer pool
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
specific Embodiment approach 1
[0034] Specific implementation mode one: the following combination figure 1 Describe this embodiment. In this embodiment, a buffer pool is set in the memory of the computer system. Multiple buffers are set in the buffer pool. All buffers are organized into a linked list for centralized management, and a download log is created for each program to record the downloaded data of the program. offset range, and set the initial value of the life cycle of the program to k, k is a natural number, and k>0, the received program download data is based on buffer pool data cache sorting online processing, including the following steps:
[0035] Step 1, receiving the program data packet downloaded by the download module, the information contained in the program data packet has program ID, starting address, number of bytes, offset,
[0036] Step 2, querying the download log of the program, and judging whether the data in the program data packet has been downloaded,
[0037] If the judgment ...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a data caching and sequencing on-line processing method based on a cache pool, belonging to the field of internet and aiming to solve the problems that the common multithreading download software can not realize on-line caching, sequencing and de-emphasizing on the download data based on cache for download data disorder and repetitive program batch concurrent download so as to be incapable of supporting real-time on-line identification, retrieving and other operations. The invention comprises the following steps: organizing all cache areas into a cache area pool, and caching the same program download data by a plurality of cache areas; querying the program download log after receiving a program data packet, abandoning the data packet if no new data exists in the data packet, otherwise: selecting a proper program existing cache area to receive data, and applying for a new cache area to receive data if no proper cache area exists; updating the download log; and taking away the well-sequenced data in the cache area by calling an analysis processing module. The steps are repeated until the program download is finished, and the cache area distributed for the program is released to the cache area pool.
Description
technical field [0001] The invention relates to an online processing method for data caching and sorting based on a buffer pool, specifically including data caching, deduplication and sorting methods, and belongs to the field of the Internet. Background technique [0002] In recent years, with the rapid development of computer technology, network technology, communication technology and multimedia technology, the data volume of multimedia information has increased sharply. There are a large amount of multimedia data on the Internet that is difficult to estimate accurately. All acquired static multimedia data also has dynamic multimedia data transmitted in the network in the form of data stream. When analyzing and processing multimedia data in the Internet, it is necessary to download the multimedia data from the Internet first, and then analyze and process the multimedia data. Moreover, in the analysis and processing of multimedia data, it is usually necessary to have conti...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Patents(China)
IPC IPC(8): H04L12/56H04L29/06H04L12/861
Inventor 郑贵滨韩纪庆姜涛郑铁然陈训逊吴震周渊刘峰张冰
Owner HARBIN INST OF TECH