Massive file guiding-type pre-fetching parallel processing acceleration method based on assembly line framework
A massive file, parallel processing technology, applied in concurrent instruction execution, machine execution devices, etc., can solve problems such as speed mismatch, slow file access performance improvement, aggravated input/output access delay, etc., to improve processing efficiency and accuracy. Effects of Sexuality and Processing Efficiency
Inactive Publication Date: 2013-08-28
凯习(北京)信息科技有限公司
View PDF3 Cites 3 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0004] With the use of multi-core or many-core processors, the computing performance of processor hardware increases rapidly. At the same time, the performance of external storage systems increases much more slowly, resulting in slower improvement of file access performance.
Therefore, the speeds of computing, storage, and input / output (i.e., I / O) are increasingly mismatched, and the input / output access delay has become one of the most important bottlenecks restricting system performance.
Massive file processing requires frequent access to the storage system, which will aggravate the access delay of input / output
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0022] The present invention will be further described in detail below in conjunction with the accompanying drawings.
[0023] see figure 1 As shown, in the serial processing of massive files, a plane coordinate system is established with the processing time as the abscissa and the ordinate as dimensionless. When processing files, first process the first file f 1 , and then process the second file f 2 ,..., and finally process the ath file f a .
[0024] The first file f 1 Sequential reading phase first subcomputation stage second subcomputation stage ..., the final subcomputation stage output stage process the first file f 1 time recorded as
[0025] The second file f 2 Sequential reading phase first subcomputation stage second subcomputation stage ..., the final subcomputation stage output stage process the second file f 2 time recorded as
[0026] The ath file f a Sequential reading phase first subcomputation stage second subcomputation...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a massive file guiding-type pre-fetching parallel processing acceleration method based on an assembly line framework. The acceleration method comprises the steps of using a course for extracting pre-fetching files from an external storage into an internal storage buffer by an application program according to pre-fetching information of massive files; calculating the pre-fetching files in a staged mode by the course; outputting a parallel processing result by the course. According to the acceleration method based on the assembly line framework, guiding-type pre-fetching is adopted to process the massive files in advance, a calculating process of the massive files is divided into a plurality of stages, the massive files are processed in a parallel mode in the stages, and execution efficiency of a system is improved. During a data pre-fetching stage, the guiding-type pre-fetching is adopted, pre-fetching behaviors of the massive files are controlled under the application program so that the pre-fetching can be accurately pre-fetched in a foreseeing mode, and therefore pre-fetching efficiency and processing efficiency of the massive files are improved.
Description
technical field [0001] The invention relates to massive data processing in a computer, in particular to a massive file processing instruction type prefetch parallel processing acceleration method based on pipeline architecture. Background technique [0002] With the development of Internet technology and the rise of cloud computing, the demand for massive data processing is increasing, among which, the processing of massive files (such as web pages, pictures, videos, audios, etc.) is especially increasing. These processing steps are complicated, require frequent access to the file system and external storage system, and at the same time need to return the processing results quickly, which has extremely high requirements for high concurrency and low latency of the system. Therefore, accelerating the processing speed of massive files is of great significance for improving the performance of massive data processing. For the convenience of description, the massive files are ref...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More IPC IPC(8): G06F9/38
Inventor刘轶刘弢师明王庆全高飞
Owner凯习(北京)信息科技有限公司