Unlock instant, AI-driven research and patent intelligence for your innovation.

Large file sorting method and system

A sorting method and large file technology, applied in special data processing applications, instruments, electrical digital data processing, etc., can solve problems affecting batch processing efficiency, restricting batch processing speed, long time, etc., to reduce CPU and memory overhead, Improve sorting speed, reduce size effect

Active Publication Date: 2016-03-30
BANK OF CHINA
View PDF2 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The sorting step takes a long time, which affects the efficiency of batch processing and becomes a key link that restricts the speed of batch processing

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Large file sorting method and system
  • Large file sorting method and system
  • Large file sorting method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0032] The present invention will be described in further detail below in conjunction with the accompanying drawings and specific embodiments, but not as a limitation of the present invention.

[0033] In this application document, a data record corresponds to a complete set of information in a data source (file), and is the basic unit for data processing. A large file refers to a data file that includes a large number of data records. Because it contains a large number of data records, generally hundreds of millions of data records are set at the level of one million or more, so the storage space it occupies is relatively large. big.

[0034] figure 1 It is a flowchart of a method for sorting large files according to an embodiment of the present invention. Such as figure 1 Shown, the large file sorting method of the present invention, comprises the steps:

[0035] (a) compression: compress each original data record included in the large file to generate corresponding new ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a large file sorting method. The method includes the steps of a, compressing each original data record comprised in a large file to obtain a corresponding new data record, and composing the new data records into a new file; b, sorting the new data records in the new file by the DFSORT tool; and c, decompressing each of the sorted new data records, and reducing into the original data records. The invention further discloses a large file sorting system used for sorting large files containing mass data records. The large file sorting method and system have the advantages that the size of the data records to be sorted and the size of the file composed of the data records are reduced by compression, CPU (central processing unit) and memory overheads are lowered, and sorting speed is higher.

Description

technical field [0001] The invention relates to a large file sorting method and system, which is mainly used to solve the performance problem of sorting massive sequential files (QSAM files) under the IBM mainframe environment and z / OS operating system. Background technique [0002] In the process of batch processing using the IBM mainframe (z / OS operating system) environment, in order to improve processing efficiency, the DFSORT tool is often used to sort a large number of sequential files. The DFSORT tool is a sorting tool software developed by IBM and is widely used in the industry. It can effectively improve the batch processing speed. However, when dealing with massive data, IBM's DFSORT tool is still insufficient, and the sorting processing time is too long, which cannot meet the application requirements. For example, the customer data of a medium-sized branch of a certain bank includes 13,227,038 customer data records, each data record is up to 2576 bytes, and the ti...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F17/30
Inventor 廖文胜徐慧斌
Owner BANK OF CHINA