Data file processing method and device and electronic equipment

A technology of data files and processing methods, applied in the field of big data, can solve the problems of high memory consumption, long import time, memory overflow, etc., and achieve the effects of improving system performance, improving import efficiency, and reducing import costs

Pending Publication Date: 2022-04-19
INDUSTRIAL AND COMMERCIAL BANK OF CHINA
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0002] In the prior art, the processing of data files, especially the processing of importing large data volume files into the system or server, has problems such as high memory consumption and frequent garbage collection by the system, such as concurrently importing large data volume Excel files into the system When it is in the middle, there will be a memory overflow problem, which will affect the system performance, resulting in a long import time, which cannot meet the needs of users for fast import

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data file processing method and device and electronic equipment
  • Data file processing method and device and electronic equipment
  • Data file processing method and device and electronic equipment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0053] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. It should be understood, however, that these descriptions are exemplary only, and are not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. Also, in the following description, descriptions of well-known structures and techniques are omitted to avoid unnecessarily obscuring the concept of the present disclosure.

[0054] The terminology used herein is for the purpose of describing particular embodiments only, and is not intended to be limiting of the present disclosure. The terms "comprising", "comprising", etc. used herein indicate the presence of stated features, ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a data file processing method which can be applied to the field of big data. The method comprises the following steps: in response to an import request, performing distributed storage on a data file corresponding to the import request; according to the attribute information of the data file, performing fragmentation definition on the data file by utilizing a scheduling server; according to the fragmentation definition and the configuration verification rule, utilizing a distributed collaborative service scheduling processing server to carry out fragmentation processing on the data file to obtain a fragmentation processing result; and verifying the fragmentation processing result by using the scheduling server, and performing fragmentation import processing on the data file by using the distributed collaborative service scheduling processing server according to the verification result. The invention further provides a data file processing device and equipment, a storage medium and a program product.

Description

technical field [0001] The present disclosure relates to the technical field of big data, in particular to the processing of big data files, and more specifically to a data file processing method, device, equipment, medium and program product. Background technique [0002] In the prior art, the processing of data files, especially the processing of importing large data volume files into the system or server, has problems such as high memory consumption and frequent garbage collection by the system, such as concurrently importing large data volume Excel files into the system When it is in the middle, there will be a memory overflow problem, which will affect the system performance, resulting in a long import time, which cannot meet the needs of users for fast import. Contents of the invention [0003] In view of the above problems, the present disclosure provides a data file processing method, device, device, medium, and program product that improve data file import efficie...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/182G06F16/16G06F9/48
CPCG06F16/182G06F16/16G06F9/4881
Inventor 谭均昌
Owner INDUSTRIAL AND COMMERCIAL BANK OF CHINA
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap