Text file processing method, device and system, electronic device, and storage medium

A text file and processing method technology, applied in file systems, multiprogramming devices, electronic digital data processing, etc., can solve the problems of file processing failure, low processing efficiency, data processing failure does not support retry, etc. Memory pressure, the effect of improving processing efficiency

Active Publication Date: 2019-02-12
JINGDONG TECH HLDG CO LTD
View PDF9 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] 1) Loading the entire file at one time in memory can easily lead to memory overflow
[0005] 2) If one piece of data in the file fails to be processed, the entire file fails to be processed
[0006] 3) Files are processed on a single machine, which strongly depends on the performance of the machine, and the processing efficiency is low
[0007] 4) Data processing failure does not support retry

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Text file processing method, device and system, electronic device, and storage medium
  • Text file processing method, device and system, electronic device, and storage medium
  • Text file processing method, device and system, electronic device, and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0043] Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0044] Furthermore, the drawings are merely schematic illustrations of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus repeated descriptions thereof will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be impleme...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a text file processing method, device and system, an electronic device and a storage medium. The text file processing method comprises the steps of: loading a plurality of pieces of data of a text file in batches; traversing the loaded plurality of pieces of data of the text file to add the plurality of pieces of data in a message queue, monitoring the message queue by a plurality of processing devices for the plurality of processing devices, and using the plurality of processing devices to consume and process the plurality of pieces of data from the message queue, respectively. The method and the device provided by the invention can reduce the pressure of the machine memory and improve the processing efficiency.

Description

technical field [0001] The present invention relates to the technical field of computer applications, in particular to a text file processing method, device, system, electronic equipment, and storage medium. Background technique [0002] It is a relatively common requirement for the system to process text files, and text files are also used as a data carrier to provide support for data transmission between systems. In the process of processing text files, various problems such as format errors, huge data volume, and complex file content business are often encountered. Processing files is often accompanied by business processing corresponding to the data in the files, and the complexity of the business varies. If only a single machine is used for processing, there will be serious performance problems. At present, most systems process text files by using a single server to obtain files through SFTP (Secure File Transfer Protocol, Secure File Transfer Protocol) and load the en...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/16G06F16/172G06F17/22G06F9/54
CPCG06F9/546G06F2209/548G06F40/194
Inventor 游刚
Owner JINGDONG TECH HLDG CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap