Batch running method, device and equipment for single-thread data reading and distribution and storage medium

A data reading, single-threaded technology, applied in the field of artificial intelligence, can solve the problem of low efficiency of file running and batching, achieve the effect of shortening the reading time, reducing the number of intervals to be sorted, and alleviating waiting anxiety

Pending Publication Date: 2022-08-09
PINGAN PUHUI ENTERPRISE MANAGEMENT CO LTD
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The present invention provides a batch running method, device, equipment and storage medium for reading and distributing single-threaded data, the main purpose of which is to solve the problem of low efficiency of file batch running

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Batch running method, device and equipment for single-thread data reading and distribution and storage medium
  • Batch running method, device and equipment for single-thread data reading and distribution and storage medium
  • Batch running method, device and equipment for single-thread data reading and distribution and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0059] It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

[0060] The embodiment of the present application provides a batch running method for reading and distributing data in a single thread. The execution subject of the batch-running method for single-threaded data reading and distribution includes, but is not limited to, at least one of devices that can be configured to execute the method provided by the embodiments of the present application, such as a server and a terminal. In other words, the batch-running method of the single-threaded data reading and distribution can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like. The server can be an independe...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to an artificial intelligence technology, and discloses a batch running method for reading and distributing single-thread data, which comprises the following steps of: configuring a batch reading size, initializing a data center, and displaying a data set which is successfully subjected to batch running processing; initializing a reading thread and a working thread pool; reading the data rows through the reading thread until the large and small data rows are read in batches or the source file is completely read, and judging whether the read data rows are in the data set or not; when the read data row is in the data set, skipping the data row; otherwise, transmitting the data row to the working thread pool; and batch processing is carried out on the data in the working thread pool, and the line number interval of the data is synchronized to a data center. In addition, the invention also relates to a block chain technology, and the source file to be batch can be stored in the node of the block chain. The invention further provides a batch running device and equipment for single-thread data reading and distribution and a storage medium. According to the invention, the file batch running efficiency can be improved.

Description

technical field [0001] The invention relates to the technical field of artificial intelligence, and in particular, to a batch-running method, device, device and storage medium for reading and distributing single-threaded data. Background technique [0002] With the digitization of people's lives, more and more enterprises have built supporting large-scale systems. However, in order to provide customers with a better user experience and improve the processing efficiency of the system, it is necessary to build a corresponding system data reading scheme for file running. batch. [0003] The existing batch running management mostly uses the SpringBacth framework to manage file reading, and then run batches of files. For example, for reading a large file, it is necessary to divide the large file into several small files, and average the small files. It is allocated to several threads for reading. In practical applications, large files need to be read in advance before they can b...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F9/50G06F9/451
CPCG06F9/5038G06F9/451G06F2209/5011G06F2209/5018Y02D10/00
Inventor 刘杰
Owner PINGAN PUHUI ENTERPRISE MANAGEMENT CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap