A method and system for batch processing of data

A batch processing and data technology, applied in the field of data processing, can solve the problems of low hardware failure rate, high cost, and difficult price to serve the general public, and achieve the effect of reducing hardware cost and improving reliability

Active Publication Date: 2022-03-25
TENPAY PAID TECH
View PDF3 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] (1) In order to ensure that running batches can be completed on time and with high quality, and to achieve the reliability of financial-level data processing, it is necessary to execute batch running tasks on high-performance hardware, and exchange higher hardware costs for lower hardware failure rates. Take a computer composed of X86 architecture processor chip components as an example. The price is at the level of 100,000 RMB. The high hardware cost leads to high financial service prices, and it is difficult to serve the general public at an affordable price.
[0004] (2) Interest calculation services cannot be provided to users during the batch running period, so 7*24 real-time interest calculation services cannot be provided, which reduces user experience

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A method and system for batch processing of data
  • A method and system for batch processing of data
  • A method and system for batch processing of data

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0039] A data batching method such as figure 1 As shown, the method includes:

[0040] S101. Read data items for batch processing from the data center in sequence according to a preset sequence.

[0041] The data center can accept the read and write operations of data items by multiple batch processes at the same time, and record the processing results of the data items by marking the processing status of the data items.

[0042] S102. According to the processing status of the data entry, determine whether the data entry has been successfully processed.

[0043] S103. If not, process the data entry.

[0044] S104. If yes, skip the data entry and continue processing the next data entry. If all data items have been read, stop reading data items.

[0045] Specifically, in the process of multiple batch processing processes interacting with the data center, the data center does not perform read protection on data items used for batch processing, that is, all batch processing pr...

Embodiment 2

[0056] The data batch processing method provided in this embodiment is applied to the execution of financial batch running tasks. figure 2 As shown, the method includes:

[0057] S201. Read data items for batch processing from the data center in sequence according to a preset sequence.

[0058] The data center can accept the read and write operations of the data items by multiple batch processes at the same time, and record the processing results of the data items by marking the processing status of the data items. In this embodiment, the data entry is the relevant data used to calculate the interest of a certain user, and reading the data entry is the process of calculating the interest to obtain the preparation data, and the processing status is used to mark whether the user is running on the current day. Interest calculation has been successfully performed in the batch.

[0059] S202. Determine whether the interest on the day has been calculated.

[0060] S203. If not, ...

Embodiment 3

[0068] In the financial field, batch running tasks are usually performed within a fixed period of time every day, and real-time interest calculation cannot be provided for users during batch running. Experience, the data batch processing method provided in this embodiment can provide 7*24 hours full real-time interest calculation service for the financial field.

[0069] The data batch processing method provided in this embodiment is the same as that in Embodiment 2. In addition, a real-time interest calculation method is also provided. The real-time interest calculation is performed by a separate processing process, such as image 3 shown, including:

[0070] S301. A separate processing request for a certain user's interest is detected;

[0071] S302. Read the data entry for calculating the user interest;

[0072] S303. According to the processing state of the data entry, determine whether the data entry has been successfully processed:

[0073] S304. If not, calculate the...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The present invention proposes a data batch processing method and system, in which multiple batch processing processes preemptively process the data to be processed in the data center, and the failure of a certain batch processing process does not affect the completion of the batch processing task; and, During the process of batch processing, real-time data processing can be performed without the interference of batch processing by executing a separate processing process, so that full real-time data computing services can be provided. The present invention is especially suitable for batch running tasks of credit products, which can not only reduce hardware costs, but also achieve financial-level data processing reliability, and can also provide 7*24 hours of full real-time data services during the batch running process.

Description

technical field [0001] The invention relates to the field of data processing, in particular to a data batch processing method and system. Background technique [0002] For the calculation of the interest of credit products, the interest of the new day is generally calculated by running batches every morning, and the batch running is the calculation of the user's interest in batches. The traditional method of running batches to calculate interest has the following shortcomings: [0003] (1) In order to ensure that batch running can be completed on time and with high quality and achieve the reliability of financial-level data processing, batch running tasks need to be executed on high-performance hardware, and higher hardware costs are exchanged for lower hardware failure rates. For example, a computer composed of X86 architecture processor chip components is priced at the level of 100,000 yuan. The high hardware cost leads to a high price of financial services, and it is dif...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F9/50G06Q40/02
CPCG06F9/5027G06Q40/03
Inventor 郑伟涛郭懿心韦德志
Owner TENPAY PAID TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap