Unlock instant, AI-driven research and patent intelligence for your innovation.

Batch data processing method, electronic device, computer equipment and storage medium

A processing method and batch data technology, which are applied in the field of computer equipment and storage media, electronic devices, and batch data processing methods, can solve problems such as large workload, invisible batch processing, invisible processing progress, and abnormal operation, and achieve saving Manpower, the effect of reducing the degree of coupling

Pending Publication Date: 2020-02-21
CHINA PING AN PROPERTY INSURANCE CO LTD
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, in the daily operation and maintenance process, the batch processing is invisible, including the processing progress, abnormal operation and other information, so that there is very little information for positioning, which brings a lot of workload to the operation and maintenance. Increased the work difficulty of operation and maintenance personnel

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Batch data processing method, electronic device, computer equipment and storage medium
  • Batch data processing method, electronic device, computer equipment and storage medium
  • Batch data processing method, electronic device, computer equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

example

[0102] Load 100 raw data files to be processed into the database temporary table, and record the number of raw data files loaded into the database temporary table to calculate the file loading progress, for example, when the number of raw data files loaded into the database temporary table B When it is 10, the completion progress of file loading is 10%, and the remaining 90% is not loaded.

[0103] Decompose the original data files loaded into the temporary table of the database into unit modules of different granularity. For the convenience of description, the following is an example of each original data file being decomposed into 4 unit modules. Among them, the first original data file is decomposed into Unit module a1, unit module b1, unit module c1 and unit module d1, the second raw data file is decomposed into unit module a2, unit module b2, unit module c2 and unit module d2, and so on.

[0104] The four grid nodes in the batch processor, grid node A, grid node B, grid n...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a batch data processing method, an electronic device, computer equipment and a storage medium. The batch data processing method comprises the following steps: decomposing an original data file loaded to a database temporary table into unit modules with different granularities according to data contents; sending the unit module to a grid node corresponding to a batch processor for processing according to the granularity of the unit module; monitoring the processing state of the unit module; and forming a target data file according to the unit modules processed by the grid nodes and outputting the target data file. According to the method, a to-be-processed original data file can be decomposed into unit modules with different granularities; the coupling degree betweenthe unit modules is reduced as much as possible; therefore, the grid nodes of the batch processor can simultaneously process the corresponding unit modules in parallel and monitor and display the processing state of each unit module, so that a user can visually know the processing state of the data and conveniently and quickly position the data with abnormal operation, the abnormal condition in batch processing can be quickly positioned and solved, and the manpower is saved.

Description

technical field [0001] The present invention relates to the technical field of batch data processing, in particular to a batch data processing method, electronic device, computer equipment and storage medium. Background technique [0002] With the increase of insurance business year by year, the core system generates more and more data, and there is a large amount of data that needs to be processed. At present, the processing and synchronization of large amounts of data are realized by batch processing. However, in the daily operation and maintenance process, the batch processing is invisible, including the processing progress, abnormal operation and other information, so that there is very little information for positioning, which brings a lot of workload to the operation and maintenance. Increased the work difficulty of operation and maintenance personnel. Contents of the invention [0003] In view of this, the present invention proposes a batch data processing method, ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F16/2455G06F9/38G06F11/32G06Q40/08
CPCG06F16/24568G06F16/24561G06F9/3885G06F11/324G06Q40/08
Inventor 陈志超
Owner CHINA PING AN PROPERTY INSURANCE CO LTD