Unlock instant, AI-driven research and patent intelligence for your innovation.

Data processing method, device, medium and electronic equipment

A data processing and data set technology, applied in the field of data processing, can solve problems such as rollback, difficulty in ensuring data consistency, and data impact

Active Publication Date: 2021-06-11
TAIKANG LIFE INSURANCE CO LTD
View PDF9 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In the batch data submission mechanism, the abnormality of a piece of data will cause the transaction rollback of the entire batch of data unit sets. If the abnormal data is swallowed, it will be difficult to ensure the consistency of the data, and it will affect the data submitted in batches by the entire queue, thereby improving Difficulty and cost of operation and maintenance

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method, device, medium and electronic equipment
  • Data processing method, device, medium and electronic equipment
  • Data processing method, device, medium and electronic equipment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0043] Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and fully convey the concept of example embodiments to those skilled in the art.

[0044] Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided in order to give a thorough understanding of embodiments of the invention. However, those skilled in the art will appreciate that the technical solutions of the present invention may be practiced without one or more of the specific details, or other methods, components, means, steps, etc. may be employed. In other instances, well-known methods, apparatus, implem...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The present invention provides a data processing method, comprising: obtaining the data set of each thread in the thread pool; processing the data set of each thread according to the business logic of each thread: when the processing result shows that the When there is data abnormality in the data set, the data set is split and processed according to the preset rules to obtain the abnormal data of the data set, and the abnormal data is marked; the processed data is processed according to the preset database operation type The data of the data set is added to the corresponding queue, so that the abnormal data of the data set can be accurately and quickly located, effectively avoiding the difficulty of ensuring the consistency of the data if the abnormal data is swallowed, and the entire queue will be submitted in batches The flaws that have an impact on the data. The invention also provides a data processing device, medium and electronic equipment.

Description

technical field [0001] The present invention relates to the technical field of data processing, in particular, to a data processing method, device, medium and electronic equipment. Background technique [0002] At present, the multi-thread processing framework can give full play to its parallel computing capabilities in the CPU-intensive data processing scenarios with a large amount of data, and improve the timeliness of data processing. At the same time, the batch submission mechanism for relational database storage within the thread can make full use of database resources. , reduce I / O waiting time, and complete data submission efficiently. According to the current test results, the batch data submission mode is at least 5 times more efficient than the one-by-one data processing. In high concurrency, I / O waiting for relational databases is one of the bottlenecks restricting database performance. Therefore, integrating the submission mode of batch data into the multi-threa...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/23
Inventor 杨旺明
Owner TAIKANG LIFE INSURANCE CO LTD