A data processing method and device

A data processing and data block technology, applied in the field of data processing, can solve the problems of persistent service processing pressure
CN106294477BActive Publication Date: 2019-10-01ALIBABA GRP HLDG LTD
4 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN Β· China
Patent Type
Patents(China)
Current Assignee / Owner
ALIBABA GRP HLDG LTD
Publication Date
2019-10-01

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

Disclosed are a data processing method and apparatus. A processing device reads data blocks from a read data block queue; generates confirmation objects that are in a one-to-one association relationship with the data blocks, the confirmation objects being used for recording confirmation states of the data blocks having the association relationship; and establishes a confirmation queue comprising N confirmation objects, and executes persistent operations on the read and confirmed data blocks according to the confirmation states of the confirmation objects in the confirmation queue. Accordingly, the processing device determines, according to the confirmation states of the confirmation objects in the confirmation queue and the arrangement order of the confirmation queue, a previous data block of a data block whose confirmation state is unconfirmed and that is early read by the processing device, and it is not required to perform persistent operations on each data block that is read and confirmed, thereby reducing the processing pressure of persistent services in the case in which the data read precision is ensured.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention relates to the field of data processing, in particular to a data processing method and device. Background technique

[0002] In Internet data processing, processing devices often need to read data blocks from a database or other data storage units, and process or confirm the read data blocks. The data to be read in a data storage unit such as a database is generally in the form of one or more data block queues, and the reading device sequentially reads and confirms data blocks from the one or more data block queues.

[0003] In the big data scenario, the data volume of the data block queue is very large, and the efficiency of serial reading or single-threaded reading of data blocks is too low. At present, reading devices generally adopt a method of reading data blocks in parallel to improve reading efficiency. The way of parallel reading will cause the order of reading data blocks from the data block queue to be different from the order...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More