A data processing method, apparatus, electronic device, and storage medium

Abstract

This application relates to a block storage system, and discloses a data processing method, apparatus, electronic device, and storage medium. The method includes: obtaining the access frequency of a disk; when the access frequency of the disk is lower than a preset access frequency, stopping writing a first byte order from a first task queue to a first cache; writing the first byte order from the first task queue to a second cache, where the position of the first byte order in the first task queue is after the position of a second byte order, and the position of the first byte order in the first task queue is adjacent to the position of the second byte order, and the second byte order is the last byte order of the first cache; after all the byte orders stored in the first cache are written to the disk, writing the byte orders stored in the second cache to the disk. Implementing the embodiments of this application can improve the disk writing efficiency.

Description

Technical Field

[0001] This application relates to the field of blockchain technology, and in particular, to a data processing method, apparatus, electronic device, and storage medium. Background Art

[0002] Currently, in a blockchain system, blocks need to be stored on a disk. Generally, the blocks are first written into a cache and then written from the cache to the disk. In this process, there may be a problem of excessive disk utilization. In this case, since the disk flushes the blocks written from the cache all at once, this causes the disk utilization to continuously increase and the disk write efficiency to continuously decrease. Therefore, there is an urgent need for a solution to improve the disk write efficiency when the disk utilization is too high. Summary of the Invention

[0003] Embodiments of this application provide a data processing method, apparatus, electronic device, and storage medium. Implementing the embodiments of this application can improve the disk write efficiency.

[0004] The first aspect of this application provides a data processing method, including:

[0005] Obtaining the access frequency of the disk;

[0006] When the access frequency of the disk is lower than a preset access frequency, stopping writing the first byte order from the first task queue into the first cache;

[0007] Writing the first byte order from the first task queue into the second cache, where the position of the first byte order in the first task queue is after the position of the second byte order and adjacent to the position of the second byte order in the first task queue, and the second byte order is the last byte order of the first cache;

[0008] After all the byte orders stored in the first cache are written to the disk, writing the byte orders stored in the second cache to the disk.

[0009] The second aspect of this application provides a data processing apparatus, including an obtaining module and a processing module, where

[0010] The obtaining module is configured to obtain the access frequency of the disk;

[0011] The processing module is configured to stop writing the first byte order from the first task queue to the first cache when the access frequency of the disk is lower than a preset access frequency; write the first byte order from the first task queue to the second cache, where the position of the first byte order in the first task queue is after the position of the second byte order, and the position of the first byte order in the first task queue is adjacent to the position of the second byte order, and the second byte order is the last byte order of the first cache; after all the byte orders stored in the first cache are written to the disk, write the byte orders stored in the second cache to the disk.

[0012] A third aspect of the present application provides an electronic device for data processing, including a processor, a memory, a communication interface, and one or more programs, where the one or more programs are stored in the memory and are generated to be executed by the processor to execute the instructions for performing the steps in any one of the methods of a data processing method.

[0013] A fourth aspect of the present application provides a computer-readable storage medium for storing a computer program, and the stored computer program is executed by the processor to implement any one of the methods of a data processing method.

[0014] It can be seen that in the above technical solution, when the access frequency of the disk is lower than the preset access frequency, that is, when the disk utilization rate is too high, stop writing the first byte order from the first task queue to the first cache, avoiding the problem that the disk utilization rate increases again when refreshing the disk due to too many byte orders cached in the first cache. At the same time, by reducing the cache size of the first cache and increasing the second cache, when refreshing the disk, the byte orders stored in the first cache can be written to the disk first, and then the byte orders stored in the second cache can be written to the disk, thereby realizing reducing the amount of data refreshed from the cache to the disk at one time when the disk utilization rate is too high, and further improving the disk writing efficiency. Description of the Drawings

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0016] Among them:

[0017] Figure 1 It is a schematic diagram of a data processing system provided by an embodiment of the present application;

[0018] Figure 2Schematic flowchart of a data processing method provided by an embodiment of the present application;

[0019] Figure 3 Schematic diagram of the relationship between a first byte order and a second byte order provided by an embodiment of the present application;

[0020] Figure 4 Schematic flowchart of another data processing method provided by an embodiment of the present application;

[0021] Figure 5 Schematic diagram of a data processing device provided by an embodiment of the present application;

[0022] Figure 6 Schematic diagram of the structure of an electronic device in the hardware operating environment related to the embodiment of the present application. Detailed implementation manners

[0023] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0024] The following will be described in detail respectively.

[0025] The terms "first" and "second" in the specification and claims of the present application and the above accompanying drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products or devices.

[0026] First, refer to Figure 1 , Figure 1A schematic diagram of a data processing system provided by an embodiment of the present application. The data processing system 100 includes a data processing device 110. The data processing device 110 is used to process and store data blocks. The data processing system 100 may include an integrated single device or multiple devices. For the convenience of description, the data processing system 100 is collectively referred to as a blockchain node in the present application. Obviously, the blockchain node may include various handheld devices, vehicle-mounted devices, wearable devices, computing devices or other processing devices connected to a wireless modem with wireless communication functions, as well as various forms of user equipment (User Equipment, UE), mobile station (Mobile Station, MS), terminal device, and so on.

[0027] Further, in order to make the embodiments of the present application clearer, some concepts or contents in the embodiments of the present application are briefly introduced here.

[0028] A blockchain is a chain-like data structure that connects data blocks in chronological order and is a distributed ledger guaranteed by cryptography to be tamper-proof and unforgeable. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and so on.

[0029] Further, the characteristics of the blockchain include open, consensus, decentralized, trustless, transparent, two-party anonymity, tamper-proof, and traceable, etc. Among them, open and transparent mean that anyone can participate in the blockchain network, each device can act as a node, and each node is allowed to obtain a complete copy of the database. Nodes jointly maintain the entire blockchain through competitive computing based on a set of consensus mechanisms. If any node fails, the remaining nodes can still work normally. Among them, decentralized and trustless mean that the blockchain is composed of numerous nodes to form an end-to-end network, without a centralized device and management institution. Data exchange between nodes is processed through digital signature technology, and there is no need to trust each other. As long as it is carried out according to the established rules of the system, nodes cannot and will not deceive other nodes. Among them, transparent and two-party anonymity mean that the operating rules of the blockchain are public, and all data information is also public. Therefore, each transaction is visible to all nodes. Since there is no trust between nodes, there is no need to disclose identities between nodes, and each participating node is anonymous. Among them, tamper-proof and traceable mean that the modification of the database by each or even multiple nodes cannot affect the databases of other nodes, unless more than 51% of the nodes in the entire network can be controlled to modify simultaneously, which is almost impossible to happen. In the blockchain, each transaction is concatenated with the adjacent two blocks through cryptographic methods, so any transaction record can be traced.

[0030] Specifically, blockchain can use a block-chain data structure to process and store data, use a distributed node consensus algorithm to generate and update data, use cryptography to ensure the security of data transmission and access, and use smart contracts composed of automated script code to program and operate data, which is a brand-new distributed infrastructure and computing method. Therefore, the immutable characteristic of blockchain technology has fundamentally changed the way of creating centralized credit, effectively improving the immutability and security of data. Among them, because smart contracts make all terms written as programs, these terms can be automatically executed on the blockchain, ensuring that when there are conditions to trigger the smart contract, the blockchain can enforce the execution according to the content in the smart contract without being blocked by any external force, thus ensuring the effectiveness and enforceability of the contract, which can not only greatly reduce costs but also improve efficiency. Each node on the blockchain has the same ledger, which can ensure that the ledger recording process is open and transparent. Blockchain technology can achieve a peer-to-peer, open and transparent direct interaction, making a high-efficiency, large-scale, and non-centralized proxy information interaction method a reality.

[0031] In addition, when writing transaction data into the blockchain, it is necessary to process the transaction data. Only when the data processing passes can the transaction data be written into the blockchain. In the existing solutions, when processing transaction data, only single transaction data can be processed. For example, when processing transaction data 1, transaction data 2 cannot be processed. After all data processing of transaction data 1 is completed, transaction data 2 can be processed. Therefore, this data processing method has low efficiency.

[0032] Based on this, the embodiments of the present application propose a data processing method to solve the above problems, and the embodiments of the present application will be introduced in detail below.

[0033] See Figure 2 , Figure 2 which is a schematic flowchart of a data processing method provided by the embodiments of the present application. This data processing method can be applied to a blockchain node. As Figure 2 shown, the method includes:

[0034] 201. Obtain the access frequency of the disk.

[0035] 202. When the access frequency of the disk is lower than the preset access frequency, stop writing the first byte order from the first task queue into the first cache.

[0036] Among them, the remaining storage space of the first cache is greater than zero.

[0037] Among them, the first byte order is the binary sequence corresponding to the first block.

[0038] Among them, the preset access frequency can be set by the administrator or configured in the configuration file of the blockchain node, and there is no limitation here.

[0039] Among them, the first task queue is used to store multiple binary sequences corresponding to multiple blocks. Further, the multiple blocks include the first block.

[0040] 203. Write the first byte order from the first task queue to the second cache. The position of the first byte order in the first task queue is after the position of the second byte order, and the position of the first byte order in the first task queue is adjacent to the position of the second byte order. The second byte order is the last byte order of the first cache.

[0041] Among them, the multiple blocks further include a second block. Further, the second byte order is the binary sequence corresponding to the second block.

[0042] Among them, the storage space of the second cache is smaller than that of the first cache. Further, the difference between the storage space of the first cache and the storage space of the second cache is determined according to the access frequency of the disk.

[0043] For example, see Figure 3 , Figure 3 is a schematic diagram of the relationship between the first byte order and the second byte order provided by an embodiment of the present application. As Figure 3 shown, the first task queue includes the first byte order and the second byte order. The position of the first byte order in the first task queue is after the position of the second byte order, and the position of the first byte order in the first task queue is adjacent to the position of the second byte order. It can be understood that when reading the byte order from the first task queue, there is still remaining storage space in the first cache, that is, the blockchain node can write the second byte order into the first cache. Since, at this time, the access frequency of the disk is lower than the preset access frequency, stop writing the first byte order from the first task queue into the first cache. Further, the blockchain node can write the first byte order into the second cache. It should be noted that in the first cache, the second byte order is the last byte order of the first cache, and there is still remaining storage space in the first cache. In the second cache, the first byte order is the first byte order of the first cache. In addition, from Figure 3 it can be seen that the storage space of the first cache is larger than that of the second cache.

[0044] 204. After all the byte orders stored in the first cache are written to the disk, write the byte orders stored in the second cache to the disk.

[0045] It can be seen that in the above technical solution, when the access frequency of the disk is lower than the preset access frequency, that is, when the disk utilization rate is too high, writing the first byte order from the first task queue to the first cache is stopped, avoiding the problem that the disk utilization rate increases again when flushing the disk due to too many byte orders cached in the first cache. At the same time, by reducing the cache size of the first cache and increasing the second cache, when flushing the disk, the byte order stored in the first cache can be written to the disk first, and then the byte order stored in the second cache can be written to the disk, thereby realizing reducing the amount of data flushed from the cache to the disk at one time when the disk utilization rate is too high, and further improving the disk writing efficiency.

[0046] See Figure 4 , Figure 4 which is a schematic flowchart of another data processing method provided by an embodiment of the present application. This data processing method can be applied to a blockchain node, where, as Figure 4 shown, before stopping writing the first byte order from the first task queue to the first cache when the access frequency of the disk is lower than the preset access frequency, the method further includes:

[0047] 401. Obtain N blocks.

[0048] Wherein, N is an integer greater than 1.

[0049] 402. Serialize the N blocks in parallel on the serialization process to obtain N byte orders, and the N blocks correspond to the N byte orders one by one.

[0050] Wherein, each byte order in the N byte orders is a binary byte order.

[0051] 403. Store the N byte orders in the second task queue.

[0052] 404. Detect whether the sorting process is in a busy state.

[0053] If not, execute step 405.

[0054] In a possible implementation manner, the method further includes: if so, obtaining the N byte orders from the second task queue when the sorting process is in an idle state; obtaining the N block heights corresponding to the N blocks; sorting the N byte orders in ascending order of the N block heights on the sorting process to obtain the sorted N byte orders; storing the sorted N byte orders in the first task queue.

[0055] It can be seen that in the above technical solution, it is realized that when the sorting process is in an idle state, the sorted byte orders are stored in the task queue, preparing for subsequent fast writing to the disk.

[0056] 405. Obtain the N byte orders from the second task queue.

[0057] 406. Obtain the N block heights corresponding to the N blocks.

[0058] 407. Sort the N byte orders on the sorting process in ascending order of the N block heights to obtain the sorted N byte orders.

[0059] 408. Store the sorted N byte orders into the first task queue.

[0060] It can be seen that in the above technical solution, when the sorting process is busy, the sorted byte orders are stored into the task queue, preparing for subsequent fast writing to the disk.

[0061] In a possible implementation manner, after all the byte orders stored in the first cache are written to the disk, writing the byte orders stored in the second cache to the disk includes:

[0062] Obtain the length of the third task queue. The byte order at the head of the third task queue will be written to the disk. The third task queue is used to store the byte orders stored in the first cache;

[0063] Adjust the length of the fourth task queue according to the access frequency of the disk. The length of the fourth task queue is less than the length of the third task queue. The fourth task queue is used to store the byte orders stored in the second cache;

[0064] After all the byte orders stored in the third task queue are written to the disk, write the byte orders stored in the fourth task queue to the disk.

[0065] Among them, the length of the fourth task queue can be Among them, A is the access frequency of the disk, B is the preset access frequency, and C is the length of the third task queue.

[0066] It can be seen that in the above technical solution, by adjusting the queue length, when the disk utilization rate is too high, the amount of data refreshed from the cache to the disk at one time is reduced, thereby improving the disk writing efficiency.

[0067] In a possible implementation, the third task queue includes at least one task queue P, and each task queue in the at least one task queue P is used to store the byte order stored in the first cache; the at least one task queue P includes task queue P1, task queue P2, task queue P3, and task queue P4. The byte order at the head of task queue P1 is the first byte order read from the first cache, and the byte order at the tail of task queue P2 is the last byte order read from the first cache. The third byte order and the fourth byte order are adjacent in the first cache, and the third byte order is before the fourth byte order in the first cache. The byte order at the tail of task queue P3 is the third byte order, and the byte order at the head of task queue P4 is the fourth byte order. Before writing the byte order stored in the fourth task queue to the disk, the method further includes:

[0068] After writing all the byte orders stored in task queue P1 to the disk, write the byte orders stored in task queue P3 to the disk;

[0069] After writing all the byte orders stored in task queue P4 to the disk, write the byte orders stored in task queue P2 to the disk.

[0070] It can be seen that in the above technical solution, by reading the byte orders from multiple task queues and respectively flushing the read byte orders to the disk, the data volume flushed from the cache to the disk at one time is reduced when the disk utilization rate is too high, thereby improving the disk writing efficiency.

[0071] In a possible implementation, the fourth task queue may include at least one task queue Q, and each task queue Q in the at least one task queue Q is used to store the byte order stored in the second cache; the at least one task queue Q includes task queue Q1, task queue Q2, task queue Q3, and task queue Q4. The byte order at the head of task queue Q1 is the first byte order read from the second cache, and the byte order at the tail of task queue Q2 is the last byte order read from the second cache. The fifth byte order and the sixth byte order are adjacent in the second cache, and the fifth byte order is before the sixth byte order in the second cache. The byte order at the tail of task queue Q3 is the fifth byte order, and the byte order at the head of task queue Q4 is the sixth byte order. For writing the byte order stored in the second cache to the disk, the method includes:

[0072] After writing all the byte orders stored in task queue Q1 to the disk, write the byte orders stored in task queue Q3 to the disk;

[0073] After writing all the byte orders stored in the task queue Q4 to the disk, write the byte orders stored in the task queue Q2 to the disk.

[0074] It can be seen that in the above technical solution, by reading byte orders from multiple task queues and flushing the read byte orders to the disk respectively, the amount of data flushed from the cache to the disk at one time is reduced when the disk utilization rate is too high, thereby improving the disk writing efficiency.

[0075] See Figure 5 , Figure 5 which is a schematic diagram of a data processing device provided by an embodiment of the present application. Among them, as Figure 5 shown, a data processing device 500 provided by an embodiment of the present application includes an acquisition module 501 and a processing module 502, where

[0076] The acquisition module 501 is used to acquire the access frequency of the disk.

[0077] The processing module 502 is used to stop writing the first byte order from the first task queue to the first cache when the access frequency of the disk is lower than a preset access frequency; write the first byte order from the first task queue to the second cache, where the position of the first byte order in the first task queue is after the position of the second byte order and adjacent to the position of the second byte order in the first task queue, and the second byte order is the last byte order of the first cache; after all the byte orders stored in the first cache are written to the disk, write the byte orders stored in the second cache to the disk.

[0078] It can be seen that in the above technical solution, when the access frequency of the disk is lower than the preset access frequency, that is, when the disk utilization rate is too high, stop writing the first byte order from the first task queue to the first cache, avoiding the problem that the disk utilization rate increases again when flushing the disk due to too many byte orders cached in the first cache. At the same time, by reducing the cache size of the first cache and increasing the second cache, when flushing the disk, the byte orders stored in the first cache can be written to the disk first, and then the byte orders stored in the second cache can be written to the disk, thereby reducing the amount of data flushed from the cache to the disk at one time when the disk utilization rate is too high, and thus improving the disk writing efficiency.

[0079] In a possible implementation, when the access frequency of the disk is lower than a preset access frequency, before stopping writing the first byte order from the first task queue to the first cache, the obtaining module 501 is further configured to obtain N blocks, where N is an integer greater than 1; the processing module 502 is further configured to serially process the N blocks in parallel on a serialization process to obtain N byte orders, where the N blocks correspond to the N byte orders one by one; store the N byte orders in a second task queue; detect whether a sorting process is in a busy state; if not, obtain the N byte orders from the second task queue; obtain N block heights corresponding to the N blocks; sort the N byte orders in ascending order of the N block heights on the sorting process to obtain sorted N byte orders; and store the sorted N byte orders in the first task queue.

[0080] It can be seen that in the above technical solution, it is realized that when the sorting process is in a busy state, the sorted byte order is stored in the task queue to prepare for subsequent fast writing to the disk.

[0081] In a possible implementation, the obtaining module 501 is further configured to obtain N block heights corresponding to the N blocks; the processing module 502 is further configured to sort the N byte orders in ascending order of the N block heights on the sorting process to obtain sorted N byte orders; and store the sorted N byte orders in the first task queue.

[0082] It can be seen that in the above technical solution, it is realized that when the sorting process is in an idle state, the sorted byte order is stored in the task queue to prepare for subsequent fast writing to the disk.

[0083] In a possible implementation, after all the byte orders stored in the first cache are written to the disk, when writing the byte orders stored in the second cache to the disk, the obtaining module 501 is configured to obtain the length of a third task queue, where the byte order at the head of the third task queue will be written to the disk, and the third task queue is used to store the byte orders stored in the first cache; the processing module 502 is configured to adjust the length of a fourth task queue according to the access frequency of the disk, where the length of the fourth task queue is less than the length of the third task queue, and the fourth task queue is used to store the byte orders stored in the second cache; and after all the byte orders stored in the third task queue are written to the disk, write the byte orders stored in the fourth task queue to the disk.

[0084] It can be seen that in the above technical solution, by adjusting the queue length, it is realized that when the disk utilization rate is too high, the amount of data refreshed from the cache to the disk at one time is reduced, thereby improving the disk writing efficiency.

[0085] In a possible implementation, the third task queue includes at least one task queue P, and each task queue in the at least one task queue P is used to store the byte order stored in the first cache; the at least one task queue P includes task queue P1, task queue P2, task queue P3, and task queue P4. The byte order at the head of task queue P1 is the first byte order read from the first cache, and the byte order at the tail of task queue P2 is the last byte order read from the first cache. The third byte order and the fourth byte order are adjacent in the first cache, and in the first cache, the third byte order is before the fourth byte order. The byte order at the tail of task queue P3 is the third byte order, and the byte order at the head of task queue P4 is the fourth byte order. Before writing the byte order stored in the fourth task queue to the disk, the processing module 502 is further configured to write the byte order stored in task queue P3 to the disk after writing all the byte orders stored in task queue P1 to the disk; and write the byte order stored in task queue P2 to the disk after writing all the byte orders stored in task queue P4 to the disk.

[0086] It can be seen that in the above technical solution, by reading the byte order from multiple task queues and respectively flushing the read byte order to the disk, the amount of data flushed from the cache to the disk at one time is reduced when the disk utilization rate is too high, thereby improving the disk write efficiency.

[0087] In a possible implementation, the fourth task queue may include at least one task queue Q, and each task queue Q in the at least one task queue Q is used to store the byte order stored in the second cache; the at least one task queue Q includes task queue Q1, task queue Q2, task queue Q3, and task queue Q4. The byte order at the head of task queue Q1 is the first byte order read from the second cache, and the byte order at the tail of task queue Q2 is the last byte order read from the second cache. The fifth byte order and the sixth byte order are adjacent in the second cache, and in the second cache, the fifth byte order is before the sixth byte order. The byte order at the tail of task queue Q3 is the fifth byte order, and the byte order at the head of task queue Q4 is the sixth byte order. When writing the byte order stored in the second cache to the disk, the processing module 502 is configured to write the byte order stored in task queue Q3 to the disk after writing all the byte orders stored in task queue Q1 to the disk; and write the byte order stored in task queue Q2 to the disk after writing all the byte orders stored in task queue Q4 to the disk.

[0088] It can be seen that in the above technical solution, by reading the byte order from multiple task queues and respectively flushing the read byte order to the disk, the amount of data flushed from the cache to the disk at one time is reduced when the disk utilization rate is too high, thereby improving the disk writing efficiency.

[0089] See Figure 6 , Figure 6 which is a schematic structural diagram of an electronic device for the hardware operating environment related to the embodiments of the present application.

[0090] The embodiments of the present application provide an electronic device for data processing, including a processor, a memory, a communication interface, and one or more programs. Among them, the one or more programs are stored in the memory and are configured to be executed by the processor to execute instructions including the steps in any one of the data processing methods. Among them, as Figure 6 shown, the electronic device for the hardware operating environment related to the embodiments of the present application may include:

[0091] A processor 601, such as a CPU.

[0092] A memory 602. Optionally, the memory may be a high-speed RAM memory or a stable memory, such as a disk memory.

[0093] A communication interface 603, configured to implement connection communication between the processor 601 and the memory 602.

[0094] Those skilled in the art can understand that Figure 6 the structure of the electronic device shown in

[0095] does not constitute a limitation thereto, and it may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements. Figure 6 As

[0096] shown, the memory 602 may include an operating system, a network communication module, and one or more programs. The operating system is a program for managing and controlling the server hardware and software resources and supports the operation of one or more programs. The network communication module is used to implement communication between the components inside the memory 602 and communication between other hardware and software inside the electronic device. Figure 6 In the electronic device shown in

[0097] Obtain the access frequency of the disk;

[0098] When the access frequency of the disk is lower than the preset access frequency, stop writing the first byte order from the first task queue to the first cache;

[0099] Write the first byte order from the first task queue to the second cache. In the first task queue, the position of the first byte order is after the position of the second byte order, and the position of the first byte order is adjacent to the position of the second byte order in the first task queue. The second byte order is the last byte order of the first cache.

[0100] After writing all the byte orders stored in the first cache to the disk, write the byte orders stored in the second cache to the disk.

[0101] For the specific implementation of the electronic device involved in this application, refer to the embodiments of the above data processing method, which will not be elaborated here.

[0102] This application also provides a computer-readable storage medium for storing a computer program. The stored computer program is executed by the processor to implement the following steps:

[0103] Obtain the access frequency of the disk;

[0104] When the access frequency of the disk is lower than the preset access frequency, stop writing the first byte order from the first task queue to the first cache;

[0105] Write the first byte order from the first task queue to the second cache. In the first task queue, the position of the first byte order is after the position of the second byte order, and the position of the first byte order is adjacent to the position of the second byte order in the first task queue. The second byte order is the last byte order of the first cache.

[0106] After writing all the byte orders stored in the first cache to the disk, write the byte orders stored in the second cache to the disk.

[0107] For the specific implementation of the computer-readable storage medium involved in this application, refer to the embodiments of the above data processing method, which will not be elaborated here.

[0108] It should be noted that for the foregoing method embodiments, for simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should know that this application is not limited by the described action sequence, because according to this application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0109] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the various embodiments of the present application.

Claims

1. A data processing method, characterized in that, Including: Obtain the access frequency of the disk; When the access frequency of the disk is lower than the preset access frequency, stop writing the first byte order from the first task queue to the first cache; Write the first byte order from the first task queue to the second cache, where the position of the first byte order in the first task queue is after the position of the second byte order and adjacent to the position of the second byte order in the first task queue, and the second byte order is the last byte order in the first cache; After all the byte orders stored in the first cache are written to the disk, write the byte orders stored in the second cache to the disk; Among them, after all the byte orders stored in the first cache are written to the disk, writing the byte orders stored in the second cache to the disk includes: Obtain the length of the third task queue. The byte order at the head of the third task queue will be written to the disk, and the third task queue is used to store the byte orders stored in the first cache; Adjust the length of the fourth task queue according to the access frequency of the disk. The length of the fourth task queue is less than the length of the third task queue, and the fourth task queue is used to store the byte orders stored in the second cache; After all the byte orders stored in the third task queue are written to the disk, the byte orders stored in the fourth task queue are written to the disk, and the length of the fourth task queue is where A is the access frequency of the disk, B is the preset access frequency, and C is the length of the third task queue.

2. The method according to claim 1, characterized in that, Before stopping writing the first byte order from the first task queue to the first cache when the access frequency of the disk is lower than the preset access frequency, the method further includes: Obtain N blocks, where N is an integer greater than 1; Serialize the N blocks in parallel on the serialization process to obtain N byte orders, and the N blocks correspond to the N byte orders one by one; Store the N byte orders in the second task queue; Detect whether the sorting process is busy; If not, obtain the N byte orders from the second task queue; Obtain the N block heights corresponding to the N blocks; Sort the N byte orders in ascending order of the N block heights on the sorting process to obtain the sorted N byte orders; Store the sorted N byte orders in the first task queue.

3. The method according to claim 2, characterized in that, The method further includes: If so, obtain the N byte orders from the second task queue when the sorting process is idle; Obtain the N block heights corresponding to the N blocks; Sort the N byte orders in ascending order of the N block heights on the sorting process to obtain the sorted N byte orders; Store the sorted N byte orders in the first task queue.

4. The method according to claim 3, characterized in that, The third task queue includes at least one task queue P, and each task queue in the at least one task queue P is used to store the byte order stored in the first cache; the at least one task queue P includes task queue P1, task queue P2, task queue P3, and task queue P4. The byte order at the head of task queue P1 is the first byte order read from the first cache, and the byte order at the tail of task queue P2 is the last byte order read from the first cache. The third byte order and the fourth byte order are adjacent in the first cache, and the third byte order is before the fourth byte order in the first cache. The byte order at the tail of task queue P3 is the third byte order, and the byte order at the head of task queue P4 is the fourth byte order. Before writing the byte order stored in the fourth task queue to the disk, the method further includes: After writing all the byte orders stored in task queue P1 to the disk, write the byte orders stored in task queue P3 to the disk; After writing all the byte orders stored in task queue P4 to the disk, write the byte orders stored in task queue P2 to the disk.

5. The method according to claim 3, characterized in that, The fourth task queue may include at least one task queue Q, and each task queue Q in the at least one task queue Q is used to store the byte order stored in the second cache; the at least one task queue Q includes task queue Q1, task queue Q2, task queue Q3, and task queue Q4. The byte order at the head of task queue Q1 is the first byte order read from the second cache, and the byte order at the tail of task queue Q2 is the last byte order read from the second cache. The fifth byte order and the sixth byte order are adjacent in the second cache, and the fifth byte order is before the sixth byte order in the second cache. The byte order at the tail of task queue Q3 is the fifth byte order, and the byte order at the head of task queue Q4 is the sixth byte order. For writing the byte order stored in the second cache to the disk, the method includes: After writing all the byte orders stored in task queue Q1 to the disk, write the byte orders stored in task queue Q3 to the disk; After writing all the byte orders stored in task queue Q4 to the disk, write the byte orders stored in task queue Q2 to the disk.

6. A data processing device, the device is used to execute the method according to any one of claims 1-5, characterized in that, The device includes an acquisition module and a processing module, where, The acquisition module is used to acquire the access frequency of the disk; The processing module is configured to stop writing the first byte order from the first task queue to the first cache when the access frequency of the disk is lower than a preset access frequency; write the first byte order from the first task queue to the second cache, where the position of the first byte order in the first task queue is after the position of the second byte order and the position of the first byte order in the first task queue is adjacent to the position of the second byte order, and the second byte order is the last byte order of the first cache; after all the byte orders stored in the first cache are written to the disk, write the byte orders stored in the second cache to the disk.

7. The device according to claim 6, characterized in that,Before stopping writing the first byte order from the first task queue to the first cache when the access frequency of the disk is lower than a preset access frequency, The acquisition module is further configured to acquire N blocks, where N is an integer greater than 1; The processing module is further configured to serialize the N blocks in parallel on a serialization process to obtain N byte orders, where the N blocks correspond to the N byte orders one by one; store the N byte orders in a second task queue; Detect whether the sorting process is in a busy state; If not, acquire the N byte orders from the second task queue; acquire the N block heights corresponding to the N blocks; Sort the N byte orders in ascending order of the N block heights on the sorting process to obtain the sorted N byte orders; store the sorted N byte orders in the first task queue.

8. An electronic device for data processing, characterized in that, It includes a processor, a memory, a communication interface, and one or more programs, where the one or more programs are stored in the memory and are generated to be executed by the processor to execute the steps in the method according to any one of claims 1-5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program, and the stored computer program is executed by the processor to implement the method according to any one of claims 1-5.

Images