Data writing method and device, electronic equipment and storage medium

By splitting the data into sub-data and determining the write thread based on the physical space block index number, the competition problem between multiple write commands in an all-flash array is solved, and data writing performance is improved.

CN119002793APending Publication Date: 2024-11-22ZHEJIANG UNIVIEW TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310571080.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-17
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

In an all-flash array, multiple I/O write commands compete for the mapping relationship table of the data fingerprint hash to the physical spatial address, resulting in a degradation of the system I/O write performance.

Method used

Split the data to be written into sub-data, and determine the corresponding data writing thread based on its physical space block index number, and write it to the corresponding idle fingerprint node through the target thread to avoid competition from multiple threads on the mapping relationship table. .

Benefits of technology

It improves the system's data writing performance, avoids competition for metadata resources between multiple data writing threads, and improves I/O writing performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119002793A_ABST
    Figure CN119002793A_ABST
Patent Text Reader

Abstract

The invention discloses a data writing method and device, electronic equipment and a storage medium. The method comprises the following steps: determining at least two pieces of to-be-written sub-data according to to-be-written data, and determining a data writing thread identifier corresponding to each piece of to-be-written sub-data according to physical space block index number information of each piece of to-be-written sub-data; determining at least one fingerprint node block under a current target fingerprint node block management unit identifier through a target fingerprint node block management unit identifier corresponding to the target data write-in thread identifier, and determining a target idle fingerprint node according to the fingerprint node block; and writing the target to-be-written sub-data into the target idle fingerprint node through a data writing thread corresponding to the target data writing thread identifier. By means of the technical scheme, it can be avoided that a plurality of data writing threads compete for metadata resources, and the data writing performance of a system is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data writing method, device, electronic device and storage medium. Background Art

[0002] An all-flash array is an independent storage array or device composed of solid-state storage media. All-flash arrays mostly use ROW (Redirect On Write) writing mode. The physical space address is no longer related to the logical address, but there is a mapping relationship between the logical address and the data fingerprint hash, and between the data fingerprint hash and the physical space address. When processing I / O (Input / Output) write commands, the mapping relationship between the logical address and the data fingerprint hash and the mapping relationship between the data fingerprint hash and the physical space address need to be updated.

[0003] When multiple I / O write commands are processed simultaneously in an all-flash array, the multiple I / O write commands compete with each other for the mapping table from data fingerprint hash to physical space address. After a write fails, a rollback operation needs to be performed on the mapping table. The thread that performs the rollback operation also competes with other I / O write threads for the mapping table from data fingerprint hash to physical space address, which greatly reduces the system's I / O write performance. Summary of the invention

[0004] The present invention provides a data writing method, device, electronic device and storage medium, so as to avoid competition for metadata resources among multiple data writing threads and improve the data writing performance of the system.

[0005] In a first aspect, an embodiment of the present invention provides a data writing method, the method comprising:

[0006] Determine at least two sub-data to be written according to the data to be written, and determine the data writing thread identifier corresponding to each sub-data to be written according to the physical space block index number information of each sub-data to be written;

[0007] Determine at least one fingerprint node block under the current target fingerprint node block management unit identifier through the target fingerprint node block management unit identifier corresponding to the target data writing thread identifier, and determine the target idle fingerprint node according to the fingerprint node block;

[0008] The target sub-data to be written is written to the target idle fingerprint node through a data writing thread corresponding to the target data writing thread identifier.

[0009] In a second aspect, an embodiment of the present invention further provides a data writing device, the device comprising:

[0010] A data writing thread identifier determination module is used to determine at least two sub-data to be written according to the data to be written, and determine the data writing thread identifier corresponding to each sub-data to be written according to the physical space block index number information of each sub-data to be written;

[0011] A target idle fingerprint node determination module is used to determine at least one fingerprint node block under the current target fingerprint node block management unit identifier through the target fingerprint node block management unit identifier corresponding to the target data writing thread identifier, and determine the target idle fingerprint node according to the fingerprint node block;

[0012] The data writing module is used to write the target sub-data to be written into the target idle fingerprint node through the data writing thread corresponding to the target data writing thread identifier.

[0013] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a data writing method as described in any one of the embodiments of the present invention is implemented.

[0014] In a fourth aspect, an embodiment of the present invention further provides a storage medium storing computer executable instructions, wherein the computer executable instructions, when executed by a computer processor, are used to execute the data writing method as described in any one of the embodiments of the present invention.

[0015] The technical solution of the embodiment of the present invention is to split the data to be written into sub-data to be written, and respectively determine the physical space index number of each sub-data to be written, and determine the data writing thread matching each sub-data to be written according to the physical space index number of each sub-data to be written, and determine the target fingerprint node block management unit, the target idle fingerprint node block and the target idle fingerprint node through the target data writing thread matching the target sub-data to be written, and write the target sub-data to be written to the target idle fingerprint node. The problem of the mapping relationship table of data fingerprint hash to physical space address competition caused by multiple data writing threads simultaneously writing data in the prior art is solved, and the competition for metadata resources between multiple data writing threads that write data at the same time is avoided, thereby improving the data writing performance of the system.

[0016] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 is a flow chart of a data writing method provided by Embodiment 1 of the present invention;

[0019] Figure 2 It is a schematic diagram of a physical space block structure provided by the first embodiment of the present invention;

[0020] Figure 3 is a flow chart of a data writing method provided by Embodiment 2 of the present invention;

[0021] Figure 4 It is a schematic diagram of the structure of a fingerprint node table provided in the second embodiment of the present invention;

[0022] Figure 5 is a structural schematic diagram of a data writing device provided in Embodiment 3 of the present invention;

[0023] Figure 6 It is a structural schematic diagram of an electronic device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] Embodiment 1

[0027] Figure 1 A flowchart of a data writing method is provided for the first embodiment of the present invention. This embodiment is applicable to the situation where multiple I / O writing processes are performed simultaneously in an all-flash array. The method can be executed by a data writing device, which can be implemented in the form of hardware and / or software. The data writing device can be configured in an electronic device deployed with an all-flash array storage system.

[0028] like Figure 1 As shown, the method includes:

[0029] S110 . Determine at least two sub-data to be written according to the data to be written, and determine the data writing thread identifier corresponding to each sub-data to be written according to the physical space block index number information of each sub-data to be written.

[0030] In this embodiment, the data to be written is split into a plurality of sub-data to be written, and subsequently different sub-data to be written are written into different fingerprint nodes respectively. Further, the data size of the sub-data to be written matches the standard data unit corresponding to the fingerprint node. Specifically, the data to be written can be split into a plurality of sub-data to be written according to the standard data unit corresponding to the fingerprint node, and the standard data unit is used to characterize the minimum operation unit for data storage. After the data to be written is split into the sub-data to be written according to the standard data unit, the sub-data to be written are written into different fingerprint nodes. Further, the standard data unit can match the storage space of the fingerprint node.

[0031] The physical space block index number information is used to indicate the index number of the physical space block where the physical space address of the sub-data to be written is located. In this embodiment, the physical space is the storage space actually used for data writing. The physical space is divided into blocks in advance and each physical space block is numbered. Figure 2 A schematic diagram of the structure of physical space blocks is provided, such as Figure 2 As shown, the numbering of each physical space block starts from 0 and increases in sequence. It should be noted that the size of each physical space block should be the same, but this embodiment does not limit the size of each physical space block.

[0032] In this embodiment, after the data to be written is split into sub-data to be written, its physical space block index number information is determined according to the physical space address of the sub-data to be written, so that the physical space block index number information can be subsequently used as a key value for determining the data writing thread.

[0033] Specifically, the data writing thread identifier of each sub-data to be written can be determined based on the total number of data writing threads and the physical space block index number information of each sub-data to be written. The total number of data writing threads can be flexibly set according to the performance of the data writing system, and this embodiment does not limit this.

[0034] Since the I / O write of the all-flash array mostly adopts the ROW write mode, whether it is new data writing or data modification writing, the data is written to the new address instead of overwriting the original address, and the physical space block index number of each sub-data to be written is continuous. Therefore, according to the physical space block index number information of each sub-data to be written, each sub-data to be written can be sent to different data writing threads for processing.

[0035] S120: Determine at least one fingerprint node block under the current target fingerprint node block management unit identifier through the target fingerprint node block management unit identifier corresponding to the target data writing thread identifier, and determine the target idle fingerprint node according to the fingerprint node block.

[0036] For the target sub-data to be written, after the corresponding target data writing thread identifier is determined according to its physical space block index number information, the data writing thread corresponding to the target data writing thread identifier will only operate its unique corresponding target fingerprint node block management unit. The role of the target fingerprint node block management unit is to determine the idle fingerprint node block and idle fingerprint node for the target sub-data to be written.

[0037] Specifically, the data writing threads correspond to the fingerprint node block management units one by one, the total number of data writing threads is the same as the total number of fingerprint node block management units, and the data writing thread identifiers correspond to the fingerprint node block management unit identifiers one by one. Each data writing thread can be numbered, starting from 0 and increasing in sequence; similarly, the fingerprint node block management units are numbered, starting from 0 and increasing in sequence. Exemplarily, if the data writing threads include data writing thread 0, data writing thread 1, and data writing thread 2, and the fingerprint node block management units include fingerprint node block management unit 0, fingerprint node block management unit 1, and fingerprint node block management unit 2, then the data writing thread 0 uses the fingerprint node blocks managed by fingerprint node block management unit 0, the data writing thread 1 uses the fingerprint node blocks managed by fingerprint node block management unit 1, and the data writing thread 2 uses the fingerprint node blocks managed by fingerprint node block management unit 2.

[0038] The fingerprint node block management unit identifier corresponds to the numbers of at least two fingerprint node blocks, that is, each fingerprint node block management unit manages at least two fingerprint node blocks, and the fingerprint node blocks managed by the fingerprint node block management unit may be non-continuous fingerprint node blocks. Exemplarily, when the total number of data write threads and the total number of fingerprint node block management units are 3, fingerprint node block management unit 0 manages fingerprint node blocks with a sequence of 0, 3, 6, 9..., fingerprint node block management unit 1 manages fingerprint node blocks with a sequence of 1, 4, 7, 10..., and fingerprint node block management unit 2 manages fingerprint node blocks with a sequence of 2, 5, 8, 11... Each fingerprint node block managed by the fingerprint node block management unit may also be a continuous fingerprint node block. For example, taking the total number of the above-mentioned data writing threads and the total number of fingerprint node block management units as 3 as an example, the fingerprint node block management unit 0 manages the fingerprint node blocks with a sequence of 0-4, the fingerprint node block management unit 1 manages the fingerprint node blocks with a sequence of 5-9, and the fingerprint node block management unit 2 manages the fingerprint node blocks with a sequence of 10-14. When all the fingerprint nodes corresponding to each fingerprint node block are fully stored, the fingerprint node blocks with a sequence of 15-19 are allocated to the fingerprint node block management unit 0, the fingerprint node blocks with a sequence of 20-24 are allocated to the fingerprint node block management unit 1, and the fingerprint node blocks with a sequence of 25-29 are allocated to the fingerprint node block management unit 2, and so on. This embodiment does not limit the numbering format of each fingerprint node block managed by the fingerprint node block management unit.

[0039] The fingerprint node block includes at least two fingerprint nodes. Specifically, each fingerprint node can be numbered consecutively, that is, each fingerprint node block includes at least two consecutive fingerprint nodes, and the fingerprint node is used to store the data fingerprint hash value and the physical space address of the sub-data to be written. Therefore, in the technical solution of this embodiment, the mapping relationship table of the data fingerprint hash value and the physical space address is composed of multiple consecutive fingerprint node blocks, and each fingerprint node block includes multiple consecutive fingerprint nodes.

[0040] The technical solution of this embodiment is to index the fingerprint node block management unit identifier through the data writing thread identifier, then index to the idle fingerprint node block, and finally obtain the idle fingerprint node. Through this three-level indexing method, the idle fingerprint node can be quickly located and the data writing speed can be improved.

[0041] S130: Write the target to-be-written sub-data to the target idle fingerprint node through a data writing thread corresponding to the target data writing thread identifier.

[0042] The technical solution of this embodiment, on the one hand, sends each sub-data to be written to different data writing threads for processing, and each data writing thread corresponds to only a unique fingerprint node block management unit. On the other hand, the mapping relationship table between the data fingerprint hash value and the physical space address is represented by the structure of multiple fingerprint node blocks. This setting makes it possible for multiple threads that write data at the same time to write data without competing for the mapping relationship table between the data fingerprint hash value and the physical space address. Each sub-data to be written is processed by a different data writing thread and written to each independent idle fingerprint node, thereby improving the system I / O write performance in the I / O data writing multi-threaded scenario.

[0043] The technical solution of the embodiment of the present invention is to split the data to be written into sub-data to be written, and respectively determine the physical space index number of each sub-data to be written, and determine the data writing thread matching each sub-data to be written according to the physical space index number of each sub-data to be written, and determine the target fingerprint node block management unit, the target idle fingerprint node block and the target idle fingerprint node through the target data writing thread matching the target sub-data to be written, and write the target sub-data to be written to the target idle fingerprint node. The problem of the mapping relationship table of data fingerprint hash to physical space address competition caused by multiple data writing threads simultaneously writing data in the prior art is solved, and the competition for metadata resources between multiple data writing threads that write data at the same time is avoided, thereby improving the data writing performance of the system.

[0044] Embodiment 2

[0045] Figure 3 A flow chart of a data writing method provided for the second embodiment of the present invention. Based on the above embodiments, the embodiment of the present invention further specifies the process of determining the sub-data to be written, the process of determining the physical space index number of each sub-data to be written, the process of determining the data writing thread matching each sub-data to be written, the process of determining the three-level index of the idle fingerprint node, and the process of writing the sub-data to be written to the idle fingerprint node.

[0046] like Figure 3 As shown, the method includes:

[0047] S210. Determine at least two sub-data to be written according to the data to be written.

[0048] Specifically, the data to be written can be split according to the standard data unit corresponding to the fingerprint node to obtain the sub-data to be written. The standard data unit can match the storage space of the fingerprint node. For example, if the storage space of the fingerprint node is 2k, the standard data unit is determined to be 2k. If the data to be written is 100k, the data to be written is split into 50 sub-data to be written.

[0049] S220. Determine physical space block index number information matching each sub-data to be written according to the physical space address of each sub-data to be written.

[0050] The physical space block index number information is obtained by numbering each physical space block after dividing the physical space into blocks.

[0051] The specific method of dividing the physical space into blocks and determining the physical space block index number information according to the physical space address of the sub-data to be written has been explained in the above embodiment, and will not be repeated here in this embodiment.

[0052] S230, determining an intermediate value according to the physical space block index number of the target sub-data to be written and the total number of data writing threads, and determining a target data writing thread identifier that matches the target sub-data to be written according to the intermediate value.

[0053] In an optional embodiment, the physical space block index number of the target sub-data to be written can be modulo the total number of data writing threads to obtain an intermediate value, and the target data writing thread identifier is determined according to the intermediate value. Exemplarily, when the total number of data writing threads is 3, the sub-data to be written with the physical space block index number of 0 is processed by data writing thread 0, the sub-data to be written with the physical space block index number of 1 is processed by data writing thread 1, the sub-data to be written with the physical space block index number of 2 is processed by data writing thread 2, the sub-data to be written with the physical space block index number of 3 is processed by data writing thread 0... and so on.

[0054] In another optional embodiment, each sub-data to be written can also be grouped according to the physical space block index number of each sub-data to be written and the total number of data writing threads, and the number of each group of sub-data to be written is the same as the total number of data writing threads. Within a group of sub-data to be written, each number from 0 to the total number of data writing threads is randomly assigned to each sub-data to be written, and the data writing thread identifier corresponding to the number assigned to the target sub-data to be written is used as the target data writing thread identifier. Exemplarily, when the total number of data writing threads is 3, if there are 100 sub-data to be written, the sub-data to be written are divided into 34 groups, wherein the number of sub-data to be written in group 33 is 3, and the number of sub-data to be written in group 1 is 1. For the 33 groups with 3 sub-data to be written, for example, for the first group, data writing thread 0, data writing thread 1, and data writing thread 2 are randomly assigned to the sub-data to be written with a physical space block index number of 0, the sub-data to be written with a physical space block index number of 1, and the sub-data to be written with a physical space block index number of 2. Exemplarily, data writing thread 1 can be assigned to the sub-data to be written with a physical space block index number of 0, data writing thread 2 can be assigned to the sub-data to be written with a physical space block index number of 1, and data writing thread 0 can be assigned to the sub-data to be written with a physical space block index number of 2. For the sub-data to be written in the 34th group, any one of data writing thread 0, data writing thread 1, and data writing thread 2 can be assigned to it.

[0055] The above embodiment is merely an example of a process of determining an intermediate value based on the physical space block index number of the target sub-data to be written and the total number of data writing threads, and determining the target data writing thread identifier based on the intermediate value. This embodiment does not limit the specific process of determining the target data writing thread identifier corresponding to the target sub-data to be written.

[0056] S240: Determine a target fingerprint node block management unit identifier corresponding to the target data writing thread identifier.

[0057] The data writing thread identifier corresponds to the fingerprint node block management unit identifier one by one, which has been explained in the above embodiment and will not be repeated in this embodiment.

[0058] S250, determining whether the sequential writing condition of the sub-data to be written is met, if so, executing S260, otherwise executing S280.

[0059] Meeting the sequential writing condition of the sub-data to be written refers to the process of sequentially writing the fingerprint node table (i.e., the mapping relationship table of data fingerprint hash to physical space address) composed of multiple idle fingerprint node blocks. When the sequential writing condition of the sub-data to be written is met, the sub-data to be written can be sequentially written in the fingerprint node table according to the data writing thread of each sub-data to be written.

[0060] S260: Determine a target idle fingerprint node block according to an idle fingerprint node block indication parameter of the fingerprint node block management unit corresponding to the target fingerprint node block management unit identifier.

[0061] Specifically, the fingerprint node block management unit includes a fingerprint node block bitmap table and a free fingerprint node block indication parameter, and the free fingerprint node block indication parameter is used to indicate the number of the next free fingerprint node block, and is used to quickly query the free fingerprint node block when the sub-data to be written are written in sequence. Exemplarily, the free fingerprint node block indication parameter can be represented by next_free_block_index.

[0062] Furthermore, when all fingerprint nodes in the current idle fingerprint node block have been used, the idle fingerprint node block indication parameter is updated.

[0063] S270: Determine the target idle fingerprint node according to the idle fingerprint node indication parameter of the target idle fingerprint node block. Execute S2100.

[0064] The fingerprint node block includes a fingerprint node bitmap table and an idle fingerprint node indication parameter. Specifically, the fingerprint node block includes a plurality of fingerprint nodes and a fingerprint node block header. The header includes an idle fingerprint node indication parameter. The idle fingerprint node indication parameter is used to indicate the space address of the next idle fingerprint node, and is used to quickly query the space address of the idle fingerprint node when the sub-data to be written are written sequentially. Exemplarily, the idle fingerprint node indication parameter can be represented by next_fp_adrr.

[0065] Furthermore, after the sub-data to be written is written into the current fingerprint node, the idle fingerprint node indication parameter is updated.

[0066] S280: Determine a target idle fingerprint node block according to the fingerprint node block bitmap table of the fingerprint node block management unit corresponding to the target fingerprint node block management unit identifier.

[0067] The fingerprint node block bitmap table includes at least two bits corresponding to the fingerprint node blocks, and the bit value indicates whether the fingerprint node block is idle.

[0068] In this embodiment, after a round of sequential writing to the fingerprint node table is completed, the free fingerprint node is searched through the bitmap table of the fingerprint node block management unit (ie, the fingerprint node block bitmap table) and the bitmap table of the fingerprint node block.

[0069] Specifically, the fingerprint node block management unit includes a fingerprint node block bitmap table, that is, a bitmap table. Each bit in the bitmap table corresponds to a fingerprint node block, and the bit value indicates whether the fingerprint node block is free. For example, 0 can be used to indicate that the fingerprint node block is free, and 1 can be used to indicate that the fingerprint node block is used.

[0070] In this embodiment, the idle fingerprint node blocks can be quickly determined according to the value of each bit in the bitmap table of the fingerprint node block management unit.

[0071] S290: Determine a target idle fingerprint node according to the fingerprint node bitmap table of the target idle fingerprint node block.

[0072] The fingerprint node bitmap table includes at least two bits corresponding to the fingerprint nodes, and the bit value indicates whether the fingerprint node is idle.

[0073] Specifically, the block header of the fingerprint node block also includes a fingerprint node bitmap table, that is, a bitmap table. Each bit in the bitmap table corresponds to a fingerprint node, and the bit value indicates whether the fingerprint node is idle. For example, 0 can be used to indicate that the fingerprint node is idle, and 1 can be used to indicate that the fingerprint node is in use.

[0074] Figure 4 A schematic diagram of the structure of a fingerprint node table is provided, such as Figure 4 As shown in Figure 1, the fingerprint node table consists of multiple fingerprint node blocks with consecutive numbers. Figure 4 The above is only an example, and the present embodiment does not limit the number of fingerprint node blocks. A fingerprint node block is composed of a block header and a plurality of fingerprint nodes with consecutive numbers. The block header includes a bitmap table and an idle fingerprint node indication parameter. The bitmap table is represented by fp_bitmap, and the idle fingerprint node indication parameter is represented by next_fp_adrr.

[0075] S2100. Write the data fingerprint hash value and the physical space address of the target sub-data to be written into the target idle fingerprint node through the data writing thread corresponding to the target data writing thread identifier.

[0076] like Figure 4 As shown, the structure of the fingerprint node includes the digital fingerprint hash value, the physical space address and the number of times the data is referenced. The digital fingerprint hash value is represented by fp_hash, the physical space address is represented by phy_addr, and the number of times the data is referenced is represented by ref_cnt.

[0077] It needs to be explained that Figure 4This is only an example of the structure of the fingerprint node. Except for the data fingerprint hash value and the physical space address, this embodiment does not limit the number and type of parameters that the fingerprint node is used to store.

[0078] Embodiment 3

[0079] Figure 5 This is a schematic diagram of the structure of a data writing device provided by Embodiment 3 of the present invention. Figure 5 As shown, the device includes: a data writing thread identification determination module 310, a target idle fingerprint node determination module 320 and a data writing module 330. Among them:

[0080] The data writing thread identifier determination module 310 is used to determine at least two sub-data to be written according to the data to be written, and determine the data writing thread identifier corresponding to each sub-data to be written according to the physical space block index number information of each sub-data to be written;

[0081] The target idle fingerprint node determination module 320 is used to determine at least one fingerprint node block under the current target fingerprint node block management unit identifier through the target fingerprint node block management unit identifier corresponding to the target data writing thread identifier, and determine the target idle fingerprint node according to the fingerprint node block;

[0082] The data writing module 330 is used to write the target sub-data to be written into the target idle fingerprint node through the data writing thread corresponding to the target data writing thread identifier.

[0083] The technical solution of the embodiment of the present invention is to split the data to be written into sub-data to be written, and respectively determine the physical space index number of each sub-data to be written, and determine the data writing thread matching each sub-data to be written according to the physical space index number of each sub-data to be written, and determine the target fingerprint node block management unit, the target idle fingerprint node block and the target idle fingerprint node through the target data writing thread matching the target sub-data to be written, and write the target sub-data to be written to the target idle fingerprint node. The problem of the mapping relationship table of data fingerprint hash to physical space address competition caused by multiple data writing threads simultaneously writing data in the prior art is solved, and the competition for metadata resources between multiple data writing threads that write data at the same time is avoided, thereby improving the data writing performance of the system.

[0084] On the basis of the above embodiment, the data writing thread identifier corresponds to the fingerprint node block management unit identifier one by one;

[0085] The fingerprint node block management unit identifier corresponds to the serial numbers of at least two fingerprint node blocks;

[0086] The fingerprint node block includes at least two fingerprint nodes;

[0087] The data size of the sub-data to be written matches the standard data unit corresponding to the fingerprint node.

[0088] Based on the above embodiment, the data writing thread identification determination module 310 includes:

[0089] According to the physical space address of each sub-data to be written, determine the physical space block index number information matching each sub-data to be written;

[0090] The physical space block index number information is obtained by numbering each physical space block after dividing the physical space into blocks;

[0091] An intermediate value is determined according to the physical space block index number of the target sub-data to be written and the total number of data writing threads, and a target data writing thread identifier matching the target sub-data to be written is determined according to the intermediate value.

[0092] On the basis of the above embodiment, the fingerprint node block management unit includes a fingerprint node block bitmap table and an idle fingerprint node block indication parameter;

[0093] The fingerprint node block bitmap table includes at least two bits corresponding to the fingerprint node blocks, and the bit value indicates whether the fingerprint node block is idle;

[0094] The idle fingerprint node block indication parameter is used to indicate the number of the next idle fingerprint node block;

[0095] The fingerprint node block includes a fingerprint node bitmap table and an idle fingerprint node indication parameter;

[0096] The fingerprint node bitmap table includes at least two bits corresponding to the fingerprint nodes, and the bit value indicates whether the fingerprint node is idle;

[0097] The idle fingerprint node indication parameter is used to indicate the number of the next idle fingerprint node.

[0098] Based on the above embodiment, the target idle fingerprint node determination module 320 includes:

[0099] A target fingerprint node block management unit identifier determining unit, used to determine a target fingerprint node block management unit identifier corresponding to a target data writing thread identifier;

[0100] A first target idle fingerprint node block determining unit is configured to determine a target idle fingerprint node block according to an idle fingerprint node block indicating parameter of a fingerprint node block management unit corresponding to the target fingerprint node block management unit identifier if it is determined that a sequential writing condition of the sub-data to be written is met;

[0101] The first target idle fingerprint node determining unit is used to determine the target idle fingerprint node according to the idle fingerprint node indication parameter of the target idle fingerprint node block.

[0102] Based on the above embodiment, the target idle fingerprint node determination module 320 further includes:

[0103] A second target idle fingerprint node block determining unit is configured to determine a target idle fingerprint node block according to a fingerprint node block bitmap table of a fingerprint node block management unit corresponding to the target fingerprint node block management unit identifier if it is determined that the sequential writing condition of the sub-data to be written is not satisfied;

[0104] The second target idle fingerprint node determination unit is used to determine the target idle fingerprint node according to the fingerprint node bitmap table of the target idle fingerprint node block.

[0105] Based on the above embodiment, the data writing module 330 includes:

[0106] The data writing unit is used to write the data fingerprint hash value and the physical space address of the target sub-data to be written into the target idle fingerprint node through the data writing thread corresponding to the target data writing thread identifier.

[0107] The data writing device provided in the embodiment of the present invention can execute the data writing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0108] Embodiment 4

[0109] Figure 6 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0110] like Figure 6As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0111] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0112] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as a data writing method.

[0113] In some embodiments, the data writing method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the data writing method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the data writing method in any other appropriate manner (e.g., by means of firmware).

[0114] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0115] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0116] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0117] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0118] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0119] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0120] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0121] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A data writing method, characterized in that: include: Determine at least two sub-data to be written according to the data to be written, and determine the data writing thread identifier corresponding to each sub-data to be written according to the physical space block index number information of each sub-data to be written; Determine at least one fingerprint node block under the current target fingerprint node block management unit identifier through the target fingerprint node block management unit identifier corresponding to the target data writing thread identifier, and determine the target idle fingerprint node according to the fingerprint node block; The target sub-data to be written is written to the target idle fingerprint node through a data writing thread corresponding to the target data writing thread identifier.

2. The method according to claim 1, characterized in that: The data writing thread identifier corresponds to the fingerprint node block management unit identifier one by one; The fingerprint node block management unit identifier corresponds to the serial numbers of at least two fingerprint node blocks; The fingerprint node block includes at least two fingerprint nodes; The data size of the sub-data to be written matches the standard data unit corresponding to the fingerprint node.

3. The method according to claim 1, characterized in that Determining the data writing thread identifier corresponding to each sub-data to be written according to the physical space block index number information of each sub-data to be written, including: According to the physical space address of each sub-data to be written, determine the physical space block index number information matching each sub-data to be written; The physical space block index number information is obtained by dividing the physical space into blocks and numbering each physical space block; An intermediate value is determined according to the physical space block index number of the target sub-data to be written and the total number of data writing threads, and a target data writing thread identifier matching the target sub-data to be written is determined according to the intermediate value.

4. The method according to claim 2, characterized in that: The fingerprint node block management unit includes a fingerprint node block bitmap table and an idle fingerprint node block indication parameter; The fingerprint node block bitmap table includes at least two bits corresponding to the fingerprint node blocks, and the bit value indicates whether the fingerprint node block is idle; The idle fingerprint node block indication parameter is used to indicate the number of the next idle fingerprint node block; The fingerprint node block includes a fingerprint node bitmap table and an idle fingerprint node indication parameter; The fingerprint node bitmap table includes at least two bits corresponding to the fingerprint nodes, and the bit value indicates whether the fingerprint node is idle; The idle fingerprint node indication parameter is used to indicate the number of the next idle fingerprint node.

5. The method according to claim 4, characterized in that Determining at least one fingerprint node block under the current target fingerprint node block management unit identifier by using the target fingerprint node block management unit identifier corresponding to the target data writing thread identifier, including: Determine a target fingerprint node block management unit identifier corresponding to a target data writing thread identifier; If it is determined that the sequential writing condition of the sub-data to be written is met, determining the target idle fingerprint node block according to the idle fingerprint node block indication parameter of the fingerprint node block management unit corresponding to the target fingerprint node block management unit identifier; Determining a target idle fingerprint node according to the fingerprint node block includes: The target idle fingerprint node is determined according to the idle fingerprint node indication parameter of the target idle fingerprint node block.

6. The method according to claim 4, characterized in that After determining the target fingerprint node block management unit identifier corresponding to the target data writing thread identifier, the method further includes: If it is determined that the sequential writing condition of the sub-data to be written is not satisfied, determining the target idle fingerprint node block according to the fingerprint node block bitmap table of the fingerprint node block management unit corresponding to the target fingerprint node block management unit identifier; Determining a target idle fingerprint node according to the fingerprint node block includes: The target idle fingerprint node is determined according to the fingerprint node bitmap table of the target idle fingerprint node block.

7. The method according to claim 1, characterized in that Writing the target to-be-written sub-data to the target idle fingerprint node through a data writing thread corresponding to the target data writing thread identifier includes: The data fingerprint hash value and the physical space address of the target sub-data to be written are written to the target idle fingerprint node through the data writing thread corresponding to the target data writing thread identifier.

8. A data writing device, characterized in that: include: A data writing thread identifier determination module is used to determine at least two sub-data to be written according to the data to be written, and determine the data writing thread identifier corresponding to each sub-data to be written according to the physical space block index number information of each sub-data to be written; A target idle fingerprint node determination module is used to determine at least one fingerprint node block under the current target fingerprint node block management unit identifier through the target fingerprint node block management unit identifier corresponding to the target data writing thread identifier, and determine the target idle fingerprint node according to the fingerprint node block; The data writing module is used to write the target sub-data to be written into the target idle fingerprint node through the data writing thread corresponding to the target data writing thread identifier.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the data writing method as described in any one of claims 1 to 7 is implemented.

10. A storage medium storing computer executable instructions, characterized in that: When the computer executable instructions are executed by a computer processor, they are used to execute the data writing method according to any one of claims 1 to 7.