Data storage method and device, computer device and storage medium

Abstract

The application relates to the technical field of distributed storage, and discloses a data storage method and device, computer equipment and a storage medium, the method comprising the following steps: in the case of using a storage node to perform erasure code calculation, whenever receiving to-be-stored data, randomly selecting a storage node and sending the to-be-stored data to the storage node; using the storage node to receive the to-be-stored data, obtaining instruction set information of the highest level supported by a central processing unit of the storage node, obtaining an encoding function matched with the instruction set information, performing encoding processing on the to-be-stored data based on the encoding function, and obtaining an erasure code corresponding to the to-be-stored data; and associating the erasure code with the to-be-stored data. The application adaptively selects the instruction set information of the highest level supported by the central processing unit, uses the encoding function corresponding to the instruction set information to perform erasure code calculation, and obtains the erasure code, so that the calculation efficiency of the erasure code is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of distributed storage, and in particular to a data storage method and device, a computer device and a storage medium. BACKGROUND

[0002] In a distributed storage system, in order to ensure the reliability and availability of data, a redundancy strategy is usually used to store data. Among them, the erasure coding redundancy strategy is widely used because of its advantage of reducing storage space consumption.

[0003] In related technologies, the erasure coding redundancy strategy is to divide data into multiple data segments, generate redundant data blocks (i.e. erasure codes) based on multiple data segments using an encoding method, and store multiple data segments and redundant data blocks in different locations of a distributed storage system.

[0004] Executing the erasure coding redundancy strategy on a storage node is essentially to use the instruction set supported by the central processing unit (CPU) inside to calculate the erasure codes. Since different central processing units support different instruction sets, the highest level instruction set in the instruction set supported by different central processing units is often used to calculate the erasure codes.

[0005] However, this way of calculating the erasure codes makes it impossible for some central processing units to use the highest level instruction set they support, resulting in low efficiency of calculating the erasure codes. SUMMARY

[0006] Therefore, the present application provides a data storage method and device, a computer device and a storage medium to solve the problem of low efficiency of calculating the erasure codes caused by using the highest level instruction set in the instruction set supported by different central processing units to calculate the erasure codes in related technologies.

[0007] In a first aspect, the present application provides a data storage method, which comprises:

[0008] In the case of using a storage node to calculate the erasure codes, whenever receiving data to be stored, a storage node is randomly selected, and the data to be stored is sent to the storage node;

[0009] The storage node receives the data to be stored, obtains the highest level instruction set information supported by the central processing unit of the storage node, obtains an encoding function matched with the instruction set information, performs encoding processing on the data to be stored based on the encoding function, and obtains the erasure codes corresponding to the data to be stored;

[0010] The erasure codes and the data to be stored are associatedly stored.

[0011] The data storage method provided in the embodiment can dynamically select the highest level of instruction set supported by the central processor, calculate the erasure code corresponding to the to-be-stored data by using the encoding function corresponding to the instruction set information, reduce the calculation delay of the erasure code, improve the calculation efficiency of the erasure code, reduce the CPU usage, and improve the calculation bandwidth.

[0012] In an optional implementation, the method further includes:

[0013] In the case of using the client to calculate the erasure code, receiving the to-be-stored data and the erasure code corresponding to the to-be-stored data sent by the client, and storing the erasure code and the to-be-stored data in association;

[0014] The client receives the to-be-stored data, obtains the highest level of instruction set supported by the central processor of the client, obtains the encoding function matched with the instruction set information, performs encoding processing on the to-be-stored data based on the encoding function, and obtains the erasure code corresponding to the to-be-stored data, and sends the erasure code and the to-be-stored data to the distributed storage system.

[0015] The data storage method provided in the embodiment can dynamically select the highest level of instruction set supported by the central processor of the client, obtain the erasure code corresponding to the to-be-stored data based on the encoding function, adapt to the CPU instruction set support condition of the client, dynamically select the highest level of instruction set, improve the calculation efficiency of the erasure code, reduce the CPU usage, and improve the calculation performance.

[0016] In an optional implementation, the encoding processing on the to-be-stored data based on the encoding function to obtain the erasure code corresponding to the to-be-stored data includes:

[0017] segmenting the to-be-stored data into a plurality of data blocks;

[0018] performing encoding processing on the plurality of data blocks by using the encoding function to generate a plurality of check blocks, wherein the erasure code includes the check blocks;

[0019] The storing the erasure code and the to-be-stored data in association includes:

[0020] The check block and the data block are stored in association.

[0021] The data storage method provided by the embodiment improves the calculation efficiency of the erasure code and reduces the CPU usage rate by encoding the to-be-stored data based on the encoding function matching the highest-level instruction set information of the storage node. The check block and the data block are stored in association, thereby improving the security of data storage.

[0022] In an optional embodiment, the method further comprises:

[0023] In the case where any data block in the distributed storage system fails, a storage node for calculating the erasure code corresponding to the data block is determined;

[0024] An encoding function matching the highest-level instruction set information supported by the central processing unit of the storage node is obtained;

[0025] The remaining data blocks and the check block associated with the data block are obtained;

[0026] The data of the remaining data blocks and the check block are input into the encoding function to obtain the original data corresponding to the failed data block, so that data recovery is performed based on the original data.

[0027] The data storage method provided by the embodiment significantly improves the efficiency of data recovery and the fault tolerance of the system by using the decoding function matching the highest-level instruction set supported by the CPU of the storage node and combining the data recovery mechanism. Meanwhile, the data storage method reduces resource consumption, enhances the scalability and flexibility of the system, and simplifies management and maintenance work.

[0028] In an optional embodiment, the method further comprises:

[0029] In the case where the randomly selected storage node is a storage node that has performed erasure code calculation, the to-be-stored data is directly encoded by using the encoding function corresponding to the storage node to obtain the erasure code corresponding to the to-be-stored data.

[0030] The data storage method provided by the embodiment significantly improves the encoding efficiency by directly using the encoding function corresponding to the storage node that has performed erasure code calculation to encode the to-be-stored data, thereby obtaining the erasure code corresponding to the to-be-stored data, eliminating the overhead caused by automatically selecting the highest-level instruction set information of the storage node each time, improving the performance of erasure code calculation, and reducing the calculation cost.

[0031] In an optional embodiment, the obtaining of the highest-level instruction set information supported by the central processing unit of the storage node comprises:

[0032] obtaining configuration information of a central processing unit of the storage node;

[0033] determining, based on the configuration information, highest-level instruction set information supported by the central processing unit.

[0034] The data storage method provided by the embodiment can significantly improve the performance of erasure calculation and the efficiency of obtaining erasure codes by determining and using the highest-level instruction set information supported by the central processing unit.

[0035] In an optional implementation, the determining, based on the configuration information, of the highest-level instruction set information supported by the central processing unit comprises:

[0036] judging, based on the configuration information, whether the central processing unit supports a finite field instruction set and a first vector extension instruction set simultaneously;

[0037] in a case where the central processing unit supports the finite field instruction set and the first vector extension instruction set simultaneously, determining that the highest-level instruction set information supported by the central processing unit is the finite field instruction set and the first vector extension instruction set;

[0038] in a case where the central processing unit does not support the finite field instruction set and the first vector extension instruction set simultaneously, judging whether the central processing unit supports the first vector extension instruction set;

[0039] in a case where the central processing unit supports the first vector extension instruction set, determining that the highest-level instruction set information supported by the central processing unit is the first vector extension instruction set;

[0040] in a case where the central processing unit does not support the first vector extension instruction set, judging whether the central processing unit supports a finite field instruction set and a second vector extension instruction set simultaneously, wherein the second vector extension instruction set has a smaller number of operation bits than the first vector extension instruction set;

[0041] in a case where the central processing unit supports the finite field instruction set and the second vector extension instruction set simultaneously, determining that the highest-level instruction set information supported by the central processing unit is the finite field instruction set and the second vector extension instruction set;

[0042] in a case where the central processing unit does not support the finite field instruction set and the second vector extension instruction set simultaneously, judging whether the central processing unit supports the second vector extension instruction set;

[0043] in a case where the central processing unit supports the second vector extension instruction set, determining that the highest-level instruction set information supported by the central processing unit is the second vector extension instruction set;

[0044] In a case that the central processor does not support the second vector extension instruction set, it is judged whether the central processor supports a third vector extension instruction set, wherein the second vector extension instruction set is an extended version of the third vector extension instruction set;

[0045] In a case that the central processor supports the third vector extension instruction set, it is determined that the highest level instruction set information supported by the central processor is the third vector extension instruction set.

[0046] The data storage method provided by the embodiment can determine the highest level instruction set supported by the CPU by judging whether the CPU supports a specific instruction set, so as to perform erasure calculation on the to-be-stored data by using the highest level instruction set supported by the CPU, and obtain an erasure code. The execution efficiency of the erasure calculation is improved, and the processing capability of the CPU is maximized.

[0047] In a second aspect, the present application provides a data storage device, which comprises:

[0048] The sending module is configured to, in a case that the erasure code calculation is performed by using the storage node, randomly select one storage node whenever the to-be-stored data is received, and send the to-be-stored data to the storage node.

[0049] The first obtaining module is configured to receive the to-be-stored data by using the storage node, obtain the highest level instruction set information supported by the central processor of the storage node, obtain an encoding function matched with the instruction set information, perform encoding processing on the to-be-stored data based on the encoding function, and obtain the erasure code corresponding to the to-be-stored data.

[0050] The storage module is configured to store the erasure code and the to-be-stored data in association.

[0051] In a third aspect, the present application provides a computer device, which comprises a memory and a processor, the memory and the processor are communicatively connected with each other, the memory stores computer instructions, and the processor executes the computer instructions, thereby executing the data storage method of the first aspect or any of the corresponding embodiments thereof.

[0052] In a fourth aspect, the present application provides a computer readable storage medium, which stores computer instructions, and the computer instructions are used to make a computer execute the data storage method of the first aspect or any of the corresponding embodiments thereof.

[0053] In a fifth aspect, the present application provides a computer program product, which comprises computer instructions, and the computer instructions are used to make a computer execute the data storage method of the first aspect or any of the corresponding embodiments thereof. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the specific embodiments or the related art of the present application, the drawings needed to be used in the specific embodiments or the related art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0055] Figure 1 is a flowchart of a data storage method according to an embodiment of the present application;

[0056] Figure 2 is a flowchart of another data storage method according to an embodiment of the present application;

[0057] Figure 3 is a flowchart of information of the highest level of instruction set supported by the central processing unit according to an embodiment of the present application;

[0058] Figure 4 is a flowchart of still another data storage method according to an embodiment of the present application;

[0059] Figure 5 is a structural block diagram of a data storage device according to an embodiment of the present application;

[0060] Figure 6 is a hardware structure diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0062] In a distributed storage system, in order to ensure the reliability and availability of data, a redundancy strategy is usually used to store data. This strategy can ensure the normal operation of the system and the safe access of data in the case of hardware failure, network failure, etc.

[0063] There are two kinds of redundancy strategies in a distributed storage system, one is a replica redundancy strategy, and the other is an erasure redundancy strategy. Among them, the replica redundancy strategy consumes more storage space, and the erasure redundancy strategy effectively reduces the consumption of storage space and reduces the transmission bandwidth between storage nodes. Therefore, the erasure redundancy strategy is widely used in distributed storage systems.

[0064] In the related art, the erasure redundancy strategy is to divide data into multiple data segments, generate redundant data blocks (i.e., erasure codes) based on the multiple data segments using an encoding manner, and store the multiple data segments and the redundant data blocks in different locations of a distributed storage system, such as disks, storage nodes, or other locations.

[0065] In the non-kernel client file scenario, the erasure redundancy strategy is executed by using the storage node, which is essentially to perform the calculation of the erasure codes by using the instruction set supported by the CPU inside. Since different central processing units support different instruction sets, the highest level instruction set in the instruction set supported by different central processing units is often used to perform the calculation of the erasure codes. Generally, the extended version of the Advanced Vector Extensions 2 (AVX2) instruction set is used to perform the calculation of the erasure codes.

[0066] In the kernel client file scenario, after the calculation of the erasure codes is lifted from the server to the client, the erasure redundancy strategy is executed by using the client, which is essentially to perform the calculation of the erasure codes by using the instruction set supported by the CPU inside. The CPU models used by the clients are different, and the highest level instruction set in the instruction set supported by the CPUs of different clients is often used to perform the calculation of the erasure codes. Generally, the extended version of the Advanced Vector Extensions 2 (AVX2) instruction set is used to perform the calculation of the erasure codes.

[0067] The distributed storage system selects different CPUs for different scenarios, and the update and replacement of the CPUs have brought new instruction sets. For example, the X86 platform CPU has brought new single instruction multiple data (SIMD) instruction sets, including Streaming SIMD Extensions (SSE) instruction set, Advanced Vector Extensions (AVX) instruction set, AVX2 instruction set, and Advanced Vector Extensions 512 (AVX512) instruction set, and the operation bit width has been expanded from 128 bits to 256 bits and then to 512 bits. Among them, the operation bit width of the SSE instruction set is 128 bits, i.e., 128 bits are operated at a time. The operation bit width of the AVX instruction set and the AVX2 instruction set is 256 bits, i.e., 256 bits are operated at a time. The operation bit width of the AVX512 instruction set is 512 bits, i.e., 512 bits are operated at a time. Among them, the SIMD instruction set allows the processor to perform the same operation on multiple data in a single instruction cycle, thereby significantly improving the calculation efficiency.

[0068] After this, CPU also launched Galois Field New Instructions (GFNI) aiming at optimizing the calculation based on Galois field (finite field). For the erasure code based on finite field, directly using GFNI instruction set can greatly improve the calculation performance, and the calculation bandwidth is nearly doubled. For example, RS (Reed-Solomon) type erasure code, which is widely used in finite field, is a kind of encoding algorithm based on finite field, which uses the algebraic properties of finite field to realize the redundant encoding and checking of data, and protects the integrity and reliability of data.

[0069] The complex encoding and decoding operations required for the calculation of the erasure code will affect the performance of the distributed storage system. The calculation method of the erasure code in the related art makes some central processing units unable to use the highest level instruction set supported by the central processing units to perform the calculation of the erasure code, resulting in low calculation efficiency of the erasure code, high calculation pressure of the CPU and low performance.

[0070] The embodiment of the present application provides a data storage method. In the case of using a storage node to perform erasure code calculation, a storage node is randomly selected when receiving to-be-stored data, and the to-be-stored data is sent to the storage node. The highest level instruction set information supported by the central processing unit of the storage node is obtained, an encoding function matched with the instruction set information is obtained, the to-be-stored data is encoded based on the encoding function, the erasure code corresponding to the to-be-stored data is obtained, and the erasure code and the to-be-stored data are associated to achieve the effect of dynamically selecting the highest level instruction set information supported by the central processing unit of the storage node performing erasure calculation to reduce the calculation delay of the erasure code, improve the calculation efficiency of the erasure code, and improve the performance of the CPU.

[0071] According to the embodiment of the present application, a data storage method is provided. It should be noted that the steps shown in the flowchart can be executed in a computer system such as a group of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from here.

[0072] In this embodiment, a data storage method is provided, which can be used in a distributed storage system, Figure 1 The flowchart of the data storage method according to the embodiment of the present application is shown in Figure 1 As shown in the figure, the flowchart includes the following steps:

[0073] Step S101, in the case of using a storage node to perform erasure code calculation, a storage node is randomly selected when receiving to-be-stored data, and the to-be-stored data is sent to the storage node.

[0074] The storage node is used for erasure code calculation, and the application scenario is a non-kernel client file scenario. At this time, after the client receives the to-be-stored data, the to-be-stored data is sent to the distributed storage system. The distributed storage system randomly selects a storage node when receiving the to-be-stored data, and uses the storage node to calculate the erasure code of the to-be-stored data.

[0075] In step S102, the to-be-stored data is received by using the storage node, the highest level instruction set information supported by the central processing unit of the storage node is obtained, the encoding function matched with the instruction set information is obtained, and the to-be-stored data is encoded based on the encoding function to obtain the erasure code corresponding to the to-be-stored data.

[0076] The storage node receives the to-be-stored data and obtains the highest level instruction set information supported by the central processing unit.

[0077] It should be noted that different functions are packaged for different instruction sets for erasure code calculation of different instruction sets. Specifically, each instruction set corresponds to an encoding function and a decoding function. The encoding function and the decoding function corresponding to each instruction set are described below from low to high level of the instruction set. It can be understood that the higher the level of the instruction set, the higher the calculation performance.

[0078] SSE instruction set: encoding function ec_enc_sse, decoding function ec_dec_sse.

[0079] AVX instruction set: encoding function ec_enc_avx, decoding function ec_dec_avx.

[0080] AVX2 instruction set: encoding function ec_enc_avx2, decoding function ec_dec_avx2.

[0081] AVX2+GFNI instruction set: encoding function ec_enc_avx2_gfni, decoding function ec_dec_avx2_gfni.

[0082] AVX512 instruction set: encoding function ec_enc_avx512, decoding function ec_dec_avx512.

[0083] AVX512+GFNI instruction set: encoding function ec_enc_avx512_gfni, decoding function ec_dec_avx512_gfni.

[0084] After obtaining the highest level instruction set information, the encoding function matched with the instruction set information is obtained, that is, the encoding function corresponding to the instruction set information.

[0085] Wherein, initialization functions are established for the encoding function and the decoding function, that is, initialization encoding function ec_enc_init and initialization decoding function ec_dec_init, for dynamically selecting the optimal instruction set function when the storage node performs erasure code calculation for the first time, that is, for dynamically obtaining the encoding function and the decoding function corresponding to the highest level of instruction set information when the storage node performs erasure code calculation for the first time.

[0086] Specifically, after obtaining the encoding function and the decoding function corresponding to the highest level of instruction set information, the initialization encoding function is replaced by the encoding function corresponding to the highest level of instruction set information based on the address of the initialization encoding function and the address of the initialization decoding function, and the initialization decoding function is replaced by the decoding function corresponding to the highest level of instruction set information. Further, the encoding function corresponding to the highest level of instruction set information is used to encode the to-be-stored data, and the erasure code corresponding to the to-be-stored data is obtained.

[0087] Step S103: store the erasure code in association with the to-be-stored data.

[0088] Wherein, after obtaining the erasure code corresponding to the to-be-stored data, the distributed storage system disperses the erasure code and the corresponding to-be-stored data on multiple storage nodes by using the CRUSH algorithm under the scalable hashing, and saves the association between the erasure code and the corresponding to-be-stored data block.

[0089] The data storage method provided in the embodiment can dynamically select the highest level of instruction set information supported by the central processing unit, calculate the erasure code corresponding to the to-be-stored data by using the encoding function corresponding to the instruction set information, reduce the calculation delay of the erasure code, improve the calculation efficiency of the erasure code, reduce the usage rate of the CPU, and improve the calculation bandwidth.

[0090] In the embodiment, a data storage method is provided, which can be used in a distributed storage system, Figure 2 is a flowchart of the data storage method according to an embodiment of the present application, as shown in the figure, the flowchart includes the following steps: Figure 2

[0091] ​Step S201: When erasure code calculation is performed using storage nodes, a storage node is randomly selected whenever data to be stored is received, and the data to be stored is sent to the storage node. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0092] Step S202: Utilize the storage node to receive the data to be stored, obtain the highest level instruction set information supported by the central processing unit of the storage node, obtain the encoding function that matches the instruction set information, encode the data to be stored based on the encoding function, and obtain the erasure code corresponding to the data to be stored.

[0093] Specifically, encoding the data to be stored based on the encoding function in step S202 to obtain the erasure code corresponding to the data to be stored includes:

[0094] Step S2021: divide the data to be stored into several data blocks.

[0095] After receiving the data to be stored, the storage node divides the data to be stored into a number of data blocks of fixed size.

[0096] Step S2022: Encode the data blocks using an encoding function to generate a number of check blocks, wherein the erasure code includes the check blocks.

[0097] The plurality of data blocks obtained above are encoded using an encoding function corresponding to the highest level instruction set information supported by the CPU of the storage node to generate a plurality of check blocks.

[0098] Step S203: store the erasure code and the data to be stored in association with each other.

[0099] Specifically, the above step S203 includes:

[0100] Step S2031: store the check block and the data block in an associated manner.

[0101] After obtaining the check block, the distributed storage system stores the check block and several data blocks corresponding to the data to be stored in a dispersed manner to multiple storage nodes.

[0102] By assigning a unique identifier to each data block and each check block, a metadata table is created to record the association relationship between each data block and the check block, thereby realizing the associated storage of the check block and several data blocks.

[0103] The data storage method provided in this embodiment encodes the data to be stored using a coding function that matches the highest-level instruction set information to generate an erasure code. This improves the computational efficiency of the erasure code and reduces CPU usage. The associated storage of check blocks and data blocks enhances data storage security.

[0104] In some optional implementations, the data storage method further includes:

[0105] In step a1, when erasure code calculation is performed using a client, data to be stored and an erasure code corresponding to the data to be stored sent by the client are received, and the erasure code and the data to be stored are associated and stored.

[0106] Among them, the client receives the data to be stored, obtains the highest level instruction set information supported by the client's central processing unit, obtains the encoding function that matches the instruction set information, encodes the data to be stored based on the encoding function, obtains the erasure code corresponding to the data to be stored, and sends the erasure code and the data to be stored to the distributed storage system.

[0107] Erasure code calculation is performed on the client side, specifically in the kernel client file scenario. After receiving the data to be stored, the client calculates the erasure code for the data to be stored. After obtaining the erasure code, the erasure code and the data to be stored are sent to the distributed storage system for associated storage.

[0108] The data storage method provided in this embodiment obtains the highest-level instruction set information supported by the client's central processing unit (CPU), obtains a coding function that matches the instruction set information, and then, based on the coding function, obtains the erasure code corresponding to the data to be stored. This method adaptively adapts to the client's CPU's instruction set support and dynamically selects the highest-level instruction set information, thereby improving the efficiency of erasure code calculations, reducing CPU usage, and enhancing computing performance.

[0109] In some optional implementations, the data storage method further includes:

[0110] Step b1: When any data block in the distributed storage system fails, determine the storage node that calculates the erasure code corresponding to the data block.

[0111] The data block failure may be data block damage or data block loss.

[0112] The storage node for calculating the erasure code corresponding to the data block, that is, the storage node for calculating the erasure code of the data to be stored corresponding to the data block.

[0113] Step b2: Obtain a decoding function that matches the highest level instruction set information supported by the central processing unit of the storage node.

[0114] After the storage node is determined, a decoding function matching the highest level instruction set information supported by the CPU of the storage node is obtained.

[0115] Step b3: Obtain the remaining data blocks and check blocks associated with the data block.

[0116] Among them, in the event of a failure in any data block, the information of the remaining data blocks and the check block associated with the data block can be obtained by querying the metadata table, and then the remaining data blocks and check blocks associated with the data block can be obtained, which is essentially obtaining the data in the remaining data blocks and check blocks.

[0117] Step b4: input the data of the remaining data blocks and the check block into a decoding function to obtain the original data corresponding to the failed data block, so as to perform data recovery based on the original data.

[0118] After obtaining the data of the remaining data blocks and the check block associated with the failed data block, the data of the remaining data blocks and the check block are input into the decoding function to obtain the original data corresponding to the failed data block, and data recovery is performed based on the original data.

[0119] The data storage method provided in this embodiment significantly improves data recovery efficiency and system fault tolerance by using a decoding function that matches the highest-level instruction set supported by the storage node CPU, combined with a data recovery mechanism. It also reduces resource consumption, enhances system scalability and flexibility, and simplifies management and maintenance.

[0120] In some optional implementations, the data storage method further includes:

[0121] In step c1, when the randomly selected storage node is a storage node that has performed erasure code calculation, the encoding function corresponding to the storage node is directly used to encode the data to be stored to obtain the erasure code corresponding to the data to be stored.

[0122] Among them, the randomly selected storage node is a storage node that has performed erasure code calculation, which means that this is not the first time that the storage node has performed erasure code calculation. At this time, there is no need to execute the steps of obtaining the highest level instruction set information supported by the central processing unit of the storage node and obtaining the encoding function that matches the instruction set information. Based on the address of the initialized encoding function, the encoding function corresponding to the storage node is directly obtained, that is, the encoding function that matches the highest level instruction set information supported by the central processing unit of the storage node is obtained. The encoding function is used to encode the data to be stored to obtain the erasure code corresponding to the data to be stored.

[0123] It can be understood that in the case that the storage node is the first time to perform the erasure code calculation, the step of acquiring the highest level instruction set information supported by the central processor of the storage node, and acquiring the encoding function matched with the instruction set information is performed, at this time, the encoding function corresponding to the highest level instruction set information replaces the initial encoding function, and the decoding function corresponding to the highest level instruction set information replaces the initial decoding function.

[0124] In the case that the storage node is the second time and after to perform the erasure code calculation, since the initial encoding function and the initial decoding function have been replaced by the encoding function and the decoding function corresponding to the highest level instruction set information in the first time to perform the erasure code calculation, the subsequent erasure code calculation is directly performed by using the encoding function, and the data recovery is directly performed by using the decoding function, so that the step of acquiring the highest level instruction set information supported by the central processor of the storage node, and acquiring the encoding function matched with the instruction set information is not required to be performed each time the erasure code calculation is performed.

[0125] The data storage method provided by the embodiment can directly use the encoding function corresponding to the storage node which has performed the erasure code calculation to perform the encoding processing on the to-be-stored data, so as to obtain the erasure code corresponding to the to-be-stored data, significantly improve the encoding efficiency, eliminate the overhead caused by automatically selecting the highest level instruction set information of the storage node each time, improve the performance of the erasure code calculation, and reduce the calculation cost.

[0126] In some optional embodiments, the step S202 further includes:

[0127] Step d1, acquiring the configuration information of the central processor of the storage node.

[0128] The configuration information of the central processor of the storage node is acquired by using a CPU identification (CPUID) instruction.

[0129] It should be noted that the CPUID is a supplementary command for X86 architecture processor, and mainly functions to allow software to discover detailed information of the central processor and identify the capability of the target machine, such as the model, whether the AVX instruction set or the AVX512 instruction set is supported, and the like. The information of the central processor is stored in the EAX, EBX, ECX, and EDX registers, and the returned information is determined according to the values of EAX and ECX.

[0130] Step d2, determining the highest level instruction set information supported by the central processor based on the configuration information.

[0131] After the configuration information of the CPU is acquired, the highest level instruction set information supported by the central processor is determined according to the configuration information.

[0132] The data storage method provided in this embodiment can significantly improve the performance of erasure correction calculations by determining and using the highest-level instruction set information supported by the central processing unit, thereby improving the efficiency of obtaining erasure codes.

[0133] In some optional implementations, the above step d2 includes:

[0134] Step d21: Based on the configuration information, determine whether the central processing unit supports both the finite field instruction set and the first vector extension instruction set.

[0135] Figure 3 FIG. 1 is a flow chart of determining the highest level of instruction set information supported by a central processing unit according to an embodiment of the present invention. Figure 3 As shown, after obtaining the CPU configuration information, ie, the instruction set information supported by the CPU, by using the CPUID command, it is determined whether the central processing unit supports both the finite field instruction set and the first vector extension instruction set based on the configuration information.

[0136] Among them, the finite field instruction set is the GFNI instruction set, and the first vector extension instruction set is the AVX512 instruction set.

[0137] Step d22: When the central processing unit supports both the finite field instruction set and the first vector extension instruction set, determine that the highest level of instruction set information supported by the central processing unit is the finite field instruction set and the first vector extension instruction set.

[0138] If the CPU supports both the GFNI and AVX512 instruction sets, the highest level of instruction set information supported by the CPU is determined to be the GFNI and AVX512 instruction sets. The encoding and decoding functions that match the instruction set information are selected, namely, ec_enc_avx512_gfni and ec_dec_avx512_gfni. The init address is replaced with the selected function, that is, ec_enc_init of the storage node is replaced with ec_enc_avx512_gfni, and ec_dec_init is replaced with ec_dec_avx512_gfni.

[0139] Step d23: If the central processing unit does not support both the finite field instruction set and the first vector extension instruction set, determine whether the central processing unit supports the first vector extension instruction set.

[0140] If the central processing unit does not support both the GFNI instruction set and the AVX512 instruction set, it is determined whether the CPU supports the AVX512 instruction set.

[0141] Step d24, in the case that the central processor supports the first vector extension instruction set, determining that the highest level of instruction set information supported by the central processor is the first vector extension instruction set.

[0142] In the case that the CPU supports the AVX512 instruction set, it is determined that the highest level of instruction set information supported by the central processor is the AVX512 instruction set. The encoding function and the decoding function matching the instruction set information, i.e. ec_enc_avx512 and ec_dec_avx512, are selected, and the init address is replaced by the selected function, i.e. ec_enc_init of the storage node is replaced by ec_enc_avx512, and ec_dec_init is replaced by ec_dec_avx512.

[0143] Step d25, in the case that the central processor does not support the first vector extension instruction set, determining whether the central processor supports the finite field instruction set and the second vector extension instruction set at the same time, wherein the operation bit number of the second vector extension instruction set, i.e. the operation bit width, is less than the operation bit number of the first vector extension instruction set.

[0144] In the case that the CPU does not support the AVX512 instruction set, it is determined whether the CPU supports the GFNI instruction set and the AVX2 instruction set at the same time, i.e. the second vector extension instruction set is the AVX2 instruction set.

[0145] Step d26, in the case that the central processor supports the finite field instruction set and the second vector extension instruction set at the same time, determining that the highest level of instruction set information supported by the central processor is the finite field instruction set and the second vector extension instruction set.

[0146] In the case that the CPU supports the GFNI instruction set and the AVX2 instruction set at the same time, it is determined that the highest level of instruction set information supported by the central processor is the GFNI instruction set and the AVX2 instruction set. The encoding function and the decoding function matching the instruction set information, i.e. ec_enc_avx2_gfni and ec_dec_avx2_gfni, are selected, and the init address is replaced by the selected function, i.e. ec_enc_init of the storage node is replaced by ec_enc_avx2_gfni, and ec_dec_init is replaced by ec_dec_avx2_gfni.

[0147] Step d27, in the case that the central processor does not support the finite field instruction set and the second vector extension instruction set at the same time, determining whether the central processor supports the second vector extension instruction set.

[0148] In the case that the central processor does not support the GFNI instruction set and the AVX2 instruction set at the same time, it is determined whether the CPU supports the AVX2 instruction set.

[0149] Step d28, in the case that the central processor supports the second vector extension instruction set, determining that the highest level of instruction set information supported by the central processor is the second vector extension instruction set.

[0150] In the case that the CPU supports the AVX2 instruction set, it is determined that the highest level of instruction set information supported by the central processor is the AVX2 instruction set. The encoding function and the decoding function matching the instruction set information are selected, i.e. ec_enc_avx2 and ec_dec_avx2, and the init address is replaced by the selected function, i.e. ec_enc_init of the storage node is replaced by ec_enc_avx2 and ec_dec_init is replaced by ec_dec_avx2.

[0151] Step d29, in the case that the central processor does not support the second vector extension instruction set, determining whether the central processor supports a third vector extension instruction set, wherein the second vector extension instruction set is an extended version of the third vector extension instruction set.

[0152] In the case that the CPU does not support the AVX2 instruction set, it is determined whether the CPU supports the AVX instruction set, i.e. the third vector extension instruction set is the AVX instruction set.

[0153] Step d210, in the case that the central processor supports the third vector extension instruction set, determining that the highest level of instruction set information supported by the central processor is the third vector extension instruction set.

[0154] In the case that the CPU supports the AVX instruction set, it is determined that the highest level of instruction set information supported by the central processor is the AVX instruction set. The encoding function and the decoding function matching the instruction set information are selected, i.e. ec_enc_avx and ec_dec_avx, and the init address is replaced by the selected function, i.e. ec_enc_init of the storage node is replaced by ec_enc_avx and ec_dec_init is replaced by ec_dec_avx.

[0155] It can be understood that, in the case that the central processor does not support the third vector extension instruction set, it is determined whether the central processor supports a streaming extension instruction set.

[0156] In the case that the streaming extension instruction set is the SSE instruction set, if the CPU supports the SSE instruction set, it is determined that the highest level of instruction set information supported by the central processor is the SSE instruction set. The encoding function and the decoding function matching the instruction set information are selected, i.e. ec_enc_sse and ec_dec_sse, and the init address is replaced by the selected function, i.e. ec_enc_init of the storage node is replaced by ec_enc_sse and ec_dec_init is replaced by ec_dec_sse.

[0157] The data storage method provided in the embodiment can determine the highest level of instruction set supported by the CPU by judging whether the CPU supports a specific instruction set, so as to perform erasure coding calculation on the to-be-stored data by using the highest level of instruction set supported by the CPU, and obtain the erasure code. The execution efficiency of the erasure coding calculation is improved, and the processing capability of the CPU is maximized.

[0158] In some optional embodiments, a storage node is randomly selected whenever the to-be-stored data is received, and the method comprises the following steps of:

[0159] In step e1, load information of each storage node in the distributed storage system is obtained.

[0160] In the embodiment, the load information of each storage node in the distributed storage system is obtained whenever the to-be-stored data is received.

[0161] The load information can be CPU usage, memory usage, etc.

[0162] In step e2, a storage node with the lowest load is selected based on the load information of each storage node.

[0163] In the embodiment, after the load information of each storage node is obtained, a storage node with the lowest load is selected based on the load information of each storage node, and the erasure code is calculated by using the storage node with the lowest load. For example, a storage node with the lowest CPU usage is selected to calculate the erasure code.

[0164] The data storage method provided in the embodiment can avoid further overloading of the storage node with high load by assigning the task of calculating the erasure code to the storage node with low current load, so as to improve the response speed and processing capability of the entire system, improve the calculation rate of the erasure code, and more effectively utilize the resources in the cluster.

[0165] In the embodiment, a data storage method is also provided, Figure 4 is a flowchart of the data storage method according to the embodiment of the present application. As shown in Figure 4 The data storage method comprises the following steps:

[0166] Firstly, the distributed storage system receives to-be-stored data, and randomly selects a storage node to perform erasure coding, i.e., to perform erasure code calculation. ec_enc_data is the erasure code calculation on the to-be-stored data.

[0167] The embodiment is directed to the encoding function and decoding function of the erasure code calculation, and the distribution functions ec_enc_disp and ec_dec_disp are respectively established. The distribution function is used to dynamically select the highest level instruction set information of the central processor of the storage node. Wherein, the address of the distribution function is the init function, i.e. the initialization function of the storage node.

[0168] Secondly, in the case that the storage node is used to calculate the erasure code for the first time, the distribution function executes the init function, i.e. ec_enc_init, to achieve the dynamic selection of the instruction set, i.e. to obtain the highest level instruction set information supported by the central processor of the storage node.

[0169] Specifically, the CPU instruction set information, i.e. the instruction set information supported by the CPU, is obtained by executing the CPUID command. The optimal instruction set function of the CPU is obtained by comparing the highest level instruction set information, i.e. by determining the highest level instruction set information through the above steps d21 to d210, and then obtaining the encoding function and decoding function corresponding to the instruction set information.

[0170] Thirdly, the distribution function replaces the init address with the optimal instruction set function. That is, the initialization function of the storage node is replaced with the encoding function and decoding function corresponding to the highest level instruction set information.

[0171] Fourthly, the optimal instruction set function is used to calculate the erasure code, i.e. the encoding function corresponding to the highest level instruction set information is used to calculate the erasure code, and the erasure code is returned to the distributed storage system, so that the distributed storage system associates the storage data to be stored with the erasure code.

[0172] Fifthly, in the case that the storage node is used to calculate the erasure code for the second time or later, the optimal instruction set function selected by the init function is directly used to calculate the erasure code.

[0173] That is, the storage node only needs to select the instruction set when calculating the erasure code for the first time. Once the optimal instruction set function is selected, the address of the init function is replaced, and the selected optimal instruction set function is directly used in the subsequent calculation without the need to select it every time.

[0174] The data storage method provided in the embodiment dynamically selects an optimal instruction set, i.e., a highest-level instruction set, based on an instruction set supported by a central processing unit performing erasure code calculation, calculates an erasure code corresponding to the to-be-stored data by using a corresponding encoding function of the instruction set, reduces the calculation time delay of the erasure code, improves the calculation efficiency of the erasure code, and improves the performance. By recording the address of the function corresponding to the selected optimal instruction set, the function corresponding to the optimal instruction set selected for the first time is directly used for subsequent encoding and decoding requests, unnecessary selection and jump are avoided, unnecessary judgment and jump overheads are reduced, the optimal instruction set is used for each calculation of the erasure code, and the calculation efficiency of the erasure code is improved.

[0175] It should be noted that the data storage method of the embodiment is applicable to a scenario of calculating an erasure code in a storage node, such as a NAS, RGW, or RBD scenario, and is also applicable to a scenario of calculating an erasure code in a client. By using the erasure code calculation manner of the embodiment, compared with the erasure code calculation manner using an AVX2 instruction set in the related art, the CPU usage rate is reduced by 20%, the calculation time delay is reduced by 15%, and the calculation bandwidth is increased by 2 times.

[0176] In the embodiment, a data storage device is also provided, which is used to implement the above-described embodiments and preferred embodiments, and will not be described again. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, implementation of hardware or a combination of software and hardware is also possible and contemplated.

[0177] The embodiment provides a data storage device, as shown in the following Figure 5 The data storage device comprises the following modules.

[0178] The sending module 501 is configured to, in the case of calculating an erasure code by using a storage node, randomly select a storage node each time the to-be-stored data is received, and send the to-be-stored data to the storage node.

[0179] The first obtaining module 502 is configured to receive the to-be-stored data by using the storage node, obtain the highest-level instruction set information supported by the central processing unit of the storage node, obtain an encoding function matched with the instruction set information, perform encoding processing on the to-be-stored data based on the encoding function, and obtain the erasure code corresponding to the to-be-stored data.

[0180] The storage module 503 is configured to store the erasure code and the to-be-stored data in association.

[0181] In some optional embodiments, the data storage device further comprises:

[0182] The receiving module is configured to receive the to-be-stored data and the erasure code corresponding to the to-be-stored data sent by the client in the case where the client performs erasure code calculation, and store the erasure code and the to-be-stored data in association.

[0183] The client receives the to-be-stored data, obtains the highest-level instruction set information supported by a central processing unit of the client, obtains an encoding function matched with the instruction set information, performs encoding processing on the to-be-stored data based on the encoding function, obtains the erasure code corresponding to the to-be-stored data, and sends the erasure code and the to-be-stored data to the distributed storage system.

[0184] In some optional embodiments, the first obtaining module 502 includes:

[0185] The splitting unit is configured to split the to-be-stored data into a plurality of data blocks.

[0186] The generating unit is configured to perform encoding processing on the plurality of data blocks by using the encoding function to generate a plurality of check blocks, wherein the erasure code includes the check blocks.

[0187] The storage module 503 includes:

[0188] The storage unit is configured to store the check blocks and the data blocks in association.

[0189] In some optional embodiments, the data storage device further includes:

[0190] The determining module is configured to determine a storage node that calculates the erasure code corresponding to any data block in the distributed storage system in the case where the data block fails.

[0191] The second obtaining module is configured to obtain a decoding function matched with the highest-level instruction set information supported by a central processing unit of the storage node.

[0192] The third obtaining module is configured to obtain the remaining data blocks and the check blocks associated with the data block.

[0193] The fourth obtaining module is configured to input data of the remaining data blocks and the check blocks into the decoding function to obtain original data corresponding to the failed data block, so as to perform data recovery based on the original data.

[0194] In some optional embodiments, the data storage device further includes:

[0195] The encoding module is configured to, in the case where the randomly selected storage node is a storage node that has performed erasure code calculation, directly perform encoding processing on the to-be-stored data by using the encoding function corresponding to the storage node to obtain the erasure code corresponding to the to-be-stored data.

[0196] In some optional embodiments, the first obtaining module 502 includes:

[0197] The first obtaining unit is configured to obtain configuration information of a central processing unit of the storage node.

[0198] The first determining unit is configured to determine, based on the configuration information, highest-level instruction set information supported by the central processing unit.

[0199] In some optional embodiments, the first determining unit includes:

[0200] The first judging unit is configured to judge, based on the configuration information, whether the central processing unit supports a finite field instruction set and a first vector extension instruction set simultaneously.

[0201] The second determining unit is configured to determine, in a case where the central processing unit supports the finite field instruction set and the first vector extension instruction set simultaneously, that the highest-level instruction set information supported by the central processing unit is the finite field instruction set and the first vector extension instruction set.

[0202] The second judging unit is configured to judge, in a case where the central processing unit does not support the finite field instruction set and the first vector extension instruction set simultaneously, whether the central processing unit supports the first vector extension instruction set.

[0203] The third determining unit is configured to determine, in a case where the central processing unit supports the first vector extension instruction set, that the highest-level instruction set information supported by the central processing unit is the first vector extension instruction set.

[0204] The third judging unit is configured to judge, in a case where the central processing unit does not support the first vector extension instruction set, whether the central processing unit supports a finite field instruction set and a second vector extension instruction set simultaneously, wherein an operation bit number of the second vector extension instruction set is less than an operation bit number of the first vector extension instruction set.

[0205] The fourth determining unit is configured to determine, in a case where the central processing unit supports the finite field instruction set and the second vector extension instruction set simultaneously, that the highest-level instruction set information supported by the central processing unit is the finite field instruction set and the second vector extension instruction set.

[0206] The fourth judging unit is configured to judge, in a case where the central processing unit does not support the finite field instruction set and the second vector extension instruction set simultaneously, whether the central processing unit supports the second vector extension instruction set.

[0207] The fifth determining unit is configured to determine, in a case where the central processing unit supports the second vector extension instruction set, that the highest-level instruction set information supported by the central processing unit is the second vector extension instruction set.

[0208] The fifth determining unit is configured to determine whether the central processing unit supports a third vector extension instruction set when the central processing unit does not support the second vector extension instruction set, wherein the second vector extension instruction set is an extended version of the third vector extension instruction set.

[0209] The sixth determining unit is configured to determine, when the central processing unit supports the third vector extension instruction set, that the highest level of instruction set information supported by the central processing unit is the third vector extension instruction set.

[0210] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0211] The data storage device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0212] The embodiment of the present invention also provides a computer device having the above Figure 5 The data storage device shown.

[0213] See also Figure 6 , Figure 6 is a structural diagram of a computer device provided by an optional embodiment of the present invention, such as Figure 6 As shown, the computer device includes: one or more processors 601, memory 602, and the interface for connecting each component, including a high-speed interface and a low-speed interface. Each component utilizes different buses to communicate with each other and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instruction executed in the computer device, including being stored in the memory or on the memory to display the graphic information of the GUI on an external input / output device (such as, being coupled to the display device of the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Equally, multiple computer devices can be connected, and each device provides the necessary operation of part (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 601 is taken as an example.

[0214] The processor 601 can be a central processor, a network processor, or a combination thereof. The processor 601 can further include a hardware chip. The hardware chip can be an application specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device can be a complex programmable logic device, a field programmable logic device, a general array logic, or any combination thereof.

[0215] The memory 602 stores instructions executable by the at least one processor 601 to cause the at least one processor 601 to perform the methods illustrated in the above embodiments.

[0216] The memory 602 can include a program storage area and a data storage area. The program storage area can store an operating system, application programs required by at least one function, and the like. The data storage area can store data created according to the use of the computer device, and the like. In addition, the memory 602 can include a high-speed random access memory, and can further include a non-transitory memory such as at least one disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some optional embodiments, the memory 602 can optionally include a memory disposed remotely from the processor 601, and these remote memories can be connected to the computer device through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0217] The memory 602 can include a volatile memory such as a random access memory, and can also include a non-volatile memory such as a flash memory, a hard disk, or a solid state disk. The memory 602 can further include a combination of the above-mentioned types of memories.

[0218] The computer device further includes a communication interface 603 for communication of the computer device with other devices or communication networks.

[0219] The embodiment of the present invention also provides a computer-readable storage medium. The above-mentioned method according to the embodiment of the present invention can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0220] A portion of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the form in which the computer program instruction exists in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc. Accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium that can be accessed by the computer.

[0221] Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A data storage method, characterized in that: The method comprises: When erasure code calculation is performed using storage nodes, whenever data to be stored is received, a storage node is randomly selected and the data to be stored is sent to the storage node; Utilizing the storage node to receive the data to be stored, obtaining the highest-level instruction set information supported by the central processing unit of the storage node, obtaining a coding function that matches the instruction set information, encoding the data to be stored based on the coding function, and obtaining an erasure code corresponding to the data to be stored; The erasure code and the data to be stored are stored in association with each other.

2. The method according to claim 1, characterized in that The method further comprises: In the case of using the client to perform erasure code calculation, receiving data to be stored and an erasure code corresponding to the data to be stored sent by the client, and associating the erasure code with the data to be stored; Among them, the client receives the data to be stored, obtains the highest level instruction set information supported by the client's central processing unit, obtains the encoding function that matches the instruction set information, encodes the data to be stored based on the encoding function, obtains the erasure code corresponding to the data to be stored, and sends the erasure code and the data to be stored to the distributed storage system.

3. The method according to claim 1, characterized in that The encoding process of the data to be stored based on the encoding function to obtain an erasure code corresponding to the data to be stored includes: dividing the data to be stored into a plurality of data blocks; encoding the plurality of data blocks using the encoding function to generate a plurality of check blocks, wherein the erasure code includes the check blocks; The associative storage of the erasure code and the data to be stored includes: The check block and the data block are stored in association.

4. The method according to claim 3, characterized in that The method further comprises: In the event of a failure in any data block in the distributed storage system, determine the storage node that calculates the erasure code corresponding to the data block; Acquire a decoding function that matches the highest level of instruction set information supported by the central processing unit of the storage node; Obtain the remaining data blocks and check blocks associated with the data block; The data of the remaining data blocks and the check block are input into the decoding function to obtain the original data corresponding to the failed data block, so as to perform data recovery based on the original data.

5. The method according to claim 1, wherein The method further comprises: In the case that the randomly selected storage node is a storage node that has performed erasure code calculation, the data to be stored is directly encoded using the encoding function corresponding to the storage node to obtain the erasure code corresponding to the data to be stored.

6. The method according to claim 1, characterized in that The obtaining of the highest level instruction set information supported by the central processing unit of the storage node includes: Obtaining configuration information of the central processing unit of the storage node; Based on the configuration information, the highest level instruction set information supported by the central processing unit is determined.

7. The method according to claim 6, characterized in that The determining, based on the configuration information, the highest level instruction set information supported by the central processing unit includes: Based on the configuration information, determining whether the central processing unit supports both a finite field instruction set and a first vector extension instruction set; In a case where the central processing unit supports both a finite field instruction set and a first vector extension instruction set, determining that the highest level of instruction set information supported by the central processing unit is the finite field instruction set and the first vector extension instruction set; In a case where the central processing unit does not support both the finite field instruction set and the first vector extension instruction set, determining whether the central processing unit supports the first vector extension instruction set; In a case where the central processing unit supports a first vector extension instruction set, determining that the highest level of instruction set information supported by the central processing unit is the first vector extension instruction set; If the central processing unit does not support the first vector extension instruction set, determining whether the central processing unit supports both a finite field instruction set and a second vector extension instruction set, wherein the number of operands of the second vector extension instruction set is smaller than that of the first vector extension instruction set; In a case where the central processing unit supports both a finite field instruction set and a second vector extension instruction set, determining that the highest level of instruction set information supported by the central processing unit is the finite field instruction set and the second vector extension instruction set; In a case where the central processing unit does not support the finite field instruction set and the second vector extension instruction set at the same time, determining whether the central processing unit supports the second vector extension instruction set; In a case where the central processing unit supports a second vector extension instruction set, determining that the highest level of instruction set information supported by the central processing unit is the second vector extension instruction set; If the central processing unit does not support a second vector extension instruction set, determining whether the central processing unit supports a third vector extension instruction set, wherein the second vector extension instruction set is an extended version of the third vector extension instruction set; In a case where the central processing unit supports the third vector extension instruction set, it is determined that the highest level of instruction set information supported by the central processing unit is the third vector extension instruction set.

8. A data storage device, characterized in that The device comprises: a sending module, configured to randomly select a storage node whenever receiving data to be stored, and send the data to the storage node when erasure code calculation is performed using the storage node; a first acquisition module, configured to receive the data to be stored using the storage node, obtain the highest-level instruction set information supported by the central processing unit of the storage node, obtain a coding function matching the instruction set information, encode the data to be stored based on the coding function, and obtain an erasure code corresponding to the data to be stored; A storage module is used to store the erasure code and the data to be stored in an associated manner.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the data storage method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the data storage method according to any one of claims 1 to 7.

Images