Data storage method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of distributed storage, and discloses a data storage method and device, computer equipment and a storage medium, and the method comprises the steps: when to-be-stored data is received under the condition that the storage nodes are used for carrying out erasure code calculation, randomly screening out one storage node, and sending the to-be-stored data to the storage node; the method comprises the following steps: receiving to-be-stored data by utilizing a storage node, acquiring instruction set information of the highest level supported by a central processing unit of the storage node, acquiring a coding function matched with the instruction set information, and coding the to-be-stored data based on the coding function to obtain an erasure code corresponding to the to-be-stored data; and associatively storing the erasure code and the to-be-stored data. According to the method and the device, the instruction set information of the highest level supported by the central processing unit is adaptively selected, the coding function corresponding to the instruction set information is utilized to perform erasure calculation, the erasure code is obtained, and the calculation efficiency of the erasure code is improved.

Description

Technical Field

[0001] The present invention relates to the field of distributed storage technology, and in particular to a data storage method, device, computer equipment and storage medium. Background Art

[0002] In distributed storage systems, in order to ensure data reliability and availability, redundancy strategies are usually used to store data. Among them, erasure redundancy strategies are widely used because of their advantage of reducing storage space consumption.

[0003] In the related art, the erasure redundancy strategy is to divide the data into multiple data fragments, generate redundant data blocks (i.e., erasure codes) based on the multiple data fragments using an encoding method, and store the multiple data fragments and redundant data blocks in different locations of the distributed storage system.

[0004] Executing the erasure redundancy strategy on the storage node is essentially to use the instruction set supported by the central processing unit (CPU) inside the storage node to calculate the erasure code. Since different CPUs support different instruction sets, the highest level instruction set among the instruction sets supported by different CPUs is often used to calculate the erasure code.

[0005] However, this erasure code calculation method makes it impossible for some central processing units to use the highest level of instruction sets they support, resulting in low calculation efficiency of erasure codes. Summary of the invention

[0006] In view of this, the present invention provides a data storage method, apparatus, computer device and storage medium to solve the problem in the related art of using the highest level instruction set in the instruction set supported by different central processing units to calculate erasure codes, resulting in low calculation efficiency of erasure codes.

[0007] In a first aspect, the present invention provides a data storage method, the method comprising:

[0008] In the case of using storage nodes to perform erasure code calculation, whenever data to be stored is received, a storage node is randomly selected and the data to be stored is sent to the storage node;

[0009] Utilizing the storage node to receive the data to be stored, obtaining the highest level instruction set information supported by the central processing unit of the storage node, obtaining a coding function matching the instruction set information, encoding the data to be stored based on the coding function, and obtaining an erasure code corresponding to the data to be stored;

[0010] The erasure code and the data to be stored are stored in association with each other.

[0011] The data storage method provided in the present embodiment randomly selects a storage node whenever receiving data to be stored by performing erasure code calculation using a storage node, and sends the data to be stored to the storage node, obtains the highest level instruction set information supported by the central processing unit of the storage node, obtains a coding function matching the instruction set information, encodes the data to be stored based on the coding function, obtains the erasure code corresponding to the data to be stored, associates and stores the erasure code with the data to be stored, can dynamically select the highest level instruction set information supported by the central processing unit, and calculates the erasure code corresponding to the data to be stored by using the coding function corresponding to the instruction set information, thereby reducing the calculation delay of the erasure code, improving the calculation efficiency of the erasure code, reducing the CPU usage, and improving the calculation bandwidth.

[0012] In an optional embodiment, the method further includes:

[0013] In the case of using the client to perform erasure code calculation, receiving the data to be stored and the erasure code corresponding to the data to be stored sent by the client, and associating the erasure code with the data to be stored;

[0014] Among them, the client receives the data to be stored, obtains the highest level instruction set information supported by the client's central processing unit, obtains a coding function matching the instruction set information, encodes the data to be stored based on the coding function, obtains the erasure code corresponding to the data to be stored, and sends the erasure code and the data to be stored to a distributed storage system.

[0015] The data storage method provided in this embodiment obtains the highest level of instruction set information supported by the client's central processing unit when using the client to perform erasure code calculation, obtains the encoding function matching the instruction set information, and obtains the erasure code corresponding to the data to be stored based on the encoding function. This realizes the adaptive support of the instruction set of the client's CPU, dynamically selects the highest level of instruction set information, improves the calculation efficiency of the erasure code, reduces the CPU usage, and improves the calculation performance.

[0016] In an optional implementation, encoding the data to be stored based on the encoding function to obtain an erasure code corresponding to the data to be stored includes:

[0017] Dividing the data to be stored into a plurality of data blocks;

[0018] Using the encoding function to encode the plurality of data blocks to generate a plurality of check blocks, wherein the erasure code includes the check blocks;

[0019] The associative storage of the erasure code and the data to be stored includes:

[0020] The check block and the data block are stored in association.

[0021] The data storage method provided in this embodiment encodes the data to be stored based on the encoding function matching the highest level instruction set information to obtain the erasure code, thereby improving the computational efficiency of the erasure code and reducing the CPU usage. The check block and the data block are stored in an associated manner to improve the security of data storage.

[0022] In an optional embodiment, the method further includes:

[0023] In the event of a failure of any data block in the distributed storage system, determine the storage node that calculates the erasure code corresponding to the data block;

[0024] Acquire a decoding function that matches the highest level of instruction set information supported by the central processing unit of the storage node;

[0025] Obtaining the remaining data blocks and check blocks associated with the data block;

[0026] The data of the remaining data blocks and the check block are input into the decoding function to obtain the original data corresponding to the failed data block, so as to perform data recovery based on the original data.

[0027] The data storage method provided in this embodiment significantly improves the efficiency of data recovery and the fault tolerance of the system by using a decoding function that matches the highest level instruction set supported by the storage node CPU and combining it with a data recovery mechanism. At the same time, it also reduces resource consumption, enhances the scalability and flexibility of the system, and simplifies management and maintenance.

[0028] In an optional embodiment, the method further includes:

[0029] In the case where the randomly selected storage node is a storage node that has performed erasure code calculation, the data to be stored is directly encoded using the encoding function corresponding to the storage node to obtain the erasure code corresponding to the data to be stored.

[0030] The data storage method provided in this embodiment encodes the data to be stored by directly using the encoding function corresponding to the storage node that has performed erasure code calculation, and obtains the erasure code corresponding to the data to be stored, thereby significantly improving the encoding efficiency, eliminating the overhead caused by automatically selecting the highest level instruction set information of the storage node each time, improving the performance of erasure calculation, and reducing the computing cost.

[0031] In an optional implementation manner, the obtaining the highest level instruction set information supported by the central processing unit of the storage node includes:

[0032] Obtaining configuration information of a central processing unit of the storage node;

[0033] Based on the configuration information, the highest level of instruction set information supported by the central processing unit is determined.

[0034] The data storage method provided in this embodiment can significantly improve the performance of erasure calculation by determining and using the highest level instruction set information supported by the central processing unit, thereby improving the efficiency of obtaining erasure codes.

[0035] In an optional implementation, determining the highest level of instruction set information supported by the central processing unit based on the configuration information includes:

[0036] Based on the configuration information, determining whether the central processing unit supports both a finite field instruction set and a first vector extension instruction set;

[0037] In the case where the central processing unit supports both a finite field instruction set and a first vector extension instruction set, determining that the highest level of instruction set information supported by the central processing unit is the finite field instruction set and the first vector extension instruction set;

[0038] In a case where the central processing unit does not support both the finite field instruction set and the first vector extension instruction set, determining whether the central processing unit supports the first vector extension instruction set;

[0039] In a case where the central processing unit supports a first vector extension instruction set, determining that the highest level of instruction set information supported by the central processing unit is the first vector extension instruction set;

[0040] In a case where the central processing unit does not support the first vector extension instruction set, determining whether the central processing unit supports both a finite field instruction set and a second vector extension instruction set, wherein the number of operands of the second vector extension instruction set is smaller than the number of operands of the first vector extension instruction set;

[0041] In the case where the central processing unit supports both a finite field instruction set and a second vector extension instruction set, determining that the highest level of instruction set information supported by the central processing unit is the finite field instruction set and the second vector extension instruction set;

[0042] In a case where the central processing unit does not support the finite field instruction set and the second vector extension instruction set at the same time, determining whether the central processing unit supports the second vector extension instruction set;

[0043] In a case where the central processing unit supports a second vector extension instruction set, determining that the highest level of instruction set information supported by the central processing unit is the second vector extension instruction set;

[0044] In a case where the central processing unit does not support a second vector extension instruction set, determining whether the central processing unit supports a third vector extension instruction set, wherein the second vector extension instruction set is an extended version of the third vector extension instruction set;

[0045] In a case where the central processing unit supports the third vector extension instruction set, it is determined that the highest level of instruction set information supported by the central processing unit is the third vector extension instruction set.

[0046] The data storage method provided in this embodiment can determine the highest level instruction set supported by the CPU by judging whether the CPU supports a specific instruction set, so as to use the highest level instruction set supported by the CPU to perform erasure calculation on the data to be stored and obtain an erasure code. This not only improves the execution efficiency of erasure calculation, but also maximizes the processing power of the CPU.

[0047] In a second aspect, the present invention provides a data storage device, the device comprising:

[0048] A sending module, for randomly selecting a storage node whenever receiving data to be stored, and sending the data to be stored to the storage node when using the storage node to perform erasure code calculation;

[0049] A first acquisition module is used to receive the data to be stored by using the storage node, obtain the highest level instruction set information supported by the central processing unit of the storage node, obtain a coding function matching the instruction set information, encode the data to be stored based on the coding function, and obtain an erasure code corresponding to the data to be stored;

[0050] A storage module is used to store the erasure code and the data to be stored in an associated manner.

[0051] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the data storage method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0052] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the data storage method of the first aspect or any corresponding embodiment thereof.

[0053] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the data storage method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0055] Figure 1 is a schematic diagram of a flow chart of a data storage method according to an embodiment of the present invention;

[0056] Figure 2 is a flow chart of another data storage method according to an embodiment of the present invention;

[0057] Figure 3 is a schematic diagram of a flow chart of determining the highest level of instruction set information supported by a central processing unit according to an embodiment of the present invention;

[0058] Figure 4 is a flow chart of another data storage method according to an embodiment of the present invention;

[0059] Figure 5 is a structural block diagram of a data storage device according to an embodiment of the present invention;

[0060] Figure 6 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0062] In distributed storage systems, in order to ensure data reliability and availability, a redundancy strategy is usually used to store data. This strategy can ensure the normal operation of the system and secure access to data in the event of hardware failures, network failures, etc.

[0063] There are two redundancy strategies in distributed storage systems: one is the copy redundancy strategy and the other is the erasure redundancy strategy. Among them, the copy redundancy strategy consumes more storage space, and the erasure redundancy strategy effectively reduces the consumption of storage space and reduces the transmission bandwidth between storage nodes. Therefore, the erasure redundancy strategy is widely used in distributed storage systems.

[0064] In the related art, the erasure redundancy strategy is to divide the data into multiple data fragments, generate redundant data blocks (i.e., erasure codes) based on the multiple data fragments using an encoding method, and store the multiple data fragments and redundant data blocks in different locations of the distributed storage system, such as disks, storage nodes, or other locations.

[0065] In the non-kernel client file scenario, using the storage node to execute the erasure redundancy strategy is essentially to use the instruction set supported by its internal CPU to calculate the erasure code. Since different central processing units support different instruction sets, the highest level instruction set supported by different central processing units is often used to calculate the erasure code. Generally, the extended version of the Advanced Vector Extensions 2 (AVX2) instruction set is used to calculate the erasure code.

[0066] In the kernel client file scenario, after the erasure code calculation is promoted from the server to the client, the client is used to execute the erasure redundancy strategy, which is essentially to use the instruction set supported by its internal CPU to calculate the erasure code. The CPU models used by the client vary, and the erasure code calculation is often performed using the highest level instruction set in the instruction set supported by the CPUs of different clients. Generally, the extended version of the Advanced Vector Extensions 2 (AVX2) instruction set is used to calculate the erasure code.

[0067] Distributed storage systems select different CPUs for different scenarios, and new instruction sets are introduced with the upgrading of CPUs. For example, the CPU of the X86 platform has introduced new Single Instruction, Multiple Data (SIMD) instruction sets, and has successively introduced Streaming SIMD Extensions (SSE), Advanced Vector Extensions (AVX), AVX2, and Advanced Vector Extensions 512 (AVX512), and the operation bit width has been expanded from 128 bits to 256 bits and then to 512 bits. Among them, the operation bit width of the SSE instruction set is 128 bits, that is, 128 bits per operation. The operation bit width of the AVX instruction set and the AVX2 instruction set is 256 bits, that is, 256 bits per operation. The operation bit width of the AVX512 instruction set is 512 bits, that is, 512 bits per operation. Among them, the SIMD instruction set allows the processor to perform the same operation on multiple data in a single instruction cycle, thereby significantly improving computing efficiency.

[0068] After that, the CPU also launched the Galois Field New Instructions (GFNI), which aims to optimize calculations based on Galois fields (finite fields). For erasure algorithms based on finite fields, directly using the GFNI instruction set can greatly improve computing performance, and the computing bandwidth has been nearly doubled. For example, RS (Reed-Solomon) erasure codes widely use finite fields. It is a coding algorithm based on finite fields, which uses the algebraic properties of finite fields to achieve redundant coding and verification of data, and protect the integrity and reliability of data.

[0069] Calculating erasure codes requires complex encoding and decoding operations, which will affect the performance of distributed storage systems. The erasure code calculation method used in related technologies makes it impossible for some central processing units to use the highest level of instruction sets they support for erasure code calculations, resulting in low erasure code calculation efficiency, high CPU calculation pressure, and low performance.

[0070] An embodiment of the present invention provides a data storage method. When using a storage node to perform erasure code calculation, whenever data to be stored is received, a storage node is randomly selected, the data to be stored is sent to the storage node, the highest level instruction set information supported by the central processing unit of the storage node is obtained, a coding function matching the instruction set information is obtained, encoding processing is performed on the data to be stored based on the coding function, and the erasure code corresponding to the data to be stored is obtained. The erasure code and the data to be stored are stored in association with each other so as to achieve the effect of dynamically selecting the highest level instruction set information supported by the central processing unit of the storage node performing the erasure code calculation for the storage node performing the erasure code calculation, reducing the calculation delay of the erasure code, improving the calculation efficiency of the erasure code, and improving the performance of the CPU.

[0071] According to an embodiment of the present invention, a data storage method embodiment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0072] In this embodiment, a data storage method is provided, which can be used in a distributed storage system. Figure 1 is a flow chart of a data storage method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0073] Step S101, when using storage nodes to perform erasure code calculation, whenever data to be stored is received, a storage node is randomly selected and the data to be stored is sent to the storage node.

[0074] Among them, the erasure code calculation is performed using the storage node, that is, the application scenario is a non-core client file scenario. At this time, after the client receives the data to be stored, it sends the data to be stored to the distributed storage system. Every time the distributed storage system receives the data to be stored, it will randomly select a storage node and use the storage node to calculate the erasure code of the data to be stored.

[0075] Step S102, using a storage node to receive data to be stored, obtaining the highest level instruction set information supported by the central processor of the storage node, obtaining a coding function matching the instruction set information, encoding the data to be stored based on the coding function, and obtaining an erasure code corresponding to the data to be stored.

[0076] After receiving the data to be stored, the storage node obtains the highest level instruction set information supported by its central processing unit.

[0077] It should be noted that different functions are encapsulated for different instruction sets for selection, so as to be used for erasure calculations of different instruction sets. Specifically, each instruction set corresponds to an encoding function and a decoding function. The following describes the encoding function and decoding function corresponding to each instruction set from low to high levels of the instruction set. It can be understood that the higher the level of the instruction set, the higher the computing performance.

[0078] SSE instruction set: encoding function ec_enc_sse, decoding function ec_dec_sse.

[0079] AVX instruction set: encoding function ec_enc_avx, decoding function ec_dec_avx.

[0080] AVX2 instruction set: encoding function ec_enc_avx2, decoding function ec_dec_avx2.

[0081] AVX2+GFNI instruction set: encoding function ec_enc_avx2_gfni, decoding function ec_dec_avx2_gfni.

[0082] AVX512 instruction set: encoding function ec_enc_avx512, decoding function ec_dec_avx512.

[0083] AVX512+GFNI instruction set: encoding function ec_enc_avx512_gfni, decoding function ec_dec_avx512_gfni.

[0084] After obtaining the highest level instruction set information, an encoding function matching the instruction set information is obtained, that is, an encoding function corresponding to the instruction set information.

[0085] Among them, initialization functions are established for the encoding function and the decoding function respectively, namely, the initialization encoding function ec_enc_init and the initialization decoding function ec_dec_init, which are used to dynamically select the optimal instruction set function when the storage node performs erasure code calculation for the first time, that is, to dynamically obtain the encoding function and decoding function corresponding to the highest level of instruction set information when the storage node performs erasure code calculation for the first time.

[0086] Specifically, after obtaining the encoding function and decoding function corresponding to the highest level of instruction set information, based on the address of the initialization encoding function and the address of the initialization decoding function, the encoding function corresponding to the highest level of instruction set information is used to replace the initialization encoding function, and the decoding function corresponding to the highest level of instruction set information is used to replace the initialization decoding function. In this way, the encoding function corresponding to the highest level of instruction set information is used to encode the data to be stored, and the erasure code corresponding to the data to be stored is obtained.

[0087] Step S103: storing the erasure code and the data to be stored in association with each other.

[0088] Among them, after obtaining the erasure code corresponding to the data to be stored, the distributed storage system uses the controlled replication under scalable hashing (Controlled Replication Under Scalable Hashing, referred to as: CRUSH) algorithm to disperse the erasure code and the corresponding data to be stored on multiple storage nodes, and save the association relationship between the erasure code and the corresponding data block to be stored.

[0089] The data storage method provided in the present embodiment randomly selects a storage node whenever receiving data to be stored by performing erasure code calculation using a storage node, and sends the data to be stored to the storage node, obtains the highest level instruction set information supported by the central processing unit of the storage node, obtains a coding function matching the instruction set information, encodes the data to be stored based on the coding function, obtains the erasure code corresponding to the data to be stored, associates and stores the erasure code with the data to be stored, can dynamically select the highest level instruction set information supported by the central processing unit, and calculates the erasure code corresponding to the data to be stored by using the coding function corresponding to the instruction set information, thereby reducing the calculation delay of the erasure code, improving the calculation efficiency of the erasure code, reducing the CPU usage, and improving the calculation bandwidth.

[0090] In this embodiment, a data storage method is provided, which can be used in a distributed storage system. Figure 2 is a flow chart of a data storage method according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0091] Step S201: When using storage nodes to perform erasure code calculation, whenever receiving data to be stored, randomly select a storage node and send the data to be stored to the storage node. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0092] Step S202, using a storage node to receive data to be stored, obtaining the highest level instruction set information supported by the central processor of the storage node, obtaining a coding function matching the instruction set information, encoding the data to be stored based on the coding function, and obtaining an erasure code corresponding to the data to be stored.

[0093] Specifically, the encoding process of the data to be stored based on the encoding function in the above step S202 to obtain the erasure code corresponding to the data to be stored includes:

[0094] Step S2021, dividing the data to be stored into a plurality of data blocks.

[0095] After receiving the data to be stored, the storage node divides the data to be stored into a number of data blocks of fixed sizes.

[0096] Step S2022: encoding the plurality of data blocks using the encoding function to generate a plurality of check blocks, wherein the erasure code includes the check blocks.

[0097] The encoding function corresponding to the highest level instruction set information supported by the CPU of the storage node is used to encode the plurality of data blocks obtained above to generate a plurality of check blocks.

[0098] Step S203: store the erasure code and the data to be stored in an associated manner.

[0099] Specifically, the above step S203 includes:

[0100] Step S2031, storing the check block and the data block in association.

[0101] After obtaining the check block, the distributed storage system stores the check block and several data blocks corresponding to the data to be stored in a dispersed manner to multiple storage nodes.

[0102] By assigning a unique identifier to each data block and each check block, a metadata table is created to record the association relationship between each data block and the check block, so as to realize the associated storage of the check block and several data blocks.

[0103] The data storage method provided in this embodiment encodes the data to be stored based on the encoding function matching the highest level instruction set information to obtain the erasure code, thereby improving the computational efficiency of the erasure code and reducing the CPU usage. The check block and the data block are stored in an associated manner to improve the security of data storage.

[0104] In some optional implementations, the data storage method further includes:

[0105] Step a1, when using the client to perform erasure code calculation, receiving the data to be stored sent by the client and the erasure code corresponding to the data to be stored, and associating and storing the erasure code and the data to be stored.

[0106] Among them, the client receives the data to be stored, obtains the highest level instruction set information supported by the client's central processing unit, obtains the encoding function that matches the instruction set information, encodes the data to be stored based on the encoding function, obtains the erasure code corresponding to the data to be stored, and sends the erasure code and the data to be stored to the distributed storage system.

[0107] The client is used to calculate the erasure code, that is, the application scenario is the kernel client file scenario. At this time, after the client receives the data to be stored, the erasure code of the data to be stored is calculated on the client, and after obtaining the erasure code, the erasure code and the data to be stored are sent to the distributed storage system for associated storage.

[0108] The data storage method provided in this embodiment obtains the highest level of instruction set information supported by the client's central processing unit when using the client to perform erasure code calculation, obtains the encoding function matching the instruction set information, and obtains the erasure code corresponding to the data to be stored based on the encoding function. This realizes the adaptive support of the instruction set of the client's CPU, dynamically selects the highest level of instruction set information, improves the calculation efficiency of the erasure code, reduces the CPU usage, and improves the calculation performance.

[0109] In some optional implementations, the data storage method further includes:

[0110] Step b1: when any data block in the distributed storage system fails, determine the storage node for calculating the erasure code corresponding to the data block.

[0111] The data block failure may be data block damage or data block loss.

[0112] The storage node for calculating the erasure code corresponding to the data block, that is, the storage node for calculating the erasure code of the data to be stored corresponding to the data block.

[0113] Step b2: Obtain a decoding function that matches the highest level of instruction set information supported by the central processing unit of the storage node.

[0114] After the storage node is determined, a decoding function matching the highest level instruction set information supported by the CPU of the storage node is obtained.

[0115] Step b3, obtaining the remaining data blocks and check blocks associated with the data block.

[0116] Among them, when any data block fails, the information of the remaining data blocks and check blocks associated with the data block can be obtained by querying the metadata table, and then the remaining data blocks and check blocks associated with the data block can be obtained, which is essentially obtaining the data in the remaining data blocks and check blocks.

[0117] Step b4: input the data of the remaining data blocks and the check block into a decoding function to obtain the original data corresponding to the failed data block, so as to perform data recovery based on the original data.

[0118] After obtaining the data of the remaining data blocks and the check blocks associated with the failed data block, the data of the remaining data blocks and the check blocks are input into a decoding function to obtain the original data corresponding to the failed data block, and data recovery is performed based on the original data.

[0119] The data storage method provided in this embodiment significantly improves the efficiency of data recovery and the fault tolerance of the system by using a decoding function that matches the highest level instruction set supported by the storage node CPU and combining it with a data recovery mechanism. At the same time, it also reduces resource consumption, enhances the scalability and flexibility of the system, and simplifies management and maintenance.

[0120] In some optional implementations, the data storage method further includes:

[0121] Step c1, when the randomly selected storage node is a storage node that has performed erasure code calculation, the encoding function corresponding to the storage node is directly used to encode the data to be stored to obtain the erasure code corresponding to the data to be stored.

[0122] Among them, the randomly screened storage node is a storage node that has performed erasure code calculation, which means that this is not the first time that the storage node has performed erasure code calculation. At this time, there is no need to execute the steps of obtaining the highest level instruction set information supported by the central processing unit of the storage node and obtaining the encoding function that matches the instruction set information. Instead, the encoding function corresponding to the storage node is obtained directly based on the address of the initialized encoding function, that is, the encoding function that matches the highest level instruction set information supported by the central processing unit of the storage node. The encoding function is used to encode the data to be stored to obtain the erasure code corresponding to the data to be stored.

[0123] It can be understood that when the storage node performs erasure code calculation for the first time, the steps of obtaining the highest level instruction set information supported by the central processing unit of the storage node and obtaining the encoding function that matches the instruction set information are executed. At this time, the encoding function corresponding to the highest level instruction set information replaces the initialization encoding function, and the decoding function corresponding to the highest level instruction set information replaces the initialization decoding function.

[0124] When the storage node performs erasure code calculation for the second time or thereafter, since the initialization encoding function and the initialization decoding function have been replaced with the encoding function and the decoding function corresponding to the highest level of instruction set information when the erasure code calculation was performed for the first time, the encoding function can be directly used to calculate the erasure code and the decoding function can be used to recover the data. There is no need to execute the steps of obtaining the highest level of instruction set information supported by the central processing unit of the storage node and obtaining the encoding function matching the instruction set information every time the erasure code calculation is performed.

[0125] The data storage method provided in this embodiment encodes the data to be stored by directly using the encoding function corresponding to the storage node that has performed erasure code calculation, and obtains the erasure code corresponding to the data to be stored, thereby significantly improving the encoding efficiency, eliminating the overhead caused by automatically selecting the highest level instruction set information of the storage node each time, improving the performance of erasure calculation, and reducing the computing cost.

[0126] In some optional implementations, the above step S202 further includes:

[0127] Step d1, obtaining configuration information of the central processing unit of the storage node.

[0128] The configuration information of the central processing unit of the storage node is obtained by using a central processing unit identification (CPUID) instruction.

[0129] It should be noted that CPUID is a processor supplementary command for the X86 architecture. Its main function is to allow software to discover detailed information about the central processor and identify the capabilities of the target machine, such as model, whether it supports the AVX instruction set or AVX512 instruction set, etc. The information of the central processor is stored in the EAX, EBX, ECX, and EDX registers, and the information returned is determined by the values ​​of EAX and ECX.

[0130] Step d2: Determine the highest level of instruction set information supported by the central processing unit based on the configuration information.

[0131] After the configuration information of the CPU is obtained, the highest level of instruction set information supported by the central processing unit is determined according to the configuration information.

[0132] The data storage method provided in this embodiment can significantly improve the performance of erasure calculation by determining and using the highest level instruction set information supported by the central processing unit, thereby improving the efficiency of obtaining erasure codes.

[0133] In some optional implementations, the above step d2 includes:

[0134] Step d21, based on the configuration information, determining whether the central processing unit supports both the finite field instruction set and the first vector extension instruction set.

[0135] Figure 3 1 is a flow chart of determining the highest level of instruction set information supported by a central processing unit according to an embodiment of the present invention. Figure 3 As shown, after obtaining the CPU configuration information, ie, the instruction set information supported by the CPU, by using the CPUID command, it is determined whether the central processing unit supports both the finite field instruction set and the first vector extension instruction set based on the configuration information.

[0136] Among them, the finite field instruction set is the GFNI instruction set, and the first vector extension instruction set is the AVX512 instruction set.

[0137] Step d22: when the central processing unit supports both the finite field instruction set and the first vector extension instruction set, determine that the highest level of instruction set information supported by the central processing unit is the finite field instruction set and the first vector extension instruction set.

[0138] Among them, the central processor supports both the GFNI instruction set and the AVX512 instruction set, and the highest level of instruction set information supported by the central processor is determined to be the GFNI instruction set and the AVX512 instruction set. Select the encoding function and decoding function that match the instruction set information, namely ec_enc_avx512_gfni and ec_dec_avx512_gfni, and replace the init address with the selected function, that is, replace ec_enc_init of the storage node with ec_enc_avx512_gfni, and replace ec_dec_init with ec_dec_avx512_gfni.

[0139] Step d23: if the central processing unit does not support both the finite field instruction set and the first vector extension instruction set, determine whether the central processing unit supports the first vector extension instruction set.

[0140] If the central processing unit does not support both the GFNI instruction set and the AVX512 instruction set, it is determined whether the CPU supports the AVX512 instruction set.

[0141] Step d24: when the central processing unit supports the first vector extension instruction set, determine that the highest level of instruction set information supported by the central processing unit is the first vector extension instruction set.

[0142] Among them, if the CPU supports the AVX512 instruction set, it is determined that the highest level of instruction set information supported by the central processor is the AVX512 instruction set. Select the encoding function and decoding function that match the instruction set information, namely ec_enc_avx512 and ec_dec_avx512, and replace the init address with the selected function, that is, replace ec_enc_init of the storage node with ec_enc_avx512, and replace ec_dec_init with ec_dec_avx512.

[0143] Step d25, when the central processing unit does not support the first vector extension instruction set, determine whether the central processing unit supports both a finite field instruction set and a second vector extension instruction set, wherein the number of operands of the second vector extension instruction set, i.e., the operation bit width, is smaller than that of the first vector extension instruction set.

[0144] If the CPU does not support the AVX512 instruction set, it is determined whether the CPU supports both the GFNI instruction set and the AVX2 instruction set, that is, the second vector extension instruction set is the AVX2 instruction set.

[0145] Step d26, when the central processing unit supports both the finite field instruction set and the second vector extension instruction set, determining that the highest level of instruction set information supported by the central processing unit is the finite field instruction set and the second vector extension instruction set.

[0146] Among them, the CPU supports both the GFNI instruction set and the AVX2 instruction set, and the highest level of instruction set information supported by the central processor is determined to be the GFNI instruction set and the AVX2 instruction set. Select the encoding function and decoding function that match the instruction set information, namely ec_enc_avx2_gfni and ec_dec_avx2_gfni, and replace the init address with the selected function, that is, replace ec_enc_init of the storage node with ec_enc_avx2_gfni, and replace ec_dec_init with ec_dec_avx2_gfni.

[0147] Step d27, when the central processing unit does not support both the finite field instruction set and the second vector extension instruction set, determining whether the central processing unit supports the second vector extension instruction set.

[0148] If the central processing unit does not support both the GFNI instruction set and the AVX2 instruction set, it is determined whether the CPU supports the AVX2 instruction set.

[0149] Step d28: When the central processing unit supports the second vector extension instruction set, determine that the highest level of instruction set information supported by the central processing unit is the second vector extension instruction set.

[0150] Among them, if the CPU supports the AVX2 instruction set, it is determined that the highest level of instruction set information supported by the central processor is the AVX2 instruction set. Select the encoding function and decoding function that match the instruction set information, namely ec_enc_avx2 and ec_dec_avx2, and replace the init address with the selected function, that is, replace ec_enc_init of the storage node with ec_enc_avx2, and replace ec_dec_init with ec_dec_avx2.

[0151] Step d29: if the central processing unit does not support the second vector extension instruction set, determine whether the central processing unit supports the third vector extension instruction set, wherein the second vector extension instruction set is an extended version of the third vector extension instruction set.

[0152] Among them, if the CPU does not support the AVX2 instruction set, it is determined whether the CPU supports the AVX instruction set, that is, the third vector extension instruction set is the AVX instruction set.

[0153] Step d210: when the central processing unit supports the third vector extension instruction set, determine that the highest level of instruction set information supported by the central processing unit is the third vector extension instruction set.

[0154] If the CPU supports the AVX instruction set, it is determined that the highest level of instruction set information supported by the central processor is the AVX instruction set. The encoding function and decoding function that match the instruction set information, namely, ec_enc_avx and ec_dec_avx, are selected, and the init address is replaced with the selected function, that is, ec_enc_init of the storage node is replaced with ec_enc_avx, and ec_dec_init is replaced with ec_dec_avx.

[0155] It can be understood that, when the central processing unit does not support the third vector extension instruction set, it is determined whether the central processing unit supports the streaming extension instruction set.

[0156] Among them, the loss extension instruction set is the SSE instruction set. If the CPU supports the SSE instruction set, it is determined that the highest level of instruction set information supported by the central processor is the SSE instruction set. Select the encoding function and decoding function that match the instruction set information, namely ec_enc_sse and ec_dec_sse, and replace the init address with the selected function, that is, replace ec_enc_init of the storage node with ec_enc_sse, and replace ec_dec_init with ec_dec_sse.

[0157] The data storage method provided in this embodiment can determine the highest level instruction set supported by the CPU by judging whether the CPU supports a specific instruction set, so as to use the highest level instruction set supported by the CPU to perform erasure calculation on the data to be stored and obtain an erasure code. This not only improves the execution efficiency of erasure calculation, but also maximizes the processing power of the CPU.

[0158] In some optional implementations, whenever data to be stored is received, a storage node is randomly selected, including:

[0159] Step e1, obtaining load information of each storage node in the distributed storage system.

[0160] Whenever data to be stored is received, load information of each storage node in the distributed storage system is obtained.

[0161] The load information may be CPU usage, memory usage, etc.

[0162] Step e2: based on the load information of each storage node, select a storage node with the lowest load.

[0163] After the load information of each storage node is obtained, a storage node with the lowest load is selected according to the load information of each storage node, and the erasure code is calculated using the storage node with the lowest load. For example, the storage node with the lowest CPU usage rate is selected to calculate the erasure code.

[0164] The data storage method provided in this embodiment can avoid further overloading of high-load storage nodes by allocating the task of calculating erasure codes to storage nodes with lower current loads, thereby improving the response speed and processing power of the entire system, increasing the calculation rate of erasure codes, and more effectively utilizing resources in the cluster.

[0165] This embodiment also provides a data storage method. Figure 4 1 is a flow chart of a data storage method according to an embodiment of the present invention. Figure 4 As shown, the erasure code calculation is described by using a storage node, and the data storage method includes the following steps:

[0166] In the first step, the distributed storage system receives the data to be stored and randomly selects a storage node for erasure coding, i.e., erasure code calculation. ec_enc_data is the erasure code calculation for the data to be stored.

[0167] This embodiment establishes distribution functions ec_enc_disp and ec_dec_disp for the encoding function and decoding function of erasure code calculation. The distribution function is used to dynamically select the highest level instruction set information of the central processor of the storage node. The address pointed to (jumped) by the distribution function at the beginning is the init function, i.e., the initialization function of the storage node.

[0168] In the second step, when the storage node performs erasure code calculation for the first time, the distribution function executes the init function, namely ec_enc_init, to dynamically select the instruction set, that is, to obtain the highest level of instruction set information supported by the central processing unit of the storage node.

[0169] Specifically, by executing the CPUID command, the CPU instruction set information, i.e., the instruction set information supported by the CPU, is obtained. The CPU optimal instruction set function is obtained by comparison, i.e., the highest level instruction set information is determined through the above steps d21 to d210, and then the encoding function and decoding function corresponding to the instruction set information are obtained.

[0170] In the third step, the distribution function replaces the init address with the optimal instruction set function, that is, the initialization function of the storage node is replaced with the encoding function and decoding function corresponding to the highest level instruction set information.

[0171] The fourth step is to use the optimal instruction set function to calculate the erasure code, that is, use the encoding function corresponding to the highest level of instruction set information to calculate the erasure code, and return the erasure code to the distributed storage system so that the distributed storage system can associate the data to be stored with the erasure code.

[0172] In the fifth step, when the storage node calculates the erasure code for the second time or later, the optimal instruction set function selected by the init function can be directly used to calculate the erasure code.

[0173] That is to say, the storage node will only select the instruction set when calculating the erasure code for the first time. Once selected, the optimal instruction set function will replace the address of the init function. The selected optimal instruction set function will be used directly in the future without having to make a selection for each calculation.

[0174] The data storage method provided in this embodiment dynamically selects the optimal instruction set, i.e., the highest-level instruction set, based on the instruction set supported by the central processing unit for erasure code calculation, and uses the encoding function corresponding to the instruction set to calculate the erasure code corresponding to the data to be stored, thereby reducing the calculation delay of the erasure code, improving the calculation efficiency of the erasure code, and improving the performance. In addition, by recording the address of the function corresponding to the selected optimal instruction set, for subsequent encoding and decoding requests, the function corresponding to the optimal instruction set selected for the first time is directly used, avoiding multiple selection jumps, reducing unnecessary judgment jump overhead, so that the optimal instruction set is used for each calculation of the erasure code, and improving the calculation efficiency of the erasure code.

[0175] It should be noted that the data storage method of the embodiment of the present invention is applicable to scenarios where erasure codes are calculated on storage nodes, such as NAS, RGW, and RBD scenarios, and is also applicable to scenarios where erasure codes are calculated on clients. Compared with the erasure code calculation method using the AVX2 instruction set in the related art, the CPU usage rate is reduced by 20%, the calculation delay is reduced by 15%, and the calculation bandwidth is increased by 2 times by using the erasure code calculation method of the embodiment of the present invention.

[0176] In the present embodiment, a data storage device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0177] This embodiment provides a data storage device, such as Figure 5 As shown, including:

[0178] The sending module 501 is used to randomly select a storage node and send the data to be stored to the storage node whenever receiving data to be stored when using the storage node to perform erasure code calculation.

[0179] The first acquisition module 502 is used to receive the data to be stored using the storage node, obtain the highest level instruction set information supported by the central processing unit of the storage node, obtain the encoding function matching the instruction set information, encode the data to be stored based on the encoding function, and obtain the erasure code corresponding to the data to be stored.

[0180] The storage module 503 is used to associate and store the erasure codes with the data to be stored.

[0181] In some optional embodiments, the data storage device further comprises:

[0182] The receiving module is used to receive the data to be stored and the erasure code corresponding to the data to be stored sent by the client when the client is used to calculate the erasure code, and associate the erasure code with the data to be stored for storage.

[0183] Among them, the client receives the data to be stored, obtains the highest level instruction set information supported by the client's central processing unit, obtains the encoding function that matches the instruction set information, encodes the data to be stored based on the encoding function, obtains the erasure code corresponding to the data to be stored, and sends the erasure code and the data to be stored to the distributed storage system.

[0184] In some optional implementations, the first acquisition module 502 includes:

[0185] The segmentation unit is used to segment the data to be stored into a plurality of data blocks.

[0186] The generating unit is used to encode a plurality of data blocks using an encoding function to generate a plurality of check blocks, wherein the erasure code includes the check blocks.

[0187] The storage module 503 includes:

[0188] A storage unit is used to store the check block and the data block in an associated manner.

[0189] In some optional embodiments, the data storage device further comprises:

[0190] The determination module is used to determine the storage node for calculating the erasure code corresponding to any data block in the distributed storage system when a data block fails.

[0191] The second acquisition module is used to acquire a decoding function that matches the highest level instruction set information supported by the central processing unit of the storage node.

[0192] The third acquisition module is used to acquire other data blocks and check blocks associated with the data block.

[0193] The fourth acquisition module is used to input the data of the remaining data blocks and the check block into the decoding function to obtain the original data corresponding to the failed data block, so as to perform data recovery based on the original data.

[0194] In some optional embodiments, the data storage device further comprises:

[0195] The encoding module is used to directly use the encoding function corresponding to the storage node to encode the data to be stored, so as to obtain the erasure code corresponding to the data to be stored, when the randomly selected storage node is a storage node that has performed erasure code calculation.

[0196] In some optional implementations, the first acquisition module 502 includes:

[0197] The first acquisition unit is used to acquire configuration information of the central processing unit of the storage node.

[0198] The first determining unit is used to determine the highest level instruction set information supported by the central processing unit based on the configuration information.

[0199] In some optional implementations, the first determining unit includes:

[0200] The first determination unit is used to determine whether the central processing unit supports both the finite field instruction set and the first vector extension instruction set based on the configuration information.

[0201] The second determining unit is used to determine that the highest level of instruction set information supported by the central processing unit is the finite field instruction set and the first vector extension instruction set when the central processing unit supports both the finite field instruction set and the first vector extension instruction set.

[0202] The second judgment unit is used to judge whether the central processing unit supports the first vector extension instruction set when the central processing unit does not support the finite field instruction set and the first vector extension instruction set at the same time.

[0203] The third determining unit is configured to determine, when the central processing unit supports the first vector extension instruction set, that the highest level of instruction set information supported by the central processing unit is the first vector extension instruction set.

[0204] The third judgment unit is used to judge whether the central processing unit supports both a finite field instruction set and a second vector extension instruction set when the central processing unit does not support the first vector extension instruction set, wherein the number of operands of the second vector extension instruction set is smaller than that of the first vector extension instruction set.

[0205] The fourth determining unit is used to determine that the highest level of instruction set information supported by the central processing unit is the finite field instruction set and the second vector extension instruction set when the central processing unit supports both the finite field instruction set and the second vector extension instruction set.

[0206] The fourth determination unit is used to determine whether the central processing unit supports the second vector extension instruction set when the central processing unit does not support both the finite field instruction set and the second vector extension instruction set.

[0207] The fifth determining unit is configured to determine, when the central processing unit supports the second vector extension instruction set, that the highest level of instruction set information supported by the central processing unit is the second vector extension instruction set.

[0208] The fifth judgment unit is used to judge whether the central processing unit supports the third vector extension instruction set when the central processing unit does not support the second vector extension instruction set, wherein the second vector extension instruction set is an extended version of the third vector extension instruction set.

[0209] The sixth determining unit is configured to determine, when the central processing unit supports the third vector extension instruction set, that the highest level of instruction set information supported by the central processing unit is the third vector extension instruction set.

[0210] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0211] The data storage device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0212] The embodiment of the present invention also provides a computer device having the above Figure 5 The data storage device shown.

[0213] See also Figure 6 , Figure 6 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 6 As shown, the computer device includes: one or more processors 601, memory 602, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components utilize different buses to communicate with each other, and can be installed on a common mainboard or installed in other ways as required. The processor can process the instructions executed in the computer device, including instructions stored in the memory or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 601 is taken as an example.

[0214] The processor 601 may be a central processing unit, a network processor or a combination thereof. The processor 601 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0215] The memory 602 stores instructions executable by at least one processor 601 , so that the at least one processor 601 executes the method shown in the above embodiment.

[0216] The memory 602 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 602 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 602 may optionally include a memory remotely arranged relative to the processor 601, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0217] The memory 602 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 602 may also include a combination of the above types of memory.

[0218] The computer device also includes a communication interface 603, which is used for the computer device to communicate with other devices or a communication network.

[0219] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0220] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0221] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A data storage method, characterized in that: The method comprises: In the case of using storage nodes to perform erasure code calculation, whenever data to be stored is received, a storage node is randomly selected and the data to be stored is sent to the storage node; Utilizing the storage node to receive the data to be stored, obtaining the highest level instruction set information supported by the central processing unit of the storage node, obtaining a coding function matching the instruction set information, encoding the data to be stored based on the coding function, and obtaining an erasure code corresponding to the data to be stored; The erasure code and the data to be stored are stored in association with each other.

2. The method according to claim 1, characterized in that: The method further comprises: In the case of using the client to perform erasure code calculation, receiving the data to be stored and the erasure code corresponding to the data to be stored sent by the client, and associating the erasure code with the data to be stored; Among them, the client receives the data to be stored, obtains the highest level instruction set information supported by the client's central processing unit, obtains a coding function matching the instruction set information, encodes the data to be stored based on the coding function, obtains the erasure code corresponding to the data to be stored, and sends the erasure code and the data to be stored to a distributed storage system.

3. The method according to claim 1, characterized in that: The encoding process is performed on the data to be stored based on the encoding function to obtain an erasure code corresponding to the data to be stored, including: Dividing the data to be stored into a plurality of data blocks; Using the encoding function to encode the plurality of data blocks to generate a plurality of check blocks, wherein the erasure code includes the check blocks; The associative storage of the erasure code and the data to be stored includes: The check block and the data block are stored in association.

4. The method according to claim 3, characterized in that The method further comprises: In the event of a failure of any data block in the distributed storage system, determine the storage node that calculates the erasure code corresponding to the data block; Acquire a decoding function that matches the highest level of instruction set information supported by the central processing unit of the storage node; Obtaining the remaining data blocks and check blocks associated with the data block; The data of the remaining data blocks and the check block are input into the decoding function to obtain the original data corresponding to the failed data block, so as to perform data recovery based on the original data.

5. The method according to claim 1, characterized in that The method further comprises: In the case where the randomly selected storage node is a storage node that has performed erasure code calculation, the data to be stored is directly encoded using the encoding function corresponding to the storage node to obtain the erasure code corresponding to the data to be stored.

6. The method according to claim 1, characterized in that The obtaining of the highest level instruction set information supported by the central processing unit of the storage node includes: Obtaining configuration information of a central processing unit of the storage node; Based on the configuration information, the highest level of instruction set information supported by the central processing unit is determined.

7. The method according to claim 6, characterized in that The determining, based on the configuration information, the highest level of instruction set information supported by the central processing unit comprises: Based on the configuration information, determining whether the central processing unit supports both a finite field instruction set and a first vector extension instruction set; In the case where the central processing unit supports both a finite field instruction set and a first vector extension instruction set, determining that the highest level of instruction set information supported by the central processing unit is the finite field instruction set and the first vector extension instruction set; In a case where the central processing unit does not support both the finite field instruction set and the first vector extension instruction set, determining whether the central processing unit supports the first vector extension instruction set; In a case where the central processing unit supports a first vector extension instruction set, determining that the highest level of instruction set information supported by the central processing unit is the first vector extension instruction set; In a case where the central processing unit does not support the first vector extension instruction set, determining whether the central processing unit supports both a finite field instruction set and a second vector extension instruction set, wherein the number of operands of the second vector extension instruction set is smaller than the number of operands of the first vector extension instruction set; In the case where the central processing unit supports both a finite field instruction set and a second vector extension instruction set, determining that the highest level of instruction set information supported by the central processing unit is the finite field instruction set and the second vector extension instruction set; In a case where the central processing unit does not support the finite field instruction set and the second vector extension instruction set at the same time, determining whether the central processing unit supports the second vector extension instruction set; In a case where the central processing unit supports a second vector extension instruction set, determining that the highest level of instruction set information supported by the central processing unit is the second vector extension instruction set; In a case where the central processing unit does not support a second vector extension instruction set, determining whether the central processing unit supports a third vector extension instruction set, wherein the second vector extension instruction set is an extended version of the third vector extension instruction set; In a case where the central processing unit supports the third vector extension instruction set, it is determined that the highest level of instruction set information supported by the central processing unit is the third vector extension instruction set.

8. A data storage device, characterized in that: The device comprises: A sending module, for randomly selecting a storage node whenever receiving data to be stored, and sending the data to be stored to the storage node when using the storage node to perform erasure code calculation; A first acquisition module is used to receive the data to be stored by using the storage node, obtain the highest level instruction set information supported by the central processing unit of the storage node, obtain a coding function matching the instruction set information, encode the data to be stored based on the coding function, and obtain an erasure code corresponding to the data to be stored; A storage module is used to store the erasure code and the data to be stored in an associated manner.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the data storage method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the data storage method according to any one of claims 1 to 7.

Images