Data storage method and device, controller and computer storage medium

Abstract

The invention relates to a data storage method and device, a controller and a computer storage medium, which are used for preventing a large amount of data from being stored in a storage space, reducing the occupation of network bandwidth and storage resources and improving the storage service quality. Comprising the following steps: performing data segmentation on to-be-stored data in a received data storage instruction to obtain each piece of fragmented data; determining a data type of the first fragmented data according to data in the first fragmented data; wherein the first fragment data is any fragment data in the fragment data; obtaining metadata information of the first fragment data through the data type of the first fragment data; the data volume of the metadata information is smaller than that of the fragmented data; if the data type of the first fragmented data is a first specified type, storing metadata information of the first fragmented data; and if the data type of the first fragmented data is a second specified type, storing the first fragmented data and metadata information of the first fragmented data.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data storage method, device, controller and computer storage medium. Background Art

[0002] Currently, when storing data in an all-flash array controller, the amount of data stored is the same as the amount of data written by the business, which not only occupies the corresponding transmission bandwidth, but also occupies the corresponding storage space.

[0003] For some storage application scenarios, such as remote replication, volume copy, and active-active disaster recovery, a large amount of stored data will be intensively read and written with high concurrency, occupying a large amount of transmission bandwidth and storage resources. This will lead to overhead in data access timeliness. Even if a large amount of "invalid" data is written, it will greatly increase the operation time of the controller and cause write amplification, which will not only shorten the service life of the SSD, but also affect the response time of other business read and write requests on the controller, reducing the storage service quality. Summary of the invention

[0004] The present invention provides a data storage method for avoiding storing a large amount of data in a storage space, reducing the occupation of network bandwidth and storage resources, and improving the storage service quality.

[0005] In a first aspect, the present application provides a data storage method, which is applied to an all-flash array controller, and the method includes:

[0006] Receiving a data storage instruction sent by a user; wherein the data storage instruction includes data to be stored;

[0007] Slicing the data to be stored to obtain data of each slice;

[0008] Determine the data type of the first shard data according to the data in the first shard data; wherein the first shard data is any one of the shard data;

[0009] Obtaining metadata information of the first shard data according to the data type of the first shard data; wherein the metadata information is used to describe key information of the first shard data, and the data volume of the metadata information is smaller than the data volume of the shard data;

[0010] If the data type of the first shard data is a first specified type, storing metadata information of the first shard data, wherein the first specified type is a type with fixed characteristics of the data;

[0011] If the data type of the first shard data is a second specified type, the first shard data and metadata information of the first shard data are stored, wherein the second specified type is a type in which data does not have fixed characteristics.

[0012] In the above method, the data to be stored is segmented to obtain data of each shard; the data type of the first shard is determined according to the data in the first shard, and then the metadata information of the first shard is obtained according to the data type of the first shard, if the data type of the first shard is the first specified type, the metadata information of the first shard is stored; if the data type of the first shard is the second specified type, the first shard and the metadata information of the first shard are stored. Therefore, in the embodiment of the present application, only the metadata information of the shard of the first specified type is stored, and the corresponding data is not stored, so the amount of stored data can be reduced, a large amount of data can be avoided from being stored in the storage space, the occupation of network bandwidth and storage resources can be reduced, and the storage service quality can be improved.

[0013] In a possible implementation manner, the data storage instruction further includes a first starting storage address and a first data length;

[0014] After the data to be stored is segmented to obtain data of each segment, the method further includes:

[0015] Based on the first starting storage address and the first data length, obtaining first target storage addresses corresponding to the respective slice data;

[0016] The obtaining metadata information of the first shard data according to the data type of the first shard data includes:

[0017] The data type of the first slice data, the first data length and the first target storage address are determined as metadata information of the first slice data.

[0018] In the above method, by establishing a mapping relationship between the storage address of the data to be stored and the target storage address of each shard data, it is convenient to accurately obtain the corresponding data when reading the data subsequently.

[0019] In a possible implementation, storing metadata information of the first shard data includes:

[0020] Storing metadata information of the first slice data in a memory block corresponding to the first target storage address;

[0021] Storing the first shard data and metadata information of the first shard data includes:

[0022] The first shard data and metadata information of the first shard data are stored in a memory block corresponding to the first target storage address.

[0023] In the above method, metadata information of the first slice data and / or the first slice data are stored in a memory block corresponding to the target address so that the corresponding data can be obtained later.

[0024] In a possible implementation, the method further includes:

[0025] receiving a data acquisition instruction sent by the user, wherein the data acquisition instruction includes a second starting storage address and a second data length;

[0026] Obtaining metadata information of each target slice data according to the second starting storage address and the second data length;

[0027] Determine the memory block corresponding to the first target shard data through metadata information of the first target shard data, and obtain sub-data corresponding to the first target shard data from the memory block; wherein the first target shard data is any one of the target shard data;

[0028] Obtaining target data according to the sub-data corresponding to each target fragment data;

[0029] The target data is sent to the user.

[0030] In the above method, the memory block corresponding to the first target shard data is determined through the metadata information of the first target shard data, and the sub-data corresponding to the first target shard data is obtained from the memory block. The target data is obtained according to the sub-data corresponding to each target shard data, thereby ensuring the accuracy of the acquired target data.

[0031] In a possible implementation manner, determining the memory block corresponding to the first target shard data by using metadata information of the first target shard data includes:

[0032] If the data type in the metadata information is a first specified type, a target memory block corresponding to the data type of the first target shard is determined by using a preset correspondence between the data type and the memory block, and the target memory block is determined as a memory block corresponding to the first target shard data; wherein the first specified type includes a plurality of subtypes;

[0033] If the data type in the metadata information is the second specified type, the target storage address in the metadata information of the first target shard data is obtained, and the memory block corresponding to the target storage address is determined as the memory block corresponding to the first target shard data.

[0034] In the above method, if the data type in the metadata information is the first specified type, the pre-set correspondence between the data type and the memory block is used to determine the target memory block corresponding to the data type of the first target shard, and the target memory block is determined as the memory block corresponding to the first target shard data, that is, in the embodiment of the present application, each type of shard data corresponding to the first specified type has a corresponding memory block, that is, there is no need to repeatedly store multiple data, and each time for the same type of data, the corresponding data is obtained from the memory block, which can save space, reduce the occupancy of network bandwidth and storage resources, and improve the storage service quality.

[0035] In a possible implementation manner, the data to be stored is segmented to obtain data of each segment, including:

[0036] The data to be stored is segmented according to the specified length to obtain data of each segment.

[0037] In the above method, the data to be stored is segmented according to the specified length to obtain each slice data, thereby ensuring that the data length of each slice data is the same.

[0038] In a possible implementation manner, the data type includes a first type, a second type, and a third type, wherein the first type and the second type are the first specified type, and the third type is the second specified type;

[0039] The determining, according to the data in the first shard data, the data type of the first shard data includes:

[0040] If all characters in the first fragment data are the same, determining that the data type of the first fragment is the first specified type;

[0041] If the characters in the first segment data are different, the data type of the first segment data is determined to be the second specified type.

[0042] In the above method, the data type of the slice data is determined by each character in the slice data, thereby ensuring the accuracy of the determined data type of the slice data.

[0043] In a second aspect, the present application provides a data storage device, the device comprising:

[0044] A receiving module, used to receive a data storage instruction sent by a user; wherein the data storage instruction includes data to be stored;

[0045] A data segmentation module is used to segment the data to be stored to obtain data of each segment;

[0046] A data type determination module, used to determine the data type of the first shard data according to the data in the first shard data; wherein the first shard data is any one of the shard data;

[0047] A metadata information determination module, configured to obtain metadata information of the first shard data according to the data type of the first shard data; wherein the metadata information is used to describe key information of the first shard data, and the data volume of the metadata information is smaller than the data volume of the shard data;

[0048] A first storage module, configured to store metadata information of the first shard data if the data type of the first shard data is a first specified type, wherein the first specified type is a type of data with fixed characteristics;

[0049] The second storage module is used to store the first shard data and metadata information of the first shard data if the data type of the first shard data is a second specified type, wherein the second specified type is a type of data that does not have fixed characteristics.

[0050] In a possible implementation manner, the data storage instruction further includes a first starting storage address and a first data length; and the device further includes:

[0051] a target storage address determination module, configured to perform data segmentation on the data to be stored to obtain the data of each slice, and then obtain the first target storage address corresponding to each slice data based on the first starting storage address and the first data length;

[0052] The metadata information determination module is specifically used to:

[0053] The data type of the first slice data, the first data length and the first target storage address are determined as metadata information of the data.

[0054] In a possible implementation manner, the first storage module is specifically configured to:

[0055] Storing metadata information of the first slice data in a memory block corresponding to the first target storage address;

[0056] The second storage module is specifically used for:

[0057] The first shard data and metadata information of the first shard data are stored in a memory block corresponding to the first target storage address.

[0058] In a possible implementation manner, the receiving module is further configured to:

[0059] receiving a data acquisition instruction sent by the user, wherein the data acquisition instruction includes a second starting storage address and a second data length;

[0060] Obtaining metadata information of each target slice data according to the second starting storage address and the second data length;

[0061] Determine the memory block corresponding to the first target shard data through metadata information of the first target shard data, and obtain sub-data corresponding to the first target shard data from the memory block; wherein the first target shard data is any one of the target shard data;

[0062] Obtaining target data according to the sub-data corresponding to each target fragment data;

[0063] The target data is sent to the user.

[0064] In a possible implementation manner, the receiving module is further configured to:

[0065] If the data type in the metadata information is a first specified type, a target memory block corresponding to the data type of the first target shard is determined by using a preset correspondence between the data type and the memory block, and the target memory block is determined as a memory block corresponding to the first target shard data; wherein the first specified type includes multiple types;

[0066] If the data type in the metadata information is the second specified type, the target storage address in the metadata information of the first target shard data is obtained, and the memory block corresponding to the target storage address is determined as the memory block corresponding to the first target shard data.

[0067] In a possible implementation, the data segmentation module is specifically used to:

[0068] The data to be stored is segmented according to the specified length to obtain data of each segment.

[0069] In a possible implementation manner, the data type determination module is specifically configured to:

[0070] If all characters in the first fragment data are the same, determining that the data type of the first fragment is the first specified type;

[0071] If the characters in the first segment data are different, the data type of the first segment data is determined to be the second specified type.

[0072] In a third aspect, the present application provides an all-flash array controller, including:

[0073] A memory for storing program instructions;

[0074] The processor is used to call the program instructions stored in the memory, and execute the steps included in any one of the data storage methods in the first aspect according to the obtained program instructions.

[0075] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps in the above-mentioned data storage method of the present application are implemented.

[0076] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which is stored in a computer-readable storage medium; when the processor of a memory access device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the memory access device executes the steps in the above-mentioned data storage method of the present application.

[0077] For each aspect from the second to the fifth aspect and the technical effects that may be achieved by each aspect, please refer to the technical effects that can be achieved by various possible solutions in the first aspect, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0079] Figure 1 One of the flow diagrams of a data storage method provided in an embodiment of the present application;

[0080] Figure 2 A schematic diagram of metadata information corresponding to each shard data provided in an embodiment of the present application;

[0081] Figure 3 A schematic diagram of a data reading process provided in an embodiment of the present application;

[0082] Figure 4 A schematic diagram of the correspondence between various data types and memory blocks provided in an embodiment of the present application;

[0083] Figure 5 A schematic diagram of each shard data pair shard information provided in an embodiment of the present application;

[0084] Figure 6 A second flow chart of a data storage method provided in an embodiment of the present application;

[0085] Figure 7 A schematic diagram of a data storage device provided in an embodiment of the present application;

[0086] Figure 8 A schematic diagram of a controller provided in an embodiment of the present application. DETAILED DESCRIPTION

[0087] In order to make the purpose, technical solution and advantages of the present application more clear, the present application will be further described in detail below in conjunction with the accompanying drawings. The specific operation method in the method embodiment can also be applied to the device embodiment or the system embodiment.

[0088] In the description of this application, "multiple" is understood to be "at least two". "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent three situations: A exists alone, A and B exist at the same time, and B exists alone. A is connected to B, which can represent two situations: A is directly connected to B and A is connected to B through C. In addition, in the description of this application, words such as "first" and "second" are only used for the purpose of distinguishing descriptions, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying order.

[0089] In the prior art, when storing data in an all-flash array controller, the amount of data written by the business will not only occupy the corresponding transmission bandwidth, but also the corresponding storage space. A large amount of stored data will be intensively read and written with high concurrency, occupying a large amount of transmission bandwidth and storage resources. This will bring about overhead in the timeliness of data access. Even if a large amount of "invalid" data is written, it will greatly increase the operation time of the controller, causing write amplification, which will not only shorten the service life of the solid-state drive, but also affect the response time of other business read and write requests on the controller, reducing the quality of storage services.

[0090] In this regard, an embodiment of the present application provides a data storage method, which obtains each shard data by performing data segmentation on the data to be stored; determines the data type of the first shard data according to the data in the first shard data, and then obtains the metadata information of the first shard data according to the data type of the first shard data, and if the data type of the first shard data is the first specified type, stores the metadata information of the first shard data; if the data type of the first shard data is the second specified type, stores the first shard data and the metadata information of the first shard data. Therefore, in the embodiment of the present application, only the metadata information of the shard data of the first specified type is stored, and the corresponding data is not stored, so the amount of stored data can be reduced, avoiding the storage of a large amount of data in the storage space, reducing the occupation of network bandwidth and storage resources, and improving the storage service quality.

[0091] like Figure 1 The figure is a flow chart of the data storage method, which may specifically include the following steps:

[0092] Step 101: receiving a data storage instruction sent by a user; wherein the data storage instruction includes data to be stored;

[0093] The data storage instruction in the embodiment of the present application also includes a first storage address and a data length of the data to be stored.

[0094] Step 102: Slice the data to be stored to obtain data of each slice;

[0095] In one embodiment, step 102 may be specifically implemented as follows: segmenting the data to be stored according to a specified length to obtain data of each segment.

[0096] The specified length in the embodiment of the present application is a data amount of a specified size. The embodiment of the present application does not limit the specific value of the specified length. The specific value of the specified length in the embodiment of the present application can be set according to the specific actual situation.

[0097] In order to ensure that the corresponding data can be obtained subsequently, in a possible implementation, after executing step 102, the first target storage address corresponding to each of the fragmented data is obtained based on the first starting storage address and the first data length.

[0098] In the embodiment of the present application, the first data length can be divided by the number of shard data to obtain the sub-data length corresponding to any shard data, and the first target storage address of each shard data can be obtained based on the first starting storage address and the sub-data length.

[0099] That is, the first storage address is added to the sub-data length to obtain the first target storage address corresponding to the first slice data. The first target storage address corresponding to the first slice data is added to the sub-data length to obtain the first target storage address corresponding to the second slice data. The first target storage address corresponding to the second slice data is added to the sub-data length to obtain the first target storage address corresponding to the third slice data... and so on.

[0100] For example, if the first starting address is 0 and the first data length is 100, and the number of slice data is 10, it is determined that the first target storage address of slice data 1 is 0-10, the first target storage address of slice data 2 is 10-20, and the first target storage address of slice data 3 is 20-30. The first target storage address of slice data 4 is 30-40. The first target storage address of slice data 5 is 40-50. The first target storage address of slice data 6 is 50-60. The first target storage address of slice data 7 is 60-70. The first target storage address of slice data 8 is 70-80. The first target storage address of slice data 9 is 80-90. The first target storage address of slice data 10 is 90-100.

[0101] Step 103: Determine the data type of the first shard data according to the data in the first shard data; wherein the first shard data is any one of the shard data;

[0102] In one embodiment, the data type of the first fragment data is obtained by the following two methods:

[0103] Method 1: If all characters in the first fragment data are the same, determining that the data type of the first fragment is the first specified type.

[0104] The first designated type in the embodiment of the present application may include multiple subtypes, such as the first type, the second type, and the third type.

[0105] In a possible implementation, if all characters in the first fragment data are first specified characters, the data type of the first fragment is determined to be the first type.

[0106] In the embodiment of the present application, the first designated character is 0, that is, all characters in the first fragment data are 0, and the data type of the first fragment data is determined to be the first type.

[0107] For example, if the first fragment data is "000000", it is determined that the data type of the first fragment data is the first type.

[0108] In a possible implementation, if each character in the first fragment data is a second designated character, the data type of the first fragment data is determined to be the second type, wherein the first designated character and the second designated character are different.

[0109] In the embodiment of the present application, the second designated character is 1, that is, all characters in the first fragment data are 1, and the data type of the first fragment data is determined to be the second type.

[0110] For example, if the first fragment data is "111111", the data type of the second fragment data is determined to be the second type.

[0111] Method three: if the characters in the first fragment data are different, determining the data type of the first fragment data to be the second specified type.

[0112] For example, if the first fragment data is "FJKD01", it is determined that the data type of the first fragment data is the third type.

[0113] It should be noted that the first specified type in the embodiment of the present application includes the first type and the second type, which is only used for illustration and does not limit the first specified type in the embodiment of the present application. The data types in which all characters in the first segment data in the embodiment of the present application are the same belong to the first specified data type in the embodiment of the present application.

[0114] Step 104: Obtain metadata information of the first shard data according to the data type of the first shard data; wherein the metadata information is used to describe key information of the first shard data, and the data volume of the metadata information is smaller than the data volume of the shard data;

[0115] In a possible implementation manner, step 104 may be specifically implemented as: determining the data type of the first slice data, the first data length, and the first target storage address as metadata information of the first slice data.

[0116] like Figure 2 As shown in the figure, it is a schematic diagram of the metadata information corresponding to each shard data. Figure 2 As can be seen, each shard data corresponds to a metadata information.

[0117] Step 105: If the data type of the first shard data is a first specified type, then storing metadata information of the first shard data, wherein the first specified type is a type of data with fixed characteristics;

[0118] In a possible implementation, step 105 may be specifically implemented as follows: if the data type of the first shard data is a first specified type, storing metadata information of the first shard data in a memory block corresponding to the target storage address.

[0119] Step 106: If the data type of the first shard data is a second specified type, then the first shard data and metadata information of the first shard data are stored, wherein the second specified type is a type in which the data does not have fixed characteristics.

[0120] In a possible implementation, if the data type of the first shard data is the second specified type, the first shard data and metadata information of the first shard data are stored in a memory block corresponding to the target storage address.

[0121] Therefore, in the embodiment of the present application, for the first specified type of sharded data, only the metadata information of the sharded data is recorded, and the actual data is not written to the disk. Moreover, since the amount of metadata information is much smaller than that of the sharded data, only a very small metadata information space is required to describe the data of a shard, which greatly reduces the amount of data read and written from the disk. For the second specified type of sharded data, in addition to recording the metadata information, the actual data still needs to be stored from the disk. Therefore, for various mixed data, after using metadata information to describe the feature data, when a large amount of feature data is mixed in the data, the disk access overhead caused by disk IO reading and writing will be greatly reduced.

[0122] Next, the method of reading data in the embodiment of the present application is introduced. Figure 3 As shown, it is a schematic diagram of the process of reading data, which may specifically include the following steps:

[0123] Step 301: receiving a data acquisition instruction sent by the user, wherein the data acquisition instruction includes a second starting storage address and a second data length;

[0124] Step 302: Obtain metadata information of each target slice data according to the second starting storage address and the second data length;

[0125] In the embodiment of the present application, the second starting storage address and the second data length are added to obtain a second ending storage address, and metadata information of each target shard data is obtained based on the second starting storage address and the second ending storage address.

[0126] For example, if the second starting storage address is 0 and the second data length is 100, the data storage address is determined to be 0-100, and metadata information within 0-100 is obtained.

[0127] Step 303: determining a memory block corresponding to the first target shard data through metadata information of the first target shard data, wherein the first target shard data is any one of the target shard data;

[0128] In a possible implementation manner, the memory block corresponding to the first target fragment data is determined in the following two ways:

[0129] Method 1: If the data type in the metadata information is a first specified type, the target memory block corresponding to the data type of the first target shard is determined by utilizing the pre-set correspondence between the data type and the memory block, and the target memory block is determined as the memory block corresponding to the first target shard data; wherein the first specified type includes multiple types.

[0130] The first specified type in the embodiment of the present application includes a first type and a second type. In the embodiment of the present application, the first type corresponds to a memory block, and the second type corresponds to a memory block. Corresponding data is pre-stored in each memory block. For example, the data stored in the memory block corresponding to the first type may be "000000". The data stored in the memory block corresponding to the second type may be "111111".

[0131] Method 2: If the data type in the metadata information is the second specified type, obtain the target storage address in the metadata information of the first target shard data, and determine the memory block corresponding to the target storage address as the memory block corresponding to the first target shard data.

[0132] Since the second designated type is data without fixed characteristics, the data of the second designated type stores actual data, so the memory block corresponding to the target storage address needs to be determined as the memory block corresponding to the first target fragment data.

[0133] like Figure 4 As shown in the figure, it is the corresponding relationship between each data type and the memory block. Figure 4 It can be seen that the memory blocks corresponding to the first type are all the same memory block, and the memory blocks corresponding to the second type are also the same memory block. However, the memory blocks corresponding to the second specified type are not the same memory blocks, and each memory block corresponding to the second specified type is different.

[0134] Therefore, for the first specified type of shard data, when the actual read data needs to be returned to the user, the corresponding shard data can be pointed to a fixed memory block, for example, the first type of shard data points to the ZERO memory block, and the second type of shard data points to the ONE memory block. At this time, the ZERO memory block and the ONE memory block are both fixed memory areas allocated globally. When the system is initialized, they are reserved and the content in the ZERO memory block is filled with all 0 data, and the content in the ONE memory block is filled with all 1 data. Similarly, when it is necessary to expand support for other data types, it is only necessary to reserve and allocate a global memory block area and fill the corresponding data according to the data characteristics. By using a global memory block, the characteristic data corresponding to the memory block can be directly returned to the user, avoiding repeated copying of memory data.

[0135] Step 304: Obtain sub-data corresponding to the first target fragment data from the memory block;

[0136] Step 305: Obtain target data according to the sub-data corresponding to each target fragment data;

[0137] In the embodiment of the present application, each target slice data includes a corresponding identifier, which is used to indicate the order of the target slice data.

[0138] In one embodiment, step 305 may be specifically implemented as follows: sorting the sub-data corresponding to the target slice data according to the identifiers of the target slice data, and concatenating the sub-data according to the sorting to obtain the target data.

[0139] Step 306: Send the target data to the user.

[0140] In the embodiment of the present application, the data transmission between each process in the controller may only transmit the slice information of the slice data, wherein the slice information includes the data type of the slice data, and then obtain the corresponding data based on the data type of the slice data.

[0141] The method of obtaining corresponding data based on the data type in the embodiment of the present application is the same as the acquisition method described above, and the embodiment of the present application will not be repeated here.

[0142] like Figure 5 As shown in the figure, it is a schematic diagram of each shard data to shard information. Figure 5It can be seen that each shard data has corresponding shard information. Therefore, when each shard data needs to be transmitted to other processes through the network, the shard information of each shard data needs to be maintained and sent to the other process as part of the network message for parsing and processing. Since the data type of the shard data can be known from the shard information, for the first specified type of data shard, it is not necessary to carry the actual data for transmission in the network. You only need to know the data type of the shard data to obtain the actual data. Avoid memory copying and network bandwidth occupation overhead. Reduce the delay of read access and improve the service quality of timely response to users.

[0143] Next, combine Figure 6 The data storage method in the embodiment of the present application is described, which may specifically include the following steps:

[0144] Step 601: receiving a data storage instruction sent by a user; wherein the data storage instruction includes data to be stored and a first storage address;

[0145] Step 602: Slice the data to be stored to obtain data of each slice;

[0146] Step 603: based on the first starting storage address and the first data length, obtaining first target storage addresses corresponding to the respective slice data;

[0147] Step 604: Determine the data type of the first shard data according to the data in the first shard data; wherein the first shard data is any one of the shard data;

[0148] Step 605: Determine the data type of the first slice data, the first data length, and the first target storage address as metadata information of the first slice data;

[0149] The metadata information is used to describe key information of the first shard data, and the data volume of the metadata information is smaller than the data volume of the shard data;

[0150] Step 606: If the data type of the first shard data is a first specified type, storing metadata information of the first shard data;

[0151] Wherein, the first specified type is a type of data having fixed characteristics;

[0152] Step 607: If the data type of the first shard data is the second specified type, storing the first shard data and metadata information of the first shard data;

[0153] The second specified type is a type in which the data does not have fixed characteristics.

[0154] Step 608: receiving a data acquisition instruction sent by the user, wherein the data acquisition instruction includes a second starting storage address and a second data length;

[0155] Step 609: Obtain metadata information of each target slice data according to the second starting storage address and the second data length;

[0156] Step 610: If the data type in the metadata information is the first specified type, a target memory block corresponding to the data type of the first target slice is determined by using a preset correspondence between the data type and the memory block, and the target memory block is determined as a memory block corresponding to the first target slice data;

[0157] The first designated type includes a plurality of subtypes; the first target slice data is any one of the target slice data;

[0158] Step 611: if the data type in the metadata information is the second specified type, obtaining the target storage address in the metadata information of the first target shard data, and determining the memory block corresponding to the target storage address as the memory block corresponding to the first target shard data;

[0159] Step 612: Obtain sub-data corresponding to the first target fragment data from the memory block;

[0160] Step 613: Obtain target data according to the sub-data corresponding to each target fragment data;

[0161] Step 614: Send the target data to the user.

[0162] Based on the same inventive concept, the present application also provides a data storage device, see Figure 7 , the device 700 comprises:

[0163] The receiving module 710 is used to receive a data storage instruction sent by a user; wherein the data storage instruction includes data to be stored;

[0164] The data segmentation module 720 is used to segment the data to be stored to obtain data of each segment;

[0165] A data type determination module 730 is used to determine the data type of the first shard data according to the data in the first shard data; wherein the first shard data is any one of the shard data;

[0166] The metadata information determination module 740 is used to obtain metadata information of the first shard data according to the data type of the first shard data; wherein the metadata information is used to describe key information of the first shard data, and the data volume of the metadata information is smaller than the data volume of the shard data;

[0167] A first storage module 750 is configured to store metadata information of the first shard data if the data type of the first shard data is a first specified type, wherein the first specified type is a type of data with fixed characteristics;

[0168] The second storage module 760 is used to store the first shard data and metadata information of the first shard data if the data type of the first shard data is a second specified type, wherein the second specified type is a type of data without fixed characteristics.

[0169] In a possible implementation manner, the data storage instruction further includes a first starting storage address and a first data length; and the device further includes:

[0170] The target storage address determination module 770 is used for performing data segmentation on the data to be stored to obtain the data of each slice, and then obtaining the first target storage address corresponding to each slice data based on the first starting storage address and the first data length;

[0171] The metadata information determination module 740 is specifically used to:

[0172] The data type of the first slice data, the first data length and the first target storage address are determined as metadata information of the data.

[0173] In a possible implementation manner, the first storage module 750 is specifically configured to:

[0174] Storing metadata information of the first slice data in a memory block corresponding to the first target storage address;

[0175] The second storage module 760 is specifically used for:

[0176] The first shard data and metadata information of the first shard data are stored in a memory block corresponding to the first target storage address.

[0177] In a possible implementation manner, the receiving module 710 is further configured to:

[0178] receiving a data acquisition instruction sent by the user, wherein the data acquisition instruction includes a second starting storage address and a second data length;

[0179] Obtaining metadata information of each target slice data according to the second starting storage address and the second data length;

[0180] Determine the memory block corresponding to the first target shard data through metadata information of the first target shard data, and obtain sub-data corresponding to the first target shard data from the memory block; wherein the first target shard data is any one of the target shard data;

[0181] Obtaining target data according to the sub-data corresponding to each target fragment data;

[0182] The target data is sent to the user.

[0183] In a possible implementation manner, the receiving module 710 is further configured to:

[0184] If the data type in the metadata information is a first specified type, a target memory block corresponding to the data type of the first target shard is determined by using a preset correspondence between the data type and the memory block, and the target memory block is determined as a memory block corresponding to the first target shard data; wherein the first specified type includes multiple types;

[0185] If the data type in the metadata information is the second specified type, the target storage address in the metadata information of the first target shard data is obtained, and the memory block corresponding to the target storage address is determined as the memory block corresponding to the first target shard data.

[0186] In a possible implementation, the data segmentation module 720 is specifically configured to:

[0187] The data to be stored is segmented according to the specified length to obtain data of each segment.

[0188] In a possible implementation manner, the data type includes a first type, a second type, and a third type, wherein the first type and the second type are the first specified type, and the third type is the second specified type;

[0189] The data type determination module 730 is specifically used to:

[0190] If all characters in the first fragment data are first specified characters, determining that the data type of the first fragment is the first type;

[0191] If all characters in the first fragment data are second designated characters, determining that the data type of the first fragment data is the second type, wherein the first designated character and the second designated character are different;

[0192] If the characters in the first fragment data are different, it is determined that the data type of the first fragment data is the third type.

[0193] Based on the same inventive concept, an all-flash array controller is also provided in the embodiment of the present application. The all-flash array controller can realize the functions of the aforementioned data storage device, refer to Figure 8 , the all-flash array controller includes:

[0194] At least one processor 801, and a memory 802 connected to the at least one processor 801. The specific connection medium between the processor 801 and the memory 802 is not limited in the embodiment of the present application. Figure 8 In the example, the processor 801 and the memory 802 are connected via a bus 800. Figure 8 The connections between other components are shown in bold lines, and are not intended to be limiting. The bus 800 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, Figure 8 Only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. Alternatively, the processor 801 can also be called a controller, and there is no limitation on the name.

[0195] In the embodiment of the present application, the memory 802 stores instructions that can be executed by at least one processor 801. The at least one processor 801 can execute the all-flash array controller method discussed above by executing the instructions stored in the memory 802. The processor 801 can implement Figure 7 The functions of each module in the device shown.

[0196] Among them, the processor 801 is the control center of the device, and can use various interfaces and lines to connect the various parts of the entire control device. By running or executing instructions stored in the memory 802 and calling the data stored in the memory 802, the various functions of the device and process data, the device can be monitored as a whole.

[0197] In one possible design, the processor 801 may include one or more processing units, and the processor 801 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 801. In some embodiments, the processor 801 and the memory 802 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on separate chips.

[0198] Processor 801 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the data storage method disclosed in the embodiments of the present application can be directly embodied as a hardware processor to be executed, or a combination of hardware and software modules in the processor can be executed.

[0199] The memory 802 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 802 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 802 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 802 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0200] By programming the processor 801, the code corresponding to the data storage method described in the above embodiment can be fixed into the chip, so that the chip can execute the code when running. Figure 1The steps of the data storage method in the embodiment shown are as follows: How to design and program the processor 801 is a technique known to those skilled in the art and will not be described in detail here.

[0201] An embodiment of the present application also provides a computer-readable storage medium that stores computer-executable instructions required to execute the above-mentioned processor, which includes a program required to execute the above-mentioned processor.

[0202] In some possible implementations, various aspects of the data storage method provided by the present application may also be implemented in the form of a program product, which includes a program code. When the above-mentioned program product is run on an electronic device, the above-mentioned program code is used to enable the above-mentioned electronic device to execute the steps of the data storage method according to various exemplary embodiments of the present application described above in this specification.

[0203] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0204] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (apparatus), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0205] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0206] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0207] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0208] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A data storage method, characterized in that: Applied in a controller, the method comprises: Receiving a data storage instruction sent by a user; wherein the data storage instruction includes data to be stored; Slicing the data to be stored to obtain data of each slice; Determine the data type of the first shard data according to the data in the first shard data; wherein the first shard data is any one of the shard data; Obtaining metadata information of the first shard data according to the data type of the first shard data; wherein the metadata information is used to describe key information of the first shard data, and the data volume of the metadata information is smaller than the data volume of the shard data; If the data type of the first shard data is a first specified type, storing metadata information of the first shard data, wherein the first specified type is a type with fixed characteristics of the data; If the data type of the first shard data is a second specified type, the first shard data and metadata information of the first shard data are stored, wherein the second specified type is a type in which data does not have fixed characteristics.

2. The method according to claim 1, characterized in that The data storage instruction also includes a first starting storage address and a first data length; After the data to be stored is segmented to obtain data of each segment, the method further includes: Based on the first starting storage address and the first data length, obtaining first target storage addresses corresponding to the respective slice data; The obtaining metadata information of the first shard data according to the data type of the first shard data includes: The data type of the first slice data, the first data length and the first target storage address are determined as metadata information of the first slice data.

3. The method according to claim 2, characterized in that The metadata information storing the first shard data includes: Storing metadata information of the first slice data in a memory block corresponding to the first target storage address; Storing the first shard data and metadata information of the first shard data includes: The first shard data and metadata information of the first shard data are stored in a memory block corresponding to the first target storage address.

4. The method according to claim 2, characterized in that: The method further comprises: Receiving a data acquisition instruction sent by the user, wherein the data acquisition instruction includes a second starting storage address and a second data length; Obtaining metadata information of each target slice data according to the second starting storage address and the second data length; Determine the memory block corresponding to the first target shard data through metadata information of the first target shard data, and obtain sub-data corresponding to the first target shard data from the memory block; wherein the first target shard data is any one of the target shard data; Obtaining target data according to the sub-data corresponding to each target fragment data; The target data is sent to the user.

5. The method according to claim 4, characterized in that The determining, by using metadata information of the first target shard data, a memory block corresponding to the first target shard data includes: If the data type in the metadata information is a first specified type, a target memory block corresponding to the data type of the first target shard is determined by using a preset correspondence between the data type and the memory block, and the target memory block is determined as a memory block corresponding to the first target shard data; wherein the first specified type includes a plurality of subtypes; If the data type in the metadata information is the second specified type, the target storage address in the metadata information of the first target shard data is obtained, and the memory block corresponding to the target storage address is determined as the memory block corresponding to the first target shard data.

6. The method according to claim 1, characterized in that The data to be stored is segmented to obtain data of each segment, including: The data to be stored is segmented according to the specified length to obtain data of each segment.

7. The method according to claim 1, characterized in that The determining, according to the data in the first shard data, the data type of the first shard data includes: If all characters in the first fragment data are the same, determining that the data type of the first fragment is the first specified type; If the characters in the first segment data are different, the data type of the first segment data is determined to be the second specified type.

8. A data storage device, characterized in that: The device comprises: A receiving module, used to receive a data storage instruction sent by a user; wherein the data storage instruction includes data to be stored; A data segmentation module is used to segment the data to be stored to obtain data of each segment; A data type determination module, used to determine the data type of the first shard data according to the data in the first shard data; wherein the first shard data is any one of the shard data; A metadata information determination module, configured to obtain metadata information of the first shard data according to the data type of the first shard data; wherein the metadata information is used to describe key information of the first shard data, and the data volume of the metadata information is smaller than the data volume of the shard data; A first storage module, configured to store metadata information of the first shard data if the data type of the first shard data is a first specified type, wherein the first specified type is a type of data with fixed characteristics; The second storage module is used to store the first shard data and metadata information of the first shard data if the data type of the first shard data is a second specified type, wherein the second specified type is a type of data that does not have fixed characteristics.

9. A controller, characterized in that: include: A memory for storing program instructions; A processor is used to call the program instructions stored in the memory, and execute the steps included in any one of claims 1-7 according to the obtained program instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images