Data storage method and device

Abstract

The present application provides a data storage method and device. The method includes: obtaining a storage configuration request, where the storage configuration request includes: the first capacity of a first storage device; allocating, according to the first capacity, the first storage device with a capacity corresponding to the first capacity and a second storage device with a second capacity, where the memory access speed of the first storage device is lower than that of the second storage device. When an application requests to read a certain part of data, if relevant data is found in the second storage device but the accuracy of the data does not meet the accuracy requirements of the application, corresponding data can be read from the first storage device and returned to the application. Since the first storage device stores all the original data sent by the application in full, the data read from the first storage device must be accurate, thus ensuring that the accuracy of the data returned to the application always meets the accuracy requirements of the application.

Description

Technical Field

[0001] This application relates to storage technology, and in particular, to a data storage method and device. Background Art

[0002] With the advent of the era of artificial intelligence, the types of artificial intelligence applications are becoming increasingly rich, such as image applications, video applications, and voice applications. Moreover, with the continuous upgrade of applications and the increasing number of users, the relevant data to be stored has also increased exponentially. Enterprises need to spend a large amount of storage costs to store this data. However, a large part of the above-mentioned applications can tolerate a certain degree of accuracy loss. For example, in face recognition applications, even if a local area of the face is missing, it will not affect the recognition effect of the application. Based on such applications that can tolerate a certain degree of accuracy loss, approximate storage technology has emerged.

[0003] The prior art provides an approximate storage scheme based on a solid-state memory (Solid-State Drive, SSD, hereinafter referred to as flash memory). After the solid-state memory receives the original data input by the application, it performs approximate storage on the original data to generate approximate data. When the application requests to read a certain part of the data, the corresponding approximate data is directly returned to the application. However, when the accuracy of the approximate data generated by the solid-state memory does not meet the accuracy requirements of the application, the prior art cannot obtain accurate data to return to the application. Summary of the Invention

[0004] This application provides a data storage method and device, which are used to solve the problem that when the accuracy of the approximate data generated by the solid-state memory in the prior art does not meet the accuracy requirements of the application, accurate data cannot be obtained to return to the application.

[0005] In a first aspect, this application provides a data storage method, including:

[0006] Obtain a storage configuration request, where the storage configuration request includes: the first capacity of the first storage device;

[0007] According to the first capacity, allocate the first storage device with a capacity corresponding to the first capacity and a second storage device with a second capacity, where the access speed of the first storage device is lower than that of the second storage device.

[0008] In a possible design, the storage configuration request further includes: a proportional parameter;

[0009] The step of allocating the first storage device with a capacity corresponding to the first capacity and a second storage device with a second capacity according to the first capacity includes:

[0010] Determine the second capacity according to the first capacity and the proportional parameter;

[0011] Allocate a first storage device with a capacity corresponding to the first capacity and a second storage device with a capacity corresponding to the second capacity according to the first capacity and the second capacity.

[0012] In a possible design, the method further includes:

[0013] Receive a data storage request sent by an application, where the data storage request carries the original data requested to be stored by the application;

[0014] Transmit the original data to the first storage device and the second storage device, so that the first storage device stores all of the original data, and the second storage device stores data with a proportion equal to a preset ratio obtained from the original data according to a preset rule;

[0015] Wherein, the size of the space occupied by the original data is less than or equal to the first capacity, and the size of the space occupied by the data with a proportion equal to the preset ratio obtained from the original data is less than or equal to the second capacity.

[0016] In a possible design, the method further includes:

[0017] Receive a data storage request sent by an application, where the data storage request carries the original data requested to be stored by the application;

[0018] Transmit the original data to the second storage device, so that the second storage device stores data with a proportion equal to a preset ratio obtained from the original data according to a preset rule, and at the same time, the second storage device transmits the original data to the first storage device for full storage.

[0019] In a possible design, the method further includes:

[0020] Receive a data reading request sent by the application, where the data reading request carries a first identifier of the data requested to be read;

[0021] Check whether the data related to the first identifier is stored in the second storage device;

[0022] If the data related to the first identifier is not found in the second storage device, read out the data corresponding to the first identifier stored in the first storage device and return it to the application.

[0023] In a possible design, the method further includes:

[0024] Predict the data that the application will request to read next according to the first identifier;

[0025] Read the data requested by the application next time from the first storage device and store it in the second storage device.

[0026] In a possible design, the storage configuration request further includes: a precision parameter; the method further includes:

[0027] If data related to the first identifier is found in the second storage device, and the precision loss of the data related to the first identifier stored in the second storage device is greater than the precision parameter, then read out the data corresponding to the first identifier stored in the first storage device and return it to the application.

[0028] In a possible design, the method further includes:

[0029] If all the data corresponding to the first identifier is found in the second storage device, and the precision loss of all the data corresponding to the first identifier stored in the second storage device is less than or equal to the precision parameter, then read out all the data corresponding to the first identifier stored in the second storage device and return it to the application.

[0030] In a possible design, the method further includes:

[0031] If partial data corresponding to the first identifier is found in the second storage device, and the precision loss of the partial data corresponding to the first identifier stored in the second storage device is less than or equal to the precision parameter, then read out the partial data and return it to the application, and at the same time read out the remaining data except the above partial data in the data corresponding to the first identifier stored in the first storage device and store it in the second storage device; read out the remaining data stored in the second storage device and return it to the application.

[0032] In a second aspect, the present application provides a data storage device, including:

[0033] An acquisition module, configured to acquire a storage configuration request, where the storage configuration request includes: a first capacity of a first storage device;

[0034] An allocation module, configured to allocate, according to the first capacity, the first storage device with a capacity corresponding to the first capacity and a second storage device with a second capacity, where the access speed of the first storage device is lower than that of the second storage device.

[0035] In a possible design, the storage configuration request further includes: a ratio parameter; specifically, the allocation module is configured to:

[0036] Determine the second capacity according to the first capacity and the ratio parameter; allocate the first storage device with a capacity corresponding to the first capacity and the second storage device with a capacity corresponding to the second capacity according to the first capacity and the second capacity.

[0037] In a possible design, the above data storage device further includes:

[0038] A receiving module, configured to receive a data storage request sent by an application, where the data storage request carries the original data requested to be stored by the application; a processing module, configured to transmit the original data to the first storage device and the second storage device, so that the first storage device stores all of the original data, and the second storage device stores data with a proportion equal to a preset ratio obtained from the original data according to a preset rule;

[0039] Wherein, the size of the space occupied by the original data is less than or equal to the first capacity, and the size of the space occupied by the data with a proportion equal to the preset ratio obtained from the original data is less than or equal to the second capacity.

[0040] In a possible design, the processing module is further configured to:

[0041] Transmit the original data to the second storage device, so that the second storage device stores data with a proportion equal to a preset ratio obtained from the original data according to a preset rule, and at the same time, the second storage device transmits the original data to the first storage device for full storage.

[0042] In a possible design, the receiving module is further configured to receive a data reading request sent by the application, where the data reading request carries a first identifier of the data requested to be read; the processing module is further configured to: check whether there is data related to the first identifier stored in the second storage device; if no data related to the first identifier is found in the second storage device, read out the data corresponding to the first identifier stored in the first storage device and return it to the application.

[0043] In a possible design, the processing module is further configured to: predict the data requested to be read by the application next time according to the first identifier; read out the data requested to be read by the application next time from the first storage device and store it in the second storage device.

[0044] In a possible design, the processing module is further configured to: if data related to the first identifier is found in the second storage device and the precision loss of the data related to the first identifier stored in the second storage device is greater than the precision parameter, read out the data corresponding to the first identifier stored in the first storage device and return it to the application.

[0045] In a possible design, the processing module is further configured to: if all data corresponding to the first identifier is found in the second storage device and the precision loss of all data corresponding to the first identifier stored in the second storage device is less than or equal to the precision parameter, read out all data corresponding to the first identifier stored in the second storage device and return it to the application.

[0046] In a possible design, the processing module is further configured to: if partial data corresponding to the first identifier is found in the second storage device and the precision loss of the partial data corresponding to the first identifier stored in the second storage device is less than or equal to the precision parameter, read out the partial data and return it to the application, and at the same time read out the remaining data except the above partial data from the data corresponding to the first identifier stored in the first storage device and store it in the second storage device; read out the remaining data stored in the second storage device and return it to the application.

[0047] In a third aspect, the present application provides a readable storage medium, on which a computer program is stored; when the computer program is executed, the above data storage method is implemented.

[0048] In a fourth aspect, the present application provides a data storage device, including: a memory and a processor;

[0049] The memory is used to store program instructions, and the processor is used to call the program instructions in the memory to execute the above data storage method.

[0050] In a fifth aspect, the present application provides a cloud server, including the data storage device, the first storage device, and the second storage device involved in the fourth aspect, where the access speed of the first storage device is lower than that of the second storage device.

[0051] The data storage method and device provided by the present application pre-allocate a first storage device and a second storage device that meet the capacity requirements set by the user for an application. The first storage device is a slow storage device with a slow memory access speed, and the second storage device is a fast storage device with a fast memory access speed. When an application requests to read a certain part of data, if the relevant data is found in the second storage device, the data is read from the second storage device and returned to the application, ensuring efficient data reading. If the relevant data is found in the second storage device, but the accuracy of the data does not meet the accuracy requirements of the application, the corresponding data is read from the first storage device and returned to the application. Since the first storage device stores all the original data sent by the application in full, the data read from the first storage device must be accurate, thus ensuring that the accuracy of the data returned to the application always meets the accuracy requirements of the application and improving the user experience of storing and reading data through the application. Moreover, the slow storage device is inexpensive, so that the storage cost of the storage method provided by the present application does not change significantly compared with the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a schematic diagram of the data storage framework of the prior art;

[0053] Figure 2 It is a schematic diagram of the data storage framework provided by the present application;

[0054] Figure 3 It is a schematic flowchart of the first embodiment of the data storage method provided by the present application;

[0055] Figure 4 It is a schematic flowchart of the second embodiment of the data storage method provided by the present application;

[0056] Figure 5 It is a schematic diagram of the principle of determining the target resource pool from the second storage device provided by the present application;

[0057] Figure 6 It is a schematic diagram of the structure of the data storage device 600 provided by the present application;

[0058] Figure 7 It is a schematic diagram of the hardware structure of the data storage device 700 provided by the present application;

[0059] Figure 8 It is a schematic diagram of the hardware structure of the cloud server 800 provided by the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0060] To make the objectives, technical solutions, and advantages of this application clearer, the following will clearly and completely describe the technical solutions in this application in conjunction with the accompanying drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without creative efforts fall within the scope of protection of this application.

[0061] In this application, it should be explained that the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance. In addition, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects and indicates that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are an "or" relationship. "At least one (item)" or its similar expression refers to any combination of these items, including any combination of single item (item) or plural items (items). For example, at least one (item) of a, b, or c can represent: a alone, b alone, c alone, the combination of a and b, the combination of a and c, the combination of b and c, or the combination of a, b, and c, where a, b, and c can be single or multiple.

[0062] Some applications can tolerate a certain degree of precision loss. For example, in face recognition applications, even if a local area of the face is missing, it will not affect the recognition effect of the application. For such applications, approximate storage technology can be used to store relevant data. Figure 1 The following is a data storage framework diagram for such applications in the prior art provided for this application. Figure 1 The shown data storage framework diagram includes an application and a solid-state memory. In the prior art, after the solid-state memory receives the original data input by the application, it performs approximate storage on the original data to generate approximate data. When the application requests to read a certain part of the data, the corresponding approximate data is directly returned to the application. However, when the precision of the approximate data generated by the solid-state memory does not meet the precision requirements of the application, the prior art cannot obtain accurate data to return to the application.

[0063] Based on the above technical problems existing in the prior art, this application provides a data storage method, which can be based on Figure 2 the shown data storage framework for implementation. Figure 2The data storage framework shown includes: an application, a control module, a first storage device, and a second storage device. Among them, the control module, the first storage device, and the second storage device can be deployed on a cloud server. The application can be a local application installed on the terminal device held by the user, or an application deployed on the cloud server.

[0064] It should be noted that: Figure 2 The application in [[ ]] is an application that can tolerate a certain degree of precision loss. For example: image applications, video applications, voice applications, etc. In addition, Figure 2 the access speed of the first storage device in [[ ]] is lower than that of the second storage device. For example: the first storage device can be a serial hard disk, and the second storage device can be a solid-state drive.

[0065] It can be understood that the above-mentioned serial hard disk and solid-state drive are only examples. As long as there is a difference in access speed between two storage devices, the one with the slower access speed can be used as the first storage device, and the one with the faster access speed can be used as the second storage device. The present application does not limit the specific types of the first storage device and the second storage device.

[0066] Before the cloud server stores the data sent by the application, it is necessary to first allocate the first storage device and the second storage device with corresponding capacities for the application. The following method can be used to allocate the first storage device and the second storage device.

[0067] In the first possible implementation manner, first, obtain a storage configuration request. The storage configuration request includes the capacity of the first storage device. For the convenience of distinction, the capacity of the first storage device will be referred to as the first capacity hereinafter, and the first capacity is set by the user. After obtaining the above storage configuration request, allocate the first storage device with a capacity corresponding to the first capacity and the second storage device with a second capacity for the application.

[0068] The following is an example for illustration:

[0069] Suppose the first capacity of the first storage device included in the obtained storage configuration request is 100G. Then, a first storage device with a capacity of 100G can be allocated for the application, and a second storage device with a corresponding capacity can be allocated for the application according to a preset ratio. For example, a second storage device with a capacity of 10G can be allocated for the application according to a ratio of 10:1. It can be understood that the 10:1 here is only an example, and the second storage device with a corresponding capacity can also be allocated for the application according to other preset ratios. The present application is not limited thereto.

[0070] In the second possible implementation, based on the first possible implementation above, the storage configuration request further includes a ratio parameter. Similar to the first capacity above, this ratio parameter is also set by the user. In this case, after obtaining the storage configuration request, the second capacity of the second storage device allocated to the application can be determined first according to the first capacity and this ratio parameter, and then the first storage device with a capacity corresponding to the first capacity and the second storage device with a capacity corresponding to the second capacity are allocated to the application.

[0071] The following is an example for illustration:

[0072] Suppose the first capacity of the first storage device included in the obtained storage configuration request is 100G and the ratio parameter is 10:3. Then, according to the first capacity and the ratio parameter, the second capacity of the second storage device that needs to be allocated to the application can be determined to be 30G. When allocating the storage device, a 100G first storage device and a 30G second storage device can be directly allocated to the application. Since the memory access speed of the second storage device is greater than that of the first storage device, the larger the proportion of the second storage device in the ratio parameter set by the user, the faster the data storage and reading experience that the user experiences when operating the application.

[0073] In the third possible implementation, based on the first possible implementation or the second possible implementation above, the storage configuration request further includes a precision parameter. Similar to the first capacity and the ratio parameter above, this precision parameter is also set by the user. When the user requests to read a certain part of the data, this precision parameter can be used to detect whether the precision of the relevant data stored in the second storage device meets the precision requirements of the application. In the case where the judgment result is not satisfied, the control module can read the corresponding data from the first storage device and return it to the application. Since the first storage device stores all the original data sent by the application in full, the data read from the first storage device must be accurate data. Compared with the prior art, the present application can still return accurate data in the case where the precision of the relevant data stored in the second storage device does not meet the precision requirements of the application, improving the user experience of accessing and storing data through the application.

[0074] The following is an example for illustration:

[0075] Assume that the first capacity of the first storage device included in the obtained storage configuration request is 100G, the ratio parameter is 10:3, and the precision parameter is 1%. First, determine the second capacity as 30G according to the first capacity and the ratio parameter, and then directly allocate a 100G first storage device and a 30G second storage device for the application. When the user requests to read a certain part of the data later, if it is detected that the precision loss of the relevant data stored in the second storage device is greater than 1%, the control module can read the corresponding data from the first storage device and return it to the application. It can be seen that the user's setting of the precision parameter can ensure that the data returned by the control module to the application can meet the precision requirements of the application.

[0076] The following combines specific embodiments to elaborate in detail on the data storage method provided by this application based on Figure 2 the data storage framework diagram shown. This application elaborates on the data storage method provided by this application in two scenarios. These two scenarios are the data storage scenario and the data reading scenario.

[0077] Embodiment 1

[0078] For the data storage scenario, Figure 3 is a schematic flowchart of Embodiment 1 of the data storage method provided by this application. The data storage method provided in this embodiment can be executed by the Figure 2 control module in. Combining Figure 2 and Figure 3 shown, the data storage method provided in this embodiment includes:

[0079] S301. Receive a data storage request sent by the application, and this data storage request carries the original data requested by the application to be stored.

[0080] Among them, the size of the space occupied by the original data is less than or equal to the first capacity mentioned above, and the size of the space occupied by the data obtained from the original data with a proportion equal to the preset ratio is less than or equal to the second capacity mentioned above.

[0081] Specifically, after receiving the data storage request sent by the application, the original data carried in this data storage request can be stored in the first storage device and the second storage through S302 or S303.

[0082] S302. Transmit the original data to the first storage device and the second storage device, so that the first storage device stores the original data in full, and the second storage device stores the data obtained from the original data with a proportion equal to the preset ratio according to the preset rule.

[0083] In a possible implementation, the above-mentioned preset ratio can correspond to the ratio parameter in the storage configuration request. For example, if the ratio of the capacity of the first storage device to the capacity of the second storage device indicated by the ratio parameter is 10:1, then the above-mentioned preset ratio can also be set to 1:10. It can be understood that the above-mentioned preset ratio may not correspond to the ratio parameter, and this application is not limited thereto.

[0084] Optionally, the above-mentioned preset rule can be the first-in-first-out rule, the last-in-first-out rule, the least recently used rule, etc.

[0085] The following takes the preset ratio of 1:10 and the preset rule of first-in-first-out as an example to illustrate the processes of S301 and S302 above:

[0086] The control module receives a data storage request sent by the application. The data storage request carries 10G of original data, and transmits the 10G of original data to the first storage device and the second storage device at the same time. The first storage device stores the 10G of original data in full, while the second storage device obtains 1G of the original data subsequently stored in the second storage device from the 10G of original data according to the first-in-first-out rule for storage.

[0087] Optionally, the error correction code (ECC) function of the second storage device can be turned off, so that the second storage device can still continue to be used when the proportion of defective blocks reaches a certain level, indirectly increasing the life of the second storage device and reducing the cost. At the same time, because the ECC function is turned off, there is no need to check during data reading and writing, and the memory access speed of the second storage device is further improved.

[0088] S303. Transmit the original data to the second storage device, so that the second storage device obtains data with a proportion equal to the preset ratio from the original data according to the preset rule for storage, and at the same time, the second storage device transmits the original data to the first storage device for full storage.

[0089] Specifically, for the preset ratio and the preset rule, refer to the above description, and this application will not elaborate here.

[0090] The following takes the preset ratio of 1:10 and the preset rule of first-in-first-out as an example to illustrate the processes of S301 and S303 above:

[0091] The control module receives a data storage request sent by an application. The data storage request carries 10G of original data, and this 10G of original data is only transmitted to the second storage device. The second storage device obtains 1G of the original data that is subsequently stored in the second storage device from the 10G of original data according to the first-in, first-out rule for storage. At the same time, the second storage device also sends all the original data transmitted to the second storage device to the first storage device for full-volume storage.

[0092] In the data storage method provided in this embodiment, the full-volume original data is stored in the first storage device, and part of the original data is stored in the second storage device. When it is convenient for a user to read certain data later, in the case where the accuracy of the relevant data stored in the second storage device does not meet the accuracy requirements of the application, the corresponding accurate data can still be read from the first storage device and returned to the application, improving the user experience of storing and reading data through the application.

[0093] Embodiment Two

[0094] For the data reading scenario, Figure 4 is a schematic flowchart of Embodiment Two of the data storage method provided in this application. As shown in combination with Figure 2 and Figure 4 The data storage method provided in this embodiment includes:

[0095] S401. Receive a data reading request sent by an application. The data reading request carries the first identifier of the data to be read.

[0096] S402. Check whether there is data related to the first identifier stored in the second storage device.

[0097] S403. Return data to the application according to the search result.

[0098] Specifically, the above search results are divided into the following situations:

[0099] The first situation: No data related to the first identifier is found in the second storage device.

[0100] The second situation: Data related to the first identifier is found in the second storage device, but the accuracy loss of the data related to the first identifier stored in the second storage device is greater than the accuracy parameter.

[0101] The third situation: All the data corresponding to the first identifier is found in the second storage device, and the accuracy loss of all the data corresponding to the first identifier stored in the second storage device is less than or equal to the accuracy parameter.

[0102] In the fourth case, partial data corresponding to the first identifier is found in the second storage device, and the precision loss of the partial data corresponding to the first identifier stored in the second storage device is less than or equal to the precision parameter.

[0103] The implementation of returning data to the application for each of the above cases will be described below.

[0104] In the first case, if no data related to the first identifier is found in the second storage device, the data corresponding to the first identifier stored in the first storage device is read out and returned to the application.

[0105] For example, the data requested by the application to be read is pictures numbered from 1 to 100. After receiving the data read request, the control module first searches for the pictures numbered from 1 to 100 in the second storage device. If they are not found in the second storage device, since the first storage device stores all the original data, the pictures numbered from 1 to 100 stored in the first storage device are directly read out and returned to the application.

[0106] Optionally, in the case where no data related to the first identifier is found in the second storage device, the data that the application will request to read next can be predicted based on the first identifier; and the data that the application will request to read next is read out from the first storage device and stored in the second storage device.

[0107] Continuing with the above example,

[0108] The data requested by the application in the previous request to read is pictures numbered from 1 to 100. It can be predicted through a prediction algorithm that the data that the application will request to read next may be pictures numbered from 101 to 200. Then, the pictures numbered from 101 to 200 can be read out from the first storage device in advance and stored in the second storage device. So that when the application requests to read the pictures numbered from 101 to 200, they can be quickly read from the second storage device with a faster memory access speed.

[0109] In the second case, if data related to the first identifier is found in the second storage device, however, the precision loss of the data related to the first identifier stored in the second storage device is greater than the precision parameter, the data corresponding to the first identifier stored in the first storage device is read out and returned to the application.

[0110] Specifically, there are two possibilities for finding the data related to the first identifier in the second storage device. One is to find all the data corresponding to the first identifier in the second storage device, and the other is to find partial data corresponding to the first identifier in the second storage device. Regardless of which of the above possibilities, as long as it is detected that the precision loss of the data related to the first identifier stored in the second storage device is greater than the precision parameter, the data corresponding to the first identifier stored in the first storage device is read out and returned to the application.

[0111] Specifically, the following method can be used to determine the precision loss of the data related to the first identifier stored in the second storage device:

[0112] Assume that the second storage device is a flash memory, and the data related to the first identifier stored in the second storage device uses a total of N_a flash memory blocks. Among these blocks, assume the number of bad blocks is N. Then the precision loss of the data related to the first identifier in the second storage device is N / N_a. If this N / N_a is greater than the precision parameter, it is considered that the precision loss of the data related to the first identifier stored in the second storage device is greater than the precision parameter.

[0113] Optionally, before detecting whether the precision loss of the data related to the first identifier stored in the second storage device is greater than the precision parameter, it can be first detected whether the data related to the first identifier stored in the second storage device is readable. If it is readable, then continue to detect whether the precision loss of the data is greater than the precision parameter. If it is not readable, the data corresponding to the first identifier stored in the first storage device is read out and returned to the application.

[0114] It should be noted that: if the above-described storage configuration request includes the precision parameter set by the user, then when detecting whether the precision loss of the data is greater than the precision parameter, the precision parameter set by the user is directly used for comparison; if the above-described storage configuration request does not include the precision parameter, then when detecting whether the precision loss of the data is greater than the precision parameter, the preset precision parameter is used for comparison.

[0115] Optionally, after reading out the data corresponding to the first identifier stored in the first storage device and returning it to the application, these data can be further stored in the target resource pool of the second storage device, so that when the user requests to read this part of the data next time, it can be directly read from the second storage device with a faster access speed. The process of determining the target resource pool from the second storage device is introduced below.

[0116] See Figure 5As shown in the figure, in the first step, select the resource pools in all the resource pools of the second storage device whose resource block failure rate is less than the above precision parameter. Assume that the selected resource pools are Flash 1, Flash 2, Flash 3, Flash 4, and Flash 5. In the second step, sort the selected resource pools in ascending order of the resource block failure rate, and the sorting result is Flash 1, Flash 3, Flash 5, Flash 4, Flash 2. In the third step, arbitrarily select one from the top K resource pools as the target resource pool, where K is a positive integer. Figure 5 In Figure 5 , K is taken as 3 for illustration. For example, Flash 3 can be selected as the target resource pool.

[0117] In the third case, if all the data corresponding to the first identifier is found in the second storage device, and the precision loss of all the data corresponding to the first identifier stored in the second storage device is less than or equal to the precision parameter, then all the data corresponding to the first identifier stored in the second storage device is read out and returned to the application.

[0118] Corresponding to the above example, the data requested by the application to read is pictures numbered from 1 to 100. After receiving the data reading request, the control module first searches for the pictures numbered from 1 to 100 in the second storage device. If all the pictures are found in the second storage device, and the precision loss of the pictures numbered from 1 to 100 stored in the second storage device obtained by the above detection method is less than or equal to the precision parameter, then the pictures numbered from 1 to 100 stored in the second storage device are read out and returned to the application.

[0119] In the fourth case, if some of the data corresponding to the first identifier is found in the second storage device, and the precision loss of the some of the data corresponding to the first identifier stored in the second storage device is less than or equal to the precision parameter, then the some of the data is read out and returned to the application. At the same time, the remaining data in the data corresponding to the first identifier stored in the first storage device except the some of the data is read out and stored in the second storage device; the remaining data stored in the second storage device is read out and returned to the application.

[0120] Corresponding to the above example, the data requested by the application to read is pictures numbered from 1 to 100. After receiving the data reading request, the control module first searches for the pictures numbered from 1 to 100 in the second storage device. If some of the data is found in the second storage device, assume that only the pictures numbered from 1 to 50 are found, and the precision loss of the pictures numbered from 1 to 50 stored in the second storage device obtained by the above detection method is less than or equal to the precision parameter, then the pictures numbered from 1 to 50 stored in the second storage device are read out and returned to the application. At the same time, the remaining pictures numbered from 51 to 100 stored in the first storage device are read out and stored in the second storage device, and the pictures numbered from 51 to 100 are read out from the second storage device and returned to the application.

[0121] The data storage method provided in this embodiment pre-allocates a first storage device and a second storage device that meet the capacity requirements set by the user for the application. The first storage device is a slow storage device with a slow memory access speed, and the second storage device is a fast storage device with a fast memory access speed. When the application requests to read a certain part of the data, if the relevant data is found in the second storage device, the data is read from the second storage device and returned to the application, ensuring the efficient reading of the data. If the relevant data is found in the second storage device, but the accuracy of the data does not meet the accuracy requirements of the application, the corresponding data is read from the first storage device and returned to the application. Since the first storage device stores all the original data sent by the application in full, the data read from the first storage device must be accurate, thus ensuring that the data returned to the application always meets the accuracy requirements of the application and improving the user experience of storing and reading data through the application. Moreover, the slow storage device has a low cost, so that the storage method provided in this application does not change significantly in terms of storage cost compared with the prior art.

[0122] Figure 6 It is a schematic structural diagram of a data storage device 600 provided in this application. This data storage device 600 can correspond to the Figure 2 control module in, as Figure 6 shown, the data storage device 600 provided in this application includes:

[0123] An acquisition module 601, configured to acquire a storage configuration request, where the storage configuration request includes: a first capacity of a first storage device;

[0124] An allocation module 602, configured to allocate the first storage device with a capacity corresponding to the first capacity and a second storage device with a second capacity according to the first capacity, where the memory access speed of the first storage device is lower than that of the second storage device.

[0125] Optionally, the storage configuration request further includes: a proportion parameter; specifically, the allocation module 602 is configured to:

[0126] Determine the second capacity according to the first capacity and the proportion parameter; allocate the first storage device with a capacity corresponding to the first capacity and the second storage device with a capacity corresponding to the second capacity according to the first capacity and the second capacity.

[0127] Optionally, the above data storage device further includes:

[0128] A receiving module 603, configured to receive a data storage request sent by an application, where the data storage request carries the original data that the application requests to store; a processing module 604, configured to transmit the original data to the first storage device and the second storage device, so that the first storage device stores the original data in full, and the second storage device stores data whose proportion in the original data is equal to a preset proportion according to a preset rule;

[0129] Wherein, the size of the space occupied by the original data is less than or equal to the first capacity, and the size of the space occupied by the data whose proportion in the original data is equal to the preset proportion is less than or equal to the second capacity.

[0130] Optionally, the processing module 604 is further configured to:

[0131] Transmit the original data to the second storage device, so that the second storage device stores data whose proportion in the original data is equal to a preset proportion according to a preset rule, and at the same time, the second storage device transmits the original data to the first storage device for full storage.

[0132] Optionally, the receiving module 603 is further configured to receive a data reading request sent by the application, where the data reading request carries a first identifier of the data requested to be read; the processing module 604 is further configured to: check whether there is data related to the first identifier stored in the second storage device; if no data related to the first identifier is found in the second storage device, read out the data corresponding to the first identifier stored in the first storage device and return it to the application.

[0133] Optionally, the processing module 604 is further configured to: predict the data that the application will request to read next according to the first identifier; read out the data that the application will request to read next from the first storage device and store it in the second storage device.

[0134] Optionally, the processing module 604 is further configured to: if data related to the first identifier is found in the second storage device, and the precision loss of the data related to the first identifier stored in the second storage device is greater than the precision parameter, read out the data corresponding to the first identifier stored in the first storage device and return it to the application.

[0135] Optionally, the processing module 604 is further configured to: if all the data corresponding to the first identifier is found in the second storage device, and the precision loss of all the data corresponding to the first identifier stored in the second storage device is less than or equal to the precision parameter, then read out all the data corresponding to the first identifier stored in the second storage device and return it to the application.

[0136] Optionally, the processing module 604 is further configured to: if partial data corresponding to the first identifier is found in the second storage device, and the precision loss of the partial data corresponding to the first identifier stored in the second storage device is less than or equal to the precision parameter, then read out the partial data and return it to the application, and at the same time read out the remaining data except the above partial data in the data corresponding to the first identifier stored in the first storage device and store it in the second storage device; read out the remaining data stored in the second storage device and return it to the application.

[0137] The data storage device provided in this application can execute the steps in any of the above method embodiments, and its implementation principle and beneficial effects are similar, which will not be elaborated here.

[0138] Figure 7 It is a schematic hardware structure diagram of the data storage device 700 provided in this application. As Figure 7 shown, the data storage device 700 includes: a memory 701 and at least one processor 702. The memory 701 is used to store program instructions, and at least one processor 702 is used to call the program instructions in the memory 701 to execute the steps in the above method embodiments. Its implementation principle and beneficial effects are similar, which will not be elaborated here.

[0139] This application also provides a readable storage medium. An execution instruction is stored in the readable storage medium. When at least one processor 702 of the data storage device executes the execution instruction, the steps in the above method embodiments are implemented. Its implementation principle and beneficial effects are similar, which will not be elaborated here.

[0140] This application also provides a program product. The program product includes a computer program (i.e., an execution instruction), and the computer program is stored in a readable storage medium. At least one processor 702 of the data storage device can read the computer program from the readable storage medium, and at least one processor 702 executes the computer program to enable the data storage device to implement the data storage methods provided in the foregoing various embodiments.

[0141] Figure 8 It is a schematic hardware structure diagram of the cloud server 800 provided in this application. As Figure 8As shown, the cloud server includes a data storage device 700, a first storage device 801, and a second storage device 802. Among them, for the allocation process of the first storage device 801 and the second storage device 802, and the process of data storage and data reading realized through the interaction among the data storage device 700, the first storage device 801, and the second storage device 802, refer to the above embodiments and will not be elaborated here.

[0142] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces, and the indirect coupling or communication connection of devices or modules can be in electrical, mechanical, or other forms.

[0143] The modules described as separate components may or may not be physically separated. The components displayed as modules may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the embodiments of the present application.

[0144] In addition, in each embodiment of the present application, the various functional modules can be integrated in a processing unit, or each module can exist physically alone, or two or more modules can be integrated in one unit. The units formed by the above modules can be implemented in the form of hardware or in the form of a combination of hardware and software functional units.

[0145] The integrated modules implemented in the form of software functional modules can be stored in a computer-readable storage medium. The above software functional modules are stored in a storage medium and include several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) to execute some steps of the methods of the embodiments of the present application.

[0146] It should be understood that the above-mentioned processor may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the invention can be directly embodied as being executed and completed by a hardware processor, or can be executed and completed by a combination of hardware and software modules in the processor.

[0147] The memory may include high-speed RAM memory, and may also include non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk, or an optical disc, etc.

[0148] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience in representation, the buses in the drawings of this application are not limited to only one bus or one type of bus.

[0149] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions according to the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD), etc.).

Claims

1. A data storage method, characterized in that, it includes: obtaining a storage configuration request, where the storage configuration request includes: the first capacity of the first storage device and a precision parameter; allocating, according to the first capacity, the first storage device with a capacity corresponding to the first capacity and a second storage device with a second capacity, where the memory access speed of the first storage device is lower than that of the second storage device, the first storage device is used for full - volume storage of original data, and the second storage device is used for storing data with a proportion equal to a preset proportion obtained from the original data according to a preset rule; receiving a data reading request sent by an application, where the first identifier of the data to be read is carried in the data reading request; checking whether there is data related to the first identifier stored in the second storage device; if data related to the first identifier is found in the second storage device, and the precision loss of the data related to the first identifier stored in the second storage device is greater than the precision parameter, then reading out the data corresponding to the first identifier stored in the first storage device and returning it to the application.

2. The method according to claim 1, characterized in that, the storage configuration request further includes: a proportion parameter; the allocating, according to the first capacity, the first storage device with a capacity corresponding to the first capacity and a second storage device with a second capacity includes: determining the second capacity according to the first capacity and the proportion parameter; allocating, according to the first capacity and the second capacity, the first storage device with a capacity corresponding to the first capacity and the second storage device with a capacity corresponding to the second capacity.

3. The method according to claim 1 or 2, characterized in that, the method further includes: receiving a data storage request sent by an application, where the original data requested to be stored by the application is carried in the data storage request; transmitting the original data to the first storage device and the second storage device, so that the first storage device performs full - volume storage on the original data, and the second storage device stores data with a proportion equal to a preset proportion obtained from the original data according to a preset rule; wherein, the size of the space occupied by the original data is less than or equal to the first capacity, and the size of the space occupied by the data with a proportion equal to the preset proportion obtained from the original data is less than or equal to the second capacity.

4. The method according to claim 1 or 2, characterized in that, the method further includes: receiving a data storage request sent by an application, where the original data requested to be stored by the application is carried in the data storage request; transmitting the original data to the second storage device, so that the second storage device stores data with a proportion equal to a preset proportion obtained from the original data according to a preset rule, and at the same time the second storage device transmits the original data to the first storage device for full - volume storage.

5. The method according to claim 1, characterized in that, the method further includes: predicting the data to be read by the application next time according to the first identifier; Read the data to be read by the next request of the application from the first storage device and store it in the second storage device.

6. The method according to claim 1, wherein, the method further includes: If all the data corresponding to the first identifier is found in the second storage device, and the precision loss of all the data corresponding to the first identifier stored in the second storage device is less than or equal to the precision parameter, then read all the data corresponding to the first identifier stored in the second storage device and return it to the application.

7. A data storage device, wherein, it includes: An acquisition module, configured to acquire a storage configuration request, where the storage configuration request includes: a first capacity of a first storage device and a precision parameter; An allocation module, configured to allocate, according to the first capacity, a first storage device with a capacity corresponding to the first capacity and a second storage device with a second capacity, where the access speed of the first storage device is lower than that of the second storage device, the first storage device is used for full-volume storage of original data, and the second storage device is used for storing data with a proportion equal to a preset proportion acquired from the original data according to a preset rule; A receiving module, configured to receive a data reading request sent by an application, where the first identifier of the data to be read is carried in the data reading request; A processing module, configured to check whether data related to the first identifier is stored in the second storage device; If data related to the first identifier is found in the second storage device, and the precision loss of the data related to the first identifier stored in the second storage device is greater than the precision parameter, then read the data corresponding to the first identifier stored in the first storage device and return it to the application.

8. The device according to claim 7, wherein, the storage configuration request further includes: a proportion parameter; the allocation module is specifically configured to: Determine the second capacity according to the first capacity and the proportion parameter; Allocate, according to the first capacity and the second capacity, a first storage device with a capacity corresponding to the first capacity and a second storage device with a capacity corresponding to the second capacity.

9. The device according to claim 7 or 8, wherein, the data storage device further includes: A receiving module, configured to receive a data storage request sent by an application, where the original data requested to be stored by the application is carried in the data storage request; A processing module, configured to transmit the original data to the first storage device and the second storage device, so that the first storage device performs full-volume storage on the original data, and the second storage device stores data with a proportion equal to a preset proportion acquired from the original data according to a preset rule; wherein, the size of the space occupied by the original data is less than or equal to the first capacity, and the size of the space occupied by the data with a proportion equal to the preset proportion acquired from the original data is less than or equal to the second capacity.

10. The device according to claim 7 or 8, wherein, The data storage device further includes: a receiving module, configured to receive a data storage request sent by an application, where the data storage request carries the original data that the application requests to store; a processing module, configured to transmit the original data to the second storage device, so that the second storage device obtains data with a proportion equal to a preset proportion from the original data for storage according to a preset rule, and at the same time, the second storage device transmits the original data to the first storage device for full-volume storage.

11. The device according to claim 7, wherein, the processing module is further configured to: predict the data to be read by the application next time according to the first identifier; read the data to be read by the application next time from the first storage device and store it in the second storage device.

12. The device according to claim 7, wherein, the processing module is further configured to: if all the data corresponding to the first identifier are found in the second storage device, and the accuracy loss of all the data corresponding to the first identifier stored in the second storage device is less than or equal to the accuracy parameter, then read all the data corresponding to the first identifier stored in the second storage device and return them to the application.

13. A readable storage medium, wherein, a computer program is stored on the readable storage medium; when the computer program is executed, the method described in any one of claims 1-6 above is implemented.

14. A data storage device, wherein, it includes: a memory and a processor; the memory is used to store program instructions, and the processor is used to call the program instructions in the memory to execute the method described in any one of claims 1-6.

15. A cloud server, wherein, it includes the data storage device, a first storage device, and a second storage device described in claim 14, where the memory access speed of the first storage device is lower than that of the second storage device.

Images