Data storage method and device, electronic equipment and storage medium

Abstract

The present disclosure relates to a data storage method, device, electronic equipment and storage medium, the method comprising: obtaining target data and historical query times, query response times and storage space values of the target data, determining a storage value parameter of the target data according to the historical query times, query response times and storage space values of the target data, determining a storage mode of the target data according to the storage value parameter, and finally storing the target data according to the storage mode. The storage value parameter of the target data is determined according to the historical query times, query response times and storage space values of the target data, so as to determine different storage modes according to different storage value parameters, thereby balancing the relationship between the value of data and the query response time and the storage space occupation, effectively guaranteeing the utilization rate of the storage space and the timeliness of the query efficiency, and reducing the storage cost and the query load.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and more specifically, to a data storage method, apparatus, electronic device, and storage medium. Background Technology

[0002] In related technologies, data is stored in its entirety within the storage capacity of the storage device, and an index is built based on this. However, this approach ignores a large amount of low-value, low-density data, as well as the significant channel overhead and load pressure caused by querying this low-value, low-density data. This results in unnecessary high storage costs, high query times, and high query loads. Summary of the Invention

[0003] The purpose of this disclosure is to provide a data storage method, apparatus, electronic device, and storage medium that determines the storage value parameters of target data based on the historical query count, query response time, and storage space value of the target data. This allows for the determination of different storage methods based on different storage value parameters, thereby balancing the relationship between data value, query response time, and storage space usage. It effectively ensures the utilization rate of storage space and the timeliness of query efficiency, while reducing storage costs and query load.

[0004] According to a first aspect of the present disclosure, a data storage method is provided, comprising:

[0005] Obtain the target data, as well as the historical query count, query response time, and storage space value of the target data;

[0006] Based on the historical query count, query response time, and storage space value of the target data, a storage value parameter is determined for the target data. The storage value parameter characterizes the value of the target data and is positively correlated with the historical query count and the query response time, and negatively correlated with the storage space value.

[0007] Based on the storage value parameter, determine the storage method of the target data;

[0008] The target data is stored according to the storage method described above.

[0009] Optionally, the step of determining the storage value parameter of the target data based on the historical query count, query response time, and storage space value of the target data specifically includes:

[0010] The ratio of the number of historical queries to the value of the storage space is determined as the value density of the target data;

[0011] The ratio of the query response time to the storage space value is determined as the query load of the target data;

[0012] The storage value parameters of the target data are determined based on the value density and the query load.

[0013] Optionally, the target data includes multiple sub-data;

[0014] The historical query count of the target data is determined based on the sum of the historical query counts of all sub-data included in the target data; and / or,

[0015] The query response time for the target data is determined based on the sum of the query response times for all sub-data included in the target data; and / or,

[0016] The storage space value of the target data is determined based on the sum of the storage space values ​​of all the sub-data included in the target data.

[0017] Optionally, the step of determining the storage method of the target data based on the storage value parameter specifically includes:

[0018] Based on the storage value parameter, determine the target threshold range to which the storage value parameter belongs within a preset range of multiple thresholds;

[0019] The storage method of the target data is determined according to the target threshold range, wherein each threshold range corresponds to a plurality of storage methods, and each storage method includes a compression method and a storage location. At least one of the compression method and the storage location is different for any two of the plurality of storage methods. The plurality of threshold ranges includes a first threshold range, a second threshold range, a third threshold range, and a fourth threshold range. The minimum value of the second threshold range is greater than the maximum value of the first threshold range, the minimum value of the third threshold range is greater than the maximum value of the second threshold range, and the minimum value of the fourth threshold range is greater than the maximum value of the third threshold range.

[0020] Optionally, the step of determining the storage method of the target data based on the target threshold range specifically includes:

[0021] If the storage value parameter is within the first threshold range, the storage method of the target data is determined to be the first storage method. The first storage method is used to extract the summary information of the target data and compress it with a first compression ratio, and then store the compressed summary information in the archive storage.

[0022] Optionally, the step of determining the storage method of the target data based on the target threshold range specifically includes:

[0023] If the storage value parameter is within the range of the second threshold, the storage method of the target data is determined to be the second storage method. The second storage method is used to extract the summary information of the target data and compress it with the second compression ratio, and then store the compressed summary information on the disk.

[0024] Optionally, the step of determining the storage method of the target data based on the target threshold range specifically includes:

[0025] When the storage value parameter is within the range of the third threshold, the storage method of the target data is determined to be the third storage method. The third storage method is used to extract the summary information of the target data and compress the summary information corresponding to the target data in the preset format at the third compression ratio, and then store the compressed summary information and the uncompressed summary information to the solid-state drive.

[0026] Optionally, the step of determining the storage method of the target data based on the target threshold range specifically includes:

[0027] If the storage value parameter is within the range of the fourth threshold, the storage method of the target data is determined to be the fourth storage method, which is used to store the target data in random access memory and disk respectively.

[0028] According to a second aspect of the present disclosure, a data storage device is provided, comprising:

[0029] The acquisition module is configured to acquire target data as well as the historical query count, query response time, and storage space value of the target data;

[0030] The first determining module is configured to determine the storage value parameter of the target data based on the historical query count, query response time, and storage space value of the target data. The storage value parameter is used to characterize the use value of the target data, and the storage value parameter is positively correlated with the historical query count and the query response time, and negatively correlated with the storage space value.

[0031] The second determining module is configured to determine the storage method of the target data based on the storage value parameter;

[0032] The storage module is configured to store the target data according to the storage method.

[0033] Optionally, the first determining module includes:

[0034] The first determining submodule is configured to determine the value density of the target data as the ratio of the number of historical queries to the value of the storage space.

[0035] The second determining submodule is configured to determine the query load of the target data as the ratio of the query response time to the storage space value;

[0036] The third determining submodule is configured to determine the storage value parameters of the target data based on the value density and the query load.

[0037] Optionally, the target data includes multiple sub-data;

[0038] The historical query count of the target data is determined based on the sum of the historical query counts of all sub-data included in the target data; and / or,

[0039] The query response time for the target data is determined based on the sum of the query response times for all sub-data included in the target data; and / or,

[0040] The storage space value of the target data is determined based on the sum of the storage space values ​​of all the sub-data included in the target data.

[0041] Optionally, the second determining module includes:

[0042] The fourth determining submodule is configured to determine the target threshold range to which the storage value parameter belongs within a preset range of multiple thresholds, based on the storage value parameter.

[0043] The fifth determining submodule is configured to determine the storage method of the target data based on the target threshold range, wherein multiple threshold ranges correspond one-to-one with multiple storage methods, each storage method includes a compression method and a storage location, and at least one of the compression method and storage location is different for any two of the multiple storage methods, the multiple threshold ranges include a first threshold range, a second threshold range, a third threshold range and a fourth threshold range, the minimum value of the second threshold range is greater than the maximum value of the first threshold range, the minimum value of the third threshold range is greater than the maximum value of the second threshold range, and the minimum value of the fourth threshold range is greater than the maximum value of the third threshold range.

[0044] Optionally, the fifth determining submodule includes:

[0045] The first determining subunit is configured to determine the storage method of the target data as a first storage method when the storage value parameter is within the first threshold range. The first storage method is used to extract the summary information of the target data and compress it with a first compression ratio, and then store the compressed summary information in the archive storage.

[0046] Optionally, the fifth determining submodule includes:

[0047] The second determining subunit is configured to determine the storage method of the target data as a second storage method when the storage value parameter is within the range of the second threshold. The second storage method is used to extract the summary information of the target data and compress it with a second compression ratio, and then store the compressed summary information on the disk.

[0048] Optionally, the fifth determining submodule includes:

[0049] The third determining subunit is configured to determine the storage method of the target data as a third storage method when the storage value parameter is within the range of the third threshold. The third storage method is used to extract the summary information of the target data and compress the summary information corresponding to the target data in a preset format at a third compression ratio, and then store the compressed summary information and the uncompressed summary information to the solid-state drive.

[0050] Optionally, the fifth determining submodule includes:

[0051] The fourth determining subunit is configured to determine the storage method of the target data as a fourth storage method when the storage value parameter is within the range of the fourth threshold. The fourth storage method is used to store the target data in random access memory and disk respectively.

[0052] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0053] A memory on which computer programs are stored;

[0054] A processor is configured to execute the computer program in the memory to implement the steps of the data storage method described in the first aspect above.

[0055] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the data storage method described in the first aspect.

[0056] The above technical solution obtains the target data, its historical query count, query response time, and storage space value. Based on these parameters, it determines the storage value parameter of the target data, then determines the storage method, and finally stores the target data according to the chosen method. By determining the storage value parameter based on the historical query count, query response time, and storage space value, different storage methods can be selected for different data values. This balances the relationship between data value, query response time, and storage space usage, effectively ensuring storage space utilization and timely query efficiency, while reducing storage costs and query load.

[0057] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0058] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings:

[0059] Figure 1 This is a flowchart illustrating a data storage method according to an exemplary embodiment.

[0060] Figure 2 This is a flowchart illustrating a method for determining storage value parameters according to an exemplary embodiment.

[0061] Figure 3 This is a flowchart illustrating another data storage method according to an exemplary embodiment.

[0062] Figure 4 This is a flowchart illustrating another data storage method according to an exemplary embodiment.

[0063] Figure 5 This is a flowchart illustrating another data storage method according to an exemplary embodiment.

[0064] Figure 6 This is a flowchart illustrating another data storage method according to an exemplary embodiment.

[0065] Figure 7 This is a block diagram illustrating a data storage device according to an exemplary embodiment.

[0066] Figure 8 This is a block diagram illustrating an electronic device according to an exemplary embodiment.

[0067] Figure 9This is a block diagram illustrating another electronic device according to an exemplary embodiment. Detailed Implementation

[0068] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.

[0069] In related technologies, data is stored in its entirety within the storage capacity of the storage device, and an index is built based on this. However, this approach ignores a large amount of low-value, low-density data, as well as the significant channel overhead and load pressure caused by querying this low-value, low-density data. This results in unnecessary high storage costs, high query times, and high query loads.

[0070] To address the aforementioned technical problems, this disclosure provides a data storage method, apparatus, electronic device, and storage medium. In order to balance the relationship between data value, query response time, and storage space usage, the storage value parameter of the target data is determined by the historical query count, query response time, and storage space value of the target data. This allows for the determination of different storage methods based on different storage value parameters, effectively ensuring the utilization rate of storage space and the timeliness of query efficiency, while reducing storage costs and query load.

[0071] Figure 1 This is a flowchart illustrating a data storage method according to an exemplary embodiment, such as... Figure 1 As shown, the method may include the following steps:

[0072] In step S101, the target data, historical query count, query response time, and storage space value of the target data are obtained.

[0073] In this embodiment, the target data can be any data in the database. The historical query count of the target data can be the sum of all queries performed on the target data after its generation, or the sum of all queries performed within a preset time period. For example, the historical query count of the target data can be the sum of all queries performed within one month from the current time. The query response time of the target data can be the average of multiple response times corresponding to multiple queries on the data within the preset time period. This response time is the time taken from receiving the user's query instruction to displaying the corresponding query content to the user. The storage space value is the size of the storage space occupied by the target data.

[0074] In step S102, the storage value parameter of the target data is determined based on the historical query count, query response time, and storage space value of the target data. The storage value parameter is used to characterize the use value of the target data, and the storage value parameter is positively correlated with the historical query count and query response time, and negatively correlated with the storage space value.

[0075] In this embodiment, the value density of the target data can be determined based on the historical query count and storage space value, the query load of the target data can be determined based on the query response time and storage space value, and the storage value parameter of the target data can be determined based on the value density and query load of the target data. Specifically, the higher the historical query count, the higher the storage value parameter; the longer the query response time, the higher the storage value parameter; and the higher the storage space value, the lower the storage value parameter.

[0076] In step S103, the storage method of the target data is determined based on the storage value parameter.

[0077] In this embodiment, different storage value parameters may correspond to different storage methods. For example, multiple threshold ranges can be set, with each threshold range corresponding to a different storage method. The corresponding storage method can be determined based on the storage value parameters of the target data.

[0078] In step S104, the target data is stored according to the storage method.

[0079] In this embodiment, after determining the storage method corresponding to the target data, the target data can be stored according to the storage method corresponding to the target data. The storage method may include at least one of the following: content extraction method, compression method, and storage location.

[0080] In this embodiment, the target data, its historical query count, query response time, and storage space value are obtained. Based on these parameters, the storage value parameter of the target data is determined. Then, the storage method is determined according to the storage value parameter, and finally, the target data is stored according to the storage method. Determining the storage value parameter based on the historical query count, query response time, and storage space value allows for the selection of different storage methods based on different storage value parameters. This balances the relationship between data value, query response time, and storage space usage, effectively ensuring storage space utilization and query efficiency, while reducing storage costs and query load.

[0081] Figure 2 This is a flowchart illustrating a method for determining storage value parameters according to an exemplary embodiment, such as... Figure 2As shown, in one possible implementation, a method for determining the storage value parameter of target data based on the historical query count, query response time, and storage space value of the target data may include the following steps:

[0082] In step S201, the ratio of the number of historical queries to the storage space value is determined as the value density of the target data.

[0083] In this implementation, the number of historical queries reflects the value of the target data; the greater the number of historical queries, the greater the value of the target data. Determining the ratio of the number of historical queries to the storage space value as the value density of the target data allows for a correlation between the value of the target data and the storage method.

[0084] In one possible implementation, the target data may include multiple sub-data, and the historical query count of the target data can be determined based on the sum of the historical query counts of all the sub-data included in the target data. Specifically, the historical query count of each sub-data included in the target data can be obtained, and the sum of the historical query counts of all the sub-data included in the target data can be determined as the query response time of the target data.

[0085] The storage space value of the target data can be determined based on the sum of the storage space values ​​of all sub-data included in the target data. Specifically, the storage space value of each sub-data included in the target data can be obtained; the sum of the query response times of all sub-data included in the target data can be determined as the total storage space value; and the product of the total storage space value and the number of sub-data included in the target data can be determined as the storage space value of the target data.

[0086] For example, the target data S = {d1,…,d} n The formula for calculating the value density of the target data can be:

[0087]

[0088] in, Let V(d) be the value density of the target data, and V(d) be the number of historical queries for sub-data d in the target data. ∑ d∈S V(d) represents the number of historical queries for the target data, S′ represents the storage space value of the target data, and k represents the number of sub-data included in the target data.

[0089] In step S202, the ratio of query response time to storage space value is determined as the query load of the target data.

[0090] In this embodiment, query response time reflects the query efficiency of the target data; the shorter the query response time, the higher the query efficiency of the target data. Determining the query load of the target data as the ratio of query response time to storage space value allows linking the query efficiency of the target data with the storage method.

[0091] In one possible implementation, the storage space value of the target data can be determined based on the sum of the storage space values ​​of all sub-data included in the target data. Specifically, the query response time of each sub-data included in the target data can be obtained; the sum of the query response times of all sub-data included in the target data is determined as the query response time of the target data.

[0092] The formula for calculating the query load of the target data can be:

[0093]

[0094] in, The query load of the target data is denoted by load(d), where load(d) is the query response time of sub-data d within the target data. ∑ d∈S load(d) is the query response time of the target data, S′ is the storage space value of the target data, and k is the number of sub-data included in the target data.

[0095] In step S203, the storage value parameters of the target data are determined based on the value density and query load.

[0096] In this embodiment, the storage value parameter of the target data can be obtained by weighted summation of value density and query load.

[0097] For example, the formula for calculating the storage value parameter of the target data can be:

[0098]

[0099] in, Let α be the storage value parameter of the target data, and α be the weighting factor, where α∈[0,1].

[0100] The result obtained from the above formula The value of α is between 0 and 1, where the value of α can be 0.5.

[0101] In this embodiment, the value density of the target data is determined based on the historical query count and storage space value; the query load of the target data is determined based on the query response time and storage space value; and the storage value parameter of the target data is determined based on the value density and query load. Definitions of value density, query load, and storage value parameter are provided to link the storage value parameter with the storage method, thereby determining different storage methods based on different storage value parameters.

[0102] In one possible implementation, the method for determining the storage method of the target data based on the storage value parameter may be as follows:

[0103] Based on the storage value parameter, determine the target threshold range to which the storage value parameter belongs within a preset range of multiple thresholds;

[0104] The storage method of the target data is determined based on the target threshold range. There are multiple threshold ranges corresponding to multiple storage methods. Each storage method includes a compression method and a storage location. At least one of the compression method and storage location is different for any two storage methods. The multiple threshold ranges include a first threshold range, a second threshold range, a third threshold range, and a fourth threshold range. The minimum value of the second threshold range is greater than the maximum value of the first threshold range, the minimum value of the third threshold range is greater than the maximum value of the second threshold range, and the minimum value of the fourth threshold range is greater than the maximum value of the third threshold range.

[0105] In this embodiment, multiple threshold ranges can be set. Based on the characteristics of the data within each threshold range, a suitable storage method is determined. Each threshold range corresponds to a different storage method, and different threshold ranges correspond to different storage methods. The compression method can specifically be a different compression ratio. For example, the compression ratio corresponding to a higher threshold range is lower than that corresponding to a lower threshold range, and the access performance of the storage location corresponding to a higher threshold range is higher than that corresponding to a lower threshold range. For data with high storage value parameters, all or part of it is stored on storage media with higher access performance to improve its read and write performance; for data with low storage value parameters, it is stored on archive storage to reduce its storage cost. A storage mechanism based on the storage value parameters of the data can avoid invalid storage and transmission of low-value-density data, effectively reduce the load on storage devices, and improve data query efficiency.

[0106] Figure 3 This is a flowchart illustrating another data storage method according to an exemplary embodiment, such as... Figure 3 As shown, in one possible implementation, the multiple threshold ranges include a first threshold range that is less than or equal to a first preset threshold, and the data storage method may include the following steps:

[0107] In step S301, the target data, historical query count, query response time, and storage space value of the target data are obtained.

[0108] In step S302, the storage value parameter of the target data is determined based on the historical query count, query response time, and storage space value of the target data.

[0109] In step S303, if the storage value parameter is within the range of the first threshold, the storage method of the target data is determined to be the first storage method. The first storage method is used to extract the summary information of the target data and compress it with the first compression ratio, and then store the compressed summary information in the archive storage.

[0110] In step S304, the target data is stored according to the first storage method.

[0111] In this embodiment, multiple threshold ranges for the storage value parameter are preset. After obtaining the storage value parameter of the target data, the target threshold range corresponding to the storage value parameter can be determined based on its magnitude. If the storage value parameter is less than or equal to a first preset threshold, it is determined that the storage value parameter of the target data falls within the first threshold range, thereby determining that the storage method of the target data is the first storage method corresponding to the first threshold range. The target data is then stored according to this first storage method. Specifically, summary information of the target data can be extracted, and the extracted summary information can be compressed using a first compression ratio before being stored in the archive storage.

[0112] The first preset threshold can be 0.25. Extracting summary information from the target data refers to extracting a summary of the target data. The method for extracting summary information can be: training a corresponding summary information extraction model based on conventional machine learning methods, and processing the target data using the summary information extraction model to obtain the summary information corresponding to the target data; details will not be elaborated here. Compressing the extracted summary information at a first compression ratio refers to performing deep compression on the extracted summary information. Deep compression can be a compression ratio greater than 50%, i.e., the first compression ratio is greater than 50%, for example, the first compression ratio can be 75%. The archive storage is a high-density storage medium with low power consumption, such as magnetic tape, typically used to store cold data.

[0113] In this embodiment, by first extracting summary information from target data with a small storage value parameter, and then performing deep compression on the extracted summary information, the storage space value of the target data can be reduced, and it can be stored in the archive storage, thereby reducing storage costs.

[0114] Figure 4 This is a flowchart illustrating another data storage method according to an exemplary embodiment, such as... Figure 4 As shown, in one possible implementation, the data storage method may include the following steps:

[0115] In step S401, the target data, historical query count, query response time, and storage space value of the target data are obtained.

[0116] In step S402, the storage value parameter of the target data is determined based on the historical query count, query response time, and storage space value of the target data.

[0117] In step S403, if the storage value parameter is within the range of the second threshold, the storage method of the target data is determined to be the second storage method. The second storage method is used to extract the summary information of the target data and compress it with the second compression ratio, and then store the compressed summary information on the disk.

[0118] In step S404, the target data is stored according to the second storage method.

[0119] In this embodiment, multiple threshold ranges for the storage value parameter are preset. After obtaining the storage value parameter of the target data, the target threshold range corresponding to the storage value parameter can be determined based on its magnitude. If the storage value parameter is greater than a first preset threshold and less than or equal to a second preset threshold, it is determined that the storage value parameter of the target data falls within the second threshold range. Therefore, the storage method of the target data can be determined as the second storage method corresponding to the second threshold range. The target data is then stored according to this second storage method. Specifically, summary information of the target data can be extracted, and the extracted summary information can be compressed using a second compression ratio before being stored on the disk.

[0120] The first preset threshold can be 0.25, and the second preset threshold can be 0.5. Performing a second compression ratio on the extracted summary information constitutes a light compression of the extracted summary information. This light compression can be a compression ratio of less than 20%, i.e., the second compression ratio is less than 20%, for example, the second compression ratio can be 10%.

[0121] In this embodiment, by first extracting summary information from target data with slightly smaller storage value parameters, and then lightly compressing the extracted summary information, the storage space value of the target data can be reduced, and it can be stored on the disk, thereby reducing storage costs.

[0122] Figure 5 This is a flowchart illustrating another data storage method according to an exemplary embodiment, such as... Figure 5 As shown, in one possible implementation, the data storage method may include the following steps:

[0123] In step S501, the target data, historical query count, query response time, and storage space value of the target data are obtained.

[0124] In step S502, the storage value parameter of the target data is determined based on the historical query count, query response time, and storage space value of the target data.

[0125] In step S503, when the storage value parameter is within the range of the third threshold, the storage method of the target data is determined to be the third storage method. The third storage method is used to extract the summary information of the target data and compress the summary information corresponding to the target data in the preset format at the third compression ratio. Then, the compressed summary information and the uncompressed summary information are stored in the solid-state drive.

[0126] In step S504, the target data is stored according to the third storage method.

[0127] In this embodiment, multiple threshold ranges for the storage value parameter are preset. After obtaining the storage value parameter of the target data, the target threshold range corresponding to the storage value parameter can be determined based on its magnitude. If the storage value parameter is greater than a second preset threshold and less than or equal to a third preset threshold, it is determined that the storage value parameter of the target data falls within the third threshold range. Therefore, the storage method of the target data can be determined as the third storage method corresponding to the third threshold range. The target data is then stored according to this third storage method. Specifically, summary information of the target data can be extracted, and the summary information corresponding to the target data in a preset format can be compressed using a third compression ratio. The compressed summary information and the uncompressed summary information are then stored on a solid-state drive.

[0128] The second preset threshold can be 0.5, and the third preset threshold can be 0.75. Performing a third compression ratio on the summary information corresponding to the target data in the preset format constitutes deep compression of the extracted summary information corresponding to the target data in the preset format. Deep compression can be a compression ratio greater than 50%, that is, the first compression ratio is greater than 50%, for example, the first compression ratio can be 75%.

[0129] In this embodiment, for target data with a slightly larger storage value parameter, summary information can be extracted first. Then, the summary information corresponding to data in a preset format that occupies a large amount of storage space can be extracted in depth. For example, the preset format data can be video and image data. The summary information corresponding to data in the target data that occupies a small amount of storage space is not compressed. The compressed summary information and the uncompressed summary information are then stored on a solid-state drive. This can reduce the storage space of the target data while improving its read and write performance and increasing the query efficiency of the target data.

[0130] Figure 6 This is a flowchart illustrating another data storage method according to an exemplary embodiment, such as... Figure 6 As shown, in one possible implementation, the data storage method may include the following steps:

[0131] In step S601, the target data, historical query count, query response time, and storage space value of the target data are obtained.

[0132] In step S602, the storage value parameter of the target data is determined based on the historical query count, query response time, and storage space value of the target data.

[0133] In step S603, if the storage value parameter is within the range of the fourth threshold, the storage method of the target data is determined to be the fourth storage method, which is used to store the target data in random access memory and disk respectively.

[0134] In step S604, the target data is stored according to the fourth storage method.

[0135] In this embodiment, multiple threshold ranges for the storage value parameter are preset. After obtaining the storage value parameter of the target data, the target threshold range corresponding to the storage value parameter can be determined based on its magnitude. If the storage value parameter is greater than a third preset threshold, it is determined that the storage value parameter of the target data falls within a fourth threshold range, thereby determining that the storage method for the target data is the fourth storage method corresponding to the fourth threshold range. The target data is then stored according to this fourth storage method; specifically, the target data can be stored in random access memory and on a disk. The third preset threshold can be 0.75.

[0136] In this embodiment, for target data with high storage value parameters, the target data can be stored directly in random access memory (RAM), and a copy of the same target data can be stored on disk. By avoiding summary extraction and compression operations on the target data and storing it in high-performance RAM and disk, read / write performance can be improved, thus increasing the query efficiency of the target data.

[0137] Figure 7 This is a block diagram illustrating a data storage device according to an exemplary embodiment. (Refer to...) Figure 7 The device 700 includes an acquisition module 701, a first determination module 702, a second determination module 703, and a storage module 704.

[0138] The acquisition module 701 is configured to acquire target data and the historical query count, query response time, and storage space value of the target data.

[0139] The first determining module 702 is configured to determine the storage value parameter of the target data based on the historical query count, query response time, and storage space value of the target data;

[0140] The second determining module 703 is configured to determine the storage method of the target data based on the storage value parameter;

[0141] The storage module 704 is configured to store the target data according to the storage method.

[0142] Optionally, the first determining module includes:

[0143] The first determining submodule is configured to determine the value density of the target data as the ratio of the number of historical queries to the value of the storage space.

[0144] The second determining submodule is configured to determine the query load of the target data as the ratio of the query response time to the storage space value;

[0145] The third determining submodule is configured to determine the storage value parameters of the target data based on the value density and the query load.

[0146] Optionally, the target data includes multiple sub-data;

[0147] The historical query count of the target data is determined based on the sum of the historical query counts of all sub-data included in the target data; and / or,

[0148] The query response time for the target data is determined based on the sum of the query response times for all sub-data included in the target data; and / or,

[0149] The storage space value of the target data is determined based on the sum of the storage space values ​​of all the sub-data included in the target data.

[0150] Optionally, the second determining module includes:

[0151] The fourth determining submodule is configured to determine the target threshold range to which the storage value parameter belongs within a preset range of multiple thresholds, based on the storage value parameter.

[0152] The fifth determining submodule is configured to determine the storage method of the target data based on the target threshold range, wherein multiple threshold ranges correspond one-to-one with multiple storage methods, each storage method includes a compression method and a storage location, and at least one of the compression method and storage location is different for any two of the multiple storage methods, the multiple threshold ranges include a first threshold range, a second threshold range, a third threshold range and a fourth threshold range, the minimum value of the second threshold range is greater than the maximum value of the first threshold range, the minimum value of the third threshold range is greater than the maximum value of the second threshold range, and the minimum value of the fourth threshold range is greater than the maximum value of the third threshold range.

[0153] Optionally, the plurality of threshold ranges includes a first threshold range that is less than or equal to a first preset threshold;

[0154] The fifth determining submodule includes:

[0155] The first determining subunit is configured to determine the storage method of the target data as a first storage method when the storage value parameter is within the first threshold range. The first storage method is used to extract the summary information of the target data and compress it with a first compression ratio, and then store the compressed summary information in the archive storage.

[0156] Optionally, the fifth determining submodule includes:

[0157] The second determining subunit is configured to determine the storage method of the target data as a second storage method when the storage value parameter is within the range of the second threshold. The second storage method is used to extract the summary information of the target data and compress it with a second compression ratio, and then store the compressed summary information on the disk.

[0158] Optionally, the fifth determining submodule includes:

[0159] The third determining subunit is configured to determine the storage method of the target data as a third storage method when the storage value parameter is within the range of the third threshold. The third storage method is used to extract the summary information of the target data and compress the summary information corresponding to the target data in a preset format at a third compression ratio, and then store the compressed summary information and the uncompressed summary information to the solid-state drive.

[0160] Optionally, the fourth determining submodule includes:

[0161] The fourth determining subunit is configured to determine the storage method of the target data as a fourth storage method when the storage value parameter is within the range of the fourth threshold. The fourth storage method is used to store the target data in random access memory and disk respectively.

[0162] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0163] Figure 8 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Figure 8 As shown, the electronic device 800 may include: a first processor 801 and a first memory 802. The electronic device 800 may also include one or more of a multimedia component 803, an I / O interface 804, and a first communication component 805.

[0164] The first processor 801 controls the overall operation of the electronic device 800 to complete all or part of the steps in the data storage method described above. The first memory 802 stores various types of data to support the operation of the electronic device 800. This data may include, for example, instructions for any application or method operating on the electronic device 800, and application-related data such as contact data, sent and received messages, pictures, audio, video, etc. The first memory 802 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. Multimedia component 803 may include a screen and an audio component. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in the first memory 802 or transmitted via the first communication component 805. The audio component also includes at least one speaker for outputting audio signals. I / O interface 804 provides an interface between the first processor 801 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. The first communication component 805 is used for wired or wireless communication between the electronic device 800 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IoT, eMTC, or other 5G technologies, or combinations thereof, is not limited here. Therefore, the corresponding first communication component 805 may include: a Wi-Fi module, a Bluetooth module, an NFC module, etc.

[0165] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the data storage method described above.

[0166] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the data storage method described above. For example, the computer-readable storage medium may be the first memory 802 including the program instructions described above, which may be executed by the first processor 801 of the electronic device 800 to complete the data storage method described above.

[0167] Figure 9 This is a block diagram illustrating another electronic device according to an exemplary embodiment. For example, electronic device 900 may be provided as a server. (Refer to...) Figure 9 The electronic device 900 includes a second processor 922, which may be one or more, and a second memory 932 for storing computer programs executable by the second processor 922. The computer program stored in the second memory 932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the second processor 922 may be configured to execute the computer program to perform the data storage method described above.

[0168] Additionally, the electronic device 900 may also include a power supply component 926 and a second communication component 950. The power supply component 926 may be configured to perform power management of the electronic device 900, and the second communication component 950 may be configured to enable communication of the electronic device 900, such as wired or wireless communication. Furthermore, the electronic device 900 may also include an input / output interface 958. The electronic device 900 can operate on an operating system stored in a second memory 932.

[0169] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the data storage method described above. For example, the non-transitory computer-readable storage medium may be the second memory 932 including the program instructions, which may be executed by the second processor 922 of the electronic device 900 to complete the data storage method described above.

[0170] In another exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program executable by a programmable device, the computer program having a code portion for performing the data storage method described above when executed by the programmable device.

[0171] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.

[0172] It should also be noted that the various specific technical features described in the above embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, this disclosure will not describe the various possible combinations separately.

[0173] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.

Claims

1. A data storage method, characterized in that, include: Obtain the target data, as well as the historical query count, query response time, and storage space value of the target data; The storage value parameter of the target data is determined based on the historical query count, query response time, and storage space value of the target data. This includes: determining the value density of the target data as the ratio of the historical query count to the storage space value; determining the query load of the target data as the ratio of the query response time to the storage space value; and determining the storage value parameter of the target data based on the value density and the query load. The storage value parameter characterizes the use value of the target data, and the storage value parameter is positively correlated with both the historical query count and the query response time, and negatively correlated with the storage space value. Based on the storage value parameter, determine the storage method of the target data; The target data is stored according to the storage method described above.

2. The data storage method according to claim 1, characterized in that, The target data includes multiple sub-data; The historical query count of the target data is determined based on the sum of the historical query counts of all sub-data included in the target data; and / or, The query response time for the target data is determined based on the sum of the query response times for all sub-data included in the target data; and / or, The storage space value of the target data is determined based on the sum of the storage space values ​​of all the sub-data included in the target data.

3. The data storage method according to claim 1, characterized in that, The step of determining the storage method of the target data based on the storage value parameter specifically includes: Based on the storage value parameter, determine the target threshold range to which the storage value parameter belongs within a preset range of multiple thresholds; The storage method of the target data is determined according to the target threshold range, wherein each threshold range corresponds to a plurality of storage methods, and each storage method includes a compression method and a storage location. At least one of the compression method and the storage location is different for any two of the plurality of storage methods. The plurality of threshold ranges includes a first threshold range, a second threshold range, a third threshold range, and a fourth threshold range. The minimum value of the second threshold range is greater than the maximum value of the first threshold range, the minimum value of the third threshold range is greater than the maximum value of the second threshold range, and the minimum value of the fourth threshold range is greater than the maximum value of the third threshold range.

4. The data storage method according to claim 3, characterized in that, The step of determining the storage method of the target data based on the target threshold range specifically includes: If the storage value parameter is within the first threshold range, the storage method of the target data is determined to be the first storage method. The first storage method is used to extract the summary information of the target data and compress it with a first compression ratio, and then store the compressed summary information in the archive storage.

5. The data storage method according to claim 3, characterized in that, The step of determining the storage method of the target data based on the target threshold range specifically includes: If the storage value parameter is within the range of the second threshold, the storage method of the target data is determined to be the second storage method. The second storage method is used to extract the summary information of the target data and compress it with the second compression ratio, and then store the compressed summary information on the disk.

6. The data storage method according to claim 3, characterized in that, The step of determining the storage method of the target data based on the target threshold range specifically includes: When the storage value parameter is within the range of the third threshold, the storage method of the target data is determined to be the third storage method. The third storage method is used to extract the summary information of the target data and perform a compression operation on the summary information corresponding to the target data in the preset format at a third compression ratio, and then store the compressed summary information and the uncompressed summary information to the solid-state drive.

7. The data storage method according to claim 3, characterized in that, The step of determining the storage method of the target data based on the target threshold range specifically includes: If the storage value parameter is within the range of the fourth threshold, the storage method of the target data is determined to be the fourth storage method, which is used to store the target data in random access memory and disk respectively.

8. A data storage device, characterized in that, include: The acquisition module is configured to acquire target data as well as the historical query count, query response time, and storage space value of the target data; The first determining module is configured to determine the storage value parameter of the target data based on the historical query count, query response time, and storage space value of the target data. This includes: determining the value density of the target data as the ratio of the historical query count to the storage space value; determining the query load of the target data as the ratio of the query response time to the storage space value; and determining the storage value parameter of the target data based on the value density and the query load. The storage value parameter characterizes the use value of the target data, and the storage value parameter is positively correlated with both the historical query count and the query response time, and negatively correlated with the storage space value. The second determining module is configured to determine the storage method of the target data based on the storage value parameter; The storage module is configured to store the target data according to the storage method.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the data storage method according to any one of claims 1-7.

10. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the data storage method according to any one of claims 1-7.

Images