Data storage method and device, storage medium and processor

Abstract

The invention discloses a data storage method and device, a storage medium and a processor. The method comprises the steps that to-be-stored data are obtained from a target database, a storage structure of the to-be-stored data in the target database at least comprises first identification information of the to-be-stored data, and the first identification information is used for identifying features of the to-be-stored data; based on the first identification information, at least determining first dynamic information of the to-be-stored data in the current period and second dynamic information of the to-be-stored data in the historical period; inputting the first dynamic information and the second dynamic information into a classification model for classification to obtain a classification result; the to-be-stored data are stored in a storage space corresponding to the classification result, and the storage space comprises a cache space or a hard disk space. The technical problem of low data distribution efficiency is solved.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a data storage method, device, storage medium and processor. Background Art

[0002] In the operator network, the performance data generated by the operation of network equipment usually has the characteristics of large data volume, high concurrency and strong real-time, and the data grows very fast. Therefore, when storing and processing such data, there are high requirements for the throughput and scalability of the database.

[0003] However, in the related technologies, the data already stored in the cache is often optimized, and the data cannot be accurately labeled in combination with the actual operation of the system, thus resulting in a technical problem of low efficiency in data diversion.

[0004] Currently, no effective solution has been proposed to the above-mentioned technical problem of low efficiency of data diversion. Summary of the invention

[0005] The embodiments of the present invention provide a data storage method, device, storage medium and processor to at least solve the technical problem of low efficiency of data diversion.

[0006] According to one aspect of an embodiment of the present invention, a data storage method is provided, the method comprising: obtaining data to be stored from a target database, wherein the storage structure of the data to be stored in the target database comprises at least: first identification information of the data to be stored, the first identification information being used to identify features of the data to be stored; based on the first identification information, at least determining first dynamic information of the data to be stored in a current cycle and second dynamic information of the data to be stored in a historical cycle, wherein a moment corresponding to the historical cycle is earlier than a moment corresponding to the current cycle, the first dynamic information being used to indicate a frequency of accessing the data to be stored in the current cycle, and a probability of finding the data to be stored in the current cycle, and the second dynamic information being used to indicate a frequency of accessing the data to be stored in the historical cycle, and a probability of finding the data to be stored in the historical cycle. probability of data to be stored; inputting the first dynamic information and the second dynamic information into a classification model for classification to obtain a classification result, wherein the classification model is obtained by training a reinforcement learning model based on the first dynamic sample information and the second dynamic sample information, the first dynamic information sample is used to indicate the frequency of accessing the sample data to be stored in the current cycle, and the probability of finding the sample data to be stored in the current cycle, the second dynamic sample information is used to indicate the frequency of accessing the sample data to be stored in the historical cycle, and the probability of finding the sample data to be stored in the historical cycle, the classification result is used to indicate the type of storage operation to be performed on the data to be stored; storing the data to be stored in a storage space corresponding to the classification result, wherein the storage space includes cache space or hard disk space.

[0007] Optionally, based on the first identification information, at least the first dynamic information of the data to be stored in the current period and the second dynamic information of the data to be stored in the historical period are determined, including: parsing the data to be stored to obtain the first identification information; searching the first identification information in the dynamic information table to obtain the first dynamic information and the second dynamic information, wherein the dynamic information table stores a mapping relationship between the first identification information and the first dynamic information, and a mapping relationship between the first identification information and the second dynamic information.

[0008] Optionally, the first dynamic information and the second dynamic information are input into a classification model for classification to obtain a classification result, including: determining the data stored in the dynamic information table except the data to be stored as the remaining data; determining the second identification information based on the remaining data, wherein the second identification information is used to identify the characteristics of the remaining data; based on the second identification information, at least determining the third dynamic information of the remaining data in the current cycle and the fourth dynamic information of the remaining data in the historical cycle, wherein the third dynamic information is used to indicate the frequency of accessing the remaining data in the current cycle and the probability of finding the remaining data in the current cycle, and the fourth dynamic information is used to indicate the frequency of accessing the remaining data in the historical cycle and the probability of finding the remaining data in the historical cycle; combining the first dynamic information, the second dynamic information, the third dynamic information and the fourth dynamic information to obtain the target dynamic information; inputting the target dynamic information into the classification model for classification to obtain a classification result.

[0009] Optionally, determining the second identification information based on the remaining data includes: parsing the remaining data to obtain the second identification information.

[0010] Optionally, based on the second identification information, at least the third dynamic information of the remaining data in the current period and the fourth dynamic information in the historical period are determined, including: searching the second identification information in the dynamic information table to obtain the third dynamic information and the fourth dynamic information, wherein the dynamic information table stores a mapping relationship between the second identification information and the third dynamic information, and a mapping relationship between the second identification information and the fourth dynamic information.

[0011] Optionally, the classification result includes a first classification result and a second classification result, the first classification result is used to indicate the type of storage operation to be performed on the data to be stored in the cache space, and the second classification result is used to indicate the type of storage operation to be performed on the data to be stored in the hard disk space, and storing the data to be stored in the storage space corresponding to the classification result includes: storing the data to be stored in the cache space corresponding to the first classification result; or storing the data to be stored in the hard disk space corresponding to the second classification result.

[0012] Optionally, the method also includes: in response to the data to be stored having been stored in the cache space, storing the remaining data in the cache space; in response to the data to be stored having been stored in the hard disk space, deleting the historical data stored in the cache space, wherein the length of time the historical data is stored in the cache space is greater than the length of time the non-historical data is stored in the cache space, and the non-historical data is the data stored in the cache space other than the historical data.

[0013] Optionally, the method also includes: updating the classification model, including: updating the target parameters in the classification model, wherein the target parameters are used to adjust the classification results; inputting the first dynamic information and the second dynamic information into the updated classification model for classification to obtain the classification results.

[0014] According to one aspect of an embodiment of the present invention, a data storage device is provided, which may include: an acquisition unit, used to acquire data to be stored from a target database, wherein the storage structure of the data to be stored in the target database includes at least: first identification information of the data to be stored, the first identification information being used to identify characteristics of the data to be stored; a determination unit, used to determine at least first dynamic information of the data to be stored in a current cycle and second dynamic information of the data to be stored in a historical cycle based on the first identification information, wherein the moment corresponding to the historical cycle is earlier than the moment corresponding to the current cycle, the first dynamic information is used to indicate the frequency of accessing the data to be stored in the current cycle, and the probability of finding the data to be stored in the current cycle, and the second dynamic information is used to indicate the frequency of accessing the data to be stored in the historical cycle, and the probability of finding the data to be stored in the historical cycle. a classification unit, used for inputting the first dynamic information and the second dynamic information into a classification model for classification to obtain a classification result, wherein the classification model is obtained by training a reinforcement learning model based on the first dynamic sample information and the second dynamic sample information, the first dynamic information sample is used to indicate the frequency of accessing the sample data to be stored in the current cycle, and the probability of finding the sample data to be stored in the current cycle, the second dynamic sample information is used to indicate the frequency of accessing the sample data to be stored in the historical cycle, and the probability of finding the sample data to be stored in the historical cycle, and the classification result is used to indicate the type of storage operation to be performed on the data to be stored; a storage unit, used to store the data to be stored in a storage space corresponding to the classification result, wherein the storage space includes a cache space or a hard disk space.

[0015] According to another aspect of an embodiment of the present invention, a processor is provided, which is used to run a program, wherein when the program is run by the processor, the data storage method in the embodiment of the present invention is executed.

[0016] According to another aspect of an embodiment of the present invention, there is further provided an electronic device, comprising: a memory storing an executable program; and a processor for running the program, wherein the data storage method in each embodiment of the present invention is executed when the program is running.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the data storage method in the present invention.

[0018] According to another aspect of an embodiment of the present invention, a computer program product is further provided. The computer program product includes a computer program. When the computer program is executed by a processor, the data storage method in the embodiment of the present invention is implemented.

[0019] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the data storage method in the embodiment of the present invention is implemented.

[0020] According to another aspect of an embodiment of the present invention, an embodiment of the present application further provides a computer program, and when the computer program is executed by a processor, the data storage method in the above-mentioned embodiment of the present invention is implemented.

[0021] In an embodiment of the present invention, when storing data, the data to be stored can be obtained from the target database. According to the first identification information of the data to be stored, at least the first dynamic information of the data to be stored in the current period and the second dynamic information of the data to be stored in the historical period can be determined. The determined first dynamic information and the determined second dynamic information are input into the classification model for classification, and the classification result can be obtained, and the obtained data to be stored is stored in the storage space corresponding to the obtained classification result, thereby achieving the purpose of diverting data at the data storage stage, thereby solving the technical problem of low efficiency of data diversion, and then achieving the technical effect of improving the efficiency of data diversion. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0023] Figure 1 is a flow chart of a data storage method according to an embodiment of the present invention;

[0024] FIG2( a ) is a schematic diagram of a hot and cold data classification system for a time series database based on deep reinforcement learning according to an embodiment of the present invention;

[0025] FIG2( b ) is a flow chart of a method for classifying hot and cold data in a time series database based on deep reinforcement learning according to an embodiment of the present invention;

[0026] FIG2( c ) is a schematic diagram of a process of writing hot and cold data into a time series database based on deep reinforcement learning according to an embodiment of the present invention;

[0027] FIG2( d ) is a schematic diagram of a cold and hot data monitoring process of a time series database based on deep reinforcement learning according to an embodiment of the present invention;

[0028] FIG2( e ) is a schematic diagram of a state transition of a deep reinforcement learning model according to an embodiment of the present invention;

[0029] Figure 3 is a schematic diagram of a data storage device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only embodiments of a part of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] According to an embodiment of the present invention, a method for storing data is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0033] Figure 1 is a flow chart of a data storage method according to an embodiment of the present invention, the method may include the following steps:

[0034] Step S101, obtaining data to be stored from a target database.

[0035] In the technical solution provided in the above step S101 of the present invention, the storage structure of the above data to be stored in the target database may at least include: first identification information of the data to be stored and timestamp information of the data to be stored. The first identification information may be used to identify the characteristics of the data to be stored, and the timestamp information may be used to record the time when the data to be stored is written into the target database.

[0036] In this embodiment, the above-mentioned target database may be but is not limited to a time series database, the above-mentioned data to be stored may be but is not limited to the time series data in the time series database, and the above-mentioned first identification information may be represented by an identifier (metric or Metric), which is only used as an example here without specific limitation.

[0037] In this embodiment, the data to be stored is obtained from the target database. Optionally, in this embodiment, when the data to be stored has been written into the target database, the target database is traversed to obtain the data to be stored. For example, when the time series data to be stored has been written into the time series database, the time series data to be stored is traversed to obtain the time series data to be stored. This is only an example and is not specifically limited.

[0038] Step S102: Based on the first identification information, at least first dynamic information of the data to be stored in the current period and second dynamic information of the data in the historical period are determined.

[0039] In the technical solution provided in step S102 of the present invention, the time corresponding to the historical period may be earlier than the time corresponding to the current period. The first dynamic information may be used to indicate the frequency of accessing the data to be stored in the current period, and the probability of finding the data to be stored in the current period. The second dynamic information may be used to indicate the frequency of accessing the data to be stored in the historical period, and the probability of finding the data to be stored in the historical period.

[0040] In this embodiment, the frequency of accessing the data to be stored in the current cycle may also be referred to as the current access frequency, and the probability of finding the data to be stored in the current cycle may also be referred to as the current hit rate. The frequency of accessing the data to be stored in the historical cycle may also be referred to as the historical access frequency, and the probability of finding the data to be stored in the historical cycle may also be referred to as the historical hit rate, which is only used as an example and is not specifically limited.

[0041] In this embodiment, after obtaining the data to be stored from the target database, at least the first dynamic information of the data to be stored in the current period and the second dynamic information of the data to be stored in the historical period are determined based on the first identification information. Optionally, on the basis of obtaining the data to be stored, this embodiment parses the acquired data to be stored to obtain the first identification information. Based on the obtained first identification information and the dynamic information table, at least the first dynamic information of the data to be stored in the current period and the second dynamic information of the data to be stored in the historical period can be determined. Among them, the dynamic information table can store the mapping relationship between the first identification information and the first dynamic information, and the mapping relationship between the first identification information and the second dynamic information.

[0042] Step S103: input the first dynamic information and the second dynamic information into a classification model for classification to obtain a classification result.

[0043] In the technical solution provided in the above step S103 of the present invention, the above classification model can be obtained by training the reinforcement learning model based on the first dynamic sample information and the second dynamic sample information. Among them, the above first dynamic information sample can be used to represent the frequency of accessing the sample data to be stored in the current cycle, and the probability of finding the sample data to be stored in the current cycle, and the above second dynamic sample information can be used to represent the frequency of accessing the sample data to be stored in the historical cycle, and the probability of finding the sample data to be stored in the historical cycle. For example, the above reinforcement learning model can be a model constructed based on a deep reinforcement learning network (Dueling Double Deep Q Network, referred to as D3QN), which is only used as an example here and is not specifically limited.

[0044] In this embodiment, the above classification results can be used to indicate the type of storage operation to be performed on the data to be stored. The above classification results can include at least: a first classification result and a second classification result, the first classification result can be used to indicate the type of storage operation to be performed on the data to be stored in the cache space, and the second classification result can be used to indicate the type of storage operation to be performed on the data to be stored in the hard disk space, which is only an example and not specifically limited.

[0045] In this embodiment, after at least determining the first dynamic information of the data to be stored in the current cycle and the second dynamic information in the historical cycle based on the first identification information, the first dynamic information and the second dynamic information are input into the classification model for classification to obtain a classification result. Optionally, this embodiment inputs the determined first dynamic information and the second dynamic information into the input layer of the classification model on the basis of determining the first dynamic information of the data to be stored in the current cycle and the second dynamic information in the historical cycle, and classifies the first dynamic information and the second dynamic information using the hidden layer and the target network of the classification model to obtain a classification result, thereby achieving the purpose of determining the type of storage operation to be performed on the data to be stored.

[0046] It should be noted that the hidden layer of the above classification model can be used to process the relationship between the first dynamic information and the second dynamic information. The target network of the above classification model can be used to calculate the probability of the type of storage operation to be performed on the data to be stored. For example, based on extracting the relationship between the first dynamic information and the second dynamic information using the hidden layer of the classification model, the extracted relationship between the first dynamic information and the second dynamic information is input into the target network of the classification model, thereby, in the target network, the probability of the type of storage operation to be performed on the data to be stored can be calculated, which is only an example here and is not specifically limited.

[0047] Step S104, storing the data to be stored in the storage space corresponding to the classification result.

[0048] In the technical solution provided in the above step S104 of the present invention, the above storage space may include at least cache space or hard disk space. For example, a hard disk may also be referred to as a low-speed storage medium, which is only used as an example and is not specifically limited.

[0049] In this embodiment, after the first dynamic information and the second dynamic information are input into the classification model for classification and the classification result is obtained, the data to be stored is stored in the storage space corresponding to the classification result. Optionally, on the basis of the classification result, the embodiment judges the relationship between the obtained classification result and the first classification result and the second classification result. If it is determined that the obtained classification result is the first classification result, the data to be stored is stored in the storage space corresponding to the first classification result; if it is determined that the obtained classification result is the second classification result, the data to be stored is stored in the storage space corresponding to the second classification result.

[0050] In the above steps S101 to S104 of the present application, when storing data, the data to be stored can be obtained from the target database. According to the first identification information of the data to be stored, at least the first dynamic information of the data to be stored in the current cycle and the second dynamic information in the historical cycle can be determined. The determined first dynamic information and the determined second dynamic information are input into the classification model for classification, and the classification result can be obtained, and the obtained data to be stored is stored in the storage space corresponding to the obtained classification result, thereby achieving the purpose of diverting data at the data warehousing stage, thereby solving the technical problem of low efficiency of data diversion, and then achieving the technical effect of improving the efficiency of data diversion.

[0051] The above method of this embodiment is further introduced below.

[0052] As an optional implementation method, step S102, based on the first identification information, at least determines the first dynamic information of the data to be stored in the current period and the second dynamic information in the historical period, including: parsing the data to be stored to obtain the first identification information; in the dynamic information table, searching for the first identification information to obtain the first dynamic information and the second dynamic information.

[0053] In this embodiment, the above dynamic information table may store a mapping relationship between the first identification information and the first dynamic information, and a mapping relationship between the first identification information and the second dynamic information. For example, the above dynamic information table may be a real-time data statistics table stored in the dynamic information storage device, which is only used as an example and is not specifically limited.

[0054] In this embodiment, after obtaining the data to be stored from the target database, the data to be stored is parsed to obtain the first identification information. Optionally, based on the data to be stored, this embodiment performs identification parsing on the acquired data to be stored to obtain the first identification information. For example, by performing identification parsing on the acquired time series data to be stored, the characteristics of the time series data to be stored can be obtained, which is only an example and not specifically limited here.

[0055] In this embodiment, after the data to be stored is parsed to obtain the first identification information, the first identification information is searched in the dynamic information table to obtain the first dynamic information and the second dynamic information. Optionally, on the basis of obtaining the first identification information, if the first identification information can be found in the dynamic information table, the first dynamic information can be obtained according to the mapping relationship between the first identification information and the first dynamic information stored in the dynamic information table. At the same time, if the first identification information can be found in the dynamic information table, the second dynamic information can be obtained according to the mapping relationship between the first identification information and the second dynamic information stored in the dynamic information table.

[0056] As an optional implementation method, step S103, inputs the first dynamic information and the second dynamic information into a classification model for classification to obtain a classification result, including: determining the data stored in the dynamic information table except the data to be stored as remaining data; determining the second identification information based on the remaining data; based on the second identification information, at least determining the third dynamic information of the remaining data in the current period and the fourth dynamic information of the remaining data in the historical period; combining the first dynamic information, the second dynamic information, the third dynamic information and the fourth dynamic information to obtain target dynamic information; inputting the target dynamic information into the classification model for classification to obtain a classification result.

[0057] In this embodiment, the second identification information can be used to identify the characteristics of the remaining data. The third dynamic information can be used to indicate the frequency of accessing the remaining data in the current cycle, and the probability of finding the remaining data in the current cycle. The fourth dynamic information can be used to indicate the frequency of accessing the remaining data in the historical cycle, and the probability of finding the remaining data in the historical cycle.

[0058] In this embodiment, after at least determining the first dynamic information of the data to be stored in the current cycle and the second dynamic information in the historical cycle based on the first identification information, the data stored in the dynamic information table other than the data to be stored is determined as the remaining data. Optionally, this embodiment determines the time series data stored in the time series data stored in the real-time data statistics table other than the time series data to be stored as the remaining time series data based on determining the first dynamic information of the data to be stored in the current cycle and the second dynamic information in the historical cycle. This is only an example and is not specifically limited.

[0059] In this embodiment, after determining the data stored in the dynamic information table other than the data to be stored as the remaining data, the second identification information is determined based on the remaining data. Optionally, this embodiment can obtain the second identification information by parsing the determined remaining data on the basis of determining the remaining data. For example, the characteristics of the remaining time series data can be obtained by parsing the determined remaining time series data.

[0060] In this embodiment, after determining the second identification information based on the remaining data, at least the third dynamic information of the remaining data in the current cycle and the fourth dynamic information in the historical cycle are determined based on the second identification information. Optionally, based on the second identification information determined, this embodiment can at least determine the third dynamic information of the remaining data in the current cycle and the fourth dynamic information in the historical cycle according to the determined second identification information and the dynamic information table. The mapping relationship between the second identification information and the third dynamic information, as well as the mapping relationship between the second identification information and the fourth dynamic information can be stored in the dynamic information table.

[0061] In this embodiment, after at least determining the third dynamic information of the remaining data in the current cycle and the fourth dynamic information in the historical cycle based on the second identification information, the first dynamic information, the second dynamic information, the third dynamic information and the fourth dynamic information are combined to obtain the target dynamic information. Optionally, this embodiment combines the first dynamic information, the second dynamic information, the third dynamic information and the fourth dynamic information based on the determination of the third dynamic information and the fourth dynamic information to obtain the target dynamic information. j The target dynamic information represented by is only given as an example here and is not specifically limited.

[0062] In this embodiment, after the first dynamic information, the second dynamic information, the third dynamic information and the fourth dynamic information are combined to obtain the target dynamic information, the target dynamic information is input into the classification model for classification to obtain the classification result. Optionally, on the basis of determining the target dynamic information, this embodiment inputs the determined target dynamic information into the input layer of the classification model, and uses the hidden layer of the classification model and the target network to classify the target dynamic information to obtain the classification result, thereby achieving the purpose of determining the type of storage operation to be performed on the data to be stored.

[0063] Optionally, based on extracting the relationship between the first dynamic information and the second dynamic information in the target dynamic information using the hidden layer of the classification model, the extracted relationship between the first dynamic information and the second dynamic information is input into the target network of the classification model. Thus, in the target network, the probability of the type of storage operation to be performed on the stored data can be calculated. This is only an example and is not specifically limited.

[0064] As an optional implementation manner, determining the second identification information based on the remaining data includes: parsing the remaining data to obtain the second identification information.

[0065] In this embodiment, after determining the data stored in the dynamic information table other than the data to be stored as the remaining data, the remaining data is parsed to obtain the second identification information. Optionally, based on the determination of the remaining data, this embodiment performs identification parsing on the determined remaining data to obtain the second identification information. For example, the characteristics of the remaining time series data can be obtained by performing identification parsing on the determined remaining time series data, which is only used as an example and is not specifically limited here.

[0066] As an optional implementation method, based on the second identification information, at least the third dynamic information of the remaining data in the current period and the fourth dynamic information in the historical period are determined, including: searching the second identification information in the dynamic information table to obtain the third dynamic information and the fourth dynamic information.

[0067] In this embodiment, the dynamic information table may store a mapping relationship between the second identification information and the third dynamic information, and a mapping relationship between the second identification information and the fourth dynamic information.

[0068] In this embodiment, after determining the second identification information based on the remaining data, the second identification information is searched in the dynamic information table to obtain the third dynamic information and the fourth dynamic information. Optionally, in this embodiment, on the basis of obtaining the second identification information, if the second identification information can be found in the dynamic information table, the second dynamic information can be obtained according to the mapping relationship between the second identification information and the third dynamic information stored in the dynamic information table. At the same time, if the second identification information can be found in the dynamic information table, the fourth dynamic information can be obtained according to the mapping relationship between the second identification information and the fourth dynamic information stored in the dynamic information table.

[0069] As an optional implementation method, step S104, storing the data to be stored in the storage space corresponding to the classification result, includes: storing the data to be stored in the cache space corresponding to the first classification result; or storing the data to be stored in the hard disk space corresponding to the second classification result.

[0070] In this embodiment, the above classification results may include a first classification result and a second classification result. The first classification result may be used to indicate the type of storage operation to be performed on the data to be stored in the cache space, and the second classification result may be used to indicate the type of storage operation to be performed on the data to be stored in the hard disk space.

[0071] In this embodiment, after the first dynamic information and the second dynamic information are input into the classification model for classification and the classification result is obtained, the data to be stored is stored in the cache space corresponding to the first classification result; or, the data to be stored is stored in the hard disk space corresponding to the second classification result. Optionally, based on the classification result, the embodiment judges the relationship between the obtained classification result and the first classification result and the second classification result. If it is determined that the obtained classification result is the first classification result, the data to be stored is stored in the cache space corresponding to the first classification result; if it is determined that the obtained classification result is the second classification result, the data to be stored is stored in the hard disk space corresponding to the second classification result.

[0072] As an optional implementation, the method further includes: in response to the data to be stored having been stored in the cache space, storing the remaining data in the cache space; in response to the data to be stored having been stored in the hard disk space, deleting the historical data stored in the cache space.

[0073] In this embodiment, the length of time that the historical data is stored in the cache space may be longer than the length of time that the non-historical data is stored in the cache space. The non-historical data may be data other than the historical data in the data stored in the cache space.

[0074] In this embodiment, in response to the data to be stored being stored in the cache space, the remaining data is stored in the cache space. Optionally, in this embodiment, in response to the time series data to be stored being stored in the cache space, the remaining time series data is stored in the cache space.

[0075] In this embodiment, in response to the data to be stored being stored in the hard disk space, the historical data stored in the cache space is deleted. Optionally, in this embodiment, in response to the time series data to be stored being stored in the hard disk space, the historical time series data stored in the cache space is deleted.

[0076] As an optional implementation mode, the method also includes: updating the classification model, including: updating the target parameters in the classification model; inputting the first dynamic information and the second dynamic information into the updated classification model for classification to obtain a classification result.

[0077] In this embodiment, the above target parameters can be used to adjust the classification results.

[0078] In this embodiment, the target parameters in the classification model are updated. Optionally, based on the classification model obtained through training, the target parameters in the classification model obtained through training are updated. The updating process of the target parameters can be implemented through a mean square loss function.

[0079] In this embodiment, after the target parameters in the classification model are updated, the first dynamic information and the second dynamic information are input into the updated classification model for classification to obtain a classification result. Optionally, on the basis of completing the update of the target parameters in the classification model, this embodiment inputs the determined first dynamic information and the second dynamic information into the input layer of the updated classification model, and uses the hidden layer and the target network of the updated classification model to classify the above-mentioned first dynamic information and the above-mentioned second dynamic information to obtain a classification result, thereby achieving the purpose of determining the type of storage operation to be performed on the data to be stored.

[0080] In an embodiment of the present invention, when storing data, the data to be stored can be obtained from the target database. According to the first identification information of the data to be stored, at least the first dynamic information of the data to be stored in the current period and the second dynamic information of the data to be stored in the historical period can be determined. The determined first dynamic information and the determined second dynamic information are input into the classification model for classification, and the classification result can be obtained, and the obtained data to be stored is stored in the storage space corresponding to the obtained classification result, thereby achieving the purpose of diverting data at the data storage stage, thereby solving the technical problem of low efficiency of data diversion, and then achieving the technical effect of improving the efficiency of data diversion.

[0081] The technical solution of the embodiment of the present invention is illustrated below in conjunction with preferred implementation modes.

[0082] In the operator network, the performance data generated by the operation of network equipment usually has the characteristics of large data volume, high concurrency and strong real-time, and the data grows very fast. Therefore, when storing and processing such data, there are high requirements for the throughput and scalability of the database.

[0083] However, in the related technologies, the data already stored in the cache is often optimized, and the data cannot be accurately labeled in combination with the actual operation of the system, thus resulting in a technical problem of low efficiency in data diversion.

[0084] In order to solve the above technical problems, an embodiment of the present invention proposes a data storage method, which can at least determine the first dynamic information of the data to be stored in the current period and the second dynamic information of the data to be stored in the historical period according to the first identification information of the data to be stored. The determined dynamic information is input into the classification model for classification, and the classification result can be obtained, and the acquired data to be stored is stored in the storage space corresponding to the classification result, thereby achieving the purpose of data diversion at the data storage stage, thereby solving the technical problem of low efficiency of data diversion, and further achieving the technical effect of improving the efficiency of data diversion.

[0085] In this embodiment, through the time series database hot and cold data classification system based on deep reinforcement learning, the acquired data to be stored can be stored in the storage space corresponding to the classification result. For example, Figure 2(a) is a schematic diagram of a time series database hot and cold data classification system based on deep reinforcement learning according to an embodiment of the present invention. As shown in Figure 2(a), the system may include: a writing module 201, a storage module 202, a query record monitoring module 203 and an algorithm training module 204. Among them, the writing module 201 may include: a parsing program 2011 and a reinforcement learning classifier (agent) 2012, the storage module 202 may include: a dynamic information storage 2021, a database cache 2022, a low-speed storage medium 2023 and an experience playback buffer 2024, and the query record monitoring module 203 may include: a query monitoring program 2031 and a state information delayed completion program 2032.

[0086] In this embodiment, the above-mentioned parsing program 2011 can be used to parse the time series data to obtain the statistical information corresponding to the time series data, for example, to obtain the metric corresponding to the time series data. The above-mentioned writing module 201 can be used to encapsulate the time series data and the corresponding statistical information into a standard data structure, and input the encapsulated time series data and statistical information into the reinforcement learning classifier (agent) 2012. The above-mentioned reinforcement learning classifier (agent) 2012 can be used to perform data diversion on the time series data.

[0087] In this embodiment, the above-mentioned dynamic information storage 2021 can be used to store and update the statistical information of all metrics. Among them, the metric information can be the annotation information unique to the time series database, and a time series data can be stored according to the following structure: sampling time (timestamp), metric and record information. It should be noted that the above-mentioned index in the time series database will continue to increase, but the object metric stored therein is the only constant. Therefore, before the time series data is written into the time series database, the metric information of the time series data can be parsed, and the statistical information of the corresponding metric record can be found in the dynamic information storage, which can be used as a reference for the classification of hot and cold data.

[0088] In this embodiment, the query monitoring program 2031 can be used to determine whether the data [State1, SM, Action, _, _] reaches the state information delayed completion time T. The state information delayed completion program 2032 can be used to update the access frequency and cache hit rate of data in the system, and complete the state information generated by reinforcement learning when writing modules.

[0089] In this embodiment, the algorithm training module 204 may include a historical experience buffer, a state information completion program, and a reinforcement learning algorithm area. In the reinforcement learning algorithm area, a model based on D3QN is used, and based on the temporal difference (TD) algorithm, historical records can be extracted from the historical experience buffer, and the agent can be trained to find the optimization target.

[0090] In this embodiment, through the time series database hot and cold data classification method based on deep reinforcement learning, the acquired data to be stored can be stored in the storage space corresponding to the classification result. For example, Figure 2(b) is a flowchart of a time series database hot and cold data classification method based on deep reinforcement learning according to an embodiment of the present invention. As shown in Figure 2(b), the method may include the following steps:

[0091] Step S201, using a parsing program to parse the data stream.

[0092] In the technical solution provided in the above step S201 of the present invention, the above data stream is written into the time series database in the form of data blocks, and the metric in each time series data in the data block is parsed using the parsing program in the writing module.

[0093] In this embodiment, the general storage format of the above time series database can be shown in the following Table 1:

[0094] Table 1 Storage table of time series database

[0095] index Annotation information Timestamp Value 1 Value Value N Index1 Metric1 Timestamp Value1 ... ValueN Index2 Metric2 Timestamp Value1 ... ValueN Index3 Metric1 Timestamp Value1 ... ValueN

[0096] After parsing the data stream using the parsing program, the process proceeds to step S202, where the statistical information MS of all metrics in the data block can be obtained from the dynamic information storage using the writing module. j .

[0097] In the technical solution provided in step S202 of the present invention, the statistical information MS j It may include: current access frequency, historical access frequency, current hit rate, and historical hit rate.

[0098] Get the statistics of all metrics in the data block MS j Then, enter step S203 to collect the statistical information of all metrics in the data block MS j Combined with the relevant statistical information SM of all metrics at the current moment in the dynamic information storage, it is State1 j .

[0099] In the data block, all the metric statistics MSj Combined with the relevant statistical information SM of all metrics at the current moment in the dynamic information storage, it is State1 j Then, step S204 is entered, and the type of storage operation (Action) is selected by the agent to classify the written time series data to obtain a classification result. For example, it can be determined whether to store the written time series data in the cache space or in the hard disk space.

[0100] In this embodiment, the statistical information MS of all metrics in the above data block j It can include current real-time statistics and historical statistics, MS j It can be composed of the mean, top-k values, target variance, valley value, and remaining cache capacity of the current time series database of all metric statistics in the current system. The actions that the agent can execute can include the following types:

[0101] Type 1, Action 1: classify the time series data into the cache;

[0102] Type 2, Action 2: Classify the time series data into low-speed storage media;

[0103] Type three, Action 3: While classifying the time series data into the cache, update the latest N data into the cache;

[0104] Type 4, Action 4: While classifying the time series data into the hard disk, delete the oldest N data in the cache.

[0105] In this embodiment, MS j Set the following dimensions to evaluate the information of the target metric: x1: the size of the data block where the metric is located; x2: the average access frequency of the target metric within T time intervals; x3: the hit rate of the target metric within T' time; x4 to x7: the access frequency in the last 4 time intervals; x8: the access peak value in T time intervals; x9: the valley value range in T time intervals; x10: the hit rate in the last 4 time intervals. SM sets the following dimensions to evaluate the information of all metrics: x11: the average access frequency of all active metics in the system; x12: the average hit rate of all active metics in the system; x13: the remaining cache capacity of the system; x14: the variance of the system access frequency; x15: the variance of the system hit rate; x16 to x19: the top 4 access peaks in the system within T time, x20 to x23: the last 4 access valleys in the active metrics in the system within T time. Finally, State1 will include MS j23 dimensions combined with SM.

[0106] After obtaining the classification result, go to step S205 and store the data [State1, SM, Action, _, _] in the dynamic information storage in json format.

[0107] In the technical solution provided in step S205 of the present invention, the current system status statistical information is copied as SM', and after waiting for the statistical window time T, the real-time information MS of the target metric is combined. j ', recalculate SM' and record it as SM". Combine SM" with the real-time information MS of the target metric j ', combined into State2 j .

[0108] After storing the data [State1, SM, Action, _, _] in the dynamic information storage in json format, proceed to step S206 and step S207 to query all key-value access requests recorded by the monitoring module and perform data statistics according to the corresponding metric.

[0109] After storing the data [State1, SM, Action, _, _] in the dynamic information storage device in json format, proceed to step S208 to call the data encapsulation monitoring interface to monitor the stored data [State1, SM, Action, _, _].

[0110] After monitoring the stored data [State1, SM, Action, _, _], proceed to step S209 to determine whether the stored data [State1, SM, Action, _, _] reaches the state information delay completion time T.

[0111] If it is determined that the stored data [State1, SM, Action, _, _] has not reached the state information delay completion time T, then return to step S208. If it is determined that the stored data [State1, SM, Action, _, _] has reached the state information delay completion time T, then enter step S210, call the training data encapsulation interface, and convert the data [State1 j , State2 j , Action j , Reward j , done j ] is stored in the historical experience buffer. The subscript j can be used to indicate a single data structure identifier to be completed or being completed after the statistical time T stored in the dynamic information storage device, done j Can be used to indicate whether this interaction has reached a terminal state.

[0112] In the technical solution provided in step S210 of the present invention, based on the current operation information of the system, State2 can be calculated. j and Reward j Among them, State2 j It can be used to indicate the current state of the system, represented by the number of visits and the hit rate of the key. j Can be used to represent the reward function.

[0113] When calling the training data encapsulation interface, the data [State1 j , State2 j , Action j , Reward j , done j ]After being stored in the historical experience buffer, step S211 and step S212 are entered to use D3QN to train the agent and learn the environment agent, and update the parameters therein to the classification environment agent.

[0114] In the technical solution provided in step S211 of the present invention, when using D3QN to train the agent, the neural network parameters of the agent in the current writing module are copied to the learning environment agent, and the learning environment agent can obtain a number (for example, batch_size) of data from the historical experience buffer [State1 i , State2 i , Action i , Reward i , done i ], thereby updating the neural network. The subscript i can be used to represent the data stored in the experience replay buffer, and the reinforcement learning algorithm randomly samples data from the experience replay buffer for model training (e.g., parameter update).

[0115] It should be noted that [State1, State2, Action, Reward, done] can be a set of standard reinforcement learning data structures. Among them, from the perspective of Markov process, State1 can be used to represent the initial state, and Action can be used to represent the action decision made by the model under State1. Since this decision produces a causal effect, the state space can be changed to State2. Assuming that after the model executes an action, there will be a mechanism to evaluate the quality of the action, and it will be generated immediately. Then, under the condition of State1, after the model executes the Action, it will immediately get an evaluation of the quality of the action (Reward). Done can be used to indicate whether this Markov process has ended, and it is given by the system environment.

[0116] In this embodiment, the process of updating the neural network can be as follows: the network parameters θ of the online classification agent are updated α Copy it to the training environment, and the parameters of the training environment agent are θ. Randomly sample m samples from the experience replay buffer D, and each sample is recorded as [State1 i , State2 i , Action i , Reward i , done i ], and calculate the y of the current Q network by the following formula i :

[0117] y i =Reward i +γQ'((φ(State2 i ),argmaxxa'Q(φ(State2 i ),Action i ,θ),θ')

[0118] Among them, θ can be used to represent the parameters of the target network, θ' can be used to represent the parameters of the verification network, and the target network target can have an update delay of t batches compared to the parameters of the verification network.

[0119] In this embodiment, the network parameters can be updated by using the mean square loss function, where the mean square loss function can be expressed as follows:

[0120]

[0121] If the training cycle is an integer multiple of t, θ is changed to θ'.

[0122] In this embodiment, the learning environment agent updates the parameters in the classification environment agent. For example, after T' time, the updated θ of the learning environment agent is copied to the online classification agent, that is, θ is changed to θ α .

[0123] In this embodiment, the acquired data to be stored can be written into the classification model through a method for writing hot and cold data to a time series database based on deep reinforcement learning. For example, FIG2(c) is a schematic diagram of a process for writing hot and cold data to a time series database based on deep reinforcement learning according to an embodiment of the present invention. As shown in FIG2(c), during the writing process, the dynamic information storage device can be a storage structure placed in an additional cache. After receiving a write request for new time series data, the database write module parses the metric bound to a time series data of a data block through a parsing program in the write module. Then, in the real-time data statistics table, the dynamic statistical information of the metric can be found, wherein the dynamic statistical information of the metric can include: MS consisting of the access frequency of the metric and the hit rate of the metric, etc. j . Then combine the relevant statistical information of all metrics in the data block into SM. It should be noted that if the writing module detects that it is a new metric, a new monitoring field is opened in the real-time data statistics table of the dynamic information storage, and the time series data is classified into the time series database for caching.

[0124] In this embodiment, through the time series database hot and cold data monitoring method based on deep reinforcement learning, the access frequency and cache hit rate of the time series data in the system can be updated, and the state information generated by reinforcement learning when writing the module can be completed. For example, Figure 2 (d) is a schematic diagram of a time series database hot and cold data monitoring process based on deep reinforcement learning according to an embodiment of the present invention. As shown in Figure 2 (d), in this monitoring process, when the query record monitoring module is used, when accessing the target keyword (key) through the time series database, it is preferentially searched from the cache. After the target key cannot be found in the cache, it is transferred to the low-speed storage medium to search for the target key. Since the time series database is a non-relational database, generally only one metric is accessed each time a query is made. The query record monitoring module will record the access frequency of the metric and the cache hit rate of the metric in each query.

[0125] In this embodiment, the state transition of the deep reinforcement learning model can be shown in Figure 2(e). For example, Figure 2(e) is a schematic diagram of the state transition of a deep reinforcement learning model according to an embodiment of the present invention. State2 can be composed of MS' and SM". MS' can be the statistical information of the metric at the current sampling time, SM" can be calculated by SM' by observing the changes in the statistical information of the metric, and SM' can be a copy of the previous system state. Among them, x11-x13, x16-x23 can all be calculated and modified based on the basic statistical information. The calculation of the variance of x14 and x15 can be achieved in the following way: Assuming that the number of system metrics is large enough, and the number is N (N can be any natural number), after a period of time, the access frequency of the target metric is m / s. j’ becomes m j ”, the access frequency of the system is given by becomes The variance σ of the system access frequency is x14. At this time, after a period of time, the variance of the system access frequency is σ'. Among them, σ' can be calculated by the following formula (2):

[0126]

[0127] When the data set is large enough, σ' can be approximately calculated by the following formula (3):

[0128]

[0129] In this embodiment, the state transition from State1 to State2 complies with the Markov Decision Process (MDP) condition. That is, the next state depends only on the current state (the high-order Markov case is not considered for the time being) and is not affected by any previous state. Among them, Reward can be calculated by the following formula (4):

[0130]

[0131] Among them, α, β and γ can be used to represent the scaling factor, and α+β=1 can be used to adjust the relative importance between the state of each target metric and the overall operation state of the system. k can be used to represent the scaling factor between the hit rate and the access frequency. m_s and acc s_m It can be used to represent the access hit rate of the target metric and the access hit rate of the system's overall activity metric. m_s and v s_m It can be used to represent the recent access frequency of the target metric and the average recent access frequency of the system's overall activity metric.total and M arrange Can be used to indicate the total cache capacity and the allocated cache capacity respectively.

[0132] In this embodiment, according to the first identification information of the data to be stored, at least the first dynamic information of the data to be stored in the current cycle and the second dynamic information of the data to be stored in the historical cycle can be determined. The determined dynamic information is input into the classification model for classification, and the classification result can be obtained, and the acquired data to be stored is stored in the storage space corresponding to the classification result, thereby achieving the purpose of data diversion at the data storage stage, thereby solving the technical problem of low efficiency of data diversion, and further achieving the technical effect of improving the efficiency of data diversion.

[0133] According to an embodiment of the present invention, a data storage device is also provided. It should be noted that the data storage device can be used to execute a data storage method in the embodiment.

[0134] Figure 3 Schematic diagram of a data storage device according to an embodiment of the present invention. Figure 3 As shown, the data storage device 300 may include: an acquisition unit 301 , a determination unit 302 , a classification unit 303 and a storage unit 304 .

[0135] The acquisition unit 301 is used to acquire the data to be stored from the target database, wherein the storage structure of the data to be stored in the target database at least includes: first identification information of the data to be stored, and the first identification information is used to identify the characteristics of the data to be stored.

[0136] The determination unit 302 is used to determine at least first dynamic information of the data to be stored in the current period and second dynamic information in the historical period based on the first identification information, wherein the moment corresponding to the historical period is earlier than the moment corresponding to the current period, the first dynamic information is used to indicate the frequency of accessing the data to be stored in the current period, and the probability of finding the data to be stored in the current period, and the second dynamic information is used to indicate the frequency of accessing the data to be stored in the historical period, and the probability of finding the data to be stored in the historical period.

[0137] The classification unit 303 is used to input the first dynamic information and the second dynamic information into the classification model for classification to obtain a classification result, wherein the classification model is obtained by training the reinforcement learning model based on the first dynamic sample information and the second dynamic sample information, the first dynamic information sample is used to indicate the frequency of accessing the sample data to be stored in the current cycle, and the probability of finding the sample data to be stored in the current cycle, the second dynamic sample information is used to indicate the frequency of accessing the sample data to be stored in the historical cycle, and the probability of finding the sample data to be stored in the historical cycle, and the classification result is used to indicate the type of storage operation to be performed on the data to be stored.

[0138] The storage unit 304 is used to store the data to be stored in a storage space corresponding to the classification result, wherein the storage space includes a cache space or a hard disk space.

[0139] Optionally, the determination unit 302 may include: a parsing module, used to parse the data to be stored to obtain first identification information; a search module, used to search the first identification information in the dynamic information table to obtain first dynamic information and second dynamic information, wherein the dynamic information table stores a mapping relationship between the first identification information and the first dynamic information, and a mapping relationship between the first identification information and the second dynamic information.

[0140] Optionally, the classification unit 303 may include: a first determination module, used to determine the data stored in the dynamic information table other than the data to be stored as the remaining data; a second determination module, used to determine the second identification information based on the remaining data, wherein the second identification information is used to identify the characteristics of the remaining data; a third determination module, used to determine at least the third dynamic information of the remaining data in the current cycle and the fourth dynamic information in the historical cycle based on the second identification information, wherein the third dynamic information is used to indicate the frequency of accessing the remaining data in the current cycle and the probability of finding the remaining data in the current cycle, and the fourth dynamic information is used to indicate the frequency of accessing the remaining data in the historical cycle and the probability of finding the remaining data in the historical cycle; a combination module, used to combine the first dynamic information, the second dynamic information, the third dynamic information and the fourth dynamic information to obtain the target dynamic information; a classification module, used to input the target dynamic information into the classification model for classification to obtain a classification result.

[0141] Optionally, the second determination module may include: a parsing submodule, configured to parse the remaining data to obtain the second identification information.

[0142] Optionally, the third determination module may include: a search submodule, used to search for the second identification information in the dynamic information table to obtain third dynamic information and fourth dynamic information, wherein the dynamic information table stores a mapping relationship between the second identification information and the third dynamic information, and a mapping relationship between the second identification information and the fourth dynamic information.

[0143] Optionally, the classification result includes a first classification result and a second classification result, the first classification result is used to indicate the type of storage operation to be performed on the data to be stored in the cache space, and the second classification result is used to indicate the type of storage operation to be performed on the data to be stored in the hard disk space. The storage unit 304 may include: a storage module, used to store the data to be stored in the cache space corresponding to the first classification result; or, to store the data to be stored in the hard disk space corresponding to the second classification result.

[0144] Optionally, the data storage device 300 may also include: a first response unit, used to store the remaining data in the cache space in response to the data to be stored having been stored in the cache space; a second response unit, used to delete the historical data stored in the cache space in response to the data to be stored having been stored in the hard disk space, wherein the length of time the historical data is stored in the cache space is greater than the length of time the non-historical data is stored in the cache space, and the non-historical data is the data stored in the cache space other than the historical data.

[0145] Optionally, the data storage device 300 may also include: an updating unit for updating the classification model, including: updating the target parameters in the classification model, wherein the target parameters are used to adjust the classification results; a first classification unit for inputting the first dynamic information and the second dynamic information into the updated classification model for classification to obtain a classification result.

[0146] In this embodiment, an acquisition unit is used to acquire the data to be stored from the target database, wherein the storage structure of the data to be stored in the target database at least includes: first identification information of the data to be stored, the first identification information is used to identify the characteristics of the data to be stored; a determination unit is used to determine at least first dynamic information of the data to be stored in the current cycle and second dynamic information of the data to be stored in the historical cycle based on the first identification information, wherein the moment corresponding to the historical cycle is earlier than the moment corresponding to the current cycle, the first dynamic information is used to indicate the frequency of accessing the data to be stored in the current cycle, and the probability of finding the data to be stored in the current cycle, and the second dynamic information is used to indicate the frequency of accessing the data to be stored in the historical cycle, and the probability of finding the data to be stored in the historical cycle; a classification unit is used to input the first dynamic information and the second dynamic information into a classification model for classification, and obtain Classification result, wherein the classification model is obtained by training the reinforcement learning model based on the first dynamic sample information and the second dynamic sample information, the first dynamic information sample is used to indicate the frequency of accessing the sample data to be stored in the current cycle, and the probability of finding the sample data to be stored in the current cycle, the second dynamic sample information is used to indicate the frequency of accessing the sample data to be stored in the historical cycle, and the probability of finding the sample data to be stored in the historical cycle, the classification result is used to indicate the type of storage operation to be performed on the data to be stored; a storage unit is used to store the data to be stored in the storage space corresponding to the classification result, wherein the storage space includes cache space or hard disk space, thereby achieving the purpose of diverting data in the data warehousing stage, thereby solving the technical problem of low efficiency of data diversion, and further achieving the technical effect of improving the efficiency of data diversion.

[0147] According to an embodiment of the present invention, a processor is further provided. The processor is used to run a program, wherein the program executes the data storage method in the embodiment when the program is run by the processor.

[0148] According to an embodiment of the present invention, there is further provided an electronic device, comprising: a memory storing an executable program; and a processor for running the program, wherein the method for storing data in the embodiment is executed when the program is running.

[0149] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the data storage method in the embodiment.

[0150] According to an embodiment of the present invention, a computer program product is further provided. The computer program product includes a computer program. When the computer program is executed by a processor, the data storage method in the embodiment is implemented.

[0151] According to an embodiment of the present invention, a computer program product is also provided, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the data storage method in the embodiment is implemented.

[0152] According to an embodiment of the present invention, a computer program is further provided. When the computer program is executed by a processor, the data storage method in the embodiment is implemented.

[0153] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0154] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0155] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0156] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed over multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0157] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0158] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the relevant technology or the whole or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the various embodiments of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, referred to as Read-Only Memory), random access memory (RAM, referred to as Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0159] The above are only preferred embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A data storage method, characterized in that: include: Acquire the data to be stored from the target database, wherein the storage structure of the data to be stored in the target database at least includes: first identification information of the data to be stored, the first identification information being used to identify the characteristics of the data to be stored; Based on the first identification information, at least first dynamic information of the data to be stored in the current cycle and second dynamic information of the data to be stored in the historical cycle are determined, wherein the time corresponding to the historical cycle is earlier than the time corresponding to the current cycle, the first dynamic information is used to indicate the frequency of accessing the data to be stored in the current cycle, and the probability of finding the data to be stored in the current cycle, and the second dynamic information is used to indicate the frequency of accessing the data to be stored in the historical cycle, and the probability of finding the data to be stored in the historical cycle; Inputting the first dynamic information and the second dynamic information into a classification model for classification to obtain a classification result, wherein the classification model is obtained by training a reinforcement learning model based on the first dynamic sample information and the second dynamic sample information, the first dynamic information sample is used to indicate the frequency of accessing the sample data to be stored in the current cycle, and the probability of finding the sample data to be stored in the current cycle, the second dynamic sample information is used to indicate the frequency of accessing the sample data to be stored in the historical cycle, and the probability of finding the sample data to be stored in the historical cycle, and the classification result is used to indicate the type of storage operation to be performed on the data to be stored; The data to be stored is stored in a storage space corresponding to the classification result, wherein the storage space includes a cache space or a hard disk space.

2. The method according to claim 1, characterized in that: Based on the first identification information, at least determining first dynamic information of the data to be stored in a current period and second dynamic information in a historical period includes: Parsing the data to be stored to obtain the first identification information; In the dynamic information table, the first identification information is searched to obtain the first dynamic information and the second dynamic information, wherein the dynamic information table stores a mapping relationship between the first identification information and the first dynamic information, and a mapping relationship between the first identification information and the second dynamic information.

3. The method according to claim 2, characterized in that Inputting the first dynamic information and the second dynamic information into a classification model for classification, and obtaining a classification result, including: Determine the data stored in the dynamic information table, except the data to be stored, as remaining data; Based on the remaining data, determining second identification information, wherein the second identification information is used to identify a feature of the remaining data; Based on the second identification information, at least third dynamic information of the remaining data in the current period and fourth dynamic information of the remaining data in the historical period are determined, wherein the third dynamic information is used to indicate the frequency of accessing the remaining data in the current period and the probability of finding the remaining data in the current period, and the fourth dynamic information is used to indicate the frequency of accessing the remaining data in the historical period and the probability of finding the remaining data in the historical period; combining the first dynamic information, the second dynamic information, the third dynamic information and the fourth dynamic information to obtain target dynamic information; The target dynamic information is input into the classification model for classification to obtain the classification result.

4. The method according to claim 3, characterized in that Determining second identification information based on the remaining data includes: The remaining data is parsed to obtain the second identification information.

5. The method according to claim 3, characterized in that: Determining at least third dynamic information of the remaining data in the current period and fourth dynamic information of the remaining data in the historical period based on the second identification information includes: In the dynamic information table, search for the second identification information to obtain the third dynamic information and the fourth dynamic information, wherein the dynamic information table stores a mapping relationship between the second identification information and the third dynamic information, and a mapping relationship between the second identification information and the fourth dynamic information.

6. The method according to claim 1, characterized in that The classification result includes a first classification result and a second classification result, wherein the first classification result is used to indicate the type of storage operation to be performed on the data to be stored in the cache space, and the second classification result is used to indicate the type of storage operation to be performed on the data to be stored in the hard disk space, and storing the data to be stored in the storage space corresponding to the classification result includes: storing the data to be stored in the cache space corresponding to the first classification result; or, The data to be stored is stored in the hard disk space corresponding to the second classification result.

7. The method according to claim 3, characterized in that The method further comprises: In response to the data to be stored having been stored in the cache space, storing the remaining data in the cache space; In response to the data to be stored having been stored in the hard disk space, the historical data stored in the cache space is deleted, wherein the length of time the historical data is stored in the cache space is greater than the length of time non-historical data is stored in the cache space, and the non-historical data is data stored in the cache space other than the historical data.

8. The method according to any one of claims 1 to 7, characterized in that The method further comprises: Updating the classification model includes: updating a target parameter in the classification model, wherein the target parameter is used to adjust the classification result; The first dynamic information and the second dynamic information are input into the updated classification model for classification to obtain the classification result.

9. A data storage device, characterized in that: include: an acquisition unit, configured to acquire the data to be stored from a target database, wherein the storage structure of the data to be stored in the target database at least includes: first identification information of the data to be stored, the first identification information being used to identify a feature of the data to be stored; a determination unit, configured to determine at least first dynamic information of the data to be stored in a current period and second dynamic information of the data to be stored in a historical period based on the first identification information, wherein the time corresponding to the historical period is earlier than the time corresponding to the current period, the first dynamic information is used to indicate the frequency of accessing the data to be stored in the current period and the probability of finding the data to be stored in the current period, and the second dynamic information is used to indicate the frequency of accessing the data to be stored in the historical period and the probability of finding the data to be stored in the historical period; a classification unit, used for inputting the first dynamic information and the second dynamic information into a classification model for classification to obtain a classification result, wherein the classification model is obtained by training a reinforcement learning model based on the first dynamic sample information and the second dynamic sample information, the first dynamic information sample is used to indicate the frequency of accessing the sample data to be stored in the current cycle, and the probability of finding the sample data to be stored in the current cycle, the second dynamic sample information is used to indicate the frequency of accessing the sample data to be stored in the historical cycle, and the probability of finding the sample data to be stored in the historical cycle, and the classification result is used to indicate the type of storage operation to be performed on the data to be stored; A storage unit is used to store the data to be stored in a storage space corresponding to the classification result, wherein the storage space includes a cache space or a hard disk space.

10. A processor, characterized in that: The processor is used to run a program, wherein the program, when run by the processor, executes the data storage method described in any one of claims 1 to 8.

11. An electronic device, characterized in that: include: A memory storing an executable program; A processor, used to run the program, wherein the program executes the data storage method described in any one of claims 1 to 8 when running.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is run, the device where the storage medium is located is controlled to execute the data storage method described in any one of claims 1 to 8.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the data storage method according to any one of claims 1 to 8 is implemented.

Images