Storage release model training method, storage release method and device

Abstract

The present invention discloses a training method, storage release method, and device for a storage release model. The method comprises: obtaining multiple sets of sample data and their labels, each set of sample data including the access frequency of multiple storage objects and the cost of the storage objects, and the labels being the target released storage objects in each set of sample data; inputting each set of sample data into a storage release model, and outputting the predicted released storage objects corresponding to each set of sample data; performing a loss function calculation based on the predicted released storage objects and the corresponding target released storage objects, and updating the parameters of the storage release model to determine the target storage release model. The target storage release model constructed by the present invention optimizes the file release strategy for Alluxio cross-media tiered storage. On the basis of considering the frequency of file usage in the released space, it also considers the storage medium space release cost and the data access efficiency of different media, thereby reducing the block release cost and improving the system operation efficiency.

Description

Technical Field

[0001] The present invention relates to the field of data storage processing technology, and in particular to a storage release model training method, a storage release method and a device. Background Art

[0002] Alluxio is a memory-based virtual distributed storage system commonly used in big data platforms. Data in this system is typically stored as file blocks between the compute and storage layers, allowing the compute layer to access relevant data more quickly and efficiently through Alluxio. In practice, Alluxio's tiered storage across storage media (such as MEM, SSD, and HDD) is one of its more typical application scenarios. The general process is as follows:

[0003] 1. Set Alluxio to a three-tier storage model with MEM, SSD, and HDD as the target storage targets for release.

[0004] 2. Adopt the storage rule from high to low level. When the space of the storage object released by the high-level target is sufficient, the data will be stored in the storage object released by the high-level target by default.

[0005] 3. When the high-level release storage object overflows, storage space is sought in the secondary release storage object. If there is no storage space on the MEM, SSD, or HDD, the Alluxio system's default space release algorithm is used to release the target release storage object and allocate storage space for new data.

[0006] The default space release algorithm of the Alluxio system is as follows:

[0007] The least recently used storage objects are released sequentially based on access frequency. After a write operation is initiated, if the Alluxio master node detects that there is insufficient space for allocation in all three layers of target release storage objects, the space release algorithm will allocate the space occupied by the least recently used storage objects in the target release storage objects without distinguishing them from each other and will only consider the access frequency of the storage objects.

[0008] However, only considering the access frequency to release the data in the target release storage object, when there is data to be stored, the data to be stored is randomly stored in the target release object, resulting in an increase in the cost of data release and low reliability of storage release. Summary of the Invention

[0009] In view of this, an embodiment of the present invention provides a storage release model training method, a storage release method and a device, aiming to solve the problem of low reliability of storage release.

[0010] According to a first aspect, an embodiment of the present invention provides a method for training a storage release model, comprising:

[0011] Acquire multiple groups of sample data and their labels, each group of the sample data including access frequencies of multiple storage objects and costs of the storage objects, and the labels being target released storage objects in each group of the sample data;

[0012] Inputting each group of sample data into a storage release model, and outputting a predicted released storage object corresponding to each group of sample data;

[0013] A loss function is calculated based on the predicted released storage object and the corresponding target released storage object, and parameters of the storage release model are updated to determine a target storage release model.

[0014] The training method of the storage release model provided in an embodiment of the present invention is applied to the Alluxio system. The data of the system is usually stored between the computing layer and the storage layer in the form of storage objects, so that the computing layer can use the Alluxio system to generate a target storage release model based on deep learning through multiple groups of sample data and labels. The predicted released storage object can be calculated based on the access frequency of the storage object and the cost of the storage object, and then the loss function is calculated for the predicted released object and the target released object. The parameters of the storage release model are updated based on the calculation results, and finally the target storage release model is determined. With reference to the two influencing factors of access frequency and cost, the accuracy of the sample data released by the target released object can be improved, so that the release cost of the target released object is minimized, and the target storage release model is input into the Alluxio system, so that the computing layer can access related data more quickly and efficiently through the Alluxio system.

[0015] In conjunction with the first aspect, in a first implementation of the first aspect, the storage release model includes a first input node, a second input node, and an output node, and is characterized in that inputting each group of sample data into the storage release model and outputting the predicted released storage object corresponding to each group of sample data includes:

[0016] For each set of sample data, inputting the access frequency of each storage object into the storage release model through the first input node;

[0017] inputting the cost of each storage object into the storage release model through the second input node;

[0018] Obtaining a first weight corresponding to the first input node and a second weight corresponding to the second input node;

[0019] The predicted released storage object output by the output node is determined according to the first weight and the access frequency, and the second weight and the cost.

[0020] The training method of the storage release model provided by the embodiment of the present invention, on the basis of considering the access frequency of the storage objects in the target released storage object, adds consideration of the cost of the storage objects and the access efficiency of different storage objects. Therefore, there are two factors in the sample data that affect the determination of the final target storage release model. Because the influence of the two factors is different, the first weight is configured for the access frequency and the second weight is configured for the cost. The predicted released storage objects output according to the sample data and the corresponding target released storage objects are calculated, and the weights configured for the two factors are adjusted based on the calculation results.

[0021] In combination with the first implementation manner of the first aspect, in the second implementation manner of the first aspect, the performing of a loss function calculation based on the predicted released storage object and the corresponding target released storage object, and updating parameters of the storage release model to determine a target storage release model, includes:

[0022] Calculate the loss between the predicted released storage object and the target released storage object by using a target loss function and calculate the gradient;

[0023] The first weight and the second weight are adjusted based on the gradient.

[0024] The training method of the storage release model provided by an embodiment of the present invention, on the basis of considering the access frequency of storage objects in the target released storage object, adds consideration of the cost of storage objects and the access efficiency of different storage objects, and makes adjustments according to the weights of various aspects, so as to determine the target released storage object, reduce the release cost of storage objects, and improve the operating efficiency of the system.

[0025] In combination with the second implementation manner of the first aspect, in the third implementation manner of the first aspect, the adjusting according to the first weight and the second weight based on the gradient includes:

[0026] analyzing changes in the first weight and the second weight determined in each training;

[0027] When the changes of the first weight and the second weight are within a preset range, a first target weight and a second target weight are generated.

[0028] The training method of the storage release model provided by an embodiment of the present invention continuously analyzes multiple groups of sample data and their labels, and then continuously adjusts the changes of the first weight and the second weight, so that the output predicted release storage object becomes closer and closer to the target release storage object. Therefore, when the changes of the first weight and the second weight are within a preset range, it indicates that they are gradually stabilizing, and the finally stabilized first weight and second weight generate the first target weight and the second target weight.

[0029] According to a second aspect, an embodiment of the present invention further provides a storage release method, the method comprising:

[0030] Obtain access frequencies and costs of multiple storage objects to be released;

[0031] Inputting access frequencies and costs of the plurality of storage objects to be released into a target storage release model to determine a target storage object among the plurality of storage objects to be released, wherein the target storage release model is trained according to the storage release model training method described above;

[0032] Release the target storage object.

[0033] The storage release method provided in an embodiment of the present invention is applied to the Alluxio system. The data of this system is usually stored between the computing layer and the storage layer in the form of storage objects. This facilitates the computing layer to obtain the target storage object through the Alluxio system based on the access frequency and cost of the storage object through the target storage release module, and then release the target storage object, so that the storage object to be released has free storage space, thereby facilitating the computing layer to access related data more quickly and efficiently through the Alluxio system.

[0034] In conjunction with the second aspect, in the first implementation of the second aspect, the method further includes:

[0035] Get the data to be stored;

[0036] The data to be stored is stored in the target storage object.

[0037] The storage release method provided in an embodiment of the present invention stores the data to be stored in the storage space of the target storage object, making it easier for the computing layer to access the relevant data more quickly and efficiently through the Alluxio system.

[0038] According to a third aspect, an embodiment of the present invention further provides a training device for a storage release model, comprising:

[0039] A first acquisition module is configured to acquire multiple groups of sample data and labels thereof, wherein each group of sample data includes access frequencies of multiple storage objects and costs of the storage objects, and the labels are target released storage objects in each group of sample data;

[0040] A first input module is configured to input each group of sample data into a storage release model and output a predicted released storage object corresponding to each group of sample data;

[0041] The target module is configured to calculate a loss function based on the predicted released storage object and the corresponding target released storage object, and update parameters of the storage release model to determine a target storage release model.

[0042] The storage release model training device provided by an embodiment of the present invention is applied to the Alluxio system. The data of the system is usually stored between the computing layer and the storage layer in the form of storage objects, so that the computing layer can use the Alluxio system to generate a target storage release model based on deep learning through multiple groups of sample data and labels. The predicted released storage object can be calculated based on the access frequency of the storage object and the cost of the storage object, and then the loss function is calculated for the predicted released object and the target released object. The parameters of the storage release model are updated based on the calculation results, and finally the target storage release model is determined. With reference to the two influencing factors of access frequency and cost, the accuracy of the sample data released by the target released object can be improved, so that the release cost of the target released object is minimized, and the target storage release model is input into the Alluxio system, so that the computing layer can access related data more quickly and efficiently through the Alluxio system.

[0043] According to a fourth aspect, an embodiment of the present invention further provides a storage release device, comprising:

[0044] A second acquisition module is used to obtain access frequencies and costs of multiple storage objects to be released;

[0045] a second input module, configured to input the access frequencies and costs of the plurality of storage objects to be released into a target storage release model, and determine a target storage object among the plurality of storage objects to be released, wherein the target storage release model is trained according to the storage release model training method described above;

[0046] A release module is used to release the target storage object.

[0047] The storage release device provided in an embodiment of the present invention is applied to the Alluxio system. The data of the system is usually stored between the computing layer and the storage layer in the form of storage objects, so that the computing layer can obtain the target storage object through the Alluxio system based on the access frequency and cost of the storage object through the target storage release module, and then release the target storage object, so that there is free storage space with the released storage object, thereby facilitating the computing layer to access related data more quickly and efficiently through the Alluxio system.

[0048] According to the fifth aspect, an embodiment of the present invention provides an electronic device, comprising a memory and a processor, wherein the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the training method of the storage release model described in the first aspect or any one embodiment of the first aspect, or executes the storage release method in the second aspect or any one embodiment of the second aspect by executing the computer instructions.

[0049] According to the sixth aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the training method of the storage release model described in the first aspect or any one of the embodiments of the first aspect, or execute the storage release method in the second aspect or any one of the embodiments of the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 Flowchart of a training method for a storage release model provided by an embodiment of the present invention;

[0052] Figure 2 1 is a flow chart of a storage release method according to an embodiment of the present invention;

[0053] Figure 3 This is a functional module diagram of a training device using the storage-release model provided by an embodiment of the present invention;

[0054] Figure 4 This is a functional module diagram of a storage release device provided by an embodiment of the present invention;

[0055] Figure 5It is a schematic diagram of the hardware structure of the electronic device provided by the embodiment of the present invention. DETAILED DESCRIPTION

[0056] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0057] It should be noted that the method for training the storage release model provided in the embodiment of the present application, its execution subject can be a device for training the storage release model, and the device for training the storage release model can be implemented as part or all of a computer device through software, hardware, or a combination of software and hardware, wherein the computer device can be a server or a terminal, wherein the server in the embodiment of the present application can be a single server or a server cluster composed of multiple servers, and the terminal in the embodiment of the present application can be a smart phone, a personal computer, a tablet computer, a wearable device, an intelligent robot, and other intelligent hardware devices. In the following method embodiments, the execution subject is an electronic device as an example for explanation.

[0058] In one embodiment of the present application, Figure 1 As shown, a training method for a storage release model is provided, which is illustrated by taking the application of the method to an electronic device as an example, and includes the following steps:

[0059] S100 , obtaining multiple groups of sample data and labels thereof, each group of the sample data including access frequencies of multiple storage objects and costs of the storage objects, and the labels being target released storage objects in each group of the sample data.

[0060] In this embodiment, the storage objects are set as three-level storage objects: MEM, SSD, and HDD, and the release costs of the storage objects are recorded as cost 1, cost 2, and cost 3. When the storage space of the three-level storage objects is full of sample data, it is necessary to evaluate the optimal storage object based on the access frequency of the storage object and the cost of the storage object, that is, the target released storage object.

[0061] When the storage object is full of data, the data in the storage object is accessed, and the number of times each data is accessed will be recorded. Therefore, within a certain period of time, the access frequency of each data can be obtained. When the data in the storage object is full, the data with a lower access frequency in the storage object needs to be released so that subsequent data can continue to be stored. Due to the different storage objects, the data processing efficiency will also be different. Taking the above-mentioned storage objects as MEM, SSD and HDD as an example, since the efficiency of storing data in MEM, SSD and HDD decreases in sequence, releasing the data in MEM and storing new data in MEM is the fastest and the cost is relatively the lowest.

[0062] Explain the sample data. For example, consider three sets of sample data, each containing three storage objects. Now, one storage object needs to be released from these three storage objects. The target release storage object is the actual storage object that should be released. The storage release model is trained so that the model predicts the same release object as the target release storage object.

[0063] S200: Input each group of sample data into a storage release model, and output a predicted released storage object corresponding to each group of sample data.

[0064] In this embodiment, each group of sample data is input into the storage release model, and the predicted released storage object corresponding to each group of sample data is output. Multiple data samples can be obtained through the storage release model, and the sample data facilitates the verification of the subsequently generated storage release model.

[0065] In an optional embodiment, taking the above-mentioned storage objects as MEM, SSD and HDD as an example, for example, a group of sample data with the lowest access frequency, which data exists in MEM, SSD and HDD, the sample data is input into the storage release model. When the output predicted released storage object is MEM, it is the target released storage object. If the output predicted released storage object is SSD or HDD, the predicted released storage object deviates from the target released storage object.

[0066] S300 , performing loss function calculation based on the predicted released storage object and the corresponding target released storage object, and updating parameters of the storage release model to determine a target storage release model.

[0067] The training method for the storage release model provided in an embodiment of the present invention is applied to the Alluxio system. The data of the system is usually stored between the computing layer and the storage layer in the form of storage objects, so that the computing layer can use the Alluxio system to generate a target storage release model based on deep learning through multiple sets of sample data and labels. The predicted released storage object can be calculated based on the access frequency of the storage object and the cost of the storage object, and then the loss function is calculated between the predicted released object and the target released object. The parameters of the storage release model are updated based on the calculation results, and the target storage release model is finally determined. The target storage release model is input into the Alluxio system, so that the computing layer can access relevant data more quickly and efficiently through the Alluxio system.

[0068] In this embodiment, the learning rate, dropout rate, number of network layers, optimizer, number of neurons and other parameters of the storage release model are optimized and adjusted through a validation set generated by multiple groups of sample data, so that the network has better feature fitting and decision-making capabilities. At this time, the accuracy of the storage release model output and the true label should reach more than 95%. The specific accuracy can be set according to actual conditions and is not limited here. The test set is input into the storage release model with the best performance on the validation set for a final performance evaluation. The quality of the storage release model is measured by the test results. When the results meet the test requirements, the storage release model parameters are saved and the storage release model is introduced into the Alluxio system to assist the system in selecting the target release storage object at the lowest cost, so that the computing layer can access related data more quickly and efficiently through Alluxio.

[0069] In an optional embodiment of the present application, Figure 1 As shown, the storage release model includes a first input node, a second input node, and an output node. The above S200 of "inputting each group of sample data into the storage release model and outputting the predicted released storage objects corresponding to each group of sample data" includes:

[0070] (1) For each set of sample data, the access frequency of each storage object is input into the storage release model through the first input node;

[0071] (2) inputting the cost of each storage object into the storage release model through the second input node;

[0072] (3) obtaining a first weight corresponding to the first input node and a second weight corresponding to the second input node;

[0073] (4) Determine the predicted released storage object output by the output node based on the first weight and the access frequency, and the second weight and the cost.

[0074] In this embodiment, the first weight is set to x, and the second weight is set to y. According to the formula: optimal release storage object = min (access frequency * x + cost * y), the values ​​of x and y can be calculated through multiple sets of sample data.

[0075] The training method of the storage release model provided by the embodiment of the present invention, on the basis of considering the access frequency of the storage objects in the target released storage object, adds consideration of the cost of the storage objects and the access efficiency of different storage objects. Therefore, there are two factors in the sample data that affect the determination of the final target storage release model. Because the influence of the two factors is different, the first weight is configured for the access frequency and the second weight is configured for the cost. The predicted released storage objects output according to the sample data and the corresponding target released storage objects are calculated, and the weights configured for the two factors are adjusted based on the calculation results.

[0076] In an optional embodiment of the present application, Figure 1 As shown, the above-mentioned step S300 of "calculating a loss function based on the predicted released storage object and the corresponding target released storage object, and updating the parameters of the storage release model to determine a target storage release model" includes:

[0077] (1) calculating the loss between the predicted released storage object and the target released storage object by using a target loss function and calculating the gradient;

[0078] (2) Adjusting the first weight and the second weight based on the gradient.

[0079] In this embodiment, the target loss function adopts the cross-entropy loss calculation method. The cross-entropy loss is used to calculate the loss between the output of the predicted released storage object and the target released storage object and calculate the gradient. The first weight and the second weight in the target storage release model are optimized through the gradient feedback. When the output of the target storage release model differs from the true label by less than 10%, the target storage release model is output.

[0080] The training method of the storage release model provided by an embodiment of the present invention, on the basis of considering the access frequency of storage objects in the target released storage object, adds consideration of the cost of storage objects and the access efficiency of different storage objects, and makes adjustments according to the weights of various aspects, so as to determine the target released storage object, reduce the release cost of storage objects, and improve the operating efficiency of the system.

[0081] In an optional embodiment of the present application, Figure 1 As shown, the above “adjusting the first weight and the second weight based on the gradient” includes:

[0082] (1) analyzing changes in the first weight and the second weight determined in each training;

[0083] (2) When the changes in the first weight and the second weight are within a preset range, a first target weight and a second target weight are generated.

[0084] The training method of the storage release model provided by an embodiment of the present invention draws a parameter trend graph of the first weight and the second weight. Due to the continuous analysis of multiple groups of sample data and their labels, the changes in the first weight and the second weight are continuously adjusted, so that the output predicted release storage object is closer and closer to the target release storage object. Therefore, when the changes in the first weight and the second weight are within a preset range, it indicates that they are gradually stabilizing. Finally, the stabilized first weight and second weight generate the first target weight and the second target weight.

[0085] In one embodiment of the present application, Figure 2 As shown, a storage release method is also provided, comprising the following steps:

[0086] S400, obtaining access frequencies and costs of multiple storage objects to be released;

[0087] S500: Inputting access frequencies and costs of the plurality of storage objects to be released into a target storage release model to determine a target storage object among the plurality of storage objects to be released, wherein the target storage release model is trained according to the storage release model training method described above;

[0088] S600: Release the target storage object.

[0089] The storage release method provided in an embodiment of the present invention is applied to the Alluxio system. The data of this system is usually stored between the computing layer and the storage layer in the form of storage objects. This facilitates the computing layer to obtain the target storage object through the Alluxio system based on the access frequency and cost of the storage object through the target storage release module, and then release the target storage object, so that the storage object to be released has free storage space, thereby facilitating the computing layer to access related data more quickly and efficiently through the Alluxio system.

[0090] In an optional embodiment of the present application, Figure 2 As shown, the method further includes:

[0091] (1) Obtaining data to be stored;

[0092] (2) Storing the data to be stored in the target storage object.

[0093] The storage release method provided in an embodiment of the present invention stores the data to be stored in the storage space of the target storage object, making it easier for the computing layer to access the relevant data more quickly and efficiently through the Alluxio system.

[0094] In one embodiment of the present application, Figure 3 As shown, a training device for a storage release model is also provided, comprising a first acquisition module 1, a first input module 2 and a target module 3, wherein:

[0095] A first acquisition module 1 is configured to acquire multiple sets of sample data and labels thereof, wherein each set of sample data includes access frequencies of multiple storage objects and costs of the storage objects, and the labels are target released storage objects in each set of sample data;

[0096] A first input module 2 is configured to input each group of sample data into a storage release model and output a predicted released storage object corresponding to each group of sample data;

[0097] The target module 3 is configured to calculate a loss function based on the predicted released storage object and the corresponding target released storage object, and update the parameters of the storage release model to determine a target storage release model.

[0098] The storage release model training device provided by an embodiment of the present invention is applied to the Alluxio system. The data of the system is usually stored between the computing layer and the storage layer in the form of storage objects, so that the computing layer can use the Alluxio system to generate a target storage release model based on deep learning through multiple groups of sample data and labels. The predicted released storage object can be calculated based on the access frequency of the storage object and the cost of the storage object, and then the loss function is calculated for the predicted released object and the target released object. The parameters of the storage release model are updated based on the calculation results, and finally the target storage release model is determined. With reference to the two influencing factors of access frequency and cost, the accuracy of the sample data released by the target released object can be improved, so that the release cost of the target released object is minimized, and the target storage release model is input into the Alluxio system, so that the computing layer can access related data more quickly and efficiently through the Alluxio system.

[0099] In one embodiment of the present application, Figure 4 As shown, a storage release device is also provided, comprising a second acquisition module 4, a second input module 5 and a release module 6, wherein:

[0100] A first acquisition module 4 is used to obtain access frequencies and costs of multiple storage objects to be released;

[0101] An input module 5 is configured to input the access frequencies and costs of the plurality of storage objects to be released into a target storage release model to determine a target storage object among the plurality of storage objects to be released, wherein the target storage release model is trained according to the storage release model training method described above;

[0102] The release module 6 is configured to release the target storage object.

[0103] The storage release device provided in an embodiment of the present invention is applied to the Alluxio system. The data of the system is usually stored between the computing layer and the storage layer in the form of storage objects, so that the computing layer can obtain the target storage object through the Alluxio system based on the access frequency and cost of the storage object through the target storage release module, and then release the target storage object, so that there is free storage space with the released storage object, thereby facilitating the computing layer to access related data more quickly and efficiently through the Alluxio system.

[0104] In an optional embodiment of the present application, Figure 3 As shown, it also includes a second acquisition module and a storage module, wherein:

[0105] A second acquisition module is used to acquire data to be stored;

[0106] The storage module is used to store the data to be stored in the target storage object.

[0107] It should be understood that although Figure 1 、 Figure 2 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1 as well as Figure 2 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The order of execution of these steps or stages is not necessarily one by one, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0108] The specific limitations and beneficial effects of the storage release device can be found in the limitations of the storage release method above and will not be further elaborated here. Each module in the above-mentioned storage release device can be implemented in whole or in part through software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in an electronic device in hardware form, or can be stored in a memory in the electronic device in software form, so that the processor can call and execute the corresponding operations of each of the above modules.

[0109] An embodiment of the present invention further provides an electronic device having the above Figure 3 The training device of the storage release model shown and Figure 4 Storage release shown.

[0110] like Figure 5 As shown, Figure 5 is a structural diagram of an electronic device provided by an optional embodiment of the present invention, such as Figure 5 As shown, the electronic device may include: at least one processor 71, such as a CPU (Central Processing Unit), at least one communication interface 73, a memory 74, and at least one communication bus 72. The communication bus 72 is used to realize the connection and communication between these components. The communication interface 73 may include a display screen (Display), a keyboard (Keyboard), and the optional communication interface 73 may also include a standard wired interface and a wireless interface. The memory 74 may be a high-speed RAM memory (Random Access Memory, volatile random access memory) or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 74 may optionally be at least one storage device located away from the aforementioned processor 71. The processor 71 may be combined with Figure 3 The training device of the storage release model shown and Figure 4 In the storage release device shown, the memory 74 stores an application program, and the processor 71 calls the program code stored in the memory 74 to execute any of the above method steps.

[0111] The communication bus 72 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The communication bus 72 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0112] Among them, the memory 74 may include a volatile memory (English: volatile memory), such as a random-access memory (English: random-access memory, abbreviated: RAM); the memory may also include a non-volatile memory (English: non-volatile memory), such as a flash memory (English: flash memory), a hard disk drive (English: hard disk drive, abbreviated: HDD) or a solid-state drive (English: solid-state drive, abbreviated: SSD); the memory 74 may also include a combination of the above types of memory.

[0113] The processor 71 may be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and a NP.

[0114] The processor 71 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0115] Optionally, the memory 74 is also used to store program instructions. The processor 71 can call the program instructions to implement the Figure 1 The training method of the storage release model shown in the embodiment or Figure 2 The storage release method shown in the embodiment.

[0116] The embodiment of the present invention further provides a non-transitory computer storage medium, wherein the computer storage medium stores computer executable instructions, which can execute the training method of the storage release model in any of the above method embodiments or Figure 2The storage release method shown in the embodiment. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD); the storage medium may also include a combination of the above types of memory.

[0117] Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A training method for a storage-release model, characterized in that: The following steps are involved: Acquire multiple groups of sample data and their labels, each group of the sample data including access frequencies of multiple storage objects and costs of the storage objects, the labels being target release storage objects in each group of the sample data, the target release storage objects being storage objects actually to be released; Inputting each group of the sample data into a storage release model, and outputting a predicted released storage object corresponding to each group of the sample data, wherein the predicted released storage object is a released object predicted by the storage release model; A loss function is calculated based on the predicted released storage object and the corresponding target released storage object, and parameters of the storage release model are updated to determine a target storage release model.

2. The method according to claim 1, wherein the storage release model comprises a first input node, a second input node, and an output node, wherein: The step of inputting each group of sample data into a storage release model and outputting a predicted released storage object corresponding to each group of sample data includes: For each set of sample data, inputting the access frequency of each storage object into the storage release model through the first input node; inputting the cost of each storage object into the storage release model through the second input node; Obtaining a first weight corresponding to the first input node and a second weight corresponding to the second input node; The predicted released storage object output by the output node is determined according to the first weight and the access frequency, and the second weight and the cost.

3. The method according to claim 2, characterized in that The calculating a loss function based on the predicted released storage object and the corresponding target released storage object, and updating parameters of the storage release model to determine a target storage release model, includes: Calculate the loss between the predicted released storage object and the target released storage object by using a target loss function and calculate the gradient; The first weight and the second weight are adjusted based on the gradient.

4. The method according to claim 3, characterized in that The adjusting the first weight and the second weight based on the gradient includes: analyzing changes in the first weight and the second weight determined in each training; When the changes of the first weight and the second weight are within a preset range, a first target weight and a second target weight are generated.

5. A storage release method, characterized in that: include: Obtain access frequencies and costs of multiple storage objects to be released; inputting access frequencies and costs of the plurality of storage objects to be released into a target storage release model, and determining a target storage object among the plurality of storage objects to be released, wherein the target storage release model is trained according to the storage release model training method according to any one of claims 1 to 4; Release the target storage object.

6. The storage release method according to claim 5, characterized in that: The method further comprises: Get the data to be stored; The data to be stored is stored in the target storage object.

7. A training device for a storage and release model, characterized in that: include: A first acquisition module is configured to acquire multiple groups of sample data and labels thereof, wherein each group of sample data includes access frequencies of multiple storage objects and costs of the storage objects, and the labels are target release storage objects in each group of sample data, wherein the target release storage objects are storage objects that are actually to be released; a first input module, configured to input each group of sample data into a storage release model, and output a predicted released storage object corresponding to each group of sample data, wherein the predicted released storage object is a released object predicted by the storage release model; The target module is configured to calculate a loss function based on the predicted released storage object and the corresponding target released storage object, and update parameters of the storage release model to determine a target storage release model.

8. A storage release device, characterized in that: include: A second acquisition module is used to obtain access frequencies and costs of multiple storage objects to be released; a second input module, configured to input access frequencies and costs of the plurality of storage objects to be released into a target storage release model, and determine a target storage object among the plurality of storage objects to be released, wherein the target storage release model is trained according to the storage release model training method according to any one of claims 1 to 4; A release module is used to release the target storage object.

9. An electronic device, characterized in that: It includes a memory and a processor, wherein the memory stores computer instructions, and the processor executes the training method of the storage release model described in any one of claims 1 to 4, or executes the storage release method shown in claim 5 or 6 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the training method of the storage release model described in any one of claims 1 to 4, or to execute the storage release method described in claim 5 or 6.

Images