Object storage management method, system and device

Abstract

The invention discloses an object storage management method, system and device, belongs to the technical field of data storage, and is used for solving the problem of lack of storage management of stored objects in related technologies. The method comprises the following steps: acquiring first feature information of a stored object in a target data storage module; the target data storage module comprises a plurality of data storage spaces; the first feature information comprises calling feature information and scene feature information of the stored object; according to the first feature information and a preset prediction strategy, predicting a calling condition of the stored object, and obtaining a first calling prediction value corresponding to the stored object; determining a target storage space corresponding to the first calling predicted value according to a first corresponding relationship between a preset calling predicted value and a data storage space; and under the condition that the target storage space is inconsistent with the first storage space where the stored object is located, scheduling the stored object to the target storage space.

Description

Technical Field

[0001] The present application belongs to the field of data storage technology, and specifically relates to an object storage management method, system, and device. Background Art

[0002] With the development of 5G (fifth-generation mobile communication technology) and gigabit optical networks, as well as the widespread application of AI (artificial intelligence) in video detection, video surveillance projects are increasing across various industries. Storage accounts for the largest portion of the cost of video surveillance projects, and with the advancement of high-definition video, storage costs are increasing. However, the percentage of video surveillance data retrieved and played back after being written to storage devices is very low. In other words, the majority of video surveillance data is never retrieved and played back after being written to storage devices, and is overwritten upon expiration. Therefore, reducing the storage cost of video surveillance data is a key goal for cost reduction and efficiency improvement in such projects.

[0003] However, due to the current lack of storage management solutions for stored video surveillance data, existing technologies have difficulty achieving the goal of reducing storage costs from the perspective of stored video surveillance data, which greatly limits the implementation of storage cost reduction solutions. Therefore, it is particularly necessary to provide a storage management solution for video surveillance data. Summary of the Invention

[0004] The embodiments of the present application provide an object storage management method, system, and device, which can solve the problem of lack of storage management of stored objects in related technologies.

[0005] In a first aspect, an embodiment of the present application provides an object storage management method, including: obtaining first characteristic information of a stored object in a target data storage module; the target data storage module includes multiple data storage spaces; the first characteristic information includes call characteristic information and scene characteristic information of the stored object; based on the first characteristic information and a preset prediction strategy, predicting the call situation of the stored object to obtain a first call prediction value corresponding to the stored object; based on a first correspondence between the preset call prediction value and the data storage space, determining the target storage space corresponding to the first call prediction value; when the target storage space is inconsistent with the first storage space where the stored object is located, scheduling the stored object to the target storage space.

[0006] In the second aspect, an embodiment of the present application provides an object storage management system, including a target data storage module, a prediction processing module and a scheduling processing module; wherein the target data storage module is used to obtain and store data generated in real time by the target platform to obtain a stored object; the target platform is connected to the target data storage module; the target data storage module includes multiple data storage spaces; the prediction processing module is used to obtain the first feature information of the stored object; the first feature information includes the call feature information and scene feature information of the stored object; based on the first feature information and the preset prediction strategy, the call situation of the stored object is predicted to obtain the first call prediction value corresponding to the stored object; based on the first correspondence between the preset call prediction value and the data storage space, the target storage space corresponding to the first call prediction value is determined; the scheduling processing module is used to schedule the stored object to the target storage space when the target storage space is inconsistent with the first storage space where the stored object is located.

[0007] In a third aspect, an embodiment of the present application provides an object storage management device, comprising: a first acquisition module, for acquiring first characteristic information of a stored object in a target data storage module; the target data storage module comprises a plurality of data storage spaces; the first characteristic information comprises call characteristic information and scene characteristic information of the stored object; a first prediction module, for predicting the call status of the stored object based on the first characteristic information and a preset prediction strategy, and obtaining a first call prediction value corresponding to the stored object; a first determination module, for determining a target storage space corresponding to the first call prediction value based on a first correspondence between a preset call prediction value and a data storage space; a first scheduling module, for scheduling the stored object to the target storage space when the target storage space is inconsistent with the first storage space where the stored object is located.

[0008] In a fourth aspect, an embodiment of the present application provides an electronic device comprising a processor; and a memory arranged to store computer-executable instructions, wherein the computer-executable instructions are configured to be executed by the processor, and the computer-executable instructions are executed by the processor to implement the steps of the object storage management method as described in the first aspect.

[0009] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, which is used to store computer-executable instructions. When the computer-executable instructions are executed by a processor, the steps of the object storage management method described in the first aspect are implemented.

[0010] In a sixth aspect, an embodiment of the present application provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the object storage management method as described in the first aspect.

[0011] In a seventh aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run executable instructions to implement the steps of the object storage management method as described in the first aspect.

[0012] In an embodiment of the present application, by obtaining the first characteristic information of the stored object in the target data storage module, the target data storage module includes multiple data storage spaces, and the first characteristic information includes the call characteristic information and scene characteristic information of the stored object, so that the call situation of the stored object is predicted based on the first characteristic information and the preset prediction strategy, and the first call prediction value corresponding to the stored object can be obtained, thereby achieving the effect of predicting the call possibility of the stored object. Furthermore, based on the first corresponding relationship between the preset call prediction value and the data storage space, the target storage space corresponding to the first call prediction value is determined, and when the target storage space is inconsistent with the first storage space where the stored object is located, the stored object is scheduled to the target storage space. It can be seen that this technical solution can schedule the stored objects according to the call possibility of the stored objects, and achieves the effect of clustering the stored objects according to the call possibility. This is conducive to centralized management of stored objects that are easy to be called, thereby facilitating better management of stored objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a schematic block diagram of an object storage management system provided in an embodiment of the present application; Figure 2 This is a flowchart of an object storage management method provided by an embodiment of the present application; Figure 3 Schematic diagram of the data transmission relationship between the video surveillance platform and each data storage space provided in an embodiment of the present application; Figure 4 This is a schematic block diagram of a time window provided in an embodiment of the present application; Figure 5 This is a schematic diagram of the implementation principle of the model training process provided in the embodiment of the present application; Figure 6 This is a schematic diagram of the scheduling process of stored objects provided in an embodiment of the present application; Figure 7 This is a schematic diagram of a scheduling process for stored objects provided by another embodiment of the present application; Figure 8This is a schematic diagram of a scheduling process for stored objects provided by another embodiment of the present application; Figure 9 This is a flowchart of an object storage management method provided by another embodiment of the present application; Figure 10 This is a schematic diagram of the structure of an object storage management device provided by an embodiment of the present application; Figure 11 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0014] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0015] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.

[0016] The object storage management method, system, and device provided in the embodiments of the present application are described in detail below with reference to specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0017] Figure 1 is a schematic block diagram of an object storage management system provided in an embodiment of the present application, such as Figure 1 As shown, the object storage management system includes a target data storage module 110, a prediction processing module 120, and a scheduling processing module 130. The target data storage module 110 is connected to the prediction processing module 120 and the scheduling processing module 130 respectively. The prediction processing module 120 is connected to the scheduling processing module 130.

[0018] In this embodiment, the target data storage module 110 is used to obtain and store data generated in real time by the target platform 20 to obtain a stored object. The target platform 20 is connected to the target data storage module 110, and the target data storage module includes a plurality of data storage spaces. The prediction processing module 120 is used to obtain the first feature information of the stored object, and the first feature information includes the call feature information and scene feature information of the stored object, so as to predict the call situation of the stored object based on the first feature information and the preset prediction strategy, obtain the first call prediction value corresponding to the stored object, and determine the target storage space corresponding to the first call prediction value based on the first correspondence between the preset call prediction value and the data storage space. The scheduling processing module 130 is used to schedule the stored object to the target storage space when the target storage space is inconsistent with the first storage space where the stored object is located.

[0019] In this embodiment, prediction processing module 120 is further configured to input call feature information into a pre-trained first prediction model to perform call status prediction processing, outputting a first prediction value corresponding to the stored object; and input scene feature information into a pre-trained second prediction model to perform call status prediction processing, outputting a second prediction value corresponding to the stored object. The first prediction value and the second prediction value are used to represent the call status of the stored object. Thus, a first call prediction value corresponding to the stored object is determined based on the first prediction value and the second prediction value.

[0020] In this embodiment, the prediction processing module 120 is also used to obtain the first sample feature information of the sample object and the sample call value corresponding to the sample object. The first sample feature information includes the first sample call feature information and the first sample scene feature information of the sample object. Thus, the first sample call feature information is input into the first prediction model to be trained for call situation prediction processing, and the first sample prediction value corresponding to the sample object is output; and the first sample scene feature information is input into the second prediction model to be trained for call situation prediction processing, and the second sample prediction value corresponding to the sample object is output. Furthermore, based on the first sample prediction value and the second sample prediction value, the first sample call prediction value corresponding to the sample object is determined, and the first sample call prediction value and the sample call value are input into the reinforcement learning model to be trained for iterative training until the reinforcement learning model meets the iteration termination condition, thereby obtaining the trained first prediction model and the trained second prediction model.

[0021] In this embodiment, the prediction processing module 120 is further configured to perform simulated scheduling processing on the sample object based on the first sample call prediction value using the reinforcement learning model to be trained. The simulated scheduling processing is used to simulate scheduling the sample object to the data storage space corresponding to the first sample call prediction value. Thus, after the simulated scheduling processing, second sample feature information of the sample object is obtained. The second sample feature information includes second sample call feature information and second sample scenario feature information of the sample object. Furthermore, the second sample call feature information is input into the first prediction model to be trained to perform call situation prediction processing, and a third sample prediction value corresponding to the sample object is output; and the second sample scenario feature information is input into the second prediction model to be trained to perform call situation prediction processing, and a fourth sample prediction value corresponding to the sample object is output. Further, based on the third sample prediction value and the fourth sample prediction value, the second sample call prediction value corresponding to the sample object is determined, and based on the second sample call prediction value and the sample call value, the value of the simulated scheduling processing is evaluated to obtain a value evaluation result corresponding to the simulated scheduling processing, and based on the value evaluation result, the model parameters of the first prediction model to be trained, the second prediction model to be trained and the reinforcement learning model to be trained are updated until the reinforcement learning model to be trained meets the iteration termination condition, and the trained first prediction model and the trained second prediction model are obtained.

[0022] In this embodiment, the target data storage module 110 is also used to determine the first data heat information of the data generated in real time by the target platform before obtaining the first feature information of the stored object in the target data storage module. The target platform is connected to the target data storage module, and the first data heat information is determined based on the generation time of the data. Thus, according to the second corresponding relationship between the preset data heat information and the data storage space, the second storage space corresponding to the first data heat information is determined, the data is scheduled to the second storage space for storage processing, and after the storage processing, the metadata information corresponding to the data is obtained. The metadata information includes at least one of the data identification information, generation time, data size, data heat information, and the data storage space in which it is located. Then, the metadata information is stored in the metadata storage space.

[0023] In this embodiment, metadata information includes the data generation time. The target data storage module 110 is further configured to retrieve metadata information for each piece of data from the metadata storage space. This module then performs time window partitioning on all data based on the generation time of each piece of data and a preset time window partitioning strategy, resulting in multiple time windows, each containing at least one piece of data. Furthermore, for each time window, first feature information of a stored object is determined based on target feature information for each piece of data within the time window. The target feature information includes call feature information and scene feature information for the data. The stored object includes all data within the time window.

[0024] In this embodiment, the scheduling processing module 130 is further configured to, if the target storage space is inconsistent with the first storage space where the stored object is located, schedule the stored object to the target storage space if the stored object satisfies a preset scheduling condition. The preset scheduling condition includes: a difference between a first call prediction value corresponding to the stored object and a second call prediction value that triggered the previous scheduling process is greater than a first preset threshold; and / or a time interval between the current scheduling process and the previous scheduling process is greater than a first time threshold.

[0025] The following describes in detail the operations performed by the object storage management system during the object storage management process. Figure 2 An object storage management method provided by one embodiment of the present application is shown. The method can be executed by an electronic device, which may include: a server and / or a terminal device, where the terminal device may be, for example, a vehicle-mounted terminal or a mobile phone terminal. In other words, the method can be executed by software or hardware installed in the electronic device, and the method includes the following steps: Step 202: Acquire first characteristic information of a stored object in a target data storage module.

[0026] The target data storage module includes a plurality of data storage spaces. The first feature information includes call feature information and scene feature information of the stored object.

[0027] It is understood that the object storage management method provided in the embodiments of this application can be applied to scenarios requiring data storage, such as video surveillance, recording, and data backup. For ease of explanation, the following embodiments will use the video surveillance scenario as an example to describe the various embodiments of this application in detail.

[0028] In video surveillance scenarios, the stored object can be video surveillance data. Call feature information can include the data storage space the time window currently resides in, the creation time of the time window (i.e., the start time of the first video file within the time window), the time of the most recent call to the time window, the call frequency of the time window, the number of times the time window has been scheduled, and so on. Scene feature information can include whether the time window records footage during the day or at night, whether the time window falls on a weekday or a weekend, whether the time window records footage indoors or outdoors, whether the time window records footage statically or dynamically, and whether the time window records footage containing people. A time window can contain multiple video files, and the time window is the smallest scheduling unit in this application scenario. The following embodiments will detail how to divide the time window, and this will not be discussed here. Scene feature information primarily includes recorded video content information. It is understood that existing video surveillance equipment has the ability to tag this information when generating video files. That is, video files can carry this information without the need for parsing the video files to obtain it.

[0029] Optionally, data storage spaces may include metadata storage spaces (e.g., metadata buckets), warm data storage spaces (e.g., warm storage buckets), cold data storage spaces (e.g., cold storage buckets), and hot data storage spaces (e.g., hot storage buckets). The metadata storage spaces may be associated with the warm data storage spaces, cold data storage spaces, and hot data storage spaces, respectively. The warm data storage spaces, cold data storage spaces, and hot data storage spaces are used to store stored objects. After storage, metadata information for the stored objects is stored in the metadata storage spaces. Therefore, stored objects stored in the warm data storage spaces, cold data storage spaces, or hot data storage spaces can be accessed based on the metadata information in the metadata storage spaces.

[0030] Step 204 : predicting the call status of the stored object based on the first feature information and a preset prediction strategy, and obtaining a first call prediction value corresponding to the stored object.

[0031] The first call prediction value is used to characterize the call status of the stored object.

[0032] Step 206 : Determine a target storage space corresponding to the first call prediction value according to a preset first correspondence between the call prediction value and the data storage space.

[0033] Step 208 : If the target storage space is inconsistent with the first storage space where the stored object is located, schedule the stored object to the target storage space.

[0034] Optionally, the scheduling frequency may be fixed once a day, and the time period for executing the scheduling may be selected from a time period that does not affect the business, such as the nighttime off-peak hours.

[0035] In an embodiment of the present application, by obtaining the first characteristic information of the stored object in the target data storage module, the target data storage module includes multiple data storage spaces, and the first characteristic information includes the call characteristic information and scene characteristic information of the stored object, so that the call situation of the stored object is predicted based on the first characteristic information and the preset prediction strategy, and the first call prediction value corresponding to the stored object can be obtained, thereby achieving the effect of predicting the call possibility of the stored object. Furthermore, based on the first corresponding relationship between the preset call prediction value and the data storage space, the target storage space corresponding to the first call prediction value is determined, and when the target storage space is inconsistent with the first storage space where the stored object is located, the stored object is scheduled to the target storage space. It can be seen that this technical solution can schedule the stored objects according to the call possibility of the stored objects, and achieves the effect of clustering the stored objects according to the call possibility. This is conducive to centralized management of stored objects that are easy to be called, thereby facilitating better management of stored objects.

[0036] In one implementation, before obtaining the first feature information of the stored object in the target data storage module (i.e., step 102), the following steps A1 to A5 can be performed to store the data generated in real time by the target platform, and obtain and store metadata information corresponding to each data.

[0037] Step A1: determining first data heat information of the data generated in real time by the target platform.

[0038] The target platform is connected to the target data storage module. The first data heat information can be determined based on the generation time of the data. The first data heat information can be hot data, warm data, or cold data.

[0039] In the case of video surveillance, the target platform is a video surveillance platform, and the data generated by the target platform in real time is video surveillance data. In this embodiment, the video surveillance data generated by the video surveillance platform in real time can be divided into hot data, warm data, and cold data according to the generation time of the video surveillance data.

[0040] For example, according to the generation time of the video surveillance data, video surveillance data generated within 7 days can be determined as hot data, video surveillance data generated within 8 to 30 days can be determined as warm data, and video surveillance data generated more than 30 days can be determined as cold data.

[0041] Step A2: determining a second storage space corresponding to the first data heat information according to a preset second correspondence between the data heat information and the data storage space.

[0042] Optionally, online storage can be selected for warm data, and offline archival storage can be selected for cold data. Accordingly, hot data can be stored in hot data storage space, warm data can be stored in warm data storage space, and cold data can be stored in cold data storage space.

[0043] In practice, there are typically two data centers: the Eastern Data Center and the Western Data Center. The Eastern Data Center stores data generated in the Eastern region, while the Western Data Center stores data generated in the Western region. Because the Western Data Center benefits from green energy and lower temperatures, storage costs are significantly reduced. Online storage costs can be two-thirds of those in the Eastern Data Center, and offline archival storage costs can be one-quarter. Therefore, to reduce data storage costs, for data generated in the Eastern region, hot data can be stored in a hot bucket in the Eastern Data Center, warm data in a warm bucket in the Western Data Center, and cold data in a cold bucket in the Western Data Center. Hot data in the hot bucket can be directly accessed without compression, and the disks are always powered on. Warm data in the warm bucket can be directly accessed without compression, and the disks are always powered on. Cold data in the cold bucket needs to be thawed before access, which takes a certain amount of time (e.g., 10 minutes). Data is losslessly compressed, and the disks are powered off when not being accessed. This allows for East-West data storage through the target data storage module, eliminating the need for target platform modifications and effectively reducing data storage costs.

[0044] Step A3: dispatch the data to the second storage space for storage processing.

[0045] In this step, hot data may be dispatched to hot data storage space for storage processing, warm data may be dispatched to warm data storage space for storage processing, and cold data may be dispatched to cold data storage space for storage processing.

[0046] Step A4: After the storage process, obtain metadata information corresponding to the data.

[0047] The metadata information may include at least one of data identification information, generation time, data size, data popularity information, and the data storage space where the data is located.

[0048] When applied to a video surveillance scenario, the data identification information may be the file name of the video file, the generation time may be the file start time of the video file, and the data size may be the file length of the video file.

[0049] Step A5: storing the metadata information in the metadata storage space.

[0050] For example, if the video surveillance data generated in real time by the video surveillance platform in the eastern region is stored in a cold storage bucket located in the western data center, the warm data is stored in a warm storage bucket located in the western data center, and the hot data is stored in a hot storage bucket located in the eastern data center, and the metadata information includes the file name, file start time, file length, data heat information and data storage space of the video file, and the metadata information is stored in the metadata bucket, then, when the current time is 2024-8-3 14:00:00, the metadata information of the video files named Aaaaa.mp4, Bbbbb.mp4, Ccccc.mp4 and Ddddd.mp4 in the metadata bucket can be as shown in Table 1.

[0051] Table 1

[0052] It is understandable that the metadata information stored in Table 1 can be arranged in ascending order according to the file start time, and in addition to the four files listed above, it also includes multiple video files not shown. In this embodiment, the data transmission direction between the video monitoring platform 310, metadata bucket 320, hot storage bucket 330, warm storage bucket 340, and cold storage bucket 350 is as follows: Figure 3 As shown, the video surveillance platform 310 is only connected to the metadata bucket 320, and the metadata bucket 320 is associated with the hot storage bucket 330 located in the eastern data center, and the warm storage bucket 340 and the cold storage bucket 350 located in the western data center.

[0053] In this embodiment, for data generated in real time by the target platform, by determining the first data heat information of the data, the second storage space corresponding to the first data heat information is determined based on the second correspondence between the preset data heat information and the data storage space, and then the data is dispatched to the second storage space for storage processing. After the storage processing, the metadata information corresponding to the data can be obtained, and the metadata information is stored in the metadata storage space, achieving the effect of storing the data in each data storage space according to the data heat. In addition, in this technical solution, the target platform is only connected to the metadata storage space, and the metadata storage space is connected to the hot data storage space, warm data storage space, and cold data storage space in various places. In this way, the storage status of each data can be clarified through the metadata information, which is conducive to the orderly management of the data generated by the target platform, and is conducive to the user's rapid call of hot data, warm data, and cold data stored in various places based on the index information (i.e., metadata information) in the metadata storage space.

[0054] In one implementation, the metadata information includes the generation time of the data. Through the following steps B1 to B3, all stored data can be divided into time windows to obtain multiple stored objects, thereby determining the first feature information of each stored object.

[0055] Step B1: Obtain metadata information of each piece of data from the metadata storage space.

[0056] The metadata information may include data identification information, generation time, data size, data popularity information and the data storage space.

[0057] Step B2: divide all data into time windows according to the generation time of each data item and the preset time window division strategy to obtain multiple time windows. Each time window includes at least one data item.

[0058] The preset time window division strategy may be to divide all data into time windows according to a preset duration, for example, dividing a day into multiple time windows with each window lasting 10 minutes.

[0059] In the case of video surveillance, video files are stored as x minutes (or seconds) per file, and the time window must be an integer multiple of the video file length, that is, a time window can contain n complete files. Figure 4 As shown, time window 1 includes file 1, file 2, file 3 and file 4, time window 2 includes file 5, file 6, file 7 and file 8, time window 3 includes file 9, file 10, file 11 and file 12, and so on.

[0060] Step B3: for each time window, determine the first feature information of the stored object according to the target feature information of each piece of data in the time window.

[0061] The target feature information may include the call feature information and scene feature information of the data. The stored object includes all data within a time window. The first feature information of the stored object may include the call feature information and scene feature information of all data within the time window.

[0062] In this embodiment, by obtaining metadata information of each piece of data, all data are divided into time windows according to the generation time of each piece of data to obtain multiple time windows, and then the first feature information of the stored object is determined according to the calling feature information and scene feature information of each piece of data in the time window, which is conducive to scheduling or calling the stored data according to the time window.

[0063] In one implementation, based on the first feature information and a preset prediction strategy, predicting the call status of the stored object and obtaining a first call prediction value corresponding to the stored object (i.e., step 204) can be performed as follows: Steps C1 to C2: In step C1, the call feature information is input into a pre-trained first prediction model for call situation prediction processing, and a first prediction value corresponding to the stored object is output; and the scene feature information is input into a pre-trained second prediction model for call situation prediction processing, and a second prediction value corresponding to the stored object is output.

[0064] The first prediction value and the second prediction value are used to characterize the call status of the stored object. Optionally, the first prediction model can be a recurrent neural network (RNN), such as an LSTM (Long Short-Term Memory). The second prediction model can be a classification / regression model, such as a GBDT (GradientBoosting Decision Tree).

[0065] The following uses LSTM as an example to illustrate how the first prediction model processes the call feature information to predict the call status and outputs the first prediction value corresponding to the stored object. The expression of the first prediction model is shown in formula (1): . (1) in, Is the time window t (i.e. stored objects t )’s call feature information, is in the time window t The predicted value of the call situation (i.e. the first predicted value), The previous time window t A hidden state of -1.

[0066] The core of LSTM lies in three gating mechanisms: the forget gate, the input gate, and the output gate. Each gate controls the flow of information using a specific mathematical formula. The following are the mathematical formulas for each gating unit: The forget gate is used to control which information in the memory unit needs to be retained or forgotten. Its calculation process is shown in formula (2): . (2) in, is the output of the forget gate, is the weight matrix of the forget gate, is the bias vector of the forget gate, It is the sigmoid activation function, which limits the value to between 0 and 1.

[0067] The input gate is used to determine the extent to which the current input information is written into the memory unit. The calculation process consists of two parts: First, as shown in formula (3), use sigmoid The function generates the write ratio; secondly, as shown in formula (4), a new candidate value is generated by tanh Function to activate.

[0068] . (3) . (4) in, is the value of the input gate, is the candidate memory cell value.

[0069] Then, as shown in formula (5), by combining the results of the forget gate and the input gate, the updated memory cell state is obtained .

[0070] . (5) The output gate is used to determine the output information of the current time step. Its calculation process is shown in formula (6): . (6) Afterwards, as shown in formula (7), the memory cell state is passed tanh The activation function is processed and then multiplied by the value of the output gate to obtain the final hidden state output .

[0071] . (7) The following uses the GBDT algorithm as an example to illustrate how the second prediction model performs call prediction processing on scene feature information and outputs the second prediction value corresponding to the stored object. The expression of the second prediction model is shown in formula (8): . (8) The GBDT algorithm can obtain the final prediction result by superimposing the prediction results of multiple decision trees. The calculation process is as follows: First, the algorithm is initialized as shown in formula (9).

[0072] . (9) Secondly, as shown in formula (10), M classification and regression trees are established to classify the time window t =1, 2, ..., N, calculate the negative gradient of the loss function corresponding to the mth tree: . (10) Then, fitting the data to get the leaf node area of the mth tree is , for k = 1, 2, ..., K, the best fitting value is calculated by formula (11): . (11) Then, update by formula (12) : . (12) Finally, the final classification regression tree is obtained by formula (13): . (13) in, Is the time window t (i.e. stored objects t ) scene feature information, is in the time window t The predicted value of the call situation (i.e. the second predicted value), are model parameters, is the loss function, is a constant.

[0073] Step C2: Determine a first call prediction value corresponding to the stored object according to the first prediction value and the second prediction value.

[0074] Optionally, the first call prediction value corresponding to the stored object can be calculated by the following formula (14): : . (14) The first predicted value and the second predicted value can be substituted into the , and These two parameters need to be continuously updated in the iterative process of the reinforcement learning model described below until the iterative termination condition is met, and then the training is stopped to obtain the updated parameters.

[0075] In this embodiment, it is clarified how to predict the call status of the stored object based on the first feature information and the corresponding prediction model, thereby obtaining the first call prediction value corresponding to the stored object, which is conducive to accurately implementing the call status prediction processing of the stored object.

[0076] In practice, call behavior is divided into two categories: the first is sudden calls; the second is periodic, regular calls, such as regular inspections. For the second type of call behavior, steps C1 and C2 can be used to predict the first call prediction value for stored objects. This facilitates scheduling of stored objects based on this first call prediction value, ensuring that when a stored object is called, it is likely to be in a hot or warm storage bucket. This allows direct calls to stored objects without waiting for them to be thawed, thus avoiding long waits for users.

[0077] The first type of call behavior cannot be accurately predicted. If the stored object being called is in a cold storage bucket, it must be thawed in real time before being presented to the user. Optionally, when thawing the first piece of data in a stored object in real time, subsequent data can be thawed in batches to ensure that the user only has to wait once. It should be noted that the thawing process can be implemented by a thawing processing module in the object storage management system, which is connected to the target data storage module.

[0078] In one implementation, sample data can be used in advance to train the first prediction model and the second prediction model that meet the requirements of the embodiment of the present application. After the object storage management method provided in the embodiment of the present application is put into use, a period of time can be set as a testing phase. The data generated during the testing phase is the model training data. The model is continuously optimized through these data to obtain an object storage management method suitable for the usage scenario.

[0079] In this embodiment, the sample data is used to train the first prediction model and the second prediction model through the following steps D1 to D4. Figure 5 As shown, the training process of the first prediction model and the second prediction model is shown, including: Step D1: Acquire first sample feature information of a sample object and a sample call value corresponding to the sample object.

[0080] The first sample feature information includes first sample call feature information and first sample scene feature information of the sample object. The sample object is selected from the stored objects in the target data storage module.

[0081] In step D2, the first sample call feature information is input into the first prediction model to be trained for call situation prediction processing, and the first sample prediction value corresponding to the sample object is output; and the first sample scene feature information is input into the second prediction model to be trained for call situation prediction processing, and the second sample prediction value corresponding to the sample object is output.

[0082] The first sample prediction value and the second sample prediction value are used to characterize the call status of the sample object. The specific process of the first prediction model performing call status prediction processing on the first sample call feature information to obtain the first sample prediction value corresponding to the sample object, and the specific process of the second prediction model performing call status prediction processing on the first sample scene feature information to obtain the second sample prediction value corresponding to the sample object, can be referred to above step C1 and will not be repeated here.

[0083] Step D3: Determine the first sample call prediction value corresponding to the sample object according to the first sample prediction value and the second sample prediction value.

[0084] The specific implementation process of this step can be found in the above step C2 and will not be repeated here.

[0085] In step D4, the first sample call prediction value and the sample call value are input into the reinforcement learning model to be trained for iterative training until the reinforcement learning model meets the iteration termination condition, thereby obtaining the trained first prediction model and the trained second prediction model.

[0086] Optionally, the reinforcement learning model can use the Actor-Critic algorithm. The Actor model learns the scheduling strategy. , used to determine the probability distribution of the strategy taken in each state S, and optimized by the cross entropy weighted by the TD error. are the parameters of the Actor network, is the TD error. By modifying the objective function, the objective function gradient of the Actor model is obtained as shown in formula (15): . (15) The Critic model learns the state value , optimized by TD error. S is the scheduling module in the scheduling strategy The state reached after L iterations (simulated scheduling) is is the parameter of the Critic network. The objective function corresponding to the Critic model is shown in formula (16): . (16) In this embodiment, by training the first prediction model and the second prediction model, a model basis is provided for quickly and accurately predicting the call status of the stored object based on the first feature information thereof.

[0087] In one implementation, the first sample call prediction value and the sample call value are input into the reinforcement learning model to be trained for iterative training until the reinforcement learning model satisfies the iteration termination condition, thereby obtaining a trained first prediction model and a trained second prediction model (i.e., step D4). The following steps D41 to D46 can be executed: Step D41 , using the reinforcement learning model to be trained, simulates scheduling processing on the sample object according to the first sample call prediction value.

[0088] Optionally, the Actor model can output a corresponding simulated scheduling strategy based on the first sample call prediction value, thereby sending the simulated scheduling strategy to the scheduling processing module, so that the scheduling processing module performs simulated scheduling processing on the sample object. The simulated scheduling processing is used to simulate scheduling the sample object to the data storage space corresponding to the first sample call prediction value. The simulated scheduling strategy may include a scheduling start position and a scheduling target position.

[0089] Step D42: After the simulation scheduling process, obtain the second sample feature information of the sample object.

[0090] The second sample feature information includes the second sample calling feature information and the second sample scene feature information of the sample object.

[0091] In step D43, the second sample call feature information is input into the first prediction model to be trained for call situation prediction processing, and the third sample prediction value corresponding to the sample object is output; and the second sample scene feature information is input into the second prediction model to be trained for call situation prediction processing, and the fourth sample prediction value corresponding to the sample object is output.

[0092] The third sample prediction value and the fourth sample prediction value are used to characterize the call status of the sample object. The specific process of the first prediction model performing call status prediction processing on the second sample call feature information to obtain the third sample prediction value corresponding to the sample object, and the specific process of the second prediction model performing call status prediction processing on the second sample scene feature information to obtain the fourth sample prediction value corresponding to the sample object, can be referred to above step C1 and will not be repeated here.

[0093] Step D44: Determine the second sample call prediction value corresponding to the sample object based on the third sample prediction value and the fourth sample prediction value.

[0094] The specific implementation process of this step can be found in the above step C2 and will not be repeated here.

[0095] Step D45, evaluating the value of the simulated scheduling process based on the second sample call prediction value and the sample call value, and obtaining a value evaluation result corresponding to the simulated scheduling process.

[0096] Optionally, the value of the simulated scheduling process can be evaluated by a Critic model based on the error value between the second sample call prediction value and the sample call value to obtain a value evaluation result corresponding to the simulated scheduling process.

[0097] Step D46: Update the model parameters of the first prediction model to be trained, the second prediction model to be trained, and the reinforcement learning model to be trained according to the value evaluation result, until the reinforcement learning model to be trained meets the iteration termination condition, and obtain the trained first prediction model and the trained second prediction model.

[0098] The iteration termination condition may include the Critic model reaching a preset number of iterations and / or the loss function converging.

[0099] In this embodiment, a reinforcement learning model is adopted, and the error value between the call prediction value and the actual call value is used as an excitation signal to continuously optimize the model parameters of the first prediction model to be trained, the second prediction model to be trained, and the reinforcement learning model to be trained, and finally the trained first prediction model and the trained second prediction model are obtained, thereby ensuring the accuracy of the prediction model and helping to improve the accuracy of the model prediction results.

[0100] In one implementation, the preset first correspondence between call prediction values and data storage spaces may include a correspondence between each data storage space and a corresponding call prediction value interval. Thus, determining the target storage space corresponding to the first call prediction value (i.e., step 206) based on the preset first correspondence between the call prediction values and the data storage spaces may be performed by determining the target call prediction value interval to which the first call prediction value belongs, and determining the data storage space corresponding to the target call prediction value interval as the target storage space.

[0101] Exemplarily, when the data storage space includes a hot storage bucket, a warm storage bucket, and a cold storage bucket, it can be preset that when the call prediction value is greater than a first call threshold (such as 70%), the corresponding data storage space is a hot storage bucket; when the preset call prediction value is less than the first call threshold and greater than a second call threshold (such as 30%), the corresponding data storage space is a warm storage bucket; when the preset call prediction value is less than the second call threshold, the corresponding data storage space is a cold storage bucket.

[0102] In one implementation, when the target storage space is inconsistent with the first storage space where the stored object is located, scheduling the stored object to the target storage space (i.e., step 208) can be performed as follows: when the target storage space is inconsistent with the first storage space where the stored object is located, if the stored object meets the preset scheduling conditions, scheduling the stored object to the target storage space.

[0103] Among them, the preset scheduling conditions include: the difference between the first call prediction value corresponding to the stored object and the second call prediction value that triggered the last scheduling process is greater than the first preset threshold; and / or, the time interval between the current scheduling process and the last scheduling process is greater than the first time threshold.

[0104] In this embodiment, by setting the sensitivity for triggering scheduling (i.e., the first preset threshold), it is possible to prevent repeated scheduling caused by the call prediction value fluctuating around the critical point. Specifically, scheduling is triggered only when the difference between the call prediction values for two triggering schedules is greater than the sensitivity. By setting a threshold for the time interval between two schedules, it is possible to prevent repeated scheduling within a time window within a short period of time.

[0105] In implementation, when the target storage space is inconsistent with the first storage space where the stored object is located, there are three scheduling modes for scheduling the stored object based on the difference between the target storage space and the first storage space: Scheduling mode 1: the stored objects are scheduled from the hot bucket to the warm bucket, e.g. Figure 6 shown.

[0106] In this scheduling mode, the target storage space is a warm storage bucket, and the first storage space is a hot storage bucket. Each file in the stored object needs to be migrated from the hot storage bucket to the warm storage bucket. Afterwards, the metadata information in the metadata storage space needs to be updated according to the data storage space where the stored object is located after scheduling.

[0107] Scheduling mode 2: stored objects are dispatched from hot or warm buckets to cold buckets, such as Figure 7 shown.

[0108] In this scheduling mode, the target storage space is a cold storage bucket, and the first storage space is a hot storage bucket or a warm storage bucket. Each file in the stored object needs to be migrated from the hot storage bucket or the warm storage bucket to the cold storage bucket. Afterwards, the metadata information in the metadata storage space needs to be updated according to the data storage space where the stored object is located after scheduling.

[0109] Scheduling mode 3: the stored objects are dispatched from the cold storage bucket to the warm storage bucket after being thawed. Figure 8 shown.

[0110] In this scheduling mode, the target storage space is a warm storage bucket, and the first storage space is a cold storage bucket. First, the files in the stored objects need to be thawed, and then the files in the thawed stored objects are migrated from the cold storage bucket to the warm storage bucket. After that, the metadata information in the metadata storage space needs to be updated according to the data storage space where the stored objects are located after scheduling.

[0111] In recent years, with the development of 5G and gigabit optical networks, as well as the widespread application of AI in video detection, video surveillance projects across various industries have increased significantly. Storage accounts for the largest portion of the cost of video surveillance projects, exceeding one-third. With the advancement of high-definition video, this proportion is expected to increase. However, video storage utilization is very low. According to statistics, less than 10% of video surveillance data written to storage devices is retrieved for playback. The majority of data is never played back after being written and is overwritten upon expiration, simply to meet regulatory requirements. Therefore, reducing storage costs for video surveillance projects is a key area for cost reduction and efficiency improvement.

[0112] Currently, methods exist to reduce video surveillance storage costs. However, existing video surveillance storage methods have the following drawbacks: Building storage in a local data center requires a large, one-time investment; the construction cycle is long, typically taking three months for procurement, construction, and commissioning; high-level operational and maintenance skills are required, requiring a deep understanding of platforms, networks, and storage; and fault recovery takes a long time, with rebuilding an 8TB hard drive taking up to 10 hours. While using a public cloud reduces the one-time investment and construction cycle, and eases operational and maintenance challenges, standard object storage fees in the public cloud are double the cost of self-built storage. Cold storage, however, lacks automatic defrost and cannot be directly integrated with the video surveillance platform.

[0113] In this regard, the present application embodiment proposes a new solution, such as Figure 9 As shown, in this technical solution, the video surveillance platform connects only to metadata buckets, which are then used to call hot, warm, and cold storage buckets in various locations. Video files are divided into time windows, and machine learning and reinforcement learning algorithms are used to predict the call prediction value for each time window within a specified time period. This determines the target storage bucket for the video file and performs data scheduling. This technical solution addresses the pain points of existing video surveillance storage construction: self-built storage requires large capital investment, a long construction cycle, high O&M skills requirements, and long fault recovery times; standard object storage rates in the cloud are twice as high as self-built storage, and using cold storage does not allow for direct integration with video surveillance.

[0114] Figure 9 The object storage management method may include the following steps: Step 901: Determine first data heat information of data generated in real time by a video surveillance platform.

[0115] The video surveillance platform is connected to a target data storage module. The target data storage module includes a plurality of data storage spaces. The first data heat information is determined based on the generation time of the data.

[0116] Step 902 : Determine a second storage space corresponding to the first data heat information according to a preset second correspondence between data heat information and data storage space.

[0117] The data storage space may include warm storage buckets and cold storage buckets located in the western data center, and hot storage buckets located in the eastern data center.

[0118] Step 903: dispatch the data to the second storage space for storage processing.

[0119] Step 904: After storage processing, obtain metadata information corresponding to the data.

[0120] The metadata information may include at least one of data identification information, generation time, data size, data popularity information, and the data storage space where the data is located.

[0121] Step 905: Store the metadata information into the metadata storage space.

[0122] Step 906: Obtain metadata information of each piece of data from the metadata storage space.

[0123] Step 907 : performing time window division processing on all data according to the generation time of each piece of data and a preset time window division strategy to obtain multiple time windows.

[0124] Each time window includes at least one piece of data.

[0125] Step 908 : For each time window, determine the first feature information of the stored object based on the target feature information of each piece of data in the time window.

[0126] The target feature information includes the call feature information and scene feature information of the data. The stored object includes all data within the time window. The first feature information includes the call feature information and scene feature information of the stored object.

[0127] Step 909 : predicting the call status of the stored object based on the first feature information and the preset prediction strategy, and obtaining a first call prediction value corresponding to the stored object.

[0128] Step 910 : Determine a target storage space corresponding to the first call prediction value according to a preset first correspondence between the call prediction value and the data storage space.

[0129] Step 911 : If the target storage space is inconsistent with the first storage space where the stored object is located, the stored object is scheduled to the target storage space.

[0130] Step 912: Update the metadata information in the metadata storage space according to the data storage space where the stored object is located after scheduling.

[0131] The specific process from step 901 to step 912 has been described in detail in the above embodiment and will not be repeated here.

[0132] It should be noted that the object storage management method provided in the embodiments of the present application can be executed by an object storage management device, or a control module within the object storage management device for executing the object storage management method. The object storage management device provided in the embodiments of the present application is described using the object storage management device executing the object storage management method as an example.

[0133] Figure 10 This is a schematic diagram of the structure of an object storage management device provided by an embodiment of the present application. Figure 10 As shown, the object storage management device includes: a first acquisition module 1010 , a first prediction module 1020 , a first determination module 1030 and a first scheduling module 1040 .

[0134] The first acquisition module 1010 is used to obtain the first characteristic information of the stored object in the target data storage module; the target data storage module includes multiple data storage spaces; the first characteristic information includes the call characteristic information and scene characteristic information of the stored object; the first prediction module 1020 is used to predict the call status of the stored object according to the first characteristic information and the preset prediction strategy, and obtain the first call prediction value corresponding to the stored object; the first determination module 1030 is used to determine the target storage space corresponding to the first call prediction value according to the first corresponding relationship between the preset call prediction value and the data storage space; the first scheduling module 1040 is used to schedule the stored object to the target storage space when the target storage space is inconsistent with the first storage space where the stored object is located.

[0135] In one implementation, the first prediction module 1020 includes: a first prediction unit and a first determination unit.

[0136] The first prediction unit is used to input the call feature information into a pre-trained first prediction model for call situation prediction processing, and output a first prediction value corresponding to the stored object; and input the scene feature information into a pre-trained second prediction model for call situation prediction processing, and output a second prediction value corresponding to the stored object; the first prediction value and the second prediction value are used to characterize the call situation of the stored object; the first determination unit is used to determine the first call prediction value corresponding to the stored object based on the first prediction value and the second prediction value.

[0137] In one implementation, the object storage management device further includes: a second acquisition module, a second prediction module, a second determination module, and an iterative training module.

[0138] The second acquisition module is used to obtain the first sample feature information of the sample object and the sample call value corresponding to the sample object; the first sample feature information includes the first sample call feature information and the first sample scene feature information of the sample object; the second prediction module is used to input the first sample call feature information into the first prediction model to be trained for call situation prediction processing, and output the first sample prediction value corresponding to the sample object; and, input the first sample scene feature information into the second prediction model to be trained for call situation prediction processing, and output the second sample prediction value corresponding to the sample object; the second determination module is used to determine the first sample call prediction value corresponding to the sample object based on the first sample prediction value and the second sample prediction value; the iterative training module is used to input the first sample call prediction value and the sample call value into the reinforcement learning model to be trained for iterative training until the reinforcement learning model meets the iteration termination condition, and the trained first prediction model and the trained second prediction model are obtained.

[0139] In one implementation, the iterative training module includes: a simulation scheduling processing unit, an acquisition unit, a second prediction unit, a second determination unit, a value evaluation unit, and a model parameter training unit.

[0140] The simulation scheduling processing unit is used to perform simulation scheduling processing on the sample object according to the first sample call prediction value through the reinforcement learning model to be trained; the simulation scheduling processing is used to simulate the scheduling of the sample object to the data storage space corresponding to the first sample call prediction value; the acquisition unit is used to obtain the second sample feature information of the sample object after the simulation scheduling processing; the second sample feature information includes the second sample call feature information and the second sample scene feature information of the sample object; the second prediction unit is used to input the second sample call feature information into the first prediction model to be trained to perform call situation prediction processing, and output a third sample prediction value corresponding to the sample object; and the second sample scene feature information is input into the second prediction model to be trained The model performs call situation prediction processing in the model and outputs a fourth sample prediction value corresponding to the sample object; the second determination unit is used to determine the second sample call prediction value corresponding to the sample object according to the third sample prediction value and the fourth sample prediction value; the value evaluation unit is used to evaluate the value of the simulation scheduling processing according to the second sample call prediction value and the sample call value, and obtain the value evaluation result corresponding to the simulation scheduling processing; the model parameter training unit is used to update the model parameters of the first prediction model to be trained, the second prediction model to be trained and the reinforcement learning model to be trained according to the value evaluation result, until the reinforcement learning model to be trained meets the iteration termination condition, and the trained first prediction model and the trained second prediction model are obtained.

[0141] In one implementation, the object storage management device further includes: a third determination module, a fourth determination module, a second scheduling module, a third acquisition module, and a storage module.

[0142] The third determination module is used to determine the first data heat information of the data generated in real time by the target platform before obtaining the first feature information of the stored object in the target data storage module; the target platform is connected to the target data storage module; the first data heat information is determined based on the generation time of the data; the fourth determination module is used to determine the second storage space corresponding to the first data heat information according to the second corresponding relationship between the preset data heat information and the data storage space; the second scheduling module is used to schedule the data to the second storage space for storage processing; the third acquisition module is used to obtain metadata information corresponding to the data after storage processing; the metadata information includes at least one of the data identification information, generation time, data size, data heat information, and the data storage space in which it is located; the storage module is used to store the metadata information in the metadata storage space.

[0143] In one implementation, the metadata information includes the generation time of the data; the object storage management device further includes: a fourth acquisition module, a time window division processing module and a fifth determination module.

[0144] The fourth acquisition module is used to obtain metadata information of each data from the metadata storage space; the time window division processing module is used to perform time window division processing on all data according to the generation time of each data and the preset time window division strategy to obtain multiple time windows; each time window includes at least one data; the fifth determination module is used to determine the first feature information of the stored object for each time window according to the target feature information of each data in the time window; the target feature information includes the call feature information and scene feature information of the data; the stored object includes all data in the time window.

[0145] In one implementation, the first scheduling module 1040 includes a scheduling unit.

[0146] A scheduling unit is used to schedule a stored object to a target storage space when the target storage space is inconsistent with a first storage space where a stored object is located, if the stored object meets a preset scheduling condition; the preset scheduling condition includes: the difference between a first call prediction value corresponding to the stored object and a second call prediction value that triggered the last scheduling process is greater than a first preset threshold; and / or, the time interval between the current scheduling process and the last scheduling process is greater than a first time threshold.

[0147] In an embodiment of the present application, by obtaining the first characteristic information of the stored object in the target data storage module, the target data storage module includes multiple data storage spaces, and the first characteristic information includes the call characteristic information and scene characteristic information of the stored object, so that the call situation of the stored object is predicted based on the first characteristic information and the preset prediction strategy, and the first call prediction value corresponding to the stored object can be obtained, thereby achieving the effect of predicting the call possibility of the stored object. Furthermore, based on the first corresponding relationship between the preset call prediction value and the data storage space, the target storage space corresponding to the first call prediction value is determined, and when the target storage space is inconsistent with the first storage space where the stored object is located, the stored object is scheduled to the target storage space. It can be seen that this technical solution can schedule the stored objects according to the call possibility of the stored objects, and achieves the effect of clustering the stored objects according to the call possibility. This is conducive to centralized management of stored objects that are easy to be called, thereby facilitating better management of stored objects.

[0148] The object storage management device in the embodiments of the present application can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, the mobile electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), while the non-mobile electronic device can be a server, network attached storage (NAS), personal computer (PC), television, ATM, or self-service kiosk, etc., without specific limitations in the embodiments of the present application.

[0149] The object storage management device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0150] The object storage management device provided in the embodiment of the present application can achieve Figures 2 to 9 To avoid repetition, the various processes implemented in the method embodiment will not be described again here.

[0151] Based on the same technical concept, an embodiment of the present application further provides an electronic device for executing the above-mentioned object storage management method. Figure 11 The following is a schematic diagram of the structure of an electronic device for implementing various embodiments of the present application. Electronic devices may vary significantly due to different configurations or performance, and may include a processor 1110, a communications interface 1120, a memory 1130, and a communication bus 1140. The processor 1110, the communications interface 1120, and the memory 1130 communicate with each other via the communication bus 1140. The processor 1110 may invoke a computer program stored in the memory 1130 and executable on the processor 1110 to perform the following steps: Obtain first characteristic information of a stored object in a target data storage module; the target data storage module includes multiple data storage spaces; the first characteristic information includes call characteristic information and scene characteristic information of the stored object; based on the first characteristic information and a preset prediction strategy, predict the call situation of the stored object to obtain a first call prediction value corresponding to the stored object; based on a first correspondence between the preset call prediction value and the data storage space, determine the target storage space corresponding to the first call prediction value; when the target storage space is inconsistent with the first storage space where the stored object is located, schedule the stored object to the target storage space.

[0152] In an embodiment of the present application, by obtaining the first characteristic information of the stored object in the target data storage module, the target data storage module includes multiple data storage spaces, and the first characteristic information includes the call characteristic information and scene characteristic information of the stored object, so that the call situation of the stored object is predicted based on the first characteristic information and the preset prediction strategy, and the first call prediction value corresponding to the stored object can be obtained, thereby achieving the effect of predicting the call possibility of the stored object. Furthermore, based on the first corresponding relationship between the preset call prediction value and the data storage space, the target storage space corresponding to the first call prediction value is determined, and when the target storage space is inconsistent with the first storage space where the stored object is located, the stored object is scheduled to the target storage space. It can be seen that this technical solution can schedule the stored objects according to the call possibility of the stored objects, and achieves the effect of clustering the stored objects according to the call possibility. This is conducive to centralized management of stored objects that are easy to be called, thereby facilitating better management of stored objects.

[0153] The specific execution steps can refer to the various steps of the above-mentioned object storage management method embodiment, and can achieve the same technical effect. To avoid repetition, they will not be repeated here.

[0154] It should be noted that the electronic devices in the embodiments of the present application include: servers, terminals, or other devices other than terminals.

[0155] The above electronic device structure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. For example, the input unit may include a graphics processing unit (GPU) and a microphone, and the display unit may be configured as a display panel in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit includes at least one of a touch panel and other input devices. A touch panel is also called a touch screen. Other input devices may include, but are not limited to, a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, and a joystick, which will not be detailed here.

[0156] Memory can be used to store software programs and various data. The memory may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store an operating system, applications or instructions required for at least one function (such as sound playback or image playback), and the like. Furthermore, the memory may include volatile memory or non-volatile memory, or both. Non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct RAM bus random access memory (DRRAM).

[0157] The processor may include one or more processing units; optionally, the processor may integrate an application processor and a modem processor, wherein the application processor primarily handles operations related to the operating system, user interface, and application programs, and the modem processor primarily processes wireless communication signals, such as a baseband processor. It is understood that the modem processor may not be integrated into the processor.

[0158] An embodiment of the present application also provides a computer-readable storage medium, which is used to store computer-executable instructions. When the computer-executable instructions are executed by a processor, the various processes of the above-mentioned object storage management method embodiment are implemented and can achieve the same technical effect. To avoid repetition, they are not repeated here.

[0159] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk.

[0160] An embodiment of the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the various processes of the above-mentioned object storage management method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0161] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned object storage management method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0162] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0163] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0164] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a more preferred embodiment. Based on this understanding, the technical solution of this application, or the part that contributes to the existing technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of this application.

[0165] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. An object storage management method, characterized in that: include: Acquire first feature information of a stored object in a target data storage module; The target data storage module includes a plurality of data storage spaces; the first feature information includes call feature information and scene feature information of the stored object; Predicting, based on the first feature information and a preset prediction strategy, a call status of the stored object to obtain a first call prediction value corresponding to the stored object; Determining a target storage space corresponding to the first call prediction value according to a preset first correspondence between the call prediction value and the data storage space; In a case where the target storage space is inconsistent with the first storage space where the stored object is located, the stored object is scheduled to the target storage space.

2. The method according to claim 1, characterized in that The step of predicting the call status of the stored object based on the first feature information and a preset prediction strategy to obtain a first call prediction value corresponding to the stored object includes: Inputting the call feature information into a pre-trained first prediction model to perform call situation prediction processing, and outputting a first prediction value corresponding to the stored object; and inputting the scene feature information into a pre-trained second prediction model to perform call situation prediction processing, and outputting a second prediction value corresponding to the stored object; the first prediction value and the second prediction value are used to characterize the call situation of the stored object; The first call prediction value corresponding to the stored object is determined according to the first prediction value and the second prediction value.

3. The method according to claim 2, characterized in that The method further comprises: Acquire first sample feature information of a sample object and a sample call value corresponding to the sample object; the first sample feature information includes first sample call feature information and first sample scene feature information of the sample object; Inputting the first sample call feature information into a first prediction model to be trained to perform call situation prediction processing, and outputting a first sample prediction value corresponding to the sample object; and inputting the first sample scene feature information into a second prediction model to be trained to perform call situation prediction processing, and outputting a second sample prediction value corresponding to the sample object; Determining a first sample call prediction value corresponding to the sample object according to the first sample prediction value and the second sample prediction value; The first sample call prediction value and the sample call value are input into the reinforcement learning model to be trained for iterative training until the reinforcement learning model meets the iteration termination condition, thereby obtaining a trained first prediction model and a trained second prediction model.

4. The method according to claim 3, characterized in that The step of inputting the first sample call prediction value and the sample call value into the reinforcement learning model to be trained for iterative training until the reinforcement learning model satisfies an iteration termination condition, thereby obtaining a trained first prediction model and a trained second prediction model, including: Performing a simulated scheduling process on the sample object according to the first sample call prediction value through the reinforcement learning model to be trained; the simulated scheduling process is used to simulate scheduling the sample object to the data storage space corresponding to the first sample call prediction value; After the simulation scheduling process, obtaining second sample feature information of the sample object; the second sample feature information includes second sample call feature information and second sample scene feature information of the sample object; Inputting the second sample call feature information into the first prediction model to be trained to perform call situation prediction processing, and outputting a third sample prediction value corresponding to the sample object; and inputting the second sample scene feature information into the second prediction model to be trained to perform call situation prediction processing, and outputting a fourth sample prediction value corresponding to the sample object; Determining a second sample call prediction value corresponding to the sample object according to the third sample prediction value and the fourth sample prediction value; Evaluate the value of the simulated scheduling process based on the second sample call prediction value and the sample call value, and obtain a value evaluation result corresponding to the simulated scheduling process; According to the value assessment result, the model parameters of the first prediction model to be trained, the second prediction model to be trained, and the reinforcement learning model to be trained are updated until the reinforcement learning model to be trained meets the iteration termination condition, thereby obtaining the trained first prediction model and the trained second prediction model.

5. The method according to claim 1, wherein Before obtaining the first feature information of the object stored in the target data storage module, the method further includes: Determining first data heat information of data generated in real time by a target platform; the target platform is connected to the target data storage module; the first data heat information is determined based on the generation time of the data; determining a second storage space corresponding to the first data heat information according to a preset second correspondence between the data heat information and the data storage space; dispatching the data to the second storage space for storage processing; After storage processing, metadata information corresponding to the data is obtained; the metadata information includes at least one of data identification information, generation time, data size, data popularity information, and data storage space of the data; The metadata information is stored in the metadata storage space.

6. The method according to claim 5, characterized in that The metadata information includes the generation time of the data; the method further includes: Acquire the metadata information of each piece of data from the metadata storage space; According to the generation time of each piece of data and the preset time window division strategy, all data are divided into time windows to obtain multiple time windows; each time window includes at least one piece of data; For each time window, the first feature information of the stored object is determined based on the target feature information of each data in the time window; the target feature information includes the call feature information and scene feature information of the data; the stored object includes all data in the time window.

7. The method according to claim 1, characterized in that When the target storage space is inconsistent with the first storage space where the stored object is located, scheduling the stored object to the target storage space includes: In the case that the target storage space is inconsistent with the first storage space where the stored object is located, if the stored object meets a preset scheduling condition, scheduling the stored object to the target storage space; The preset scheduling conditions include: the difference between the first call prediction value corresponding to the stored object and the second call prediction value that triggered the last scheduling process is greater than a first preset threshold; and / or, the time interval between the current scheduling process and the last scheduling process is greater than the first time threshold.

8. An object storage management system, characterized in that: It includes target data storage module, prediction processing module and scheduling processing module; wherein, The target data storage module is used to obtain and store data generated in real time by the target platform to obtain a stored object; the target platform is connected to the target data storage module; the target data storage module includes a plurality of data storage spaces; The prediction processing module is configured to obtain first feature information of the stored object; the first feature information includes call feature information and scene feature information of the stored object; predict the call status of the stored object based on the first feature information and a preset prediction strategy to obtain a first call prediction value corresponding to the stored object; and determine a target storage space corresponding to the first call prediction value based on a first correspondence between the preset call prediction value and the data storage space; The scheduling processing module is configured to schedule the stored object to the target storage space when the target storage space is inconsistent with the first storage space where the stored object is located.

9. An object storage management device, characterized in that: include: A first acquisition module is used to acquire first feature information of a stored object in a target data storage module; The target data storage module includes a plurality of data storage spaces; the first feature information includes call feature information and scene feature information of the stored object; a first prediction module, configured to predict a call status of the stored object based on the first feature information and a preset prediction strategy, and obtain a first call prediction value corresponding to the stored object; a first determining module, configured to determine a target storage space corresponding to the first call prediction value according to a preset first correspondence between the call prediction value and the data storage space; The first scheduling module is configured to schedule the stored object to the target storage space when the target storage space is inconsistent with the first storage space where the stored object is located.

10. An electronic device, characterized in that: include: processor; as well as A memory arranged to store computer-executable instructions, wherein the computer-executable instructions are configured to be executed by the processor, and the computer-executable instructions are executed by the processor to implement the object storage management method according to any one of claims 1 to 7.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store computer-executable instructions, and when the computer-executable instructions are executed by a processor, the object storage management method according to any one of claims 1 to 7 is implemented.

12. A computer program product, characterized in that The invention comprises a computer program, which implements the object storage management method according to any one of claims 1 to 7 when executed by a processor.

Images