Data level-to-level management method and device based on spatio-temporal information and storage medium

Abstract

The invention discloses a data level-to-level management method and device based on spatio-temporal information and a storage medium. The data level-to-level management method comprises the steps that the value level of initial data in a data storage system is acquired; determining target data in the initial data according to the current space utilization rate and the value level of the data storage system; and in response to triggering of the data deletion task, deleting the target data according to the stored time of the target data. According to the scheme, the effective utilization rate of storage resources can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data management, in particular to a data hierarchical management method and device based on space-time information and a storage medium. BACKGROUND

[0002] Data storage refers to recording data in a storage device in a certain format for subsequent access, processing and use. However, with the development of information technology and the explosive growth of data, more and more data is generated. Therefore, an effective data management method is one of the important technologies to solve the problem of data storage.

[0003] Taking streaming data as an example, current streaming data storage systems mostly use a single-dimensional strategy based on time or capacity for data management. For example, a traditional scheme deletes the earliest written data through a FIFO (First In First Out) algorithm, or deletes the data with the lowest access frequency through an LRU (Least Recently Used) algorithm.

[0004] However, the traditional methods all have obvious defects, such as invalid data redundancy, single coverage strategy, and contradiction between coverage period and storage efficiency. SUMMARY

[0005] The present application provides at least a data hierarchical management method and device based on space-time information and a computer readable storage medium.

[0006] The first aspect of the present application provides a data hierarchical management method based on space-time information, comprising: obtaining a value level of initial data in the data storage system; determining target data in the initial data according to a current space usage rate of the data storage system and the value level; and performing deletion processing on the target data according to a stored time of the target data in response to a data deletion task trigger.

[0007] In an embodiment, the determining of the target data in the initial data according to the current space usage rate of the data storage system and the value level comprises: obtaining a space threshold interval of the data storage system, each space threshold interval corresponding to each value level; determining a target value level corresponding to the current space usage rate according to a space threshold interval in which the current space usage rate is located; and determining initial data at the target value level as the target data.

[0008] In an embodiment, before the deleting the target data according to the stored time of the target data, the method further comprises: judging whether the current space usage is greater than a preset space threshold; if yes, triggering the data deletion task.

[0009] In an embodiment, the deleting the target data according to the stored time of the target data comprises: comparing the stored time of the target data with a time threshold; deleting the target data with the stored time greater than the time threshold.

[0010] In an embodiment, after the deleting the target data with the stored time greater than the time threshold, the method further comprises: updating the current space usage according to the used space and the total space of the data storage system to obtain an updated current space usage; adjusting the time threshold according to the updated current space usage, the current space usage being negatively correlated with the time threshold.

[0011] In an embodiment, the initial data comprises newly stored data and / or historically stored data, and the obtaining the value level of the initial data in the data storage system comprises: in response to obtaining the newly stored data, performing value rating processing on the newly stored data to obtain the value level of the newly stored data; in response to a value updating task being triggered, performing value updating processing on the value level of the historically stored data.

[0012] In an embodiment, the performing value rating processing on the newly stored data to obtain the value level of the newly stored data comprises: analyzing data content of the newly stored data; and performing value rating on the newly stored data according to the data content to obtain the value level.

[0013] In an embodiment, the performing value updating processing on the value level of the historically stored data comprises: analyzing similarity of each historically stored data; and performing updating processing on the value level of each historically stored data according to the similarity.

[0014] The second aspect of the present application provides a data hierarchical management device based on space-time information, comprising: an obtaining module configured to obtain a value level of initial data in a data storage system; a data determining module configured to determine target data in the initial data according to a current space usage of the data storage system and the value level; and a data deleting module configured to, in response to a data deletion task being triggered, perform deletion processing on the target data according to a stored time of the target data.

[0015] The third aspect of the present application provides an electronic device, comprising a memory and a processor, the processor is used to execute program instructions stored in the memory to realize the data hierarchical management method based on space-time information.

[0016] The fourth aspect of the present application provides a computer readable storage medium, which stores program instructions, the program instructions are executed by the processor to realize the data hierarchical management method based on space-time information.

[0017] The above scheme can determine the data value of each initial data by obtaining the value level of the initial data in the data storage system, and low-value data will be deleted preferentially compared with high-value data; the target data in the initial data is determined according to the current space utilization rate and the value level of the data storage system, and the target data to be deleted preferentially can be determined according to the current space utilization rate of the data storage system; if the data deletion task is triggered, the low-value data stored for a long time can be deleted according to the stored time of the target data. Thus, the data management based on the double threshold standards of time information and space information is realized, which not only can improve the effective utilization rate of storage resources, but also can avoid the misdeletion of high-value data.

[0018] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, but not limiting the present application. BRIEF DESCRIPTION OF DRAWINGS

[0019] The drawings incorporated into the specification and form part of the specification, which show the embodiments consistent with the present application, and together with the specification used to illustrate the technical solutions of the present application.

[0020] Figure 1 is a flowchart of an exemplary embodiment of the data hierarchical management method based on space-time information of the present application;

[0021] Figure 2 is an exemplary arrangement flowchart of the data hierarchical management method based on space-time information of the present application;

[0022] Figure 3 is a block diagram of the data hierarchical management device based on space-time information of an exemplary embodiment of the present application;

[0023] Figure 4 is a structural diagram of an embodiment of the electronic device of the present application;

[0024] Figure 5 is a structural diagram of an embodiment of the computer readable storage medium of the present application. DETAILED DESCRIPTION

[0025] The scheme of the embodiments of the present application will be described in detail below with reference to the drawings of the specification.

[0026] In the following description, for purposes of explanation and not limitation, specific details are set forth such as particular architectures, interfaces, techniques, etc. in order to provide a thorough understanding of the application.

[0027] The term "and / or", merely describes association relationship of associated objects, which means that there can be three relationships, for example, A and / or B, which can represent three cases of A alone, A and B together, and B alone. In addition, the character " / " in this paper generally represents that the front and rear associated objects are a "or" relationship. In addition, "multiple" in this paper means two or more than two. In addition, the term "at least one" in this paper means any one of multiple or any combination of at least two of multiple, for example, including at least one of A, B and C, which can mean including any one or more elements selected from the set consisting of A, B and C.

[0028] For the sake of understanding, the application scenarios of the present application are now exemplarily described. The data hierarchical management method of the present application mainly realizes hierarchical management according to the value level of the data stored in the data storage system. These data can be in various forms and formats, which are not limited by the present application. For example, the method of the present application can be used to manage video data, image data, text data and / or audio data, etc. In the subsequent embodiments, the storage scenario of video data (streaming data) is mainly taken as an example for description.

[0029] The method of the present application can solve the problems of high-value data miscovering and redundant data occupying resources caused by the inability of traditional data management methods to dynamically balance data value and storage period, especially in the scenario of limited storage space, the effect of the method of the present application is more obvious.

[0030] Please refer to Figure 1 , Figure 1 is a flowchart of an exemplary embodiment of the data hierarchical management method based on space-time information of the present application. The method can be applied to a data storage system. Specifically, it can include the following steps:

[0031] Step S110, obtaining the value level of the initial data in the data storage system.

[0032] Among them, the initial data can include the data newly written into the data storage system, or the historical storage data already written into the storage in the data storage system, which is not limited here.

[0033] Exemplarily, for the newly stored data, it can include but is not limited to pre-rating processing when its data file is generated, assigning a value level to each newly generated data file, that is, the value level of the initial data can be obtained.

[0034] Similarly, for the historical stored data, the historical stored data is also pre-rated when it is just written into the data storage system, so the historical stored data also has a corresponding value level. In addition, optionally, the application can also set a value updating process, for example, a value level updating of the historical stored data is performed once every preset rating updating period, so that the value level of the historical stored data can be dynamically adjusted according to the actual situation.

[0035] It should be noted that the value level of the data in the application can be obtained according to the data attribute (such as data size, data type) and / or data content and other data related information, which is not limited here. The value level of the application can have various setting modes according to the needs of the actual application scene, which is not limited here. For example, the value level of the application can be set as low value and high value, or can also be set as L1 to LN, a total of N value levels (N is an integer greater than 1), which is not repeated here.

[0036] Step S120, determining the target data in the initial data according to the current space usage rate and the value level of the data storage system.

[0037] Among them, the current space usage rate refers to the space usage rate of the current storage space of the data storage system. Therefore, the current space usage rate obtained at different times can be the same or different, which is not limited here. The target data refers to the to-be-deleted data selected from the initial data.

[0038] Exemplarily, when the storage space of the data storage system is insufficient, the problem of space utilization needs to be considered. Therefore, the method of the application will preferentially determine the data with lower value as the target data to be deleted. When the storage space of the data storage system is sufficient, the problem of space utilization is usually not considered, so the low-value data or part of the low-value data can be retained. Therefore, a certain corresponding relationship between the space usage rate and the value level can be preset in the application, and the data with lower value (i.e. the target data) currently recommended to be deleted can be determined according to the current space usage rate and the value level.

[0039] Step S130, in response to the data deletion task trigger, performing deletion processing on the target data according to the stored time of the target data.

[0040] In combination with the foregoing, the trigger condition of the data deletion task can be one or more, which can be set as needed according to the actual application scenario, and is not limited herein. For example, the data deletion task can be triggered after the target data is determined, or the data deletion task can be triggered after receiving a data deletion instruction issued by a user (or other electronic device), or the data deletion task can be triggered according to the space usage rate of the data storage system, or the data deletion task can be triggered according to a preset data deletion period, and the like.

[0041] Specifically, after the data deletion task is triggered, the target data can be deleted. Alternatively, the target data can be further selected and then deleted.

[0042] For example, the target data can be deleted according to the stored time of the target data, which can include but is not limited to: the target data is deleted in sequence according to the stored time of the target data, so that data with a relatively long stored time in the target data can be preferentially deleted, and data with a relatively short stored time can be deleted later. Alternatively, the target data to be deleted in this data deletion task can also be determined from the target data according to a preset time threshold T and the stored time of the target data (for example, the target data with a stored time greater than the time threshold is deleted, and the target data with a stored time greater than the time threshold is deleted), which can be deleted in sequence or not in sequence as in the foregoing example, and is not limited herein.

[0043] It should be further noted that the data management method of the present application can be not only deleting the data, but also covering the data, and the like, and is not limited herein. The difference between the deletion processing and the covering processing mainly lies in the data retention state. For example, the deletion processing usually refers to deleting the data index (marking the data as deleted or coverable), but not actually erasing the data on the storage medium, and the data can be recovered through certain data recovery technology; the covering processing usually refers to directly writing new data to the original data storage location, replacing the original data content. If the new data completely occupies the storage space of the original data, the original data cannot be recovered, and the covering processing is more common in the processing scene of streaming data, and is not described in detail. Therefore, for the process of implementing the deletion processing mentioned in the present application, other data management methods can also be replaced as needed, and is not described in detail herein.

[0044] For example, for a video file, the deletion processing can be to delete the stored video recording data from the directory list of the system; and the covering processing is to rewrite the data in the area where the video recording data is stored. Therefore, another embodiment of step S130 can also include: in response to the data covering task trigger, covering the target data according to the stored time of the target data.

[0045] It can be seen that the application can determine the data value of each initial data by obtaining the value level of the initial data in the data storage system, and low-value data is preferentially deleted compared with high-value data; target data in the initial data can be determined according to the current space usage rate and the value level of the data storage system, and the target data that needs to be preferentially deleted can be determined through the current space usage rate of the data storage system; if the data deletion task is triggered, the low-value data stored for a long time can be deleted according to the stored time of the target data. Thus, the data management based on the double threshold standards of time information and space information is realized, which not only can improve the effective utilization rate of storage resources, but also can avoid the misdeletion of high-value data.

[0046] On the basis of the above-mentioned embodiments, the application embodiment describes the step of determining the target data in the initial data according to the current space usage rate and the value level of the data storage system. Specifically, the method of the embodiment comprises the following steps:

[0047] obtaining the space threshold interval of the data storage system, each space threshold interval corresponding to each value level; determining the target value level corresponding to the current space usage rate according to the space threshold interval in which the current space usage rate is located; and determining the initial data in the target value level as the target data.

[0048] In combination with the foregoing embodiments, the application can set several value levels as needed, and therefore one or more space threshold intervals corresponding to each value level can be preset in the data storage system, which can be one-to-one correspondence or one-to-many correspondence, which is not limited here. For example, the value level can be divided into L1 to L5, a total of 5 levels, and the space threshold interval can be [0, 0.2), [0.2, 0.4), [0.4, 0.6), [0.6, 0.8), [0.8, 1], a total of 5 intervals, and the five value levels are one-to-one corresponding to the five space threshold intervals.

[0049] By setting the corresponding space threshold interval for each value level, the deletable value level (i.e., the target value level) can be determined according to the space threshold interval in which the current space usage rate of the data storage system is located, that is, the initial data corresponding to the target value level can be determined as the target data.

[0050] Exemplarily, the current space usage rate (or the space threshold interval) can be positively correlated with the target value level. For example, the larger the current space usage rate, the larger the space threshold interval it is located in, that is, the more data that needs to be deleted, and therefore the target value level of the deletable data can be determined, and the initial data lower than or equal to the target value level is determined as the deletable target data.

[0051] Specifically, if the current space usage rate is 0.75, the space threshold interval is [0.6, 0.8), and the corresponding value level is L4, the initial data of L1-L3 or L1-L4 (data with a relatively low value level in the current space usage rate scenario) can be determined as the target data.

[0052] Further, in the subsequent deletion processing of the target data, it can be out-of-order deletion or in-order deletion. The deletion method provided in step S130 can be referred to, and details are not described herein. It should be noted that the out-of-order deletion and in-order deletion in the present application refer to whether the deletion is performed in a specified order according to a preset rule, and the logical order that must be strictly followed in principle when a computer performs a deletion task in some scenarios is not changed or adjusted, and details are not described herein.

[0053] In addition, the value level of the target data can also be considered in the in-order deletion process. That is, in addition to being deleted in order according to the stored time of the target data, the target data can also be deleted according to the value level (for example, data with a relatively low value level in the target data is preferentially deleted), or the stored time and the value level of the target data can be weighted and summed according to a preset weighting coefficient to obtain the deletion priority of the target data, and then the target data is deleted in order according to the deletion priority. The setting method of the weighting coefficient of the stored time and the weighting coefficient of the value level can be set as needed according to the specific application scenario, and details are not described herein. If the storage time of the data needs to be considered, the weighting coefficient of the stored time can be greater than the weighting coefficient of the value level; if the value level of the data needs to be considered, the weighting coefficient of the stored time can be less than the weighting coefficient of the value level, and details are not limited. In this way, the deletion process of the data can be avoided from being affected by a single deletion rule, so that relatively important data is not deleted.

[0054] It should be noted that although the target data is data with a relatively low value level determined from the initial data, even in the target data, it can be further divided into target data with a relatively high value level and target data with a relatively low value level. Especially in the application scenario where the data amount of the target data is large, after deleting part of the target data, the current space usage rate of the data storage system can be significantly affected (data deletion can reduce the current space usage rate), that is, deleting data can actually make the storage space of the data storage system gradually sufficient, and thus the target value level of the data to be deleted also decreases, which means that in some special scenarios, some target data with a relatively high value level can actually not be deleted. Therefore, deleting the target data in order can preferentially delete target data with a relatively low value, and with the deletion of data and the release of storage space, target data with a relatively high value can also be preserved to some extent.

[0055] It should be noted that the method of performing the ordered deletion or unordered deletion can be pre-set or flexibly selected. The method of ordered deletion can be centered on data value, and can avoid deleting data with high value as much as possible during data management. The method of unordered deletion is centered on management efficiency, and can improve the data deletion efficiency compared with the method of ordered deletion. In the flexible selection process, whether to perform the ordered deletion or unordered deletion can be determined according to the data size of the target data. If the data size of the target data is greater than the data size threshold (deleting data will significantly affect the space usage rate), the ordered deletion can be selected. If the data size of the target data is less than or equal to the data size threshold, the unordered deletion can be selected.

[0056] In addition, the space difference can also be obtained by comparing the difference between the obtained current space usage rate and the left boundary of the space threshold interval in which the current space usage rate is located. If the data size of the target data is greater than the data size corresponding to the space difference, it indicates that deleting all the target data will release more storage space, that is, the ordered deletion can be selected until the current space usage rate decreases to a lower level of the space threshold interval, and then the target data with higher value corresponding to the original space threshold interval does not need to be deleted. For example, the initial obtained current space usage rate is 0.75 (located in the space threshold interval [0.6, 0.8), and the target value level corresponding to the space threshold interval is L1 to L4), and the difference between the left boundary is 0.15. If the data size corresponding to the usage rate of 0.15 is 15 GB, and the data size of the target data is 20 GB (of which the data size of L1 to L3 is 16 GB, the data size of L4 is 4 GB, and the data size of L5 is not discussed at this time), then after deleting the target data of L1-L3, the current space usage rate decreases to the space threshold interval [0.4, 0.6), that is, the target data of L4 level does not need to be deleted, thereby retaining the data with relatively high value level in the target data. Similarly, if the data size of the target data is less than or equal to the data size corresponding to the space difference, it indicates that deleting all the target data will not release more storage space, that is, the unordered deletion can be selected.

[0057] On the basis of the above embodiment, the embodiment of the present application describes the steps before the target data is deleted according to the stored time of the target data in response to the triggering of the data deletion task. Specifically, the method of the embodiment includes the following steps:

[0058] It is judged whether the current space usage rate is greater than the preset space threshold. If yes, the data deletion task is triggered.

[0059] With the foregoing embodiments, the method provided by the present application can have multiple trigger conditions for triggering the data hierarchical management task (such as the data deletion task, the data covering task, etc.). The present embodiment mainly takes the comparison result between the current space usage rate and the preset space threshold as the trigger condition for example.

[0060] Specifically, the preset space threshold is used to represent whether the remaining storage space of the data storage system is short. When the current space usage rate is less than or equal to the preset space threshold, it represents that there is more remaining storage space in the data storage system, that is, the data deletion task does not need to be triggered. When the current space usage rate is greater than the preset space threshold, it represents that there is less remaining storage space in the data storage system, and the data deletion task needs to be triggered.

[0061] On the basis of the foregoing embodiments, the present embodiment describes the step of deleting the target data according to the stored time of the target data. Specifically, the method of the present embodiment comprises the following steps:

[0062] Comparing the stored time of the target data with a time threshold; deleting the target data whose stored time is greater than the time threshold.

[0063] With the foregoing embodiments, the method of deleting the target data in the present application can have multiple methods. The present application mainly takes the method of selectively deleting the target data according to the comparison result between the stored time and the time threshold for example.

[0064] The stored time of the present application is mainly used to distinguish the old and new data in the data storage system. The calculation method of the stored time can include but is not limited to being calculated based on the data write time or being calculated based on the time when the data is completed to be written, etc. The specific calculation method can be set as needed according to the actual application scene, which is not limited here.

[0065] For example, the stored time is compared with the time threshold. If the stored time is greater than the time threshold, the old data in the target data can be determined, and such data can be preferentially deleted. Similarly, if the stored time is less than or equal to the time threshold, the new data in the target data can be determined, and such data can be retained.

[0066] In addition, the application can further select data from the target data for deletion according to the stored time, and can further select data from the target data for deletion according to the recent use time. The recent use time refers to the time when the data is recently used (accessed). The recent use time is compared with the time threshold value. If the recent use time is greater than the time threshold value, it can be determined that the target data in the target data is not recently used (equivalent to idle data), and such data can be preferentially deleted. Similarly, if the recent use time is less than or equal to the time threshold value, it can be determined that the target data in the target data is recently used, and such data can be retained.

[0067] It should be noted that the time threshold values adopted by the two examples in the embodiment can be the same or different, which is not limited here. In order to distinguish, the time threshold value selected according to the stored time can be referred to as a storage time threshold value, and the time threshold value selected according to the recent use time can be referred to as an idle time threshold value.

[0068] On the basis of the above embodiment, the embodiment of the application describes the step after deleting the target data with the stored time greater than the time threshold value. Specifically, the method of the embodiment comprises the following steps:

[0069] Updating the current space usage rate according to the used space and the total space of the data storage system to obtain an updated current space usage rate; and adjusting the time threshold value according to the updated current space usage rate, the current space usage rate being negatively correlated with the time threshold value.

[0070] In combination with the foregoing embodiment, the space usage rate of the application can be determined based on the used space of the data storage system and the total space of the data storage system, for example, can be represented by the ratio between the used space and the total space, and in other application scenarios, can also be represented by the difference between the used space and the total space. Here, it is not described in detail, and can be set as needed according to the actual application scenario.

[0071] It should be noted that the current space usage rate of the data storage system is determined based on the current storage state of the data storage system, for example, based on the comparison between the currently acquired used space and the total space, so the current space usage rate acquired at different times can be the same or different.

[0072] For example, the process of updating the current space usage rate can be performed after each time of processing data (deletion, overwrite, etc.), or can be selected to be performed before each time of processing data (deletion, overwrite, etc.) (for example, acquired before step S120), or can be real-time updated (acquired before and after processing data) or periodically updated, which is not limited here. For example, as shown in Figure 2 Figure 2 ​is an exemplary arrangement flow diagram of the data hierarchical management method based on space-time information of the present application. It should be noted that, due to data deletion and data writing, the space usage rate before and after updating can be the same or different, which is not limited here.

[0073] Another exemplary, in the present application, the time threshold for screening target data can also be adjusted according to the current space usage rate obtained each time. The current space usage rate and the time threshold can be negatively correlated. For example, the higher the space usage rate, the shorter the time threshold, that is, when deleting the target data stored for more than the time threshold, more target data will be selected for deletion. Similarly, if the space usage rate is lower, the time threshold is longer, that is, when deleting the target data stored for more than the time threshold, less target data will be selected for deletion. Thus, the time threshold can be dynamically adjusted according to the system running state.

[0074] On the basis of the above embodiments, the present application embodiment describes the step of obtaining the value level of the initial data in the data storage system. Specifically, the method of the present embodiment comprises the following steps:

[0075] In response to obtaining new storage data, the new storage data is subjected to value rating processing to obtain the value level of the new storage data; in response to the value update task triggering, the value level of the historical storage data is subjected to value update processing.

[0076] In combination with the foregoing embodiments, it is continued to be described that the initial data of the present application can include data newly stored into the system and / or historical storage data already stored into the system. The value level of different types of data can be obtained by the same or different ways.

[0077] Exemplarily, when the new storage data is obtained (for example, when a new file is generated in the data storage system), the new storage data is subjected to pre-rating processing (value rating processing), and each newly generated file is assigned a value level, so that the new storage data in the data storage system also has a corresponding value level. The method of value rating processing can be various, which can be set as needed by referring to the existing method and combining with the actual application scenario, and is not described here.

[0078] As for the historical storage data (the historical storage data is equivalent to the completed new storage data), the data storage system can be preset with a periodic value update task. During the normal operation of the data storage system, the value level of the historical storage data can be updated when the value update task is triggered. The method of value update processing can be various, which can be set as needed by referring to the method of value rating processing and / or other existing methods and combining with the actual application scenario, and is not described here.

[0079] On the basis of the foregoing embodiments, the embodiments of the present application exemplarily illustrate the step of performing value rating processing on the newly stored data to obtain the value level of the newly stored data. Specifically, the method of the present embodiment comprises the following steps:

[0080] analyzing the data content of the newly stored data; and performing value rating on the newly stored data according to the data content to obtain the value level.

[0081] In combination with the foregoing embodiments, it is specified that the pre-rating processing on the newly stored data in the present application can be obtained by performing value rating processing on the data content of the newly stored data, or the value level can be marked according to the level marking instruction input by the user when the newly stored data is written (then the value rating is not needed).

[0082] It is specified that the newly stored data can include but is not limited to video data, image data, audio data and / or text data and the like. For the convenience of illustration, the video data is mainly taken as an example for explanation in the subsequent examples.

[0083] Specifically, the value level of the newly stored data can be determined by analyzing the data content of the newly stored data and comparing the data content with a preset content level template. The content level template is a value level template preset according to the data content that can exist in the newly stored data. Different content level templates can correspond to the same or different value levels, which are not limited herein. In addition, the value rating model can be obtained by pre-training the neural network, the data content of the newly stored data is extracted and analyzed by the value rating model, and then the value level of the newly stored data is output. In addition to the data content, the data attribute can also be used for rating, or the data content and the data attribute can be combined for rating according to the preset weight, which is not described herein. In the foregoing embodiments, it is mentioned that the data attribute can include the data size, in addition to which, for different data types, the video data can refer to the video code rate, resolution, etc., the audio data can refer to the audio code rate, etc., the image data can refer to the image resolution, etc., and the text data can refer to the logical coherence, etc. The specific settings can be made according to the actual application scenario, which are not limited herein.

[0084] Exemplarily, as video data, the common application scenarios can focus on one or more events (i.e. data content) occurring in the video data. For example, whether a target object appears in a single video data, whether a target motion trajectory appears, whether a target behavior appears, and / or whether it is a still picture (moving picture), etc. It can also be determined whether there is an overlapping picture area and / or repeated picture content between multiple video data. For example, video data without a target object can be rated as L1, video data with a target object can be rated as L3, video data with a target object and the target object is occurring a target behavior can be rated as L5, etc.

[0085] Another exemplary, for video data, it is usually stored in the form of a video file in the data storage system. For a video file, especially a video file with a long time span (long video time), it can include low-value data content and high-value data content. For example, the data content of a video in the prologue is mostly in a still picture, a small part of the data content appears a target object, and then there is a part of the data content returning to the still picture after the target object leaves the picture. In this regard, the method of the present application can rate the video file as L3 according to the data content (existence of the target object). In addition, the video file (video data) can also be segmented according to the data content analysis result of the video file, to obtain a first sub-file (corresponding to the still picture in the prologue), a second sub-file (corresponding to the picture with the target object), and a third sub-file (corresponding to the still picture in the subsequent), and then the value rating of each sub-file (sub-data) is performed. Thus, part of the important content in the whole file can be segmented and stored. In which, the need for processing and rating according to the segmentation method in the foregoing example can be selected by comparing the proportion of important content (data content of the event occurring) in the whole file. If the proportion of important content is greater than 50% or other proportion threshold, the whole file can be rated directly without segmentation. Otherwise, the whole file can be rated after segmentation. For example, video data, audio data can be judged by the time proportion of important content, text data can be judged by the proportion of the number of important content, and image data can be judged by the proportion of the number of important content (e.g. cutting and retaining the image with the target object). Thus, each data can be optionally segmented, taking into account the influence on storage space and data processing efficiency.

[0086] In addition, the newly stored data can be set as a lock attribute, that is, no matter what value level the newly stored data is, it will not be deleted or covered by the automatic management method.

[0087] On the basis of the above-mentioned embodiments, the embodiments of the present application exemplarily illustrate the steps of value updating processing of the value levels of the historical storage data. Specifically, the method of the present embodiment comprises the following steps:

[0088] analyzing the similarity of each historical storage data; and updating the value level of each historical storage data according to the similarity.

[0089] In the foregoing embodiments, the data content of each data file is mainly rated independently, while in the present embodiment, the data content of multiple data files is mainly rated in association.

[0090] In combination with the foregoing embodiments, when the value of the historical storage data is updated in the present application, it can be obtained by reanalyzing and rating the data content of the historical storage data, or by analyzing the similarity between each historical storage data in the data storage system to update the value.

[0091] Specifically, the similarity of each historical storage data can be analyzed through the data content of each historical storage data. Among them, all historical storage data can be compared for similarity, or part of the historical storage data can be compared for similarity (for example, historical storage data of the same data type are compared for similarity, etc.), which is not limited here. Through similarity comparison, redundant data in the historical storage data can be determined, and at least one relatively higher value data file can be selected for retention, reducing the redundant space occupation.

[0092] Exemplarily, for video data, when the video is recorded, it may include a main stream file and an auxiliary stream file, and the two stream files are generated by different encoding methods. Among them, the main stream file is commonly used for picture display, and the auxiliary stream file is commonly used for picture detection, which is equivalent to the quality of the main stream file is usually better than that of the auxiliary stream file. Therefore, in the process of value updating processing of the video data, the main and auxiliary stream files corresponding to the same video file can be marked, and different value levels can be assigned to the main and auxiliary stream files. Among them, in the application scenario of quality priority, the value level of the main stream file of the same video file can be higher than that of the auxiliary stream file, so that the auxiliary stream file is preferentially deleted and covered in the data management process. If in the application scenario of volume priority, the value level of the main stream file of the same video file can be lower than that of the auxiliary stream file. Specifically, it can be set as needed according to the actual application scenario, which is not limited here.

[0093] Another example, during video recording, multiple video acquisition devices may be deployed in the same scene to record videos, which may result in multiple redundant video files containing the same pictures. In this regard, the method of the present application can also determine two or more historical storage data with similarity greater than a preset similarity threshold as redundant video files by analyzing the similarity of the historical storage data, assign a lower value level to the video file with lower quality in the similar data, and assign a higher value level to the video file with higher quality. The specific method of analyzing similarity is not the focus of the present application, and can refer to the existing related technology, for example, the method of analyzing similarity can include but is not limited to intelligent secondary analysis, AI model assisted image analysis, cross-file hash comparison and deduplication, etc., which are not limited here.

[0094] It should be further explained that the execution subject of the data hierarchical management method based on spatiotemporal information can be a data hierarchical management device based on spatiotemporal information, for example, the data hierarchical management method based on spatiotemporal information can be executed by a terminal device or a server or other processing device, wherein the terminal device can be a user equipment (User Equipment, UE), a computer, a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementation manners, the data hierarchical management method based on spatiotemporal information can be realized by a processor calling computer readable instructions stored in a memory.

[0095] Figure 3 is a block diagram of a data hierarchical management device based on spatiotemporal information according to an example embodiment of the present application. As shown in Figure 3 The example data hierarchical management device based on spatiotemporal information 300 includes an acquisition module 310, a data determination module 320, and a data deletion module 330.

[0096] Specifically:

[0097] The acquisition module 310 is configured to acquire a value level of initial data in a data storage system.

[0098] The data determination module 320 is configured to determine target data in the initial data according to the current space usage rate and the value level of the data storage system.

[0099] The data deletion module 330 is configured to perform deletion processing on the target data according to the stored time of the target data in response to a data deletion task trigger.

[0100] In the exemplary data hierarchical management device based on space-time information, the data value of each initial data can be determined by obtaining the value level of the initial data in the data storage system, and low-value data can be deleted preferentially compared with high-value data; the target data in the initial data can be determined according to the current space usage of the data storage system and the value level, and the target data to be preferentially deleted can be determined according to the current space usage of the data storage system; if the data deletion task is triggered, the low-value data stored for a long time can be deleted according to the stored time of the target data. Thus, the data management based on the double threshold standards of time information and space information is realized, which not only can improve the effective utilization of storage resources, but also can avoid the misdeletion of high-value data.

[0101] It should be noted that the apparatus provided in the above embodiments and the method provided in the above embodiments belong to the same concept, wherein the specific manner in which each module and unit performs operations has been described in detail in the method embodiments, and will not be described here. In actual application, the functions of the apparatus can be allocated by different functional modules according to needs, that is, the internal structure of the apparatus is divided into different functional modules to complete all or part of the functions described above, and this is not limited herein.

[0102] The functions of each module can be referred to the data hierarchical management method based on space-time information embodiments, and will not be described here.

[0103] Please refer to Figure 4 , Figure 4 is a structural schematic diagram of an embodiment of an electronic device. The electronic device 100 includes a memory 101 and a processor 102, and the processor 102 is configured to execute program instructions stored in the memory 101 to implement the steps in any of the above data hierarchical management methods based on space-time information. In one specific implementation scenario, the electronic device 100 can include but is not limited to a microcomputer, a server, and in addition, the electronic device 100 can also include a notebook computer, a tablet computer, and other mobile devices, which are not limited here.

[0104] Specifically, the processor 102 is configured to control itself and the memory 101 to implement the steps in any of the above embodiments of the method for managing data based on space-time information. The processor 102 can also be referred to as a CPU (Central Processing Unit). The processor 102 can be an integrated circuit chip with processing capability. The processor 102 can also be a general purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general purpose processor can be a microprocessor or the processor can also be any conventional processor. In addition, the processor 102 can be implemented by an integrated circuit chip.

[0105] In the exemplary electronic device, by obtaining the value levels of the initial data in the data storage system, the data values of the initial data can be determined, and low-value data can be deleted in priority to high-value data; the target data in the initial data is determined according to the current space usage of the data storage system and the value levels, and the target data to be deleted in priority can be determined according to the current space usage of the data storage system; if the data deletion task is triggered, the low-value data stored for a long time can be deleted according to the stored time of the target data. Thus, the data management based on the double threshold standards of time information and space information is implemented, which not only improves the effective utilization of storage resources, but also avoids the misdeletion of high-value data.

[0106] Please refer to Figure 5 , Figure 5 is a structural schematic diagram of an embodiment of the computer readable storage medium of the present application. The computer readable storage medium 110 stores program instructions 111 capable of being executed by a processor, and the program instructions 111 are used to implement the steps in any of the above embodiments of the method for managing data based on space-time information.

[0107] In the exemplary storage medium, by running the program instructions in the storage medium, the value levels of the initial data in the data storage system are obtained, the data values of the initial data can be determined, and low-value data is preferentially deleted compared with high-value data; target data in the initial data is determined according to the current space usage of the data storage system and the value levels, the target data that needs to be preferentially deleted can be determined through the current space usage of the data storage system; if the data deletion task is triggered, the low-value data stored for a long time can be deleted according to the stored time of the target data. Thus, the data management based on the double threshold standards of time information and space information is realized, which not only can improve the effective utilization of storage resources, but also can avoid the misdeletion of high-value data.

[0108] In some embodiments, the apparatus provided by the embodiments of the present disclosure has functions or includes modules that can be used to perform the methods described in the above method embodiments, and the specific implementation can refer to the description of the above method embodiments. For brevity, it will not be repeated here.

[0109] The above description of each embodiment tends to emphasize the differences between each embodiment, and the same or similar parts can be mutually referred to, and for brevity, will not be repeated here.

[0110] In several embodiments provided in the present application, it should be understood that the disclosed method and device can be implemented by other ways. For example, the above-described device implementation is only schematic, for example, the division of modules or units is only a logical function division, and actual implementation can have another division manner, for example, a unit or component can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed mutual ones can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0111] In addition, each of the function units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit. When the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application, essentially or in the form of a contribution to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor execute all or part of the steps of the methods in each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

Claims

1. A data hierarchical management method based on spatiotemporal information, characterized in that, The method is applied to a data storage system, and the method includes: Obtain the value level of the initial data in the data storage system; The target data in the initial data is determined based on the current space utilization rate of the data storage system and the value level. In response to the triggering of a data deletion task, the target data is deleted based on its storage time.

2. The method according to claim 1, characterized in that, The step of determining the target data in the initial data based on the current space utilization rate of the data storage system and the value level includes: Obtain the spatial threshold range of the data storage system, with each spatial threshold range corresponding to a value level; Based on the space threshold range in which the current space utilization rate falls, determine the target value level corresponding to the current space utilization rate; The initial data that is at the target value level is determined as the target data.

3. The method according to claim 1, characterized in that, Before deleting the target data based on its storage time in response to a data deletion task, the method further includes: Determine whether the current space utilization rate is greater than a preset space threshold; If so, the data deletion task will be triggered.

4. The method according to claim 1, characterized in that, The step of deleting the target data based on its storage time includes: The stored time of the target data is compared with a time threshold; Delete the target data whose stored time exceeds the time threshold.

5. The method according to claim 4, characterized in that, After deleting the target data whose storage time exceeds the time threshold, the method further includes: The current space utilization rate is updated based on the used space and total space of the data storage system to obtain the updated current space utilization rate. The time threshold is adjusted based on the updated current space utilization rate, and the current space utilization rate is negatively correlated with the time threshold.

6. The method according to claim 1, characterized in that, The initial data includes new stored data and / or historical stored data, and obtaining the value level of the initial data in the data storage system includes: In response to the acquisition of new stored data, the new stored data is subjected to value rating processing to obtain the value level of the new stored data; In response to the triggering of the value update task, the value level of the historical stored data is updated.

7. The method according to claim 6, characterized in that, The process of performing value rating on the newly stored data to obtain the value level of the newly stored data includes: Analyze the data content of the newly stored data; The newly stored data is valued based on its content to obtain the value level.

8. The method according to claim 6, characterized in that, The value update process for the historical stored data includes: Analyze the similarity of various historical stored data; The value level of each historical stored data is updated based on the similarity.

9. An electronic device, characterized in that, The method includes a memory and a processor, the processor being configured to execute program instructions stored in the memory to implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having program instructions stored thereon, characterized in that, When the program instructions are executed by the processor, they implement the method described in any one of claims 1 to 8.

Images