A processing method and apparatus, electronic device, and storage medium

Abstract

The application discloses a processing method and device, electronic equipment and storage medium. The method comprises the following steps: obtaining a target storage duration of second data in first data in a target node, wherein the first data is data corresponding to a target service type in target user data of a target user; if the target storage duration is less than or equal to a first preset duration corresponding to the target user, determining to-be-processed data in the first data according to a storage space size occupied by the target user data; and processing the to-be-processed data. The method sets the first preset duration corresponding to the target user, and when the target storage duration of the second data in the first data in the target node is less than or equal to the first preset duration corresponding to the target user, the to-be-processed data is processed according to the storage space size occupied by the target user data, so that the control demand of each user is met on the basis of realizing the first data life cycle management.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and in particular to a processing method and device, electronic equipment and storage medium. BACKGROUND

[0002] Elasticsearch (ES for short) is an open source search engine based on Apache Lucene. When storing data, ES usually divides the data into hot data and cold data and stores them in corresponding storage spaces. As time goes by, the data in ES will gradually accumulate and increase. At this time, the stored hot data and cold data need to be dynamically adjusted to reduce the negative pressure of ES and improve the current query efficiency.

[0003] In the existing technical solution, ES can manage the life cycle of hot data and cold data according to the index. For example, when the hot data of a certain user reaches the set storage time, the entire index containing the hot data will be migrated. This management method is relatively rough and cannot meet the control needs of each user. SUMMARY

[0004] The present application provides a processing method and device, electronic equipment and storage medium to meet the control needs of each user on the basis of realizing the life cycle management of hot data and cold data.

[0005] According to an aspect of the present application, a processing method is provided, comprising:

[0006] obtaining a target storage duration of second data in first data in a target node, the first data being data corresponding to a target business type in target user data of a target user;

[0007] if the target storage duration is less than or equal to a first preset duration corresponding to the target user, determining to-be-processed data in the first data according to the size of the storage space occupied by the target user data;

[0008] processing the to-be-processed data.

[0009] According to another aspect of the present application, a processing device is provided, comprising:

[0010] an obtaining module configured to obtain a target storage duration of second data in first data in a target node, the first data being data corresponding to a target business type in target user data of a target user;

[0011] determining, if the target storage duration is less than or equal to the first preset duration corresponding to the target user, the to-be-processed data in the first data according to a size of a storage space occupied by the target user data;

[0012] processing the to-be-processed data.

[0013] According to another aspect of the present application, an electronic device is provided, which comprises:

[0014] at least one processor; and

[0015] a memory connected to the at least one processor in communication; wherein

[0016] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the processing method according to any one of the embodiments of the present application.

[0017] According to another aspect of the present application, a computer readable storage medium is provided, which stores computer instructions for enabling a processor to execute the processing method according to any one of the embodiments of the present application when executed.

[0018] The embodiments of the present application provide a processing method, device, electronic device and storage medium. The method comprises: obtaining a target storage duration of second data in first data in a target node, the first data being data corresponding to a target service type in target user data of a target user; if the target storage duration is less than or equal to a first preset duration corresponding to the target user, determining to-be-processed data in the first data according to a size of a storage space occupied by the target user data; and processing the to-be-processed data. By setting the first preset duration corresponding to the target user, the to-be-processed data can be processed according to the size of the storage space occupied by the target user data when the target storage duration of the second data in the first data in the target node is less than or equal to the first preset duration corresponding to the target user, so as to meet the control requirements of each user on the basis of realizing the first data life cycle management.

[0019] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to make the technical solutions in the embodiments of the present application clearer, the accompanying drawings needed in the embodiments will be briefly introduced. Obviously, the accompanying drawings in the following description only show some embodiments of the present application, and all other embodiments obtained by those of ordinary skill in the art without any creative effort based on the accompanying drawings should belong to the protection scope of the present application.

[0021] Figure 1 is a flow chart of a processing method according to the first embodiment of the present application;

[0022] Figure 2 is a flow chart of a processing method according to the second embodiment of the present application;

[0023] Figure 3 is a flow chart of a processing method according to the second embodiment of the present application;

[0024] Figure 4 is a flow chart of a processing method according to the third embodiment of the present application;

[0025] Figure 5 is a flow chart of a processing method according to the third embodiment of the present application;

[0026] Figure 6 is a structural schematic diagram of a processing device according to the fourth embodiment of the present application;

[0027] Figure 7 is a structural schematic diagram of an electronic device according to the fifth embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the technical solutions in the embodiments of the present application clearer, the accompanying drawings needed in the embodiments will be briefly introduced. Obviously, the accompanying drawings in the following description only show some embodiments of the present application, and all other embodiments obtained by those of ordinary skill in the art without any creative effort based on the accompanying drawings should belong to the protection scope of the present application.

[0029] It should be noted that the terms "first", "second", and the like in the description and in the claims of the present application and the above-described accompanying drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] Embodiment one

[0031] is a flowchart of a processing method according to an embodiment one of the present application, the embodiment can be applicable to the case of processing the data stored by the user in the ES, the method can be executed by a processing device, which can be realized in the form of hardware and / or software, and the processing device can be configured in an electronic device. As shown in Figure 1 , the method comprises:

[0032] S110, obtaining a target storage duration of second data in the first data in the target node, the first data being data corresponding to a target service type in target user data of a target user.

[0033] Wherein, the first data can be considered as the data corresponding to the target service type in the target user data of the target user, the target user can refer to a user who stores data in the target node; the target user data is the data stored by the target user in the target node, the size and content of the target user data are not limited, the target user data can correspond to one or more service types, the target service type can be understood as a certain service type in the plurality of service types, and the target service type is used to represent the service type of the first data, such as cloud firewall service.

[0034] The target node can be understood as a server for storing user data of a plurality of users, and in the embodiment, the types of user data stored in different types of servers are different, such as the user data (i.e. hot data) with higher access frequency of each user can be stored in a server with higher performance (i.e. hot node), so as to facilitate the user to query; and the user data (i.e. cold data) with lower access frequency of each user can be stored in a server with lower performance (i.e. cold node, such as hard disk), for recovery and backup.

[0035] Optionally, the second data can be data with the earliest storage time in the first data, and the target storage duration is a duration of storage of the second data in the target node.

[0036] In this step, first, the target storage duration of the second data in the first data in the target node can be acquired, so as to facilitate subsequent processing of data in the target node. The specific method of acquiring the target storage duration of the second data in the first data is not limited, for example, the second data in the first data can be determined first, and then the target storage duration of the second data in the target node can be determined based on the second data.

[0037] S120, if the target storage duration is less than or equal to the first preset duration corresponding to the target user, determining the to-be-processed data in the first data according to a size of a storage space occupied by the target user data.

[0038] The first preset duration can be a maximum storage duration of the first data preset by the user. The first preset durations corresponding to different users can be the same or different, and the user can set them according to actual conditions. In this embodiment, the user can also set different first preset durations according to different first data, for example, when the first data is hot data, the target user can set the first preset duration to 3 months; when the first data is cold data, the target user can set the first preset duration to one year, etc. The to-be-processed data can be considered as data to be processed.

[0039] Specifically, after acquiring the target storage duration of the second data in the first data, the target storage duration can be compared with the first preset duration corresponding to the target user, and the to-be-processed data can be determined according to the comparison result. For example, when the target storage duration is less than or equal to the first preset duration corresponding to the target user, it indicates that the storage duration of the data with the earliest storage time in the first data does not exceed the critical duration preset by the user, and at this time, the to-be-processed data can be determined according to the size of the storage space occupied by the user data of the target user. The specific steps of determining the to-be-processed data can be different according to the difference between the storage space size and the target storage duration.

[0040] S130, processing the to-be-processed data.

[0041] After the to-be-processed data in the first data is determined, the to-be-processed data needs to be processed, and the specific means for processing the to-be-processed data are not limited and can be different according to different first data. For example, when the first data contains hot data, after the to-be-processed data in the first data is determined, the to-be-processed data can be migrated. The process of migration is not limited in the embodiment. When the first data contains cold data, after the to-be-processed data in the first data is determined, the to-be-processed data can be deleted to reserve more idle storage space for the target user.

[0042] The processing method provided in the embodiment one of the application comprises the following steps: obtaining a target storage time length of second data in first data in a target node, wherein the first data is data corresponding to a target service type in target user data of a target user; if the target storage time length is less than or equal to a first preset time length corresponding to the target user, determining to-be-processed data in the first data according to a storage space size occupied by the target user data; and processing the to-be-processed data. By setting the first preset time length corresponding to the target user, the to-be-processed data can be processed according to the storage space size occupied by the target user data when the target storage time length of the second data in the first data in the target node is less than or equal to the first preset time length corresponding to the target user. Thus, the control requirements of each user are met on the basis of realizing the first data life cycle management.

[0043] In one embodiment, before the step of obtaining the target storage time length of the second data in the first data in the target node, the method further comprises the following steps of:

[0044] The second data in the first data is determined according to the storage time.

[0045] In this step, the second data in the first data can be determined according to the storage time of the data. Different first data can correspond to different obtaining methods. For example, when the first data is hot data, after the first data is determined, the earliest storage time can be determined according to the order of the storage time of each data in the first data, and the data corresponding to the earliest storage time is taken as the second data. When the first data is cold data, the earliest storage time can be determined according to the cold data related information recorded in a table (for example, a first preset table) of a certain database, and the data corresponding to the earliest storage time is taken as the second data.

[0046] Embodiment two

[0047] Figure 2 is a flowchart of the processing method provided in the embodiment two of the application. The embodiment two is optimized on the basis of the above-mentioned embodiments. In the embodiment, the first data and the target node are further specified as follows: the first data comprises hot data, and the target node comprises a hot node.

[0048] Meanwhile, the case that the processing of the to-be-processed data is further included before the processing of the to-be-processed data is embodied as follows: if the target storage duration is greater than the first preset duration corresponding to the target user, the first to-be-processed data in the first data is taken as the to-be-processed data.

[0049] The details of the embodiment not yet described can refer to the embodiment one.

[0050] As shown in Figure 2 The processing method provided by the embodiment two of the present disclosure includes the following steps:

[0051] S210, obtaining a target storage duration of second data in the first data in a target node, the first data being data corresponding to a target service type in target user data of a target user, the first data including hot data, and the target node including a hot node.

[0052] In the embodiment, the hot data can be online data that needs to be frequently accessed, for example, the hot data can be user data within half a year, or user data within three months, and the specific duration can be set according to the actual business scenario. The hot node can be a server with higher configuration, and the hot node has better disk throughput and better CPU.

[0053] When the first data includes hot data and the target node includes a hot node, that is, the target storage duration of the second data in the hot data of the target user in the hot node can be obtained first. The method of obtaining is not limited, for example, the index corresponding to the target service type can be obtained in the hot node first; then the first data of the target user is searched in the index corresponding to the target service type, and the earliest storage time is determined according to the time unit in the first data, or the document of the target user is searched in the index corresponding to the target service type, and the earliest storage time is determined according to the document attribute; finally, the data corresponding to the earliest storage time is taken as the second data, and the target storage duration of the second data in the first data in the target node is obtained.

[0054] S220, determining whether the target storage duration is less than or equal to a first preset duration corresponding to the target user, if yes, performing S230; if no, performing S240.

[0055] In the embodiment, the first preset time length can be considered as the storage time length of the hot data preset by the user, and therefore, after the target storage time length of the second data in the first data is obtained, the target storage time length can be compared with the first preset time length corresponding to the target user. When the target storage time length is less than or equal to the first preset time length corresponding to the target user, it indicates that the storage time length of the earliest data in the first data does not exceed the preset storage time length of the hot data, and therefore, the to-be-processed data can be determined according to the storage space size occupied by the user data of the target user, that is, step S230 is performed. When the target storage time length is greater than the first preset time length corresponding to the target user, it indicates that the storage time length of the earliest data in the first data has exceeded the preset storage time length of the hot data, and therefore, the first to-be-processed data in the first data can be taken as the to-be-processed data, that is, step S240 is performed.

[0056] S230, determining the to-be-processed data in the first data according to the storage space size occupied by the target user data.

[0057] In the step, the manner of determining the to-be-processed data according to the current storage space size occupied by the target user is not limited, for example, the storage space size can be compared with the first preset threshold value, and the to-be-processed data can be directly determined according to the comparison result. Alternatively, the storage space size can be compared with the first preset threshold value, and the to-be-processed data can be determined according to the comparison result and the target storage time length of the second data. The embodiment does not limit this. The means of obtaining the storage space size occupied by the target user data can be determined according to actual processing conditions, for example, the storage space size occupied by the user data of each user can be statistically counted by date and hour in a table (for example, a second preset table) of a certain database, and then the target user data of the target user is accumulated by hour to obtain the storage space size occupied by the target user data.

[0058] S240, taking the first to-be-processed data in the first data as the to-be-processed data.

[0059] The first to-be-processed data can be considered as the data with a longer storage time length in the first data, for example, the first to-be-processed data can include the second data, and the specific determination manner can be determined by relevant personnel, which is not further expanded here.

[0060] S250, processing the to-be-processed data.

[0061] After the data to be processed is determined, the data to be processed can be processed. For example, in an embodiment, the data to be processed can be migrated to other storage spaces for storage, such as cold nodes. The migration method is not limited and can be determined according to actual conditions. In another embodiment, the data to be processed can be directly deleted, and the present embodiment is not limited in this regard.

[0062] The processing method provided in Embodiment Two can determine the data to be processed according to the comparison result of the target storage duration of the second data in the hot data of the target user and the first preset duration when the first data includes the hot data and the target node includes the hot node, thereby achieving processing of the data to be processed.

[0063] In an embodiment, the determining of the data to be processed in the first data according to the size of the storage space occupied by the target user data includes:

[0064] If the size of the storage space occupied by the target user data is greater than a first preset threshold and the target storage duration of the second data is greater than a second preset duration, the first data to be processed in the first data is taken as the data to be processed.

[0065] If the size of the storage space occupied by the target user data is greater than the first preset threshold and the target storage duration of the second data is less than or equal to the second preset duration, the second data to be processed in the first data is taken as the data to be processed.

[0066] The first preset threshold can be considered as the maximum storage space of the hot data pre-set by the target user, and the second preset duration can be considered as the critical value of the hot data storage duration. The second preset duration is less than the first preset duration, for example, the second preset duration can be two days. The second data to be processed is similar to the first data to be processed and can be considered as the data with a longer storage duration in the first data. For example, the second data to be processed can be part of the second data. The second data to be processed and the first data to be processed are only used to distinguish different objects, and the present embodiment is not limited in this regard.

[0067] Specifically, when the size of the storage space occupied by the target user data is greater than the first preset threshold and the target storage duration of the second data is greater than the second preset duration, it indicates that the size of the storage space currently occupied by the hot data of the target user has exceeded the maximum storage space of the hot data pre-set by the target user and the storage duration of the data with the earliest storage time in the hot data has exceeded the critical value of the hot data storage duration. At this time, the first data to be processed in the first data can be taken as the data to be processed.

[0068] When the storage space size occupied by the target user data is greater than the first preset threshold and the target storage time length of the second data is less than or equal to the second preset time length, it indicates that the storage space size currently occupied by the hot data of the target user has exceeded the maximum storage space of the hot data set by the target user in advance, but the storage time length of the earliest data in the hot data has not exceeded the critical value of the hot data storage time length, and the second to-be-processed data in the first data can be taken as to-be-processed data. The specific manner of determining the second to-be-processed data can be determined by relevant personnel, and will not be further expanded here.

[0069] In one embodiment, the first to-be-processed data in the first data is taken as to-be-processed data, including:

[0070] determining a target hot data index corresponding to the target business type and first data stored in the target hot data index;

[0071] taking first to-be-processed data with a storage time located within a first preset time period corresponding to the first data as to-be-processed data.

[0072] The target hot data index can refer to an index in a hot node corresponding to the target business type. It can be considered that the target hot data index stores user data with a plurality of storage time lengths corresponding to the target business type of a plurality of users, such as data corresponding to the target business type in the target user data of the target user, i.e., the first data. The first preset time period can be considered as a preset time period, such as a day.

[0073] In this step, when determining to-be-processed data, first, a target hot data index corresponding to the target business type and first data stored in the target hot data index are determined; then, according to the storage time of the data in the first data, data with a storage time located within a first preset time period is taken as to-be-processed data. For example, when a day is taken as the first preset period, data with a storage time located within the earliest day is taken as to-be-processed data. For example, assuming that the storage date of the data with the earliest storage time in the first data is y month d, data with a storage time located within y month d in the first data is taken as to-be-processed data.

[0074] In one embodiment, the second to-be-processed data in the first data is taken as to-be-processed data, including:

[0075] determining a target hot data index corresponding to the target business type and first data stored in the target hot data index;

[0076] The second to-be-processed data with a storage time within the first n preset time granularities corresponding to the first data are taken as the to-be-processed data, where n is a positive integer, and the total time length corresponding to the first n preset time granularities is less than the cycle length of the first preset time cycle.

[0077] The preset time granularity can be a pre-set time granularity for determining the second to-be-processed data. For example, the preset time granularity can be an hour.

[0078] In the step of taking the second to-be-processed data in the first data as the to-be-processed data, the target hot data index corresponding to the target service type and the first data stored in the target hot data index can be determined first. Then, the second to-be-processed data with a storage time within the first n preset time granularities corresponding to the first data can be taken as the to-be-processed data according to the storage time of the data in the first data. For example, when the preset time granularity is an hour, the data with a storage time within the first n hours corresponding to the first data can be taken as the to-be-processed data according to the storage time of the data in the first data after the first data is determined. The value of n can be determined by an experience value.

[0079] In one embodiment, the processing of the to-be-processed data includes:

[0080] The to-be-processed data is migrated into a first cold data index in the cold node, and the first cold data index corresponds to the target service type and the current first preset time cycle.

[0081] The first cold data index can be considered as an index in the cold node corresponding to the target service type and the current first preset time cycle.

[0082] In one embodiment, after the to-be-processed data is determined, the to-be-processed data can be migrated into the first cold data index in the cold node. For example, the to-be-processed data can be directly migrated into the first cold data index, or the to-be-processed data can be grouped and migrated, and the present embodiment does not limit this.

[0083] In one embodiment, the migration of the to-be-processed data into the first cold data index in the cold node includes:

[0084] The to-be-processed data is grouped according to a preset time granularity and a storage time, where the time length corresponding to the preset time granularity is less than the cycle length of the first preset time cycle;

[0085] Each group of to-be-processed data is sequentially migrated into the first cold data index in the cold node in an order from early to late according to the storage time.

[0086] Specifically, in the process of migrating the to-be-processed data into the first cold data index in the cold node, first, the to-be-processed data can be grouped according to the preset time granularity and the storage time, and then each group of to-be-processed data is migrated into the first cold data index in the cold node in the order from early to late according to the storage time.

[0087] In one embodiment, before the migrating the to-be-processed data into the first cold data index in the cold node, further comprising:

[0088] If the first cold data index does not exist in the cold node, a cold data index corresponding to the target business type and the current first preset time period is created as the first cold data index.

[0089] It can be understood that when the to-be-processed data is migrated into the first cold data index in the cold node, if the first cold data index does not exist in the cold node, a corresponding cold data index needs to be created in the cold node according to the target business type and the current first preset time period as the first cold data index; and when the first cold data index exists in the cold node, the migration of the to-be-processed data can be directly performed.

[0090] Figure 3 is a flowchart of a processing method according to Embodiment Two of the present application, as shown in Figure 3 The earliest document can be obtained from the ES (i.e., the target storage time length of the second data in the first data in the target node is obtained), and then it is judged whether the storage time length of the document exceeds the hot data retention time (i.e., the first preset time length corresponding to the target user), if yes, the hot data is migrated by day, such as the data of the earliest document on the same day is divided into an hourly queue and migrated to the cold node (i.e., the first to-be-processed data whose storage time is within the first preset time period corresponding to the first data is taken as the to-be-processed data; the to-be-processed data is grouped according to the preset time granularity and the storage time; each group of to-be-processed data is migrated into the first cold data index in the cold node in the order from early to late according to the storage time), after the migration is completed, the hot data space usage size is re-counted, the record of the cold data space usage size in the first preset table of the database is triggered, and the operation is ended.

[0091] If not, the hot data space usage size is obtained, and it is determined whether the hot data space usage size exceeds the hot data space limit (i.e., the to-be-processed data in the first data is determined according to the storage space size occupied by the target user data). If the hot data space limit is exceeded, it is determined whether the earliest document time is the current day or the previous day. If the earliest document time is not the current day or the previous day, the hot data is migrated by day (i.e., if the storage space size occupied by the target user data is greater than the first preset threshold and the target storage duration of the second data is greater than the second preset duration, the first to-be-processed data in the first data is taken as the to-be-processed data). If the earliest document time is the current day or the previous day (i.e., if the storage space size occupied by the target user data is greater than the first preset threshold and the target storage duration of the second data is less than or equal to the second preset duration), the current day data is migrated by hour, that is, the target hot data index corresponding to the target business type and the first data stored in the target hot data index are determined. The second to-be-processed data stored within the first n preset time granularities corresponding to the first data is taken as the to-be-processed data, and the to-be-processed data is processed.

[0092] Embodiment Three

[0093] Figure 4 is a flowchart of a processing method according to Embodiment Three of the present application. Embodiment Three is an optimization based on the above-mentioned embodiments. In this embodiment, the first data and the target node are further specified as: the first data includes cold data, and the target node includes a cold node.

[0094] Meanwhile, the case before the to-be-processed data is processed further includes: if the target storage duration is greater than the first preset duration corresponding to the target user, the earliest user data storage duration of each user in the second cold data index is obtained. If each of the earliest user data storage durations is greater than the first preset duration corresponding to the corresponding user, the second cold data index is taken as the to-be-processed data, otherwise, the third to-be-processed data in the first data is taken as the to-be-processed data.

[0095] The details of this embodiment are not described herein and can be referred to Embodiment One.

[0096] As shown in Figure 4 , a processing method according to Embodiment Three of the present application includes the following steps:

[0097] S310, obtaining a target storage duration of second data in the first data in a target node, the first data being data corresponding to a target business type in target user data of a target user, the first data including cold data, and the target node including a cold node.

[0098] In the embodiment, the cold data can refer to offline data that is not frequently accessed, such as user data that is more than three years old, and the specific time length can be set according to an actual business scenario. The cold node can refer to a server with a lower configuration, such as a hard disk, and is used for recovery and backup data.

[0099] When the first data includes cold data and the target node includes a cold node, the target storage time length of the second data in the target user cold data in the cold node can be obtained first, and the obtaining method is not limited further and can be determined according to an actual job situation.

[0100] S320, determining whether the target storage time length is less than or equal to a first preset time length corresponding to the target user, if yes, performing S330; if no, performing S340-S350.

[0101] In the embodiment, the first preset time length can be considered as a storage time length of cold data that is set by a user in advance, and therefore, after the target storage time length of the second data in the first data is obtained, the target storage time length can be compared with the first preset time length corresponding to the target user. When the target storage time length is less than or equal to the first preset time length corresponding to the target user, it is indicated that the storage time length of the earliest data in the first data does not exceed the preset storage time length of cold data, and at this time, the to-be-processed data can be determined according to the storage space size occupied by the user data of the target user, that is, step S330 is performed. When the target storage time length is greater than the first preset time length corresponding to the target user, it is indicated that the storage time length of the earliest data in the first data has exceeded the preset storage time length of cold data, and at this time, step S340 can be performed.

[0102] S330, determining the to-be-processed data in the first data according to the storage space size occupied by the target user data.

[0103] In this step, the manner of determining the to-be-processed data according to the current storage space size occupied by the target user is not limited, for example, the storage space size can be compared with a second preset threshold value, and the to-be-processed data can be directly determined according to the comparison result. The means of obtaining the storage space size occupied by the target user data can be determined according to an actual processing situation, for example, the storage space size occupied by the target user data can be obtained by performing real-time statistics on the storage space size occupied by the user data of each user in a table of a certain database by date and hour, and then accumulating the target user data of the target user by date and hour. Alternatively, the storage space size occupied by the target user data can be obtained by performing real-time statistics on the storage space size occupied by the user data of each user in a table of a certain database by date, and then accumulating the target user data of the target user by date.

[0104] S340, acquire the earliest user data storage duration of each user in the second cold data index.

[0105] The second cold data index can refer to an index corresponding to the target business type in the cold node. It can be considered that the second cold data index stores user data corresponding to the target business type of multiple users, such as target user data of a target user stored in the second cold data index. The earliest user data storage duration can be understood as the storage duration corresponding to the data stored earliest in the user data of a certain user, that is, each user corresponds to an earliest user data storage duration.

[0106] In this step, the earliest user data storage duration of each user in the second cold data index can be acquired for subsequent processing. The method of acquiring the earliest user data storage duration is not limited, such as acquiring the storage duration corresponding to the data stored earliest in the user data of each user according to the time unit stored in the user data of each user, that is, the earliest user data storage duration.

[0107] S350, determine whether the earliest user data storage duration of each user is greater than the first preset duration corresponding to the user, if yes, execute S3501; if no, execute S3502.

[0108] After acquiring the earliest user data storage duration, the earliest user data storage duration is compared with the first preset duration corresponding to the user to determine whether the earliest user data storage duration of each user is greater than the first preset duration corresponding to the user. If the earliest user data storage duration of each user is greater than the first preset duration corresponding to the user, it means that the earliest user data storage duration of each user exceeds the maximum storage duration set by the corresponding user. At this time, the second cold data index containing the user data can be used as the data to be processed for processing. If the earliest user data storage duration of each user is not greater than the first preset duration corresponding to the user, it means that there is a user data whose storage duration does not exceed the maximum storage duration set by the corresponding user. At this time, step S3502 can be executed.

[0109] S3501, use the second cold data index as the data to be processed.

[0110] S3502, use the third data to be processed in the first data as the data to be processed.

[0111] The third data to be processed can be considered as data with a longer storage duration in the cold data, such as the second data or part of the second data. The specific determination method can be determined by relevant personnel, which will not be further expanded here.

[0112] S360, processing the to-be-processed data.

[0113] The processing method provided in Embodiment Three of the present application can determine the corresponding to-be-processed data according to the comparison result of the target storage duration of the second data in the cold data of the target user and the first preset duration, when the first data includes cold data and the target node includes a cold node, by setting the first preset duration corresponding to the target user, thereby realizing processing of the to-be-processed data.

[0114] In one embodiment, the determining the to-be-processed data in the first data according to the storage space size occupied by the target user data comprises:

[0115] If the storage space size occupied by the target user data is greater than the second preset threshold, the third to-be-processed data in the first data is taken as the to-be-processed data.

[0116] The second preset threshold can be considered as the maximum storage space of the cold data pre-set by the target user.

[0117] Specifically, when the storage space size occupied by the target user data is greater than the second preset threshold, it indicates that the storage space size currently occupied by the cold data of the target user has exceeded the maximum storage space of the cold data pre-set by the target user, and at this time, the third to-be-processed data in the first data can be taken as the to-be-processed data; when the storage space size occupied by the target user data is less than or equal to the second preset threshold, the operation is ended.

[0118] In one embodiment, after the obtaining the target storage duration of the second data in the first data in the target node, the method further comprises:

[0119] If the cold data index corresponding to the business type corresponding to the second data and the storage time of the second data exists in the target node, the cold data index corresponding to the business type corresponding to the second data and the storage time of the second data is recorded as a second cold data index.

[0120] It can be considered that, after the obtaining the target storage duration of the second data in the first data in the target node, if the cold data index corresponding to the business type corresponding to the second data and the storage time of the second data exists in the cold node, the cold data index corresponding to the business type corresponding to the second data and the storage time of the second data can be recorded as a second cold data index, so as to facilitate subsequent processing operation; if the cold data index corresponding to the business type corresponding to the second data and the storage time of the second data does not exist in the cold node, it indicates that such index has been deleted, and the operation is ended.

[0121] In an embodiment, the step of taking the third to-be-processed data in the first data as to-be-processed data comprises:

[0122] determining the first data stored in the second cold data index;

[0123] taking the third to-be-processed data with a storage time within the first m preset time granularities corresponding to the first data as to-be-processed data, wherein m is a positive integer, and the total length of the first m preset time granularities is less than the cycle length of the first preset time cycle.

[0124] In the embodiment, when the step of taking the third to-be-processed data in the first data as to-be-processed data is performed, the first data stored in the second cold data index can be determined first, and then the third to-be-processed data with a storage time within the first m preset time granularities corresponding to the first data can be taken as to-be-processed data according to the storage time of the data in the first data. For example, when the preset time granularity is an hour, after the first data is determined, the data with a storage time within the first m hours corresponding to the first data can be taken as to-be-processed data according to the storage time of the data in the first data, and the value of m can be determined by an experience value.

[0125] In an embodiment, the step of processing the to-be-processed data comprises:

[0126] deleting the to-be-processed data.

[0127] In the embodiment, after the to-be-processed data is determined, the determined to-be-processed data can be deleted. The step of deleting can be different according to different to-be-processed data, which is not limited in the embodiment.

[0128] In an embodiment, the step of deleting the to-be-processed data comprises:

[0129] if the to-be-processed data is the third to-be-processed data, grouping the to-be-processed data according to a preset time granularity and a storage time, wherein the length of the preset time granularity is less than the cycle length of the first preset time cycle;

[0130] deleting each group of to-be-processed data in turn according to the order of storage time from early to late.

[0131] Specifically, when the to-be-processed data is the third to-be-processed data, the to-be-processed data can be grouped according to a preset time granularity and a storage time first, and then each group of to-be-processed data can be deleted in turn according to the order of storage time from early to late, so as to reduce the resource usage rate of the cold node and ensure the stable operation of the ES.

[0132] In an embodiment, the step of deleting the to-be-processed data comprises:

[0133] If the data to be processed is a second cold data index, then delete the second cold data index.

[0134] It can be assumed that when the data to be processed is the second cold data index, the second cold data index can be directly deleted. On this basis, the user data of all users in the entire second cold data index is deleted, freeing up more space for cold nodes.

[0135] Figure 5 This is a flowchart of a processing method provided according to Embodiment 3 of the present invention, as follows: Figure 5 As shown, firstly, the earliest "active" time is obtained from the database (i.e., the target storage duration of the second data in the first data within the target node). Then, it is determined whether an index exists in Elasticsearch (i.e., whether a cold data index exists in the target node corresponding to the business type and storage duration of the second data). If an index exists, it is determined whether the earliest "active" time exceeds the data storage duration (i.e., the first preset duration corresponding to the target user). If so (i.e., the target storage duration exceeds the first preset duration corresponding to the target user), the maximum storage time for all users is obtained (i.e., the earliest user data storage duration in the second cold data index). Finally, it is determined whether all users have exceeded the storage time. If all users have exceeded the storage time... If the storage time is specified, the entire index is deleted directly (i.e., if the storage time of each earliest user is greater than the first preset time corresponding to the user, the second cold data index is used as data to be processed; the data to be processed is deleted), and the status of the corresponding record in the database is set to "deleted", and the operation ends; if not all users have exceeded the storage time, the old data corresponding to the index is deleted hourly (i.e., the first data stored in the second cold data index is determined; the third data to be processed whose storage time is within the first m preset time granularities corresponding to the first data is used as data to be processed; the data to be processed is grouped according to the preset time granularity and storage time; and the data to be processed in each group is deleted in order of storage time from earliest to latest).

[0136] If not (i.e., if the target storage duration is less than or equal to the first preset duration corresponding to the target user), then the size of the cold data space used (i.e., the size of the storage space occupied by the target user data) is counted, and it is determined whether the size of the cold data space used exceeds the cold data space limit (i.e., the second preset threshold). If it exceeds, then the old data corresponding to the index is deleted hourly (i.e., if the size of the storage space occupied by the target user data is greater than the second preset threshold, then the third data to be processed in the first data is taken as the data to be processed).

[0137] The processing method provided by the embodiments of the present invention will be described exemplarily below:

[0138] (1) Cold and hot separation method: first create a new index to the cold node according to the day, in the format {business name-date} (that is, create a cold data index corresponding to the target business type and the current first preset time period as the first cold data index), such as cfw_ips-20220418. Then migrate the data of X days ago (determined by the time-space management method) of the hot node to the index marked with the corresponding date according to the tenant and period (such as hour).

[0139] The benefit of migrating data according to the small force period of time is to reduce the resource usage of the hot node and ensure stable operation of the business.

[0140] For example, in the hot node, the index cfw_ips stores the hot data corresponding to the target business type (i.e. cfw) in the target user data of the target user, and the current date is 2022-4-19.

[0141] First, create an index cfw_ips-20220419 (that is, create a cold data index corresponding to the target business type and the current first preset time period as the first cold data index) in the cold node according to the date, for storing cold data; the hot data saving time configured by the tenant user1 is 5 days (i.e. the first preset time period corresponding to the target user), assuming that the storage space has not reached the limit, then if the target storage time of the second data of the tenant in cfw_ips exceeds 5 days (i.e. if the target storage time is greater than the first preset time period corresponding to the target user), then the data of the tenant user1 five days ago needs to be migrated to the index cfw_ips-20220419 according to a certain period (assuming hourly) (that is, the first data within the first preset time period corresponding to the storage time is treated as the first data to be processed; the first data to be processed is grouped according to the preset time granularity and storage time; each group of data to be processed is migrated to the first cold data index in the cold node in order from early to late according to the storage time).

[0142] (2) Time-space management method: first define the control dimensions, mainly the cold and hot data space of each tenant (i.e. the first preset threshold and the second preset threshold), the cold and hot data saving time (i.e. the first preset time period); then manage the hot and cold data by day, if the hot data of the tenant reaches the hot data space limit (i.e. the first preset threshold) or the hot data saving time (i.e. the first preset time period), then migrate the hot data to the cold data by day; if the cold data of the tenant reaches the cold data space limit (i.e. the second preset threshold) or the cold data saving time (i.e. the first preset time period), then delete the cold data by day.

[0143] The space usage size of the cold data can be counted according to a statistical table of the hot data, that is, if the hot data is counted by hours in the statistical table, then the cold data of a day can be directly obtained by summing up each hour segment of the hot data of a day; if the hot data is counted by days in the statistical table, then the space size of the hot data of a day can be directly reused.

[0144] In addition, if there is a case that the amount of logs of the day is huge, that is, the user data of the day has exceeded the hot data space, then the exceeding part can be migrated to the cold data by a period (such as an hour).

[0145] In summary, the embodiment can realize the needs of limiting the cold and hot data space and limiting the cold and hot data saving time according to the needs of each tenant.

[0146] Embodiment Four

[0147] Figure 6 is a structural schematic diagram of a processing device provided by the embodiment four of the application, as shown in the figure, the device comprises: Figure 6

[0148] The first obtaining module 410 is configured to obtain a target storage duration of second data in the first data in a target node, the first data being data corresponding to a target business type in target user data of a target user;

[0149] The first determining module 420 is configured to determine, if the target storage duration is less than or equal to a first preset duration corresponding to the target user, the to-be-processed data in the first data according to a storage space size occupied by the target user data.

[0150] The processing module 430 is configured to process the to-be-processed data.

[0151] ​The processing device provided in the fourth embodiment of the present application comprises: a first obtaining module 410, configured to obtain a target storage duration of second data in the first data in a target node, wherein the first data is data corresponding to a target service type in target user data of a target user; a first determining module 420, configured to determine, if the target storage duration is less than or equal to a first preset duration corresponding to the target user, the to-be-processed data in the first data according to a storage space size occupied by the target user data; and a processing module 430, configured to process the to-be-processed data. By setting the first preset duration corresponding to the target user, the to-be-processed data can be processed according to the storage space size occupied by the target user data when the target storage duration of the second data in the first data in the target node is less than or equal to the first preset duration corresponding to the target user, so as to meet the control requirements of each user on the basis of realizing the first data life cycle management.

[0152] Optionally, the first data comprises hot data, and the target node comprises a hot node.

[0153] Optionally, the first determining module 420 comprises:

[0154] a first determining unit, configured to determine, if the storage space size occupied by the target user data is greater than a first preset threshold and the target storage duration of the second data is greater than a second preset duration, the first to-be-processed data in the first data as the to-be-processed data.

[0155] a second determining unit, configured to determine, if the storage space size occupied by the target user data is greater than the first preset threshold and the target storage duration of the second data is less than or equal to the second preset duration, the second to-be-processed data in the first data as the to-be-processed data.

[0156] Optionally, the processing device provided in the embodiment further comprises:

[0157] a second determining module, configured to determine, if the target storage duration is greater than the first preset duration corresponding to the target user, the first to-be-processed data in the first data as the to-be-processed data before processing the to-be-processed data.

[0158] Optionally, the first determining unit or the third determining unit is configured to:

[0159] determine a target hot data index corresponding to the target service type and first data stored in the target hot data index;

[0160] determine the first to-be-processed data stored in a first preset time period corresponding to the first data as the to-be-processed data.

[0161] Optionally, the second determining unit is specifically used for:

[0162] determining a target hot data index corresponding to the target service type and first data stored in the target hot data index;

[0163] taking second to-be-processed data, whose storage time is within a first preset time period corresponding to the target service type, as the to-be-processed data.

[0164] Optionally, the processing module 430 includes:

[0165] a migration unit, configured to migrate the to-be-processed data into a first cold data index in the cold node, the first cold data index corresponding to the target service type and a current first preset time period.

[0166] Optionally, the migration unit is specifically used for:

[0167] grouping the to-be-processed data according to a preset time granularity and storage time, wherein the preset time granularity corresponds to a time length smaller than a cycle length of the first preset time period;

[0168] migrating each group of to-be-processed data into the first cold data index in the cold node in turn according to a storage time from early to late.

[0169] Optionally, the processing module 430 further includes:

[0170] a creating unit, configured to, before the migration of the to-be-processed data into the first cold data index in the cold node, if there is no first cold data index in the cold node, create a cold data index corresponding to the target service type and the current first preset time period as the first cold data index.

[0171] Optionally, the first data includes cold data, and the target node includes a cold node.

[0172] Optionally, the first determining module 420 includes:

[0173] a fourth determining unit, configured to, if the storage space occupied by the target user data is greater than a second preset threshold, take third to-be-processed data in the first data as the to-be-processed data.

[0174] Optionally, the processing apparatus provided in the embodiment further includes:

[0175] The second obtaining module is configured to, before processing the to-be-processed data, if the target storage time length is greater than the first preset time length corresponding to the target user, obtain an earliest user data storage time length of each user in a second cold data index.

[0176] The determining module is configured to, if each of the earliest user data storage time lengths is greater than the first preset time length corresponding to the corresponding user, take the second cold data index as the to-be-processed data, or take third to-be-processed data in the first data as the to-be-processed data.

[0177] Optionally, the processing apparatus provided in the embodiment further includes:

[0178] The index determining module is configured to, after obtaining the target storage time length of the second data in the target node, if there is a cold data index corresponding to a service type corresponding to the second data and a storage time of the second data in the target node, take the cold data index corresponding to the service type corresponding to the second data and the storage time of the second data as a second cold data index.

[0179] Optionally, the fourth determining unit or the determining module includes:

[0180] The first data stored in the second cold data index is determined.

[0181] Third to-be-processed data with a storage time located within the first m preset time granularities corresponding to the first data is taken as to-be-processed data, where m is a positive integer, and a total time length corresponding to the first m preset time granularities is less than a cycle length of the first preset time cycle.

[0182] Optionally, the processing module 430 includes:

[0183] The deleting unit is configured to delete the to-be-processed data.

[0184] Optionally, the deleting unit is specifically configured to:

[0185] If the to-be-processed data is the third to-be-processed data, the to-be-processed data is grouped according to a preset time granularity and a storage time, where a time length corresponding to the preset time granularity is less than a cycle length of the first preset time cycle.

[0186] Each group of to-be-processed data is sequentially deleted in an order from early to late according to the storage time.

[0187] Optionally, the deleting unit is specifically configured to:

[0188] If the to-be-processed data is the second cold data index, the second cold data index is deleted.

[0189] Optionally, the processing apparatus provided by the embodiment further comprises:

[0190] The third determining module is configured to determine the second data in the first data according to storage time before a target storage duration of the second data in the first data in the target node.

[0191] The processing apparatus provided by the embodiment can execute the processing method provided by any of the embodiments of the application, and has the corresponding function modules and beneficial effects of the execution method.

[0192] Embodiment five

[0193] Figure 7 is a structural schematic diagram of an electronic device according to the embodiment five of the application. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown here, their connections, and their functions, are meant to be examples only, and are not intended to limit the implementations of the application described and / or claimed in this document.

[0194] As shown in Figure 7 The electronic device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11, wherein the memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0195] A plurality of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, a loudspeaker, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunications networks.

[0196] The processor 11 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, and the like. The processor 11 performs various methods and processes described above, such as the processing method.

[0197] In some embodiments, the processing method can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded onto the RAM 13 and executed by the processor 11, one or more steps of the processing method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the processing method by any other suitable means, such as by means of firmware.

[0198] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (PLD), a computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0199] Computer programs used to implement the processes of the application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed by the processor, implements the functions / acts specified in the flowcharts and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0200] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0201] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0202] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0203] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0204] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0205] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A processing method characterized by, The method comprises: acquiring a target storage duration of second data in first data in a target node, the first data being data corresponding to a target service type in target user data of a target user; if the target storage duration is less than or equal to a first preset duration corresponding to the target user, determining to-be-processed data in the first data according to a storage space size occupied by the target user data; processing the to-be-processed data; wherein, when the first data comprises hot data and the target node comprises a hot node, the determining to-be-processed data in the first data according to the storage space size occupied by the target user data comprises: if the storage space size occupied by the target user data is greater than a first preset threshold and the target storage duration of the second data is greater than a second preset duration, determining first to-be-processed data in the first data as to-be-processed data; and if the storage space size occupied by the target user data is greater than the first preset threshold and the target storage duration of the second data is less than or equal to the second preset duration, determining second to-be-processed data in the first data as to-be-processed data; the processing the to-be-processed data comprises: grouping the to-be-processed data according to a preset time granularity and storage time, wherein a time length corresponding to the preset time granularity is less than a cycle length of a first preset time cycle; and sequentially migrating each group of to-be-processed data to a first cold data index in a cold node in an order from early to late according to storage time, wherein the first cold data index corresponds to the target service type and a current first preset time cycle; when the first data comprises cold data and the target node comprises a cold node, the determining to-be-processed data in the first data according to the storage space size occupied by the target user data comprises: if the storage space size occupied by the target user data is greater than a second preset threshold, determining third to-be-processed data in the first data as to-be-processed data; the processing the to-be-processed data comprises: if the to-be-processed data is third to-be-processed data, grouping the to-be-processed data according to a preset time granularity and storage time, wherein a time length corresponding to the preset time granularity is less than a cycle length of a first preset time cycle; sequentially deleting each group of to-be-processed data in an order from early to late according to storage time; and if the to-be-processed data is a second cold data index, deleting the second cold data index.

2. The method of claim 1, wherein, before the processing the to-be-processed data, the method further comprises: if the target storage duration is greater than the first preset duration corresponding to the target user, determining first to-be-processed data in the first data as to-be-processed data.

3. The method according to claim 1 or 2, characterized in that, the determining first to-be-processed data in the first data as to-be-processed data comprises: determining a target hot data index corresponding to the target service type and first data stored in the target hot data index; determining first to-be-processed data having a storage time within a first preset time cycle corresponding to the first data as to-be-processed data.

4. The method of claim 1, wherein, The second to-be-processed data in the first data is taken as to-be-processed data, including: determining a target hot data index corresponding to the target service type and first data stored in the target hot data index; taking second to-be-processed data with a storage time within a first n preset time granularity corresponding to the first data as to-be-processed data, wherein n is a positive integer, and a total time length corresponding to the first n preset time granularity is less than a cycle length of a first preset time cycle.

5. The method of claim 1, wherein, Before migrating each group of to-be-processed data into the first cold data index in the cold node, further comprising: if the first cold data index does not exist in the cold node, creating a cold data index corresponding to the target service type and the current first preset time cycle as the first cold data index.

6. The method of claim 1, wherein, Before processing the to-be-processed data, further comprising: if the target storage duration is greater than the first preset duration corresponding to the target user, obtaining the earliest user data storage duration of each user in the second cold data index; if each earliest user data storage duration is greater than the first preset duration corresponding to the corresponding user, taking the second cold data index as to-be-processed data, otherwise, taking third to-be-processed data in the first data as to-be-processed data.

7. The method according to claim 1 or 6, characterized in that, After obtaining the target storage duration of the second data in the first data in the target node, further comprising: if the target node exists a cold data index corresponding to the service type corresponding to the second data and the storage time of the second data, the cold data index corresponding to the service type corresponding to the second data and the time to which the second data belongs is recorded as the second cold data index.

8. The method according to claim 1 or 6, characterized in that, The third to-be-processed data in the first data is taken as to-be-processed data, including: determining first data stored in the second cold data index; taking third to-be-processed data with a storage time within a first m preset time granularity corresponding to the first data as to-be-processed data, wherein m is a positive integer, and a total time length corresponding to the first m preset time granularity is less than a cycle length of a first preset time cycle.

9. The method of claim 1, wherein, Before obtaining the target storage duration of the second data in the first data in the target node, further comprising: determining the second data in the first data according to the storage time.

10. A processing device, characterized by The device comprises: an obtaining module, configured to obtain a target storage duration of second data in first data in a target node, the first data being data corresponding to a target service type in target user data of a target user; a determining module, configured to determine to-be-processed data in the first data according to a storage space size occupied by the target user data if the target storage duration is less than or equal to a first preset duration corresponding to the target user; a processing module, configured to process the to-be-processed data; wherein, when the first data comprises hot data and the target node comprises a hot node, The determining module comprises: a first determining unit, configured to determine first to-be-processed data in the first data as to-be-processed data if the size of the storage space occupied by the target user data is greater than a first preset threshold and the target storage duration of the second data is greater than a second preset duration; and a second determining unit, configured to determine second to-be-processed data in the first data as to-be-processed data if the size of the storage space occupied by the target user data is greater than the first preset threshold and the target storage duration of the second data is less than or equal to the second preset duration. The processing module comprises: a migration unit, configured to group the to-be-processed data according to a preset time granularity and storage time, wherein the time length corresponding to the preset time granularity is less than the cycle length of a first preset time cycle; and sequentially migrate each group of to-be-processed data into a first cold data index in the cold node in an order from early to late according to the storage time, wherein the first cold data index corresponds to the target business type and a current first preset time cycle. When the first data comprises cold data and the target node comprises a cold node, The determining module comprises: a fourth determining unit, configured to determine third to-be-processed data in the first data as to-be-processed data if the size of the storage space occupied by the target user data is greater than a second preset threshold. The processing module comprises: a deletion unit, configured to group the to-be-processed data according to a preset time granularity and storage time if the to-be-processed data is third to-be-processed data, wherein the time length corresponding to the preset time granularity is less than the cycle length of a first preset time cycle; sequentially delete each group of to-be-processed data in an order from early to late according to the storage time; and delete the second cold data index if the to-be-processed data is the second cold data index.

11. An electronic device, comprising: The electronic device comprises: at least one processor; and a memory connected with the at least one processor in communication; wherein The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the processing method in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for enabling the processor to execute the processing method in any one of claims 1-9 when executed.

Images