Data processing method, device and computer equipment for target object

Abstract

The application relates to the computer technical field, in particular to a target object data processing method, device and computer equipment. The target object data processing method comprises the following steps: obtaining a standard estimation model and a hierarchical estimation model of a target object, the standard estimation model is used for calculating the occupied resource amount of the target object in a standard mode, the hierarchical estimation model is used for calculating the occupied resource amount of the target object in a hierarchical mode, and in the standard mode, the target object is stored in a high-frequency layer, and in the hierarchical mode, at least part of the target object is stored in a low-frequency layer; based on the standard estimation model and the hierarchical estimation model, a balance point model is calculated, the balance point model is used for obtaining a balance function of the occupied resource amount of the target object in the standard mode and the hierarchical mode; storage information of the target object is acquired, and based on the storage information and the balance point model, one of the standard estimation model and the hierarchical estimation model is determined to be used for scheduling the target object.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a target object data processing method and device and computer equipment. BACKGROUND

[0002] In recent years, cloud computing technology has developed rapidly, and a large number of cloud computing systems have been applied. Due to the accessibility, low cost and efficient computing characteristics of cloud computing services, the number of individual, small business and large-scale industrial users has grown at an unprecedented rate in the past few years.

[0003] In the related art, data storage technology provides different levels of storage device capacity and performance, and obtains the best performance at the lowest cost. However, the access hotness of data is constantly changing, which requires real-time adjustment of data storage strategy. However, the existing scheduling algorithm does not fully consider the different needs of users for resources, and cannot well provide resources for users to meet their needs. SUMMARY

[0004] Therefore, it is necessary to provide a target object data processing method, device and computer equipment capable of quickly evaluating a data storage scheme.

[0005] In a first aspect, the present application provides a target object data processing method, the method comprising:

[0006] obtaining a standard estimation model and a hierarchical estimation model of a target object, the standard estimation model being used to calculate the resource occupation amount of the target object in a standard mode, and the hierarchical estimation model being used to calculate the resource occupation amount of the target object in a hierarchical mode, and in the standard mode, the target object is stored in a high-frequency layer, and in the hierarchical mode, at least part of the target object is stored in a low-frequency layer;

[0007] based on the standard estimation model and the hierarchical estimation model, calculating a balance point model, the balance point model being used to obtain a balance function of the resource occupation amount of the target object in the standard mode and the hierarchical mode;

[0008] obtaining storage information of the target object, and based on the storage information and the balance point model, determining to use one of the standard estimation model and the hierarchical estimation model to schedule the target object.

[0009] In one embodiment, the obtaining the storage information of the target object, and based on the storage information and the balance point model, determining to use one of the standard estimation model and the hierarchical estimation model to schedule the target object, comprises:

[0010] In the case of scheduling the target object using the hierarchical estimation model, periodically scheduling at least part of the target object to a low-frequency layer.

[0011] In one of the embodiments, the obtaining the storage information of the target object, and determining to schedule the target object using one of the standard estimation model and the hierarchical estimation model based on the balance point model and the storage information, comprises:

[0012] Obtaining any two of the average occupancy, the access frequency, and the storage time of the target object.

[0013] In one of the embodiments, the target object comprises a plurality of sub-objects.

[0014] The obtaining the standard estimation model and the hierarchical estimation model of the target object comprises:

[0015] Obtaining a hierarchical initial estimation model.

[0016] Determining the occupied resource amount of each of the sub-objects.

[0017] Based on the occupied resource amount of the sub-objects and the preset average occupancy, determining the object monitoring parameter in the hierarchical initial estimation model to obtain the hierarchical estimation model.

[0018] In one of the embodiments, the determining the object monitoring parameter in the hierarchical initial estimation model based on the occupied resource amount of the sub-objects and the preset average occupancy comprises:

[0019] In the case that the occupied resource amount of the plurality of sub-objects is not greater than the preset average occupancy, dividing the sub-objects whose occupied resource amount is not greater than the preset average occupancy into a sub-object combination.

[0020] In the case that the occupied resource amount of the sub-object combination is greater than the preset average occupancy, calculating the object monitoring parameter in the hierarchical initial estimation model based on the occupied resource amount of the sub-object combination to obtain the hierarchical estimation model.

[0021] In one of the embodiments, the determining the object monitoring parameter in the hierarchical initial estimation model based on the occupied resource amount of the sub-objects and the preset average occupancy comprises:

[0022] In the case that the occupied resource amount of the sub-objects is greater than the preset average occupancy, calculating the object monitoring parameter in the hierarchical initial estimation model based on the preset average occupancy to obtain the hierarchical estimation model.

[0023] In a second aspect, the application further provides a data processing device of a target object, which comprises:

[0024] an obtaining module, configured to obtain a standard estimation model and a hierarchical estimation model of a target object, the standard estimation model being used to calculate an occupied resource amount of the target object in a standard mode, the hierarchical estimation model being used to calculate an occupied resource amount of the target object in a hierarchical mode, and in the standard mode, the target object is stored in a high-frequency layer, and in the hierarchical mode, at least part of the target object is stored in a low-frequency layer;

[0025] a calculating module, configured to calculate a balance point model based on the standard estimation model and the hierarchical estimation model, the balance point model being used to obtain a balance function of the occupied resource amounts of the target object in the standard mode and the hierarchical mode;

[0026] a determining module, configured to obtain storage information of the target object, and determine to schedule the target object by using one of the standard estimation model and the hierarchical estimation model based on the storage information and the balance point model.

[0027] In a third aspect, a computer device is provided, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0028] obtaining a standard estimation model and a hierarchical estimation model of a target object, the standard estimation model being used to calculate an occupied resource amount of the target object in a standard mode, the hierarchical estimation model being used to calculate an occupied resource amount of the target object in a hierarchical mode, and in the standard mode, the target object is stored in a high-frequency layer, and in the hierarchical mode, at least part of the target object is stored in a low-frequency layer;

[0029] calculating a balance point model based on the standard estimation model and the hierarchical estimation model, the balance point model being used to obtain a balance function of the occupied resource amounts of the target object in the standard mode and the hierarchical mode;

[0030] obtaining storage information of the target object, and determining to schedule the target object by using one of the standard estimation model and the hierarchical estimation model based on the storage information and the balance point model.

[0031] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the following steps:

[0032] obtain a standard estimation model and a hierarchical estimation model of the target object, the standard estimation model being used to calculate an occupied resource amount of the target object in a standard mode, the hierarchical estimation model being used to calculate an occupied resource amount of the target object in a hierarchical mode, and in the standard mode, the target object is stored in a high-frequency layer, and in the hierarchical mode, at least part of the target object is stored in a low-frequency layer;

[0033] calculate a balance point model based on the standard estimation model and the hierarchical estimation model, the balance point model being used to obtain a balance function of the occupied resource amounts of the target object in the standard mode and the hierarchical mode;

[0034] obtain storage information of the target object, and determine to schedule the target object using one of the standard estimation model and the hierarchical estimation model based on the storage information and the balance point model.

[0035] In a fifth aspect, the present application further provides a computer program product, comprising a computer program which, when executed by a processor, implements the following steps:

[0036] obtain a standard estimation model and a hierarchical estimation model of the target object, the standard estimation model being used to calculate an occupied resource amount of the target object in a standard mode, the hierarchical estimation model being used to calculate an occupied resource amount of the target object in a hierarchical mode, and in the standard mode, the target object is stored in a high-frequency layer, and in the hierarchical mode, at least part of the target object is stored in a low-frequency layer;

[0037] calculate a balance point model based on the standard estimation model and the hierarchical estimation model, the balance point model being used to obtain a balance function of the occupied resource amounts of the target object in the standard mode and the hierarchical mode;

[0038] obtain storage information of the target object, and determine to schedule the target object using one of the standard estimation model and the hierarchical estimation model based on the storage information and the balance point model.

[0039] The data processing method, device, computer device, computer readable storage medium and computer program product of the target object described above, by calculating a balance point model based on a standard estimation model and a hierarchical estimation model, can obtain a more suitable estimation model, and also make the storage data process have more diversified and intelligent evaluation indexes, and thus can clearly analyze the storage strategy and the actual cost. At the same time, at least part of the target objects can be regularly scheduled to the low-frequency layer by using the hierarchical estimation model, and the storage cost can also be reduced. In addition, the related information of the target object can also be collected when used, and then the parameters of the hierarchical estimation model are recalculated, so as to timely adjust the hierarchical estimation model, and further save the storage cost. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a flowchart of the data processing method for the target object provided in the first embodiment;

[0042] Figure 2 This is a flowchart of the data processing method for the target object provided in the second embodiment;

[0043] Figure 3 This is a flowchart of the data processing method for the target object provided in the third embodiment;

[0044] Figure 4 This is a flowchart of the data processing method for the target object provided in the fourth embodiment;

[0045] Figures 5 to 14 Schematic diagrams of standard and layered modes provided in different embodiments;

[0046] Figure 15 A diagram of a target object classification selector model provided in one embodiment;

[0047] Figure 16 Here is a diagram of a scheduling center model provided in one embodiment;

[0048] Figure 17 Here is a flowchart of a hierarchical scheduling model provided in one embodiment;

[0049] Figure 18 A structural block diagram of a data processing apparatus for a target object provided in one embodiment;

[0050] Figure 19 This is an internal structural diagram of a computer device provided in one embodiment. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0052] Please see Figures 1 to 4Different embodiments of the present application provide a data processing method of target objects, which can be applied to a server, a system comprising a terminal and a server, and realized through the interaction between the terminal and the server. The terminal can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, Internet of Things devices, etc. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0053] In one embodiment, as shown in Figure 1 A data processing method of target objects is provided, and this embodiment is exemplified by the method applied to a terminal. The data processing method of target objects comprises the following steps:

[0054] Step S100: obtaining a standard estimation model and a hierarchical estimation model of target objects, the standard estimation model being used to calculate the resource occupation amount of target objects in a standard mode, the hierarchical estimation model being used to calculate the resource occupation amount of target objects in a hierarchical mode, and in the standard mode, the target objects are stored in a high-frequency layer, and in the hierarchical mode, at least part of the target objects are stored in a low-frequency layer.

[0055] Step S200: calculating a balance point model based on the standard estimation model and the hierarchical estimation model, the balance point model being used to obtain a balance function of the resource occupation amount of target objects in the standard mode and the hierarchical mode.

[0056] Step S300: obtaining storage information of target objects, and determining to schedule the target objects using one of the standard estimation model and the hierarchical estimation model based on the storage information and the balance point model.

[0057] In step S100, the target objects can include stored data. Specifically, the target objects can include images, files, videos, etc. The resource occupation amount can include the data size of the target objects, or the storage cost of the target objects, etc.

[0058] The standard estimation model can be used to calculate the resource occupation amount of target objects in the standard mode. Specifically, the standard estimation model can be used to estimate the standard storage cost of target objects. As an example, in the standard mode, the target objects are stored in the high-frequency layer. In the high-frequency layer, the target objects can have faster access speed, while the target objects also have higher storage cost.

[0059] The hierarchical estimation model can be used to calculate the resource occupation amount of the target objects in the hierarchical mode. Specifically, the hierarchical estimation model can be used to estimate the intelligent hierarchical storage cost of the target objects. As an example, in the hierarchical mode, part of the target objects are stored in the high-frequency layer, and part of the target objects are stored in the low-frequency layer (the settlement layer). The target objects in the high-frequency layer can have faster access speed, and the target objects in the low-frequency layer can have lower access speed. Moreover, part of the target objects can be regularly stored to the low-frequency layer. Of course, the storage cost of the low-frequency layer can be lower than that of the high-frequency layer.

[0060] In step S200, the balance point model is used to obtain the balance function of the resource occupation amount of the target objects in the standard mode and the hierarchical mode. Specifically, the standard estimation model and the hierarchical estimation model can be combined to calculate the function when the standard storage cost obtained by the standard estimation model is consistent with the hierarchical storage cost obtained by the hierarchical estimation model, that is, the balance point model.

[0061] In step S300, the storage information of the target objects can include any two of the average occupation amount, the access frequency, and the storage time of the target objects. Then, any two of the average occupation amount, the access frequency, and the storage time of the target objects are brought into the balance point model to determine the balance value of the third party. Moreover, based on the balance value, it can be determined whether to use the standard estimation model or the hierarchical estimation model.

[0062] Then, the obtained balance point value can be sent to the client. In the case that the standard mode is triggered, the target objects are stored in the standard mode. In the case that the hierarchical mode is triggered, at least part of the target objects are regularly scheduled to the low-frequency layer. It can be understood that in the case that it is determined to use the standard estimation model, it means that the cost of storing the target objects in the high-frequency layer is lower. In the case that it is determined to use the hierarchical estimation model, it means that the cost of storing the target objects in the high-frequency layer and the low-frequency layer is lower. Further, in the case that the hierarchical estimation model is used to schedule the target objects, at least part of the target objects are regularly scheduled to the low-frequency layer. As an example, at least part of the target objects can be regularly scheduled to the low-frequency layer every week or every month.

[0063] In this embodiment, by calculating the balance point model based on the standard estimation model and the hierarchical estimation model, a more suitable estimation model can be obtained, and the storage data process has more diversified and intelligent evaluation indexes, and the storage strategy and the actual cost can be clearly analyzed. At the same time, in this embodiment, by using the hierarchical estimation model, at least part of the target objects are regularly scheduled to the low-frequency layer, and the storage cost can also be reduced. In addition, in this embodiment, the related information of the target objects can also be collected when the hierarchical estimation model is used, and then the parameters of the hierarchical estimation model are recalculated, so that the hierarchical estimation model is adjusted in time, and the storage cost is further saved.

[0064] In one embodiment, the target object includes a plurality of sub-objects. It can be understood that the target object can include a plurality of pictures, videos, files, databases, etc. At this time, as shown in Figure 2 Step S100 can include:

[0065] Step S110: obtaining a layered initial estimation model.

[0066] Step S120: determining an occupied resource amount of each sub-object.

[0067] Step S130: determining an object monitoring parameter in the layered initial estimation model based on the occupied resource amount of the sub-objects and a preset average occupation amount, and obtaining a layered estimation model.

[0068] In step S110, the layered initial estimation model can be a basic, unrefined layered estimation model. The layered initial estimation model can only define the basic logic and structure of the layered storage estimation of the target object.

[0069] In step S120, each sub-object can have a certain occupied resource amount. The embodiment does not limit the range of the occupied resource amount of the sub-objects.

[0070] In step S130, the object monitoring parameter can be used to represent the cost required for monitoring the sub-objects. At this time, in the case of a large number of sub-objects, the cost required for monitoring the sub-objects can be large. In the case of a small number of sub-objects, the cost required for monitoring the sub-objects can be small.

[0071] In the embodiment, by analyzing the occupied resource amount of the sub-objects in detail, an accurate and reliable layered estimation model is ultimately obtained, so that a more suitable storage model can be selected.

[0072] Specifically, as shown in Figure 3 and Figure 4 Step S130 can include:

[0073] Step S131: in the case where the occupied resource amounts of the plurality of sub-objects are not greater than the preset average occupation amount, dividing the sub-objects whose occupied resource amounts are not greater than the preset average occupation amount into a sub-object combination.

[0074] Step S132: in the case where the occupied resource amount of the sub-object combination is greater than the preset average occupation amount, calculating the object monitoring parameter in the layered initial estimation model based on the occupied resource amount of the sub-object combination, and obtaining the layered estimation model.

[0075] Step S133: in the case where the occupied resource amount of the sub-object is greater than the preset average occupation amount, calculating the object monitoring parameter in the layered initial estimation model based on the preset average occupation amount, and obtaining the layered estimation model.

[0076] In steps S131 to S133, the preset average occupancy amount can be the preset occupancy amount. The embodiment does not limit the specific range of the preset average occupancy amount. In the case where the occupancy resource amount of the plurality of sub-objects is not greater than the preset average occupancy amount, it indicates that the data amount of the sub-object is small. At this time, the plurality of sub-objects whose occupancy resource amount is not greater than the preset average occupancy amount can be divided into a sub-object combination, so as to combine the plurality of sub-objects with small data amount into a sub-object combination with large data amount, and further reduce the cost required for monitoring the sub-objects. Of course, in the case where the occupancy resource amount of the sub-object is greater than the preset average occupancy amount, the calculation can be directly performed.

[0077] As an example, in the case where the occupancy resource amount of the first sub-object is not greater than the preset average occupancy amount, the first sub-object can be made to wait. Then, in the case where a second sub-object whose occupancy resource amount is not greater than the preset average occupancy amount appears, the first sub-object and the second sub-object can be divided into a sub-object combination, and the occupancy resource amount of the sub-object combination is compared with the preset average occupancy amount. In the case where the occupancy resource amount of the sub-object combination is not greater than the preset average occupancy amount, the first sub-object and the second sub-object are controlled to continue to wait. In the case where the occupancy resource amount of the sub-object combination is greater than the preset average occupancy amount, the object monitoring parameter in the hierarchical initial estimation model is calculated using the sub-object combination.

[0078] In the embodiment, by comparing the occupancy resource amount of the sub-object with the preset average occupancy amount, the cost required for monitoring the sub-objects is reduced, and further the cost of using the hierarchical estimation model is reduced.

[0079] The specific implementation of the present application is described below by way of example. It can be understood that the following functions and values are only illustrative and do not limit the protection scope of the present application.

[0080] The standard estimation model can be represented as: F=Q×1024×a 标准 ×M=1024QaM.

[0081] The hierarchical initial estimation model can be represented as:

[0082]

[0083] Wherein, Q can represent the data amount of the target object as QTB. The average size of the sub-object (file) is b GB. The storage capacity cost of the target object is a yuan / GB / month. The hierarchical object monitoring cost is c yuan per 10,000 objects. In the hierarchical mode, the storage type is converted to low-frequency access for 30 days, and the data storage time is M×30 days, where M represents the number of months of storage.

[0084] Assuming there is no access operation to the data that has been settled, a fixed number of files (set the proportion of the number to the original number as s) will be settled to the low frequency layer every month, and will not be accessed back to high frequency data, so at most months to complete the settlement of all data, the stratified estimation model can be:

[0085]

[0086] Wherein, 1024Qa standard can represent the cost of the first month, the first month there is no data sinking.

[0087] The joint standard estimation model and stratified estimation model can be obtained:

[0088]

[0089] Wherein, sM≤1 to ensure that no more than 100% of the data is settled.

[0090] Assuming the standard storage cost is a standard = 0.118 yuan / GB / month, the low frequency storage cost is a low frequency = 0.08 yuan / GB / month, and the stratified object monitoring cost is c = 0.175 yuan per 10,000 objects. If there are Q = 1TB files to be stored, the following can be obtained:

[0091]

[0092] Based on the above model, the corresponding schematic diagram can be obtained Figures 5 to 14 . Among them, the blue surface represents the storage cost in the standard mode, and the red surface represents the storage cost in the stratified mode.

[0093] First, please refer to Figure 5 and Figure 6 , in the b-axis direction, when the value of b (average file size) is relatively large Figure 5 , the influence on the curve is basically negligible, and when the value of b is relatively small Figure 6 , the influence on the cost is relatively large. Specifically, the change of b (average file size) in the range of 64KB-0.05GB is relatively meaningful.

[0094] Second, please refer to Figures 7 to 8 . In the s-axis direction, when the settlement rate is too large (for example, more than 20%), when the storage time is relatively large, the model will cause the cost reduction to be negative, which is contrary to the fact. Therefore, we set sM≤1 to ensure that no more than 100% of the data is settled.

[0095] Third, with the increase of storage time M, the cost of the two storage modes is as follows Figure 9 , Figure 10 ,Figure 11 , Figure 12 , Figure 13 and Figure 14 . Except for the first month, the cost of tiered storage will be easier to lower than the standard storage as the storage time increases.

[0096] Solve the following equation:

[0097]

[0098] The equilibrium point model is:

[0099]

[0100] The equilibrium point model shows that when two of the average file size b, access frequency s, and storage time M are known, the equilibrium point between the cost of the standard mode and the tiered mode can be calculated. For example, in the fixed quantity settlement mode, the intelligent storage cost is inversely proportional to the average file, the settlement ratio, and the storage time.

[0101] Specifically, in the first possible case, the average file size is known to be 0.005 GB, and the storage time is 12 months, s = 0.0167 = 1.67% can be calculated, which indicates that if more than 1.67% of the data is settled every month, the cost of tiered storage will be lower than that of standard storage within 12 months. In the second possible case, the average file size is known to be 0.005 GB, and 10% of the data is settled every month, M = 2.84 can be calculated, which indicates that if the storage time is greater than 2.84 months in this scenario, the cost of tiered storage will be lower than that of standard storage. In the third possible case, 5% of the data is settled every month, and the storage time is 12 months, the average file size b = 0.0016735 GB (1.714 MB) can be calculated, so as long as the average file is greater than 1.714 MB, the cost of tiered storage will be lower than that of standard storage in the case of 12 months of storage.

[0102] Further, please refer to Figure 15 , Figure 16 and Figure 17 , for the data processing system related to the target object of the present application, a classifier can be trained according to a historical data set, the classifier receives target object data collected by the client, identifies the data type, and transmits the calculated results to form a tiered estimation model. The parameters calculated are brought into the tiered initial estimation model, and the scheduler calculates the optimal cost scheme and file scheduling rules according to the model, and regularly executes the scheduling target data. The present application can also set up a monitoring center to monitor each sub-object, feedback the situation regularly, recalculate the intelligent tiering model parameters, and adjust the tiered estimation model in a timely manner.

[0103] It should be understood that although the steps in the flowcharts involved in the embodiments described above are shown in sequence according to the arrows, the steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, the execution of the steps is not strictly limited in sequence, and the steps can be executed in other sequences. Moreover, at least some of the steps in the flowcharts involved in the embodiments described above can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of the steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or steps or stages in other steps.

[0104] Based on the same inventive concept, the embodiments of the present application also provide a target object data processing device for implementing the target object data processing method described above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more target object data processing device embodiments provided below can refer to the limitations of the target object data processing method described above, and will not be repeated here.

[0105] In one embodiment, as shown in Figure 18 A target object data processing device is provided, comprising: an acquisition module, a calculation module, and a determination module, wherein:

[0106] The acquisition module is configured to obtain a standard estimation model and a hierarchical estimation model of the target object, the standard estimation model being used to calculate the resource occupancy of the target object in a standard mode, and the resource occupancy of the target object in a hierarchical mode, and in the standard mode, the target object is stored in a high-frequency layer, and in the hierarchical mode, at least part of the target object is stored in a low-frequency layer.

[0107] The calculation module is configured to calculate a balance point model based on the standard estimation model and the hierarchical estimation model, the balance point model being used to obtain a balance function of the resource occupancy of the target object in the standard mode and the hierarchical mode.

[0108] The determination module is configured to obtain storage information of the target object, and determine to schedule the target object using one of the standard estimation model and the hierarchical estimation model based on the storage information and the balance point model.

[0109] In one embodiment, the determination module is configured to periodically schedule at least part of the target object to the low-frequency layer in the case of scheduling the target object using the hierarchical estimation model.

[0110] In one embodiment, the determination module is configured to obtain any two of the average occupancy, the access frequency, and the storage time of the target object.

[0111] In one embodiment, the target object includes a plurality of sub-objects. The obtaining module is configured to obtain a hierarchical initial estimation model; determine an occupied resource amount of each sub-object; and determine an object monitoring parameter in the hierarchical initial estimation model based on the occupied resource amount of the sub-object and a preset average occupancy amount, to obtain a hierarchical estimation model.

[0112] In one embodiment, the obtaining module is configured to, in a case where the occupied resource amounts of the plurality of sub-objects are not greater than the preset average occupancy amount, divide the plurality of sub-objects whose occupied resource amounts are not greater than the preset average occupancy amount into a sub-object combination; and in a case where the occupied resource amount of the sub-object combination is greater than the preset average occupancy amount, calculate the object monitoring parameter in the hierarchical initial estimation model based on the occupied resource amount of the sub-object combination, to obtain the hierarchical estimation model.

[0113] In one embodiment, the obtaining module is configured to, in a case where the occupied resource amount of the sub-object is greater than the preset average occupancy amount, calculate the object monitoring parameter in the hierarchical initial estimation model based on the preset average occupancy amount, to obtain the hierarchical estimation model.

[0114] The above modules in the data processing apparatus of the target object can be realized by software, hardware, or a combination thereof. The above modules can be embedded in or independent of a processor in a computer device in a hardware form, or stored in a memory in a computer device in a software form, so as to be called and executed by a processor to perform operations corresponding to the above modules.

[0115] In addition, the data processing apparatus of the target object can also have other modules, such as a first module, a second module, etc., for processing other steps and methods provided in the present application.

[0116] In one exemplary embodiment, a computer device is provided, which can be a server, and an internal structure diagram of the computer device can be as shown in FIG. 1. Figure 19As shown in the figure. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through the system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the data processing data of the target object. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with the terminal outside through the network connection. The computer program is executed by the processor to realize a data processing method of a target object.

[0117] Those skilled in the art can understand that, Figure 19 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0118] In one exemplary embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the following steps:

[0119] Step S100: obtaining a standard estimation model and a hierarchical estimation model of a target object, the standard estimation model being used to calculate the resource occupation amount of the target object in a standard mode, and the hierarchical estimation model being used to calculate the resource occupation amount of the target object in a hierarchical mode, and in the standard mode, the target object is stored in a high-frequency layer, and in the hierarchical mode, at least part of the target object is stored in a low-frequency layer.

[0120] Step S200: based on the standard estimation model and the hierarchical estimation model, calculating a balance point model, the balance point model being used to obtain a balance function of the resource occupation amount of the target object in the standard mode and the hierarchical mode.

[0121] Step S300: obtaining storage information of the target object, and based on the storage information and the balance point model, determining to schedule the target object using one of the standard estimation model and the hierarchical estimation model.

[0122] In one embodiment, the processor executing the computer program further implements the following steps:

[0123] In the case of scheduling the target objects using the hierarchical estimation model, periodically schedule at least part of the target objects to the low frequency layer.

[0124] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0125] Obtain any two of the average occupancy, the access frequency and the storage time of the target objects.

[0126] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0127] Step S110: Obtain the hierarchical initial estimation model.

[0128] Step S120: Determine the occupied resource amount of each sub-object.

[0129] Step S130: Based on the occupied resource amount of the sub-objects and the preset average occupancy, determine the object monitoring parameter in the hierarchical initial estimation model to obtain the hierarchical estimation model.

[0130] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0131] Step S131: In the case that the occupied resource amount of the plurality of sub-objects is not greater than the preset average occupancy, divide the sub-objects whose occupied resource amount is not greater than the preset average occupancy into a sub-object combination.

[0132] Step S132: In the case that the occupied resource amount of the sub-object combination is greater than the preset average occupancy, based on the occupied resource amount of the sub-object combination, calculate the object monitoring parameter in the hierarchical initial estimation model to obtain the hierarchical estimation model.

[0133] Step S133: In the case that the occupied resource amount of the sub-object is greater than the preset average occupancy, based on the preset average occupancy, calculate the object monitoring parameter in the hierarchical initial estimation model to obtain the hierarchical estimation model.

[0134] In one embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program, when executed by a processor, implements the following steps:

[0135] Step S100: Obtain the standard estimation model and the hierarchical estimation model of the target objects, the standard estimation model is used to calculate the occupied resource amount of the target objects in the standard mode, the hierarchical estimation model is used to calculate the occupied resource amount of the target objects in the hierarchical mode, and in the standard mode, the target objects are stored in the high frequency layer, and in the hierarchical mode, at least part of the target objects are stored in the low frequency layer.

[0136] Step S200: calculating a balance point model based on the standard estimation model and the hierarchical estimation model, the balance point model being used to obtain a balance function of the target object occupying resource amounts in the standard mode and the hierarchical mode.

[0137] Step S300: obtaining storage information of the target object, and determining to schedule the target object using one of the standard estimation model and the hierarchical estimation model based on the storage information and the balance point model.

[0138] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0139] In the case of scheduling the target object using the hierarchical estimation model, at least part of the target objects are regularly scheduled to the low-frequency layer.

[0140] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0141] Obtaining any two of the average occupation amount, the access frequency, and the storage time of the target object.

[0142] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0143] Step S110: obtaining a hierarchical initial estimation model.

[0144] Step S120: determining the occupation resource amount of each sub-object.

[0145] Step S130: determining the object monitoring parameter in the hierarchical initial estimation model based on the occupation resource amount of the sub-object and the preset average occupation amount, and obtaining the hierarchical estimation model.

[0146] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0147] Step S131: in the case that the occupation resource amounts of the plurality of sub-objects are not greater than the preset average occupation amount, dividing the plurality of sub-objects whose occupation resource amounts are not greater than the preset average occupation amount into a sub-object combination.

[0148] Step S132: in the case that the occupation resource amount of the sub-object combination is greater than the preset average occupation amount, calculating the object monitoring parameter in the hierarchical initial estimation model based on the occupation resource amount of the sub-object combination, and obtaining the hierarchical estimation model.

[0149] Step S133: in the case that the occupation resource amount of the sub-object is greater than the preset average occupation amount, calculating the object monitoring parameter in the hierarchical initial estimation model based on the preset average occupation amount, and obtaining the hierarchical estimation model.

[0150] In one embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the following steps:

[0151] Step S100: obtaining a standard estimation model and a hierarchical estimation model of the target object, the standard estimation model being used to calculate the occupied resource amount of the target object in a standard mode, and the hierarchical estimation model being used to calculate the occupied resource amount of the target object in a hierarchical mode, and in the standard mode, the target object is stored in a high-frequency layer, and in the hierarchical mode, at least part of the target object is stored in a low-frequency layer.

[0152] Step S200: based on the standard estimation model and the hierarchical estimation model, calculating a balance point model, the balance point model being used to obtain a balance function of the occupied resource amount of the target object in the standard mode and the hierarchical mode.

[0153] Step S300: obtaining storage information of the target object, and based on the storage information and the balance point model, determining to schedule the target object using one of the standard estimation model and the hierarchical estimation model.

[0154] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0155] In the case of scheduling the target object using the hierarchical estimation model, at least part of the target object is periodically scheduled to the low-frequency layer.

[0156] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0157] Obtaining any two of the average occupied amount, the access frequency, and the storage time of the target object.

[0158] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0159] Step S110: obtaining a hierarchical initial estimation model.

[0160] Step S120: determining the occupied resource amount of each sub-object.

[0161] Step S130: based on the occupied resource amount of the sub-object and the preset average occupied amount, determining the object monitoring parameter in the hierarchical initial estimation model to obtain the hierarchical estimation model.

[0162] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0163] Step S131: in the case that the occupied resource amount of the plurality of sub-objects is not greater than the preset average occupied amount, dividing the plurality of sub-objects whose occupied resource amount is not greater than the preset average occupied amount into a sub-object combination.

[0164] Step S132: in the case that the occupied resource amount of the sub-object combination is greater than the preset average occupancy amount, the object monitoring parameter in the hierarchical initial estimation model is calculated based on the occupied resource amount of the sub-object combination, and a hierarchical estimation model is obtained.

[0165] Step S133: in the case that the occupied resource amount of the sub-object is greater than the preset average occupancy amount, the object monitoring parameter in the hierarchical initial estimation model is calculated based on the preset average occupancy amount, and a hierarchical estimation model is obtained. It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0166] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0167] The technical features of the above embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present application.

[0168] The above-described embodiments are merely illustrative of several embodiments of the present application, which are described in more detail and in a specific manner, but should not be construed as limiting the scope of the patent of the present application. It should be noted that, for those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A data processing method for a target object, characterized in that, The method includes: Obtain a standard estimation model and a hierarchical estimation model for the target object. The standard estimation model is used to calculate the resource consumption of the target object in the standard mode, and the hierarchical estimation model is used to calculate the resource consumption of the target object in the hierarchical mode. In the standard mode, the target object is stored in the high-frequency layer, and in the hierarchical mode, at least a portion of the target object is stored in the low-frequency layer. Based on the standard estimation model and the hierarchical estimation model, a balance point model is calculated. The balance point model is used to obtain a balance function of the amount of resources occupied by the target object in the standard mode and the hierarchical mode. Obtain the storage information of the target object, and based on the storage information and the equilibrium point model, determine whether to use the standard estimation model or the hierarchical estimation model to schedule the target object; The target object includes multiple sub-objects; The standard estimation model and hierarchical estimation model for obtaining the target object include: Obtain the hierarchical initial estimation model; Determine the resource consumption of each of the sub-objects; Based on the resource usage of the sub-objects and the preset average resource usage, the object monitoring parameters in the hierarchical initial estimation model are determined to obtain the hierarchical estimation model, including: when the resource usage of multiple sub-objects is not greater than the preset average resource usage, dividing the multiple sub-objects whose resource usage is not greater than the preset average resource usage into sub-object combinations; when the resource usage of the sub-object combination is greater than the preset average resource usage, calculating the object monitoring parameters in the hierarchical initial estimation model based on the resource usage of the sub-object combination to obtain the hierarchical estimation model.

2. The data processing method for the target object according to claim 1, characterized in that, The step of obtaining the storage information of the target object and determining, based on the storage information and the equilibrium point model, to schedule the target object using either the standard estimation model or the hierarchical estimation model includes: When scheduling the target objects using the hierarchical estimation model, at least a portion of the target objects will be periodically scheduled to the low-frequency layer.

3. The data processing method for the target object according to claim 1, characterized in that, The step of obtaining the storage information of the target object and determining, based on the storage information and the equilibrium point model, to schedule the target object using either the standard estimation model or the hierarchical estimation model includes: Obtain any two of the target object's average usage, access frequency, and storage time.

4. The data processing method for the target object according to claim 1, characterized in that, The determination of object monitoring parameters in the hierarchical initial estimation model based on the resource usage of the sub-object and the preset average resource usage includes: If the resource usage of a sub-object is greater than the preset average usage, the object monitoring parameters in the hierarchical initial estimation model are calculated based on the preset average usage to obtain the hierarchical estimation model.

5. A data processing device for a target object, characterized in that, The device includes: The acquisition module is used to obtain a standard estimation model and a hierarchical estimation model of the target object. The standard estimation model is used to calculate the resource consumption of the target object in the standard mode and the resource consumption of the target object in the hierarchical mode. In the standard mode, the target object is stored in the high-frequency layer, and in the hierarchical mode, at least a portion of the target object is stored in the low-frequency layer. The calculation module is used to calculate the equilibrium point model based on the standard estimation model and the hierarchical estimation model. The equilibrium point model is used to obtain the balance function of the resource consumption of the target object in the standard mode and the hierarchical mode. The determination module is used to obtain the storage information of the target object, and based on the storage information and the equilibrium point model, determine whether to use the standard estimation model or the hierarchical estimation model to schedule the target object; The target object includes multiple sub-objects; The acquisition module is further configured to acquire a hierarchical initial estimation model; determine the resource usage of each sub-object; and determine object monitoring parameters in the hierarchical initial estimation model based on the resource usage of the sub-object and a preset average usage, thereby obtaining the hierarchical estimation model. This includes: when the resource usage of multiple sub-objects is not greater than a preset average usage, dividing the multiple sub-objects with resource usage not greater than the preset average usage into sub-object combinations; and when the resource usage of a sub-object combination is greater than the preset average usage, calculating object monitoring parameters in the hierarchical initial estimation model based on the resource usage of the sub-object combination, thereby obtaining the hierarchical estimation model.

6. The apparatus according to claim 5, characterized in that, The determining module is further configured to periodically schedule at least a portion of the target objects to a low-frequency layer when scheduling the target objects using the hierarchical estimation model.

7. The apparatus according to claim 5, characterized in that, The determining module is also used to obtain any two of the target object's average usage, access frequency, and storage time.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images