Method and apparatus for processing files
Through the time series analysis model, predicting storage space requirements and deleting or segmenting files, the problem of low storage space management efficiency in Internet technology is solved and efficient utilization of storage space is achieved.
Patent Information
- Application Number
- CN201810142038.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2018-02-11
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2038-02-11
AI Technical Summary
With the development of Internet technology, the amount of data in network resources has increased, and the storage space occupied by files storing data has continued to increase. It is difficult for existing technologies to effectively and timely clean up, resulting in low storage space management efficiency.
By obtaining the historical occupied storage space increase sequence of the target storage area, the time series analysis model is used to predict future storage space requirements. If the predicted increase exceeds the remaining space, delete the file with the earliest operation time to free up space, or the timeout is over or oversized Files are split into multiple subfiles for optimal storage.
It improves file management flexibility for processing target storage space, effectively frees up storage space, avoids excessive storage space, and improves storage space utilization efficiency.
Smart Images

Figure CN110334059B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of computer technology, specifically to the field of Internet technology, and in particular to a method and device for processing files. Background Art
[0002] With the development of Internet technology, the amount of data in network resources is getting larger and larger. As time goes by, the storage space occupied by the files storing data will also continue to increase. Therefore, it is necessary to clean up the stored files in a timely manner. Most of the current solutions are to retain the data recorded within a certain time range, and delete or split the data or files whose operation time (such as the time to modify or delete the data in the file) exceeds a certain time according to a preset method. Summary of the invention
[0003] The embodiments of the present application provide a method and device for processing a file.
[0004] In a first aspect, an embodiment of the present application provides a method for processing files, the method comprising: obtaining a sequence of historical occupied storage space increases of a target storage area, wherein the historical occupied storage space increase is the increase in the size of the occupied storage space of the target storage area within a historical time period; obtaining a predicted occupied storage space increase of the target storage area within a predicted time period based on the sequence of historical occupied storage space increases and a preset time series analysis model, wherein the time series analysis model is used to characterize the correspondence between the sequence of historical occupied storage space increases and the predicted occupied storage space increase; performing the following processing steps: determining the remaining storage space size of the target storage area, and determining whether the predicted occupied storage space increase is greater than the determined remaining storage space size; in response to determining that the predicted occupied storage space increase is greater than the determined remaining storage space size, deleting the file with the earliest operation time among the files stored in the target storage area, and continuing to perform the processing steps, wherein the operation time is the time when the file was last operated on.
[0005] In some embodiments, based on a historical occupied storage space increase sequence and a preset time series analysis model, a predicted occupied storage space increase of a target storage area within a predicted time period is obtained, including: deleting historical occupied storage space increases in the historical occupied storage space increase sequence that are greater than or equal to a preset first increase threshold and / or less than or equal to a preset second increase threshold, to obtain a sequence of historical occupied storage space increases to be input, wherein the first increase threshold is greater than the second increase threshold; inputting the sequence of historical occupied storage space increases to be input into a preset time series analysis model to obtain the predicted occupied storage space increase of the target storage area within the predicted time period.
[0006] In some embodiments, after determining the remaining storage space size of the target storage area, the method further includes: in response to determining that the remaining storage space size of the target storage area is less than or equal to the remaining storage space threshold, deleting the file with the earliest operation time among the files stored in the target storage area.
[0007] In some embodiments, the method further includes: in response to determining that the difference between the current time and the operation time of the file with the earliest operation time among the files stored in the target storage area is greater than a preset first time threshold, deleting the file with the earliest operation time among the files stored in the target storage area.
[0008] In some embodiments, the method further includes: determining the write duration of a file stored in the target storage area, wherein the write duration is the difference between the time when data was last written to the file and the time when data was first written to the file; determining a file whose write duration is greater than a preset second time threshold as an over-duration file; dividing the over-duration file into at least two first sub-files and storing them, wherein the write duration of the first sub-file is less than or equal to a preset third time threshold, and the second time threshold is greater than the third time threshold.
[0009] In some embodiments, the method also includes: determining the size of storage space occupied by files stored in the target storage area; determining files whose occupied storage space size is greater than a preset first occupied storage space threshold as oversized files; dividing the oversized files into at least two second sub-files and storing them, wherein the size of storage space occupied by the second sub-file is less than or equal to a preset second occupied storage space threshold, and the first occupied storage space threshold is greater than the second occupied storage space threshold.
[0010] In a second aspect, an embodiment of the present application provides a device for processing files, the device comprising: an acquisition unit, configured to acquire a sequence of historical occupied storage space increases of a target storage area, wherein the historical occupied storage space increases are increases in the size of the occupied storage space of the target storage area within a historical time period; a calculation unit, configured to obtain a predicted occupied storage space increase of the target storage area within a predicted time period based on the sequence of historical occupied storage space increases and a preset time series analysis model, wherein the time series analysis model is used to characterize the correspondence between the sequence of historical occupied storage space increases and the predicted occupied storage space increase; a processing unit, configured to perform the following processing steps: determining the size of the remaining storage space of the target storage area, and determining whether the predicted occupied storage space increase is greater than the determined remaining storage space size; a first deletion unit, configured to delete the file with the earliest operation time among the files stored in the target storage area in response to determining that the predicted occupied storage space increase is greater than the determined remaining storage space size, and continue to perform the processing steps, wherein the operation time is the time when the file was last operated on.
[0011] In some embodiments, the calculation unit includes: a deletion module, configured to delete the historical occupied storage space increase in the historical occupied storage space increase sequence that is greater than or equal to a preset first increase threshold and / or less than or equal to a preset second increase threshold, to obtain a sequence of historical occupied storage space increase to be input, wherein the first increase threshold is greater than the second increase threshold; a calculation module, configured to input the sequence of historical occupied storage space increase to be input into a preset time series analysis model, to obtain a predicted occupied storage space increase in the target storage area within a predicted time period.
[0012] In some embodiments, the device further includes: a second deleting unit configured to delete the file with the earliest operation time among the files stored in the target storage area in response to determining that the remaining storage space size of the target storage area is less than or equal to the remaining storage space threshold.
[0013] In some embodiments, the device also includes: a third deletion unit, configured to delete the file with the earliest operation time among the files stored in the target storage area in response to determining that the difference between the current time and the operation time of the file with the earliest operation time among the files stored in the target storage area is greater than a preset first time threshold.
[0014] In some embodiments, the device also includes: a first determination unit, configured to determine the write duration of a file stored in the target storage area, wherein the write duration is the difference between the time when data was last written to the file and the time when data was first written to the file; a second determination unit, configured to determine a file whose write duration is greater than a preset second time threshold as an over-duration file; a first segmentation unit, configured to segment the over-duration file into at least two first sub-files and store them, wherein the write duration of the first sub-file is less than or equal to a preset third time threshold, and the second time threshold is greater than the third time threshold.
[0015] In some embodiments, the device also includes: a third determination unit, configured to determine the size of storage space occupied by files stored in the target storage area; a fourth determination unit, configured to determine a file whose occupied storage space size is greater than a preset first occupied storage space threshold as an oversized file; and a second segmentation unit, configured to segment the oversized file into at least two second sub-files and store them, wherein the size of storage space occupied by the second sub-file is less than or equal to a preset second occupied storage space threshold, and the first occupied storage space threshold is greater than the second occupied storage space threshold.
[0016] In a third aspect, an embodiment of the present application provides a server, comprising: one or more processors; a storage device for storing one or more programs; when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation method in the first aspect.
[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.
[0018] The method and device for processing files provided in the embodiments of the present application predict the occupied storage space of the target storage area within the prediction time period by utilizing a time series analysis model. If the predicted increase in occupied storage space is greater than the remaining storage space size of the target storage area, the files stored in the target storage space are deleted, thereby improving the flexibility of processing files stored in the target storage space. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:
[0020] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;
[0021] Figure 2is a flow chart of an embodiment of a method for processing a file according to the present application;
[0022] Figure 3 is a schematic diagram of an application scenario of the method for processing files according to the present application;
[0023] Figure 4 is a flowchart of another embodiment of a method for processing a file according to the present application;
[0024] Figure 5 is a schematic structural diagram of an embodiment of a device for processing a file according to the present application;
[0025] Figure 6 It is a structural diagram of a computer system suitable for implementing a server of an embodiment of the present application. DETAILED DESCRIPTION
[0026] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.
[0027] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0028] Figure 1 An exemplary system architecture 100 is shown to which embodiments of the method for processing a file or the apparatus for processing a file of the present application can be applied.
[0029] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.
[0030] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various client applications can be installed on terminal devices 101, 102, 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.
[0031] Terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting various communication applications (such as web browsing, online chatting, etc.), including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III, Moving Picture Experts Group Audio Layer 3), MP4 (Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Group Audio Layer 4) players, laptop computers and desktop computers, etc.
[0032] The server 105 may be a server that provides various services, such as a file processing server that stores the data generated on the terminal devices 101, 102, and 103 in the form of files. The file processing server may analyze, store, and perform other processing on the received data.
[0033] It should be noted that the method for processing files provided in the embodiments of the present application is generally executed by the server 105 , and accordingly, the device for processing files is generally set in the server 105 .
[0034] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.
[0035] Continue to refer Figure 2 , shows a process 200 of an embodiment of a method for processing a file according to the present application. The method for processing a file comprises the following steps:
[0036] Step 201: Obtain a historical sequence of increases in occupied storage space in a target storage area.
[0037] In this embodiment, the method for processing a file is executed on an electronic device (eg Figure 1The server shown in the figure can obtain the historical occupied storage space increase sequence for the target storage area remotely or locally through a wired connection or a wireless connection. The target storage area can be a storage area (such as a disk, a storage chip, etc.) set in advance by a technician in the above-mentioned electronic device, or it can be a storage area set by a technician in other devices that are communicatively connected to the above-mentioned electronic device. The historical occupied storage space increase can be the increase in the size of the occupied storage space of the target storage area within a historical time period. The historical time period can be the interval time set by the technician to scan the storage space of the target storage area. After each scan, the increase in the size of the occupied storage space of the target storage area compared to the last scan can be obtained.
[0038] As an example, assuming that each historical time period is 5 minutes, the acquired historical occupied storage space increase amount sequence includes 5 values, and the historical occupied storage space increase amount sequence represents the size of the occupied storage space increased every 5 minutes in the past 25 minutes. For example, the historical occupied storage space increase amount sequence is "0.2, 0.8, 1, 0.3, 0.1", and the historical occupied storage space increase amount sequence represents that the size of the occupied storage space increased every 5 minutes in the past 25 minutes can be 0.2GB, 0.8GB, 1GB, 0.3GB, and 0.1GB respectively.
[0039] It should be noted that the historical increase in occupied storage space may be a negative number, indicating that part of the occupied storage space of the target storage space has been released during the historical time period.
[0040] Step 202: based on the historical occupied storage space increase amount sequence and a preset time series analysis model, obtain the predicted occupied storage space increase amount of the target storage area within the predicted time period.
[0041] In this embodiment, based on the historical occupied storage space increase sequence obtained in step 201, the electronic device can obtain the predicted occupied storage space increase of the target storage area within the predicted time period based on the historical occupied storage space increase sequence and a preset time series analysis model. The time series analysis model is used to characterize the correspondence between the historical occupied storage space increase sequence and the predicted occupied storage space increase. The predicted time period can be a time period set by a technician in the electronic device. For example, the predicted time period can be set to 30 minutes, and the electronic device can predict the size of the occupied storage space increase in the target storage area within 30 minutes after the current time through the time series analysis model.
[0042] The above-mentioned time series analysis model can be used as a mathematical model established by curve fitting and parameter estimation based on the observed time series data. It can generally be established using curve fitting and parameter estimation methods (such as nonlinear least squares method). The above-mentioned time series analysis model may include but is not limited to at least one of the following: AR (AutoRegression) model, MA (Moving Average) model, ARMA (Auto RegressionMoving Average) model, ARIMA (Auto Regressive Integrated MovingAverage Model) model, etc.
[0043] Optionally, the electronic device can obtain the predicted occupied storage space increase of the target storage area within the prediction time period based on the historical occupied storage space increase sequence and the ARIMA model. Among them, the ARIMA model is a common model used for time series prediction in statistical models. It can convert non-stationary time series into stationary time series, and then regress the lagged value of the dependent variable and the present value and lagged value of the random error term to establish a model. The ARIMA model includes a moving average process (MA), an autoregressive process (AR), an autoregressive moving average process (ARMA) and an ARIMA process, depending on whether the original sequence is stationary and the different parts contained in the regression.
[0044] Usually, the ARIMA model has three parameters: p, d, and q, where p represents the number of lags of the time series data itself used in the model, also called the AR (Auto-Regressive) term; d represents the number of order differentiations that the time series data needs to undergo to become stable time series data, also called the Integrated term; q represents the number of lags of the prediction error used in the model, also called the MA (Moving Average) term. In practice, by analyzing the above-mentioned historical sequence of the increase in occupied storage space, the three parameters of the ARIMA model can be determined, and then an ARIMA model for determining the increase in the predicted occupied storage space within the prediction time period can be established. It should be noted that the above-mentioned method of establishing the ARIMA model is a well-known technology that is currently widely studied and applied, and will not be repeated here.
[0045] The electronic device may input the historical occupied storage space increase amount sequence into a preset time series analysis model to obtain the predicted occupied storage space increase amount of the target storage area within the predicted time period.
[0046] In some optional implementations of this embodiment, the electronic device may obtain the predicted increase in occupied storage space of the target storage area within the predicted time period through the following steps:
[0047] First, the electronic device deletes the historical occupied storage space increases that are greater than or equal to the preset first increase threshold and / or less than or equal to the preset second increase threshold in the historical occupied storage space increase sequence, and obtains the historical occupied storage space increase sequence to be input. The first increase threshold is greater than the second increase threshold. For example, the preset first increase threshold is 1GB, and the second increase threshold is 1MB, then the set of historical occupied storage space increases that are less than 1GB and greater than 1MB is determined as the historical occupied storage space increase sequence to be input. The obtained historical occupied storage space increase sequence to be input does not contain values of a sudden and substantial increase or decrease in the historical occupied storage space increase due to certain special circumstances, which can make the obtained predicted occupied storage space increase more accurate.
[0048] It should be understood that the first increase threshold and the second increase threshold may be values directly set by a technician, or may be values determined by the electronic device based on a preset method. For example, the electronic device may determine the first increase threshold by multiplying the average value of each historical increase in occupied storage space by a preset multiple, or may determine the second increase threshold by multiplying the average value by a preset percentage.
[0049] Then, the electronic device inputs the to-be-input historical occupied storage space increase amount sequence into a preset time series analysis model to obtain the predicted occupied storage space increase amount of the target storage area within the predicted time period.
[0050] Step 203, executing the following processing steps: determining the remaining storage space size of the target storage area, and determining whether the predicted increase in occupied storage space is greater than the determined remaining storage space size.
[0051] In this embodiment, based on the predicted increase in occupied storage space of the target storage area within the predicted time period obtained in step 202, the above-mentioned electronic device can perform the following processing steps: determine the remaining storage space size of the target storage area, and determine whether the predicted increase in occupied storage space is greater than the determined remaining storage space size.
[0052] As an example, assuming that the predicted time period is 30 minutes, the predicted increase in occupied storage space is 30GB, and the determined remaining storage space size of the current target storage area is 20GB, the above electronic device determines that the predicted increase in occupied storage space is greater than the determined remaining storage space size.
[0053] In some optional implementations of this embodiment, after the electronic device determines the remaining storage space size of the target storage area, it may further perform the following steps: in response to determining that the remaining storage space size of the target storage area is less than or equal to the remaining storage space threshold, delete the file with the earliest operation time among the files stored in the target storage area. The remaining storage space threshold may be a value preset by a technician in the electronic device, or may be a value obtained by multiplying the total storage space size of the target storage area by a preset percentage.
[0054] As an example, assuming that the remaining storage space threshold is 5G, and the remaining storage space size of the determined target storage area is 4G, the electronic device deletes the file with the earliest operation time among the files stored in the target storage area. It should be understood that the electronic device can repeatedly perform the step of deleting the file with the earliest operation time until the remaining storage space size of the target storage area is greater than the remaining storage space threshold.
[0055] Step 204, in response to determining that the predicted increase in occupied storage space is greater than the determined remaining storage space size, delete the file with the earliest operation time among the files stored in the target storage area, and continue to execute the above processing steps.
[0056] In this embodiment, based on the processing steps in step 203, the above-mentioned electronic device can delete the file with the earliest operation time among the files stored in the target storage area in response to determining that the predicted increase in occupied storage space is greater than the determined remaining storage space size, and continue to execute the above-mentioned processing steps.
[0057] The operation time is the time when the file was last operated. For example, after the electronic device operates a file (such as writing data, deleting data, modifying data, etc.), the time when the file was operated can be recorded. The recorded time when the file was last operated is the operation time of the file. The file can be a log file that records data sent by the client used by the user, or a log file that records data by the electronic device itself, or other types of files, such as text files, video files, audio files, etc.
[0058] In this embodiment, after executing step 203, the remaining storage space size of the target storage area finally determined will be less than or equal to the predicted increase in occupied storage space.
[0059] Continue to see Figure 3 , Figure 3 FIG. 1 is a schematic diagram of an application scenario of the method for processing files according to this embodiment. Figure 3In the application scenario, the server 301 first obtains the historical occupied storage space increase sequence 302 of the target storage area (for example, the hard disk of the server 301). Then, based on the historical occupied storage space increase sequence 302 and the preset time series analysis model (for example, the ARIMA model), the server 301 obtains the predicted occupied storage space increase 304 of the target storage area in the prediction time period as 5GB, wherein the prediction time period is 30 minutes after the current time (2018-1-1-12:45). Next, the server 301 determines that the remaining storage space size 305 of the target storage area is 4GB, and the server 301 determines that the predicted occupied storage space increase is greater than the remaining storage space size, and deletes the file with the earliest operation time (such as file 1 in the figure) in the file list 306 stored in the target storage area. Then, the remaining storage space size is determined again, and it is re-determined whether the predicted occupied storage space increase is greater than the remaining storage space size, and it is determined whether to delete the file with the earliest operation time in the file list 306. After multiple file deletion operations, the final predicted occupied storage space increase is less than or equal to the remaining storage space size.
[0060] The method provided by the above-mentioned embodiment of the present application predicts the occupied storage space of the target storage area within the prediction time period by utilizing a time series analysis model. If the predicted increase in the occupied storage space is greater than the remaining storage space size of the target storage area, the files stored in the target storage space are deleted, thereby improving the flexibility of processing the files stored in the target storage space.
[0061] Further references Figure 4 , which shows a process 400 of another embodiment of a method for processing a file. The process 400 of the method for processing a file comprises the following steps:
[0062] Step 401: Obtain a historical sequence of increases in occupied storage space in a target storage area.
[0063] In this embodiment, step 401 and Figure 2 Step 201 in the corresponding embodiment is basically the same and will not be described again here.
[0064] Step 402: based on the historical occupied storage space increase amount sequence and a preset time series analysis model, obtain the predicted occupied storage space increase amount of the target storage area within the predicted time period.
[0065] In this embodiment, step 402 and Figure 2 Step 202 in the corresponding embodiment is basically the same and will not be described again here.
[0066] Step 403, executing the following processing steps: determining the remaining storage space size of the target storage area, and determining whether the predicted increase in occupied storage space is greater than the determined remaining storage space size.
[0067] In this embodiment, step 403 and Figure 2 Step 203 in the corresponding embodiment is basically the same and will not be described again here.
[0068] Step 404, in response to determining that the predicted increase in occupied storage space is greater than the determined remaining storage space size, delete the file with the earliest operation time among the files stored in the target storage area, and continue to execute the above processing steps.
[0069] In this embodiment, step 404 and Figure 2 Step 204 in the corresponding embodiment is basically the same and will not be described again here.
[0070] Step 405, in response to determining that the difference between the current time and the operation time of the file with the earliest operation time among the files stored in the target storage area is greater than a preset first time threshold, deleting the file with the earliest operation time among the files stored in the target storage area.
[0071] In this embodiment, the method for processing a file is executed on an electronic device (eg Figure 1 The server shown in the figure can delete the file with the earliest operation time (i.e., the expired file) in the files stored in the target storage area in response to determining that the difference between the current time and the operation time of the file with the earliest operation time in the files stored in the target storage area is greater than a preset first time threshold. The current time is the time recorded by the above electronic device when executing this step. For example, the current time is 12:00 on January 18, 2018, and the operation time of the file with the earliest operation time in the files stored in the target storage area is 11:59 on December 19, 2017, and the first time threshold is 30 days. Then, the above electronic device determines that the difference between the current time and the operation time of the file with the earliest operation time is greater than 30 days, and then deletes the file with the operation time of 11:59 on December 19, 2017.
[0072] Step 406, determine the writing duration of the files stored in the target storage area, determine the files whose writing duration is greater than the preset second time threshold as overtime files, and divide the overtime files into at least two first sub-files and store them.
[0073] In this embodiment, the electronic device may first determine the write duration of the file stored in the target storage area. The write duration is the difference between the time when data is last written to the file and the time when data is first written to the file. Then, the electronic device determines the file whose write duration is greater than the preset second time threshold as an over-time file. Finally, the electronic device divides the over-time file into at least two first sub-files and stores them. The write duration of the first sub-file is less than or equal to the preset third time threshold, and the second time threshold is greater than the third time threshold.
[0074] Exemplarily, assuming that in a certain file stored in the target storage area, the time of first writing data into the file is 12:00 on January 1, 2018, and the time of last writing data into the file is 22:01 on January 1, 2018, then the writing duration is 10 hours and 1 minute. Assuming that the preset second time threshold is 10 hours, the file is an overtime file. The above electronic device further divides the overtime file into at least two first sub-files and stores them. Assuming that the preset third time threshold is 10 minutes, the overtime file can be finally divided into multiple first sub-files, wherein the writing duration of each first sub-file is less than or equal to 10 minutes.
[0075] Step 407, determine the storage space occupied by the files stored in the target storage area; determine the files whose storage space occupied is greater than a preset first storage space occupied threshold as oversized files; and divide the oversized files into at least two second sub-files and store them.
[0076] In this embodiment, the electronic device may first determine the size of the storage space occupied by the files stored in the target storage area. Then, the electronic device determines the file whose storage space size is greater than a preset first occupied storage space threshold as an oversized file. Finally, the electronic device divides the oversized file into at least two second sub-files and stores them. The storage space size occupied by the second sub-file is less than or equal to the preset second occupied storage space threshold, and the first occupied storage space threshold is greater than the second occupied storage space threshold.
[0077] Exemplarily, assuming that the storage space occupied by a certain file stored in the target storage area is 1GB, and the preset first occupied storage space threshold is 100MB, then the file is an oversized file. Assuming that the preset second occupied storage space threshold is 10MB, the electronic device further divides the oversized file into at least two second sub-files and stores them. The storage space occupied by each second sub-file is less than or equal to 10MB.
[0078] It should be noted that, in practice, the execution order of step 401 to step 407 can be arbitrarily configured on the premise that no logical conflict is caused when each step is executed.
[0079] from Figure 4 It can be seen that Figure 2 Compared with the corresponding embodiments, the process 400 of the method for processing files in this embodiment highlights the processing steps for overdue files, oversized files, and overlong files, thereby further improving the flexibility of processing files stored in the target storage space.
[0080] Further references Figure 5 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a device for processing a file, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.
[0081] like Figure 5 As shown, the device 500 for processing files in this embodiment includes: an acquisition unit 501, configured to acquire a sequence of historical occupied storage space increases of a target storage area, wherein the historical occupied storage space increase is an increase in the size of the occupied storage space of the target storage area in a historical time period; a calculation unit 502, configured to obtain a predicted occupied storage space increase of the target storage area in a predicted time period based on the sequence of historical occupied storage space increases and a preset time series analysis model, wherein the time series analysis model is used to characterize the corresponding relationship between the sequence of historical occupied storage space increases and the predicted occupied storage space increase; a processing unit 503, configured to perform the following processing steps: determining the remaining storage space size of the target storage area, and determining whether the predicted occupied storage space increase is greater than the determined remaining storage space size; a first deletion unit 504, configured to delete the file with the earliest operation time among the files stored in the target storage area in response to determining that the predicted occupied storage space increase is greater than the determined remaining storage space size, and continue to perform the processing steps, wherein the operation time is the time when the file was last operated.
[0082] In this embodiment, the acquisition unit 501 can remotely or locally acquire a sequence of historical occupied storage space increases for the target storage area through a wired connection or a wireless connection. The target storage area can be a storage area (such as a disk, a storage chip, etc.) pre-set by a technician in the above-mentioned device 500, or it can be a storage area set by a technician in other devices that are connected to the above-mentioned device 500. The historical occupied storage space increase can be the increase in the size of the occupied storage space of the target storage area in the historical time period. The historical time period can be the interval time set by the technician to scan the storage space of the target storage area. After each scan, the increase in the size of the occupied storage space of the target storage area in the time period between the last scan and the current scan can be obtained.
[0083] In this embodiment, based on the historical occupied storage space increase sequence obtained by the acquisition unit 501, the calculation unit 502 can obtain the predicted occupied storage space increase of the target storage area within the prediction time period based on the historical occupied storage space increase sequence and the preset time series analysis model. The time series analysis model is used to characterize the correspondence between the historical occupied storage space increase sequence and the predicted occupied storage space increase. The prediction time period can be a time period set by a technician in the device 500. For example, the prediction time period can be set to 30 minutes, and the electronic device can predict the size of the occupied storage space increase of the target storage area within the next 30 minutes through the time series analysis model.
[0084] In this embodiment, based on the predicted increase in occupied storage space of the target storage area within the predicted time period obtained by the calculation unit 502, the above-mentioned processing unit 503 can perform the following processing steps: determine the remaining storage space size of the target storage area, and determine whether the predicted increase in occupied storage space is greater than the determined remaining storage space size.
[0085] In this embodiment, based on the processing steps in the processing unit 503, the first deleting unit 504 may delete the file with the earliest operation time among the files stored in the target storage area in response to determining that the predicted increase in the occupied storage space is greater than the determined remaining storage space size, and continue to execute the above processing steps. The operation time is the time when the file was last operated.
[0086] In some optional implementations of the present embodiment, the calculation unit 502 may include: a deletion module (not shown in the figure), configured to delete the historical occupied storage space increase in the historical occupied storage space increase sequence that is greater than or equal to a preset first increase threshold and / or less than or equal to a preset second increase threshold, to obtain a sequence of historical occupied storage space increase to be input, wherein the first increase threshold is greater than the second increase threshold; a calculation module (not shown in the figure), configured to input the sequence of historical occupied storage space increase to be input into a preset time series analysis model, to obtain a predicted occupied storage space increase in the target storage area within the predicted time period.
[0087] In some optional implementations of this embodiment, the above-mentioned device may also include: a second deletion unit (not shown in the figure), configured to delete the file with the earliest operation time among the files stored in the target storage area in response to determining that the remaining storage space size of the target storage area is less than or equal to the remaining storage space threshold.
[0088] In some optional implementations of the present embodiment, the above-mentioned device may also include: a third deletion unit (not shown in the figure), configured to delete the file with the earliest operation time among the files stored in the target storage area in response to determining that the difference between the current time and the operation time of the file with the earliest operation time among the files stored in the target storage area is greater than a preset first time threshold.
[0089] In some optional implementations of the present embodiment, the above-mentioned device may also include: a first determination unit (not shown in the figure), configured to determine the write duration of the file stored in the target storage area, wherein the write duration is the difference between the time when data is last written to the file and the time when data is first written to the file; a second determination unit (not shown in the figure), configured to determine a file whose write duration is greater than a preset second time threshold as an overtime file; a first segmentation unit (not shown in the figure), configured to segment the overtime file into at least two first sub-files and store them, wherein the write duration of the first sub-file is less than or equal to a preset third time threshold, and the second time threshold is greater than the third time threshold.
[0090] In some optional implementations of the present embodiment, the above-mentioned device may further include: a third determination unit (not shown in the figure), configured to determine the size of the storage space occupied by the files stored in the target storage area; a fourth determination unit (not shown in the figure), configured to determine a file whose occupied storage space size is greater than a preset first occupied storage space threshold as an oversized file; a second segmentation unit (not shown in the figure), configured to segment the oversized file into at least two second sub-files and store them, wherein the size of the storage space occupied by the second sub-file is less than or equal to a preset second occupied storage space threshold, and the first occupied storage space threshold is greater than the second occupied storage space threshold.
[0091] The device provided by the above-mentioned embodiment of the present application predicts the occupied storage space of the target storage area within the prediction time period by utilizing a time series analysis model. If the predicted increase in the occupied storage space is greater than the remaining storage space size of the target storage area, the files stored in the target storage space are deleted, thereby improving the flexibility of processing the files stored in the target storage space.
[0092] Reference below Figure 6 , which shows a schematic diagram of the structure of a computer system 600 suitable for implementing a server of an embodiment of the present application. Figure 6 The server shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0093] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the system 600 are also stored. The CPU 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.
[0094] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.
[0095] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the central processing unit (CPU) 601, the above functions defined in the method of the present application are executed. It should be noted that the computer-readable medium described in the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, an apparatus or a device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, an apparatus or a device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.
[0096] Computer program code for performing the operations of the present application may be written in one or more programming languages or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0097] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0098] The units involved in the embodiments described in the present application may be implemented by software or by hardware. The units described may also be set in a processor, for example, it may be described as: a processor includes an acquisition unit, a calculation unit, a processing unit and a first deletion unit. The names of these units do not constitute a limitation on the units themselves in some cases, for example, the acquisition unit may also be described as "a unit for acquiring a sequence of historical occupied storage space increases in a target storage area".
[0099] As another aspect, the present application also provides a computer-readable medium, which may be included in the server described in the above embodiment; or it may exist independently and not be assembled into the server. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by the server, the server: obtains the historical occupied storage space increase sequence of the target storage area; based on the historical occupied storage space increase sequence and the preset time series analysis model, obtains the predicted occupied storage space increase of the target storage area within the predicted time period; performs the following processing steps: determines the remaining storage space size of the target storage area, and determines whether the predicted occupied storage space increase is greater than the determined remaining storage space size; in response to determining that the predicted occupied storage space increase is greater than the determined remaining storage space size, deletes the file with the earliest operation time among the files stored in the target storage area, and continues to perform the processing steps.
[0100] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.
Claims
1. A method for processing a file, comprising: Acquire a sequence of historical occupied storage space increase amounts of the target storage area, wherein the historical occupied storage space increase amount is an increase in the size of the storage space occupied by the target storage area in a historical time period, and the historical time period is an interval time for performing a storage space scan on the target storage area; Based on the historical occupied storage space increase amount sequence and a preset time series analysis model, a predicted occupied storage space increase amount of the target storage area within a predicted time period is obtained, wherein the time series analysis model is used to characterize the corresponding relationship between the historical occupied storage space increase amount sequence and the predicted occupied storage space increase amount; The following processing steps are performed: determining the remaining storage space size of the target storage area, and determining whether the predicted increase in occupied storage space is greater than the determined remaining storage space size; In response to determining that the predicted increase in occupied storage space is greater than the determined remaining storage space size, the file with the earliest operation time among the files stored in the target storage area is deleted, and the processing steps are continued, wherein the operation time is the time when the file was last operated on.
2. The method according to claim 1, wherein: The obtaining, based on the historical occupied storage space increase amount sequence and a preset time series analysis model, a predicted occupied storage space increase amount of the target storage area within a predicted time period includes: Deleting the historical occupied storage space increase amounts in the historical occupied storage space increase amount sequence that are greater than or equal to a preset first increase amount threshold and / or less than or equal to a preset second increase amount threshold, to obtain a historical occupied storage space increase amount sequence to be input, wherein the first increase amount threshold is greater than the second increase amount threshold; The to-be-input historical occupied storage space increase amount sequence is input into a preset time series analysis model to obtain the predicted occupied storage space increase amount of the target storage area within the predicted time period.
3. The method according to claim 1, wherein: After determining the remaining storage space size of the target storage area, the method further includes: In response to determining that the remaining storage space size of the target storage area is less than or equal to the remaining storage space threshold, the file with the earliest operation time among the files stored in the target storage area is deleted.
4. The method according to claim 1, wherein: The method further comprises: In response to determining that the difference between the current time and the operation time of the file with the earliest operation time among the files stored in the target storage area is greater than a preset first time threshold, the file with the earliest operation time among the files stored in the target storage area is deleted.
5. The method according to claim 1, wherein: The method further comprises: Determine the write duration of the file stored in the target storage area, wherein the write duration is the difference between the time when data is last written to the file and the time when data is first written to the file; Determine a file whose writing time is longer than a preset second time threshold as an overtime file; The overtime file is divided into at least two first sub-files and stored, wherein the writing duration of the first sub-file is less than or equal to a preset third time threshold, and the second time threshold is greater than the third time threshold.
6. The method according to any one of claims 1 to 5, wherein: The method further comprises: Determine the size of storage space occupied by files stored in the target storage area; Determine a file whose occupied storage space size is greater than a preset first occupied storage space threshold as an oversized file; The super large file is divided into at least two second sub-files and stored, wherein the storage space occupied by the second sub-files is less than or equal to a preset second occupied storage space threshold, and the first occupied storage space threshold is greater than the second occupied storage space threshold.
7. A device for processing a file, comprising: an acquisition unit configured to acquire a sequence of historical occupied storage space increase amounts of a target storage area, wherein the historical occupied storage space increase amounts are increases in the size of the storage space occupied by the target storage area in a historical time period, and the historical time period is an interval time for performing a storage space scan on the target storage area; a calculation unit configured to obtain a predicted increase in the occupied storage space of the target storage area within a predicted time period based on the historical occupied storage space increase amount sequence and a preset time series analysis model, wherein the time series analysis model is used to characterize a corresponding relationship between the historical occupied storage space increase amount sequence and the predicted occupied storage space increase amount; A processing unit configured to perform the following processing steps: determining the remaining storage space size of the target storage area, and determining whether the predicted increase in occupied storage space is greater than the determined remaining storage space size; The first deleting unit is configured to delete the file with the earliest operation time among the files stored in the target storage area in response to determining that the predicted increase in occupied storage space is greater than the determined remaining storage space size, and continue to execute the processing steps, wherein the operation time is the time when the file was last operated.
8. The device according to claim 7, wherein: The computing unit comprises: a deleting module configured to delete the historical occupied storage space increments in the historical occupied storage space increment sequence that are greater than or equal to a preset first increment threshold and / or less than or equal to a preset second increment threshold, to obtain a historical occupied storage space increment sequence to be input, wherein the first increment threshold is greater than the second increment threshold; The calculation module is configured to input the to-be-input historical occupied storage space increase amount sequence into a preset time series analysis model to obtain the predicted occupied storage space increase amount of the target storage area within the predicted time period.
9. The device according to claim 7, wherein: The device also includes: The second deleting unit is configured to delete the file with the earliest operation time among the files stored in the target storage area in response to determining that the remaining storage space size of the target storage area is less than or equal to the remaining storage space threshold.
10. The device according to claim 7, wherein: The device also includes: The third deleting unit is configured to delete the file with the earliest operation time among the files stored in the target storage area in response to determining that the difference between the current time and the operation time of the file with the earliest operation time among the files stored in the target storage area is greater than a preset first time threshold.
11. The device according to claim 7, wherein: The device also includes: A first determining unit is configured to determine a write duration of a file stored in the target storage area, wherein the write duration is a difference between a time when data is last written to the file and a time when data is first written to the file; A second determining unit is configured to determine a file whose writing time is longer than a preset second time threshold as an overtime file; The first segmentation unit is configured to segment the overtime file into at least two first sub-files and store them, wherein the writing duration of the first sub-file is less than or equal to a preset third time threshold, and the second time threshold is greater than the third time threshold.
12. The device according to any one of claims 7 to 11, wherein: The device also includes: A third determining unit, configured to determine the size of storage space occupied by the files stored in the target storage area; A fourth determining unit configured to determine a file whose occupied storage space size is greater than a preset first occupied storage space threshold as an oversized file; The second segmentation unit is configured to segment the super large file into at least two second sub-files and store them, wherein the storage space occupied by the second sub-files is less than or equal to a preset second occupied storage space threshold, and the first occupied storage space threshold is greater than the second occupied storage space threshold.
13. A server, comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.
14. A computer-readable storage medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.
