File processing method, apparatus, device, and storage medium

Abstract

The application discloses a file processing method and device, equipment and storage medium, and relates to the technical field of computer information storage, and the method comprises the steps that the configuration parameters of each file segment in a to-be-processed file are acquired, a target reserved area of the to-be-processed file is determined, the segment order of each file segment is determined according to the configuration parameters of each file segment, the target position of each file segment in the target reserved area is determined according to the segment order of each file segment, and each file segment is written into the corresponding target position in the target reserved area. According to the configuration parameters of the file segment, the segment order of each file segment is determined, and the target position of each file segment in the target reserved area is determined according to the segment order of each file segment, so that the file segments written into the target reserved area are continuous, and the internal fragmentation problem of the file is solved.

Description

Technical Field

[0001] This invention relates to the field of computer information storage technology, and specifically to a file processing method, apparatus, device, and storage medium. Background Technology

[0002] Log-structured file systems have spawned various file systems, collectively known as log-structured (LFS) file systems, such as F2FS and NILFS. The main features of this type of file system are off-site update mechanism and checkpoint mechanism. Regardless of whether the underlying storage device is a mechanical hard drive or a flash storage device, the off-site update mechanism can significantly improve random write performance. Therefore, this type of file system is widely used in mobile smart devices.

[0003] The side effect of offsite update mechanism is that, compared with other file systems that update in-situ, file fragmentation caused by file updates in LFS-like file systems will significantly reduce the performance of sequential read and sequential write operations in structured file systems.

[0004] Existing methods mainly alleviate the fragmentation problem of structured file systems through garbage collection (GC) mechanisms. However, this mechanism can only alleviate external fragmentation of the file system and cannot alleviate internal fragmentation. Summary of the Invention

[0005] This invention provides a file processing method, apparatus, device, and storage medium to alleviate the problem of file fragmentation within a structured file system.

[0006] On one hand, embodiments of the present invention provide a file processing method, the method comprising:

[0007] Obtain the configuration parameters for each file segment in the file to be processed, and determine the target reserved area for the file to be processed;

[0008] The order of each file segment is determined based on its configuration parameters.

[0009] Based on the order of the file segments, determine the target location of each file segment in the target reserved area;

[0010] Each of the aforementioned file fragments is written into its corresponding target location within the target reserved area.

[0011] On the other hand, embodiments of the present invention provide a document processing apparatus, the apparatus comprising:

[0012] The acquisition module is used to acquire the configuration parameters of each file segment in the file to be processed and to determine the target reserved area of ​​the file to be processed;

[0013] The fragment sorting module is used to determine the fragment order of each file fragment according to the configuration parameters of each file fragment;

[0014] The fragment-sensitive file address mapping module is used to determine the target location of each file fragment in the target reserved area according to the fragment order of each file fragment;

[0015] An internal anti-fragmentation recycling module is used to write each of the file fragments into its corresponding target location in the target reserved area.

[0016] On the other hand, embodiments of the present invention provide a file processing device, the device including a memory and a processor; the memory stores an application program, and the processor is used to run the application program in the memory to perform the operations in the above method.

[0017] On the other hand, embodiments of the present invention provide a storage medium storing a plurality of instructions adapted for loading by a processor to execute the steps in the above method.

[0018] This invention provides a file processing method, apparatus, device, and storage medium, relating to the field of computer information storage technology. The method involves obtaining configuration parameters for each file segment in a file to be processed, determining the target reserved area of ​​the file, determining the segment order of each file segment based on the configuration parameters, determining the target position of each file segment in the target reserved area based on the segment order, and writing each file segment to its corresponding target position in the target reserved area. This invention ensures that the file segments written to the target reserved area are continuous by determining the segment order of each file segment through configuration parameters and determining the target position of each file segment in the target reserved area based on the segment order, thereby alleviating the problem of file fragmentation. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the existing technology's remote update mechanism;

[0021] Figure 2This is a schematic diagram of existing waste recycling mechanisms in the current technology;

[0022] Figure 3 This is a schematic diagram of an embodiment of the file processing method provided by the present invention;

[0023] Figure 4 This is a schematic diagram of another embodiment of the file processing method provided by the present invention;

[0024] Figure 5 This is a schematic diagram of an embodiment of the method for determining internal fragmented data provided by the present invention;

[0025] Figure 6 This is a schematic diagram of an embodiment of the method for determining the target reserved area of ​​a file to be processed provided by the present invention;

[0026] Figure 7 This is a schematic diagram of an embodiment of the method for determining the target location of a file fragment in a target reserved area provided by the present invention;

[0027] Figure 8 This is a schematic diagram of the file internal fragmentation processing provided by the present invention;

[0028] Figure 9 This is a schematic diagram of another embodiment of the file processing method provided by the present invention;

[0029] Figure 10 This is a schematic diagram of another embodiment of the file processing method provided by the present invention;

[0030] Figure 11 This is a schematic diagram of another embodiment of the file processing method provided by the present invention;

[0031] Figure 12 This is a schematic diagram of an embodiment of the method for updating a list to be organized provided by the present invention;

[0032] Figure 13 This is a schematic diagram of the file writing structure file system provided by the present invention;

[0033] Figure 14 This is a schematic diagram of an embodiment of the document processing device provided by the present invention;

[0034] Figure 15 This is a schematic diagram of an embodiment of the document processing device provided by the present invention. Detailed Implementation

[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] As described in the background section, under the off-site update mechanism, when file segments in a file stored in the structured file system are deleted, overwritten, or parameters are updated, it will lead to discontinuity of file segments in the file and discontinuity of free storage areas in the structured file system, resulting in internal file fragmentation and external file fragmentation. Internal file fragmentation manifests as discontinuity of file segments in the file, while external file fragmentation manifests as discontinuity of free storage areas in the structured file system, i.e., fragmentation of the remaining space in the structured file system.

[0037] For example, such as Figure 1 As shown, Figure 1 This is a schematic diagram of a remote update mechanism in existing technology. Figure 1 The file structure system shown includes 16 storage areas, such as Figure 1 As shown in Figure (a), in the initial state, file A consists of four file segments A1, A2, A4, and A5, and file A occupies four consecutive storage areas in the file structure system. File B consists of four file segments B1, B2, B3, and B4, and file B occupies four consecutive storage areas in the file structure system. The remaining storage areas in the file structure system are contiguous. When file segments A2, B3, A2, and B2 are updated off-site in sequence, the updated file segment A2 is written to free storage area 08, the updated file segment B3 is written to free storage area 09, the updated file segment A2 is written to free storage area 10, and the updated file segment B2 is written to free storage area 11. At the same time, the file segments stored in storage areas 01, 03, 05, and 06 are erased. Figure 1As shown in Figure (b), the file segments of files A and B are not contiguous, resulting in internal file fragmentation. Moreover, the number of remaining storage areas in the structured file system remains unchanged, but the remaining storage areas in the structured file system are not contiguous, resulting in external file fragmentation. When sequential reading of files A and B is required, the number of I / O operations and overhead will increase due to the contiguous file segments of files A and B. At the same time, the out-of-order storage addresses of the file segments of files A and B will degrade the performance of sequential addressing storage devices. When the C file being written consists of 8 file segments, file segment C1 is written to the remaining storage area 12, file segment C2 to the remaining storage area 13, file segment C3 to the remaining storage area 14, and file segment C4 to the remaining storage area 15. When writing file segment C5, since there is no free storage area after storage area 15, it is necessary to traverse the storage areas of the structured file system to determine the blank storage area 01 from the entire structured file system's storage areas, and write file segment C5 to storage area 01. Following the writing method of file segment C5, file segments C6, C7, and C8 are written sequentially. This increases the number of I / O operations when writing files, and as... Figure 1 As shown in Figure (c), this causes the file segments written to the C file to be discontinuous, resulting in internal file fragmentation.

[0038] To alleviate fragmentation in structured file systems, existing technologies employ garbage collection mechanisms. However, these mechanisms only mitigate fragmentation of the remaining space within the structured file system, not fragmentation within individual files. For example, ... Figure 2 As shown, Figure 2 This is a diagram illustrating the existing waste recycling mechanism, such as... Figure 2 As shown in Figure (a), in the existing garbage collection mechanism, the structured system files are divided into four storage segments: sec1, sec2, sec3, and sec4. Each storage segment includes four storage areas. Files A and B exhibit both external and internal fragmentation. Figure 2 As shown in Figures (b) and (c), the existing garbage collection mechanism traverses each storage segment, writing the data of the used storage area in each segment into contiguous storage segments of free storage area, thus ensuring that the remaining space in the structure system file is contiguous; as Figure 2As shown in Figure (d), the file segments of files A and B in the structured file system after being organized by the garbage collection mechanism are still discontinuous. It can be seen that the existing garbage collection mechanism has not alleviated the internal fragmentation of files. When sequential reading of files A and B is required, the internal fragmentation of files A and B will increase the number of I / O operations and overhead, resulting in a decrease in the performance of sequential addressing storage devices.

[0039] Based on this, in order to alleviate the problem of internal file fragmentation, embodiments of the present invention provide a file processing method, apparatus, device, and storage medium. By configuring parameters of file segments, the segment order of each file segment is determined, and the target position of each file segment in the target reserved area is determined according to the segment order of each file segment. In this way, it can be ensured that each file segment written in the target reserved area is continuous, thereby alleviating the problem of internal file fragmentation.

[0040] The implementation details of the technical solutions in the embodiments of the present invention will be described in detail below:

[0041] like Figure 3 As shown, Figure 3 This is a schematic diagram of one embodiment of the file processing method provided by the present invention. This method can be executed by an electronic device with data storage and processing capabilities, such as a server, smartphone, tablet computer, personal computer, etc., and is not specifically limited thereto.

[0042] Reference Figure 3 As shown, the method includes at least steps 301 to 304, which are explained in detail below:

[0043] Step 301: Obtain the configuration parameters of each file segment in the file to be processed, and determine the target reserved area of ​​the file to be processed.

[0044] In this invention, the file to be processed can be a file to be written received by the structured file system. For example, when a file write request is received, a file write operation is triggered, the file to be processed is obtained, and the file segments of the file to be processed are traversed to obtain the configuration parameters of each file segment. The file to be processed includes, but is not limited to, the file being written for the first time, the file segment or file to be updated, or the file already written in the structured file system. This invention does not limit the type of file to be processed.

[0045] Optionally, in some implementations, when the file write operation is the first time the file is written, the file to be written is set as the file to be processed, each file segment of the file to be processed is traversed, and the configuration parameters of each file segment are determined.

[0046] Optionally, in other embodiments, when the file write operation is a file fragment update, the file fragment to be updated is obtained. Based on the file identifier of the file fragment to be updated, a written file in the structured file system that matches the file identifier is determined. The set consisting of the file fragment to be updated and the file fragments in the written file that match the file identifier is set as the file to be processed. Each file fragment in the file to be processed is traversed, and the configuration parameters of each file fragment are determined. Here, file fragment update represents updating data on at least one file fragment in the written file in the structured file system. The data update includes, but is not limited to, file fragment overwriting and data updates within a file fragment.

[0047] Optionally, in other embodiments, when the file write operation is a file update, the file to be updated is obtained, the file to be updated is set as the file to be processed, each file segment in the file to be processed is traversed, and the configuration parameters of each file segment are determined. Here, file update represents overwriting at least one written file in the structured file system.

[0048] Optionally, in other embodiments of the present invention, the file to be processed can also be a written file in a structured file system. For example, when a defragmentation request is received, a file defragmentation operation is triggered, a target written file is selected from the structured file system, the selected target written file is set as the file to be processed, and the file segments of the file to be processed are traversed to obtain the configuration parameters of each file segment. The target written file can be any written file in the structured file system, or it can be selected from the written files in the structured file system according to preset conditions. These preset conditions include, but are not limited to, file size, file type, and number of file updates.

[0049] In this invention, configuration parameters are used to characterize the order of each file segment in the file to be processed, including but not limited to the segment encoding, creation time, and update time of each file segment in the file to be processed.

[0050] In this invention, the target reserved area is a contiguous storage area in the structured file system allocated solely for storing the file to be processed. It includes, but is not limited to, persistent storage and temporary storage. The persistent storage area indicates that the target reserved area is used solely for storing the file to be processed, and any free reserved storage space within the target reserved area is not reclaimed. The temporary storage area indicates that the target reserved area is used solely for storing the file to be processed, and any free reserved storage space within the target reserved area can be reclaimed.

[0051] Specifically, reclaiming free reserved storage areas within the target reserved area refers to the process of reclaiming these areas as free storage when the remaining space in the structured file system is insufficient, thereby increasing the remaining space in the structured file system. The remaining space in the structured file system refers to the storage space comprised of free storage areas; free storage areas refer to regular storage areas in the structured file system that have not been written to and are not part of the reserved area.

[0052] Optionally, in some embodiments of the present invention, a file identifier of the file to be processed can be obtained, and a reserved area in the structured file system that matches the file identifier can be determined as the target reserved area of ​​the file to be processed. The file identifier can be one or more of letters, numbers, and special characters, and can be used to characterize the file type of the file to be processed, including but not limited to video files, text files, audio files, and image files. For example, a contiguous free storage area in the file structure system can be divided into multiple reserved areas, and a preset file type can be determined for each reserved area. The mapping relationship between the preset file identifier and file type is queried after determining the file identifier of the file to be processed to obtain the file type corresponding to the file to be processed. Based on the preset file type corresponding to each reserved area, the reserved area whose preset file type matches the file type of the file to be processed is determined as the target reserved area of ​​the file to be processed.

[0053] Optionally, in other embodiments of the present invention, an isolation mechanism can be used to determine contiguous free storage areas in the storage region of the structured file system as the target reserved area for the file to be processed. For example, contiguous free storage areas in the structured file system can be divided into at least two storage areas: a first storage area and a second storage area. The storage area in the first storage area is set as a regular storage area, and the second storage area is set as the target reserved area for the file to be processed. The isolation mechanism means that when data in the target reserved storage area is written to, updated, or erased, the data in the regular storage area of ​​the first storage area is not affected.

[0054] Step 302: Determine the segment order of each file segment based on the configuration parameters of each file segment.

[0055] The fragment order of a file segment refers to the sequence of file segments obtained by sorting each file segment according to its configuration parameters, and the sequence number of each file segment in the file segment sequence.

[0056] In some implementations, the file segments can be sorted in descending order of configuration parameters; alternatively, they can be sorted in ascending order of configuration parameters.

[0057] Step 303: Determine the target positions of each file fragment in the target reserved area according to the fragment order of each file fragment.

[0058] The target position may be the start address of the idle reserved storage area in the target reserved area, or may be the address range of the idle reserved storage area in the target reserved area. The address herein refers to the physical address where data is stored in the structured file system.

[0059] Optionally, in some embodiments, by querying the pre-stored relationship data between fragment order and positions according to the fragment order of file fragments, the target positions associated with each fragment order are obtained, and the target position associated with each fragment order is determined as the target position of the corresponding file fragment in the target reserved area. Wherein, the relationship data between fragment order and positions includes multiple fragment orders and the corresponding position of each fragment order.

[0060] Optionally, in other embodiments, consecutive idle reserved storage areas in the target reserved area may be determined, the fragment order of each file fragment is traversed, and starting from the first idle reserved storage area in the consecutive idle reserved storage areas, the address range or start address of the first idle reserved storage area is set as the target position of the file fragment corresponding to the first fragment order in the target reserved area, and the address range or start address of the next adjacent idle reserved storage area after the first idle reserved storage area is set as the target position of the file fragment corresponding to the second fragment order in the target reserved area, until the traversal of the fragment order of all file fragments is completed.

[0061] Wherein, the first idle reserved storage area may be the idle reserved storage area with the lowest address range or start address among the consecutive idle reserved storage areas.

[0062] Step 304: write each file fragment into its corresponding target position in the target reserved area.

[0063] In the present invention, after determining the target positions of each file fragment in the target reserved area, each file fragment is written into its corresponding target position in the target reserved area.

[0064] Optionally, in some embodiments, when the file to be processed is a file to be written, according to the fragment order of each file fragment, each file fragment of the file to be processed is sequentially written into the corresponding target position of the file fragment in the target reserved area. For example, when the file to be processed includes four file fragments A1, A2, A3 and A4, and the respective fragment order of A1, A2, A3 and A4 satisfies A1 < A2 < A3 < A4, A1, A2, A3 and A4 are sequentially written into their respective corresponding target positions in the target reserved area.

[0065] Optionally, in some implementations, when the file to be processed is a written file, the file segments corresponding to each segment sequence are copied sequentially according to the segment sequence of each file segment, and the copied file segments corresponding to the segment sequence are written into the target position of the file segments corresponding to the segment sequence in the target reserved area.

[0066] The file processing method provided in this embodiment of the invention determines the fragment order of each file fragment by configuring file fragment parameters, and determines the target position of each file fragment in the target reserved area according to the fragment order of each file fragment. In this way, it can ensure that each file fragment written in the target reserved area is continuous, thereby alleviating the problem of internal file fragmentation.

[0067] based on Figure 3 In one example, to improve system performance, the files to be processed can be first fragmented. Only when the fragmentation requirements are met will the actual fragmentation process be executed. For example... Figure 4 The diagram shown is a schematic representation of another embodiment of the document processing method provided in this application. Figure 4 Another embodiment of the document processing method provided in this application, as shown, includes at least steps 401 to 406:

[0068] Step 401: Determine the internal fragmented data of the file to be processed.

[0069] In this embodiment, the files to be processed that have been written to the structured file system can be files written to the structured file system within a past period of time, such as the past day, the past three days, or the past week. Alternatively, the files to be processed that have been written to the structured file system within the past period can be files that conform to preset rules. These preset rules can be one or more of file tags, file size, and file type. For example, when the preset rule is file tags, files tagged as "not internally fragmented" can be set as files to be processed, as can files tagged as "fragmented files."

[0070] In this embodiment, internal fragmentation data is used to characterize the degree of internal fragmentation of the file to be processed, including but not limited to the degree of disorder of file segments and the degree of voids in file segments.

[0071] In some embodiments of the present invention, the internal fragmented data of the file to be processed can be determined based on the storage address of the file segments of the file to be processed. Specifically, as shown in the figure... Figure 5 As shown, the present invention provides a method for determining internal fragmented data, specifically including steps 4011 to 4014:

[0072] Step 4011: Obtain the storage address of the file segment of the file to be processed.

[0073] The storage address can be the starting address of the physical address of the file segment in the structured file system, or it can be a range of physical addresses of the file segment in the structured file system. In some implementations, the storage address of the file segment of the file to be processed can be read directly from the structured file system; in other implementations, the logical address of the file segment of the file to be processed can be read from the structured file system, and the logical storage address mapping information of the file to be processed can be obtained. The storage address mapping information includes the mapping relationship between logical addresses and storage addresses.

[0074] Optionally, in some implementations, considering that the internal fragmentation of the file to be processed is low when it is written to the structured file system for the first time, the file to be processed can be directly written to the remaining space in the structured file system when the file write operation is the first write operation. Alternatively, the target reserved area of ​​the file to be processed can be determined, and the file segments in the file to be processed can be written to the free reserved storage area in the target reserved area in sequence according to the configuration parameters of the file segments in the file to be written.

[0075] Optionally, in some implementations, considering that the file to be processed is not being written to the structured file system for the first time, i.e., when the file write operation is a file segment update, the file segment of the file to be processed may be internally fragmented due to factors such as the update of the file segment of the file to be processed and the discontinuity of the remaining space in the structured file system. Therefore, the storage address of the file segment of the file to be processed in the structured file system is obtained, and steps 4012 to 4014 are executed to determine the internally fragmented data of the file to be processed.

[0076] Step 4012: Determine the discrete data of the file to be processed based on the storage address of each file fragment.

[0077] Discrete data is used to quantify the degree of disorder of file segments in the file to be processed.

[0078] In some implementations, the logical address range of the file to be processed can be determined based on its logical address. Then, based on this logical address range and the storage addresses of each file segment, discrete data of the file to be processed can be obtained. This discrete data can be the number of file segments with out-of-order storage addresses. For example, if file A has a logical address range of LAB10–LAB13, with corresponding storage addresses PPN10, PPN12, PPN11, and PPN13, and file B has a logical address range of LAB14–LAB17, with corresponding storage addresses PPN14–PPN17, then the storage addresses of file segments with logical addresses LAB11 and LAB12 in file A are out of order, and the discrete data of the file to be processed is determined to be 2; the discrete data of the file to be processed in file B is 0.

[0079] In other implementations, the number of file segments with disordered storage addresses in the file to be processed can be counted based on the configuration parameters of each file segment and the storage address of each file segment, thereby obtaining the discrete data of the file to be processed. Specifically, the discrete data determination method includes at least steps 40121 to 40123:

[0080] Step 40121: Determine the fragment order of each file fragment based on the configuration parameters of each file fragment in the file to be processed.

[0081] Step 40122: Determine the storage order of each file segment based on the storage address of each file segment in the file to be processed.

[0082] Step 40123: Based on the storage order of each file segment and the difference information between the segment order, determine the discrete data of the file to be processed.

[0083] Optionally, in some implementations, the difference information can be the order difference or similarity between the storage order and the fragment order of each file segment. For example, taking the difference information as the order difference between the storage order and the fragment order of each file segment, for each file segment, the order difference between the storage order and the fragment order can be determined. If the order difference is not equal to 0, it is determined that the storage order of the file segment is inconsistent with the fragment order of the file segment; if the order difference is equal to 0, it is determined that the storage order of the file segment is consistent with the fragment order of the file segment. The number of file segments in the file to be processed whose storage order is inconsistent with the fragment order of the file segment is counted to obtain the discrete data of the file to be processed. For example, taking file segments A1, A2, A3, and A4 of file A as having segment encoding as the configuration parameter, and their corresponding segment encodings being 001, 002, 003, and 004 respectively, then the segment order of file segments A1, A2, A3, and A4 of file A is 1, 2, 3, and 4 respectively. When the storage order of file segments A1, A2, A3, and A4 of file A is 4, 3, 2, and 1 respectively, that is, the storage order of each file segment in file A is inconsistent with the segment order of the file segment, then the discrete data 4 of the file to be processed is determined; when the storage order of file segments A1, A2, A3, and A4 of file A is 1, 2, 3, and 4 respectively, that is, the storage order of each file segment in file A is consistent with the segment order of the file segment, then the discrete data 0 of the file to be processed is determined.

[0084] Step 4013: Determine the hole data of the file to be processed based on the state difference information between each file segment and its adjacent file segments.

[0085] Among them, the state difference information represents the difference between the state information of each file segment and the state information of its adjacent file segments. Hollow data quantifies the hollowness of the file to be processed, whereby the hollowness is used to determine whether all file segments of the file to be processed are stored in a contiguous storage area. When the hollowness is not 0, it is determined that all file segments of the file to be processed are not stored in a contiguous storage area; when the hollowness is 0, it is determined that all file segments of the file to be processed are stored in a contiguous storage area. The state information includes the file identifier of the file segment and the storage status of the file segment, where the storage status includes stored and free.

[0086] In step 4013, the state information between each file segment and its adjacent file segments can be compared to determine the state difference information between each file segment and its adjacent file segments. Based on the state difference information, the hole data of the file to be processed can be determined. Specifically, the method for determining the hole data of the file to be processed includes at least steps 40131 to 40133:

[0087] Step 40131: For each file segment, compare the status information of that file segment with the status information of its adjacent file segments.

[0088] Step 40132: The number of adjacent file segments that are inconsistent with the state information of the file segment is determined as the local hole data of the file segment.

[0089] Step 40133: Obtain the hole data of the file to be processed based on the local hole data of each file fragment.

[0090] The adjacent file segment can be either the preceding file segment or the following file segment.

[0091] For example, taking the case where the adjacent file segment is the preceding file segment adjacent to the current file segment as an example, we can start from the first file segment of the file to be processed, compare the state information of the current file segment with the state information of its adjacent file segments. If the state information of the current file segment is consistent with the state information of its adjacent file segments, then we determine that the local hole data of the current file segment is 0, and determine the local hole data of the next file segment of the current file segment; if the state information of the current file segment is inconsistent with the state information of its adjacent file segments, then we set the next adjacent file segment of the adjacent file segment as the new adjacent file segment, and set the state information of the current file segment in relation to the new adjacent file segment. The status information of adjacent files is compared. If the status information of the file segment is consistent with the status information of the new adjacent file, the local hole data of the file segment is determined to be 1, and the local hole data of the next file segment of the file segment is determined. If the status information of the file segment is inconsistent with the status information of the new adjacent file, the status information of the file segment is compared with the status information of the next adjacent file segment of the new adjacent file. This process is repeated until the status information of the new adjacent file is consistent with the status information of the file segment to be tested. The number of adjacent file segments whose status information is inconsistent with the status information of the file segment is counted, the local hole data is obtained, and the local hole data of the next file segment of the file segment is determined.

[0092] Optionally, in some implementations, the local hole data of each file fragment can be aggregated to obtain the hole data of the file to be processed.

[0093] Optionally, in other embodiments, local void data of each file segment can be statistically analyzed to obtain statistical characteristics of the local void data, and these statistical characteristics can be set as the void data of the file to be processed. These statistical characteristics include, but are not limited to, the mean, maximum, and range of the local void data.

[0094] Step 4014: Based on the discrete data and void data of the file to be processed, determine the internal fragmented data of the file to be processed.

[0095] Optionally, in some implementations, the discrete data and gap data of the file to be processed can be aggregated to obtain the internal fragmented data of the file to be processed.

[0096] Alternatively, in other implementations, the arithmetic mean or weighted average of the discrete data and the gap data of the file to be processed can be determined as the internal fragmented data of the file to be processed.

[0097] Optionally, in other embodiments, the weights corresponding to the discrete data and the void data of the file to be processed can be obtained. Based on the discrete data and void data of the file to be processed, and the weights corresponding to the discrete data and void data of the file to be processed, a weighted sum of the discrete data and void data of the file to be processed can be obtained. The weighted sum of the discrete data and void data of the file to be processed can be determined as the internal fragmented data of the file to be processed.

[0098] Step 402: If the internal fragmented data of the file to be processed is greater than or equal to the preset internal fragmentation threshold, then obtain the configuration parameters of each file segment in the file to be processed and determine the target reserved area of ​​the file to be processed.

[0099] In this embodiment, after obtaining the internal fragmentation data of the file to be processed, the internal fragmentation data of the file to be processed can be compared with a preset internal fragmentation threshold. If the internal fragmentation data of the file to be processed is greater than or equal to the preset internal fragmentation threshold, it indicates that the internal fragmentation degree of the file to be processed is high. Then, the configuration parameters of each file segment in the file to be processed are obtained, and the target reserved area of ​​the file to be processed is determined.

[0100] In this embodiment, the target reserved area for the file to be processed can be determined based on the file size, cumulative number of writes, and creation time. Specifically, as shown... Figure 6 As shown, the present invention also provides a method for determining the target reserved area of ​​a file to be processed, including steps 4021 to 4023:

[0101] Step 4021: Determine the modification frequency of the file to be processed within the interval based on the time interval between the creation time of the file to be processed and the current time, and the cumulative number of writes to the file to be processed.

[0102] In some implementations, the modification frequency of the file to be processed within the interval can be obtained by accumulating the number of writes divided by the interval duration.

[0103] Step 4022: Determine the number of reserved storage areas for the files to be processed based on the modification frequency and the file size of the files to be processed.

[0104] Optionally, in some implementations, the storage space required for the file to be processed can be obtained by multiplying the value of the modification frequency and the file size of the file to be processed, the storage space size of each preset reserved storage area can be obtained, and the number of reserved storage areas for the file to be processed can be obtained by dividing the storage space size by the storage space size of each reserved storage area.

[0105] Optionally, in other embodiments, the storage space size of each reserved storage area can be obtained, and the initial number of reserved storage areas for the file to be processed can be obtained by dividing the file size of the file to be processed by the storage space size of each reserved storage area. The number of reserved storage areas can be obtained by (initial number + (initial data volume * modification frequency value)).

[0106] Optionally, in other embodiments, the relationship between the modification frequency and the quantity coefficient can be queried based on the modification frequency to obtain the target quantity coefficient corresponding to the preset modification frequency range. The product of the target quantity coefficient and the file size of the file to be processed is used to obtain the storage space required by the file to be processed. The storage space size of each preset reserved storage area is obtained, and the number of reserved storage areas for the file to be processed is obtained by dividing the storage space size by the storage space size of each reserved storage area. Alternatively, the storage space size of each preset reserved storage area is obtained, and the initial number of reserved storage areas for the file to be processed is obtained by dividing the file size of the file to be processed by the storage space size of each reserved storage area. The number of reserved storage areas is then obtained by multiplying the initial number by (initial number + (initial data volume * target quantity coefficient)). The relationship between the modification frequency and the quantity coefficient includes multiple preset modification frequency ranges and the quantity coefficient corresponding to each preset modification frequency range.

[0107] Optionally, in other embodiments, the modification frequency and the file size of the file to be processed can be input into a preset prediction model to obtain the number of reserved storage areas for the file to be processed. The prediction model can be a mathematical model, such as a linear equation model, an exponential equation model, etc.

[0108] Step 4023: Determine the target reserved area for the files to be processed based on the number of reserved storage areas for the files to be processed.

[0109] Among them, the number of target reserved areas in the target reserved area is greater than or equal to the number of reserved storage areas for the files to be processed.

[0110] Optionally, in some implementations, the same number of contiguous free storage areas can be determined from the remaining space of the structured file system based on the number of reserved storage areas for the files to be processed, and these contiguous free storage areas can be set as the target reserved areas for the files to be processed.

[0111] Optionally, in other embodiments, it can be determined whether there is an existing target reserved area associated with the file to be processed based on the file identifier of the file to be processed; if there is no existing target reserved area associated with the file to be processed, the storage space size of each preset reserved storage area is obtained, the initial number of reserved storage areas for the file to be processed is obtained by dividing the file size of the file to be processed by the storage space size of each reserved storage area, the final number of reserved storage areas for the file to be processed is obtained by adding the number of reserved storage areas for the file to be processed to the initial number, and the continuous free storage areas in the remaining space of the structured file system that have the same number as the final number are set as the target reserved areas for the file to be processed.

[0112] Optionally, in other embodiments, if there is an existing target reserved area associated with the file to be processed, the number of free reserved storage areas in the existing target reserved area is compared with the number of reserved storage areas for the file to be processed; if the number of free reserved storage areas in the existing target reserved area is greater than or equal to the number of reserved storage areas for the file to be processed, then the existing target reserved area associated with the file to be processed is set as the target reserved area for the file to be processed; if the number of free reserved storage areas in the existing target reserved area is less than or equal to the number of reserved storage areas for the file to be processed, then the existing target reserved area is updated to obtain the target reserved area for the file to be processed.

[0113] Updating an existing target reserved area can involve increasing the number of free reserved storage areas within that area. In some implementations, a target number of free storage areas contiguous with the existing target reserved area from the remaining space of the structured file system can be set as free reserved storage areas to increase the number of free reserved storage areas in the existing target reserved area. In other implementations, data in used reserved storage areas within the existing target reserved area can be merged or erased to increase the number of free reserved storage areas in the existing target reserved area. The target number can be determined based on the difference between the number of free reserved storage areas in the existing target reserved area and the number of reserved storage areas for the file to be processed. For example, the absolute value of the difference can be set as the target number, or the sum of the absolute value of the difference and a preset parameter can be set as the target number, where the preset parameter is a positive integer greater than 1.

[0114] In other implementations, when the internal fragmented data of the file to be processed is greater than or equal to a preset internal fragmentation threshold, it can be determined whether there is an existing target reserved area associated with the file to be processed based on the file identifier of the file to be processed; if there is no existing target reserved area associated with the file to be processed, the target reserved area of ​​the file to be processed is determined according to the above steps 4021 to 4023; if there is an existing target reserved area associated with the file to be processed, the target reserved area of ​​the file to be processed is determined based on the number of free reserved storage areas in the existing target reserved area, the file size of the file to be processed, the cumulative number of writes, and the creation time.

[0115] Optionally, in some embodiments, when there is an existing target reserved area associated with the file to be processed, the number of free reserved storage areas in the existing target reserved area can be compared with a preset number threshold; if the number of free reserved storage areas in the existing target reserved area is greater than the preset number threshold, then the existing target reserved area associated with the file to be processed is set as the target reserved area of ​​the file to be processed; if the number of free reserved storage areas in the existing target reserved area is less than or equal to the preset number threshold, then the number of reserved storage areas for the file to be processed is determined according to the above steps 4021 to 4023, and the existing target reserved area is updated according to the number of reserved storage areas for the file to be processed and step 4023 to obtain the target reserved area for the file to be processed.

[0116] Step 403: Determine the segment order of each file segment based on the configuration parameters of each file segment.

[0117] This embodiment can be referred to. Figure 3 The content of step 302 in the previous section will be understood and will not be elaborated here.

[0118] Optionally, in some embodiments of the present invention, after determining the target reserved area of ​​the file to be processed, the file segments can be sorted in ascending order according to the configuration parameters of each file segment to obtain the segment order of each file segment.

[0119] In other embodiments of the present invention, considering that in the off-site update mechanism, after updating a file segment in a structured file segment, the unupdated file segment is not immediately erased, but a garbage collection mechanism is periodically triggered to perform garbage collection on the file segments in the structured file system that need to be erased. Therefore, there may be at least two file segments with the same configuration information but different write times in the structured file system. If the segment order of each file segment is determined only based on the configuration parameters of each file segment, the segment order may be inaccurate. Based on this, in order to improve the accuracy of the segment order of file segments, the segment order of each file segment can be determined based on the configuration parameters and write time of each file segment in the file to be processed. For example, taking the segment encoding of the file segment as an example, the method for determining the segment order of file segments includes at least steps 4031 to 4033:

[0120] Step 4031: Determine whether there is a first file segment with the same segment encoding among the file segments.

[0121] Step 4032: If there are first file segments with the same segment encoding, then determine the time difference between the write time of each first file segment and the current time.

[0122] Step 4033: Sort the first file segment and the second file segment with the smallest time difference according to the segment encoding to obtain the segment order.

[0123] The second file segment is the file segment other than the first file segment among all file segments.

[0124] In some implementations, if there are no first file segments with the same segment code, the file segments are sorted in ascending order of their segment codes to obtain the segment order of the file segments.

[0125] In other embodiments of the present invention, in order to improve the accuracy of the fragment order of file segments, it can be determined whether each file segment in the file to be processed has an eraser mark. If each file segment in the file to be processed does not have an eraser mark, then the file segments are sorted in ascending order of their configuration parameters to obtain the fragment order of each file segment. If a file segment in the file to be processed has an eraser mark, then the file segment with the eraser mark is removed, and the remaining file segments are sorted in ascending order of their configuration parameters to obtain the fragment order.

[0126] Step 404: Determine the target location of each file segment in the target reserved area according to the segment order of each file segment.

[0127] In this embodiment, after determining the fragment order, the target position of each file fragment in the target reserved area can be determined according to the fragment order of each file fragment and the method in step 303. Repeated content will not be described here.

[0128] In other embodiments of the present invention, after determining the fragment order, the target free reserved storage area corresponding to the fragment order of each file fragment can be determined based on the storage address of the free reserved storage area in the target reserved area and the fragment order of each file fragment. The storage address of the target free reserved storage area corresponding to the fragment order of each file fragment is then set as the target location of that file fragment in the target reserved area. Specifically, as... Figure 7 As shown, the present invention provides a method for determining the target location of a file fragment in a target reserved area, including steps 4041 to 4043:

[0129] Step 4041: Obtain the storage address of each target reserved storage area in the target reserved area.

[0130] The target reserved storage area refers to the free reserved storage area within the target reserved area.

[0131] Step 4042: Determine the storage order of each target reserved storage area based on the storage address of each reserved storage area.

[0132] In some implementations, the target reserved storage areas can be sorted in ascending order of address based on their storage addresses, and the storage order of each target reserved storage area can be determined based on the sorted target reserved storage areas.

[0133] Step 4043: For each file segment, determine the storage address of the target reserved storage area whose storage order is the same as the segment order of the file segment as the target location of the file segment in the target reserved area.

[0134] In some embodiments of the present invention, after determining the target position of each file segment in the target reserved area, each file segment is written into its corresponding target position in the target reserved area according to the target position of each file segment in the target reserved area.

[0135] Step 405: Write each file fragment to its corresponding target location in the target reserved area.

[0136] This embodiment can be referred to. Figure 3 We will understand step 304 in the previous section, and will not repeat the same content here.

[0137] In this embodiment, when the file to be processed is a file to be written to the structured file system, each file segment is written to the target position of the file segment in the target reserved area according to the target position of each file segment in the target reserved area.

[0138] Optionally, in other implementations, when the file to be processed is a file already written in the structured file system, the storage address of the file segment of the file to be processed is obtained, and each file segment is synchronized to the target position of the file segment in the target reserved area according to the storage address of each file segment and the target position of each file segment in the target reserved area.

[0139] Optionally, in some implementations, the logical address of each file segment can be mapped to the target location of the file segment in the target reserved area based on the target location of each file segment in the target reserved area, the storage address of each file segment can be updated, and each file segment can be written to its corresponding target location in the target reserved area.

[0140] Optionally, in other embodiments, the storage address of each file segment can be obtained based on the target position of each file segment in the target reserved area, the offset position of each file segment can be determined based on the storage address of each file segment and the target position of the file segment in the target reserved area, each file segment can be copied to obtain a copy file segment based on the offset position of each file segment, and the copy file segment of each file segment can be written to the target position of the file segment in the target reserved area according to the offset position of the file segment.

[0141] In other embodiments of the present invention, considering that rewriting all file fragments in the file to be processed would involve a large amount of data and increase workload, and that when the file to be processed is a file already written in a structured file system, the storage address of some file fragments in the file to be processed is the same as the target position of that file fragment in the target reserved area, if these file fragments are also rewritten, the writing operation would be invalid and increase workload, therefore, in order to reduce the amount of data written to the target reserved area, the embodiments of the present invention determine the offset position of each file fragment based on the storage address of each file fragment and the target position of the file fragment in the target reserved area, filter out the target file fragments that need to be moved based on the offset position, and write the target file fragments to the target position of the target file fragments in the target reserved area. Specifically, the method of writing file fragments to the target reserved area includes steps 4051 to 4053:

[0142] Step 4051: Obtain the storage address of each file fragment.

[0143] Step 4052: Determine the offset position of each file segment based on its storage address and target position in the target reserved area.

[0144] The offset position represents the positional difference between the storage address of each file segment and the target position of that file segment in the target reserved area.

[0145] Step 4053: Write the target file segment with an offset position greater than the preset offset position into the target position corresponding to the target file segment in the target reserved area.

[0146] Optionally, in some implementations, for each file segment, the offset position of the file segment is compared with a preset offset position. If the offset position of the file segment is less than or equal to the preset offset position, it means that the storage address of the file segment is the same as the target position of the file segment in the target reserved area, and no processing is required for the file segment. Then, it is determined whether the offset position of the next file segment to be processed is greater than the preset offset position. If the offset position of the file segment is greater than the preset offset position, it means that the storage address of the file segment is different from the target position of the file segment in the target reserved area, and processing is required for the file segment. Then, the file segment is set as the target file segment. Each target file segment is synchronized to the target position of the target file segment in the target reserved area, so as to write the target file segment into the target reserved area at the target position corresponding to the target file segment.

[0147] For example, such as Figure 8 As shown, Figure 8 This is a schematic diagram of file fragmentation processing according to an embodiment of the present invention. For file fragments A1, A2, A3, and A4 of file A, wherein, as shown... Figure 8As shown in Figure (a), file fragments A2, A3, and A4 are stored in the existing target reserved area of ​​file A, and the storage addresses of file fragments A2, A3, and A4 are out of order. File fragment A1 is stored in the regular storage area of ​​the structured file system, meaning that file A has a high degree of internal fragmentation. Therefore, according to steps 4021-4023, the target reserved area of ​​file A is determined, and according to steps 4041-4043, the target positions of file fragments A1, A2, A3, and A4 in the target reserved area are determined. Then, according to steps 4051-4053, file fragments A1, A2, A3, and A4 are synchronized sequentially to their respective target positions in the target reserved area. Figure 8 As shown in Figure (b), file segment A1 in storage area 00 is synchronized to target location Y00 in the target reserved area; file segment A2 at location Y05 in the target reserved area is synchronized to target location Y01 in the target reserved area; file segment A3 at location Y06 in the target reserved area is synchronized to target location Y02 in the target reserved area; and file segment A2 at location Y04 in the target reserved area is synchronized to target location Y03 in the target reserved area. Simultaneously, file segments in storage area 00, target reserved area Y05, target reserved area Y06, and target reserved area Y04 are erased, so that file segments A1, A2, A3, and A4 are stored consecutively in the target reserved area according to segment order. Figure 8 As shown in Figure (c).

[0148] Optionally, in some embodiments of the present invention, after writing each target file segment in the file to be processed to the target location in the target reserved area, target file segments in the file to be processed that are not at the target location can be erased to increase the remaining space in the structured file system. Alternatively, target file segments in the file to be processed that are not at the target location can be marked as to be erased so that the target file segments carrying the erase-to-be-erased marker will be erased the next time the garbage collection mechanism is triggered.

[0149] Optionally, in some embodiments of the present invention, after each file segment is written to its corresponding target location in the target reserved area, when it is detected that a file segment of the file to be processed needs to be updated, it can be updated through an in-situ update mechanism. For example, when file segment A1 in the file to be processed needs to be updated, the received file segment A1 to be updated can be newly written to the target location M1 of file segment A1 in the target reserved area, overwriting the file segment A1 at the target location M1.

[0150] Optionally, in some embodiments of the present invention, after each file segment is written to its corresponding target location in the target reserved area, when it is detected that a file segment of the file to be processed needs to be updated, an off-site update mechanism can be used for updating. For example, when file segment A1 in the file to be processed needs to be updated, the received file segment A1 to be updated can be newly written to the first free reserved storage area in the free reserved storage area of ​​the target reserved area, and the file segment A1 stored at the target location M1 in the target reserved area can be erased. Steps 404 and 405 are then executed to newly write the file segment A1 to be updated to the target location M1.

[0151] Step 406: After step 401, if the internal fragmented data of the file to be processed is less than the preset internal fragmentation threshold, then the file to be processed to be written to the structured file system is written to the structured file system, or the file to be processed that has been written to the structured file system is not processed.

[0152] Specifically, writing the file to be processed into the structured file system can mean writing the file to the remaining space in the structured file system, or writing the file to the free reserved storage area in the target reserved area of ​​the structured file system. It's understandable that when the internal fragmentation of the file to be processed is low, writing the file to the target reserved area of ​​the structured file system does not mean writing each file fragment of the file to its corresponding target location in the target reserved area.

[0153] In this embodiment of the invention, the need to perform internal fragmentation processing on a file is determined by the file's internal fragmentation data. When internal fragmentation processing is required, a target reserved area is determined based on the file size, creation time, and cumulative write count. Each file segment is then written into the target reserved area according to its offset position, so that the file segments are arranged sequentially in the target reserved area, thereby reducing internal file fragmentation and alleviating the problem.

[0154] based on Figure 3 In the illustrated embodiment, to improve system performance and reduce IO consumption during sequential file reading, the file type can be determined based on the file attribute information of the file to be processed before obtaining the configuration parameters of each file segment in the file to be processed. If the file type is a preset target type, then the target reserved area of ​​the file to be processed can be determined. For example... Figure 9 As shown, Figure 9 This is a schematic diagram of another embodiment of the document processing method provided in this application. Figure 9 Another embodiment of the document processing method provided in this application may include the following steps:

[0155] Step 901: Determine the file attribute information of the file to be processed.

[0156] The file attribute information includes, but is not limited to, file size, internal fragmented data, number of file writes, file read size, and file read parameters. The file read parameters include the number of times the file to be processed will be read according to the target read method. The target read method includes sequential read.

[0157] In some implementations, the internal fragmented data of the file to be processed can be determined according to steps 4011 to 4014 above, which will not be elaborated here.

[0158] Step 902: Determine the file type of the file to be processed based on the file attribute information.

[0159] The file types include, but are not limited to, fragile files, fragment-sensitive files, non-fragile files, non-fragment-sensitive files, and normal files. Normal files refer to files that are neither fragile nor fragment-sensitive. Fragile files are files whose storage addresses are not contiguous, and fragment-sensitive files are files that need to be read sequentially.

[0160] In some implementations, in fragile file identification, the file type of the file to be processed can be determined based on file size, internal fragmentation data, and the number of times the file has been written. Specifically, the fragile file determination method includes steps 9021A to 9022A:

[0161] Step 9021A: Compare the file size with a first preset file size threshold, compare the internal fragmented data with a preset first fragmentation threshold, and compare the number of file writes with a preset file write count threshold.

[0162] In some implementations, the file size, internal fragmented data, and number of file writes of the file to be processed can be compared with their respective thresholds. For example, the file size can be compared with a first preset file size threshold, the internal fragmented data can be compared with a preset first fragmentation threshold, and the number of file writes can be compared with a preset file write count threshold in parallel.

[0163] In other implementations, the file size, number of file writes, and internal fragmentation data of the file to be processed can be compared sequentially with their respective thresholds. For example, the file size of the file to be processed can be compared with a first preset file size threshold. If the file size is less than the first preset file size threshold, the file type is determined to be a non-fragile file. If the file size is greater than or equal to the first preset file size threshold, the number of file writes can be compared with a preset file write count threshold. If the number of file writes is less than the preset file write count threshold, the file type is determined to be a non-fragile file. If the number of file writes is greater than or equal to the preset file write count threshold, the internal fragmentation data can be compared with a preset first fragmentation threshold. If the internal fragmentation data is less than the preset first fragmentation threshold, the file type is determined to be a non-fragile file; if the internal fragmentation data is greater than or equal to the preset first fragmentation threshold, the file type is determined to be a fragile file.

[0164] Step 9022A: If the file size is greater than or equal to a preset file size threshold, the internal fragmented data is greater than or equal to a preset first fragmentation threshold, and the number of file writes is greater than or equal to a preset file write count threshold, then the file type of the file to be processed is determined to be a fragile file.

[0165] In this embodiment, the file type of the file to be processed is determined to be a non-fragile file if the file to be processed meets one or more of the following conditions: the file size is less than a preset file size threshold, the internal fragmented data is less than a preset first fragmentation threshold, and the number of file writes is less than a preset file write count threshold.

[0166] In other embodiments, it can be queried whether the file to be processed carries a fragile file mark. If the file to be processed carries a fragile file mark, the file type is determined to be a fragile file. If the file to be processed does not carry a fragile file mark, the fragile file determination method shown in steps 9021A to 9022A above is executed to determine the file type of the file to be processed.

[0167] In some implementations, fragmented file identification determines whether a file is a fragment-sensitive file based on its size, internal fragmentation data, file read size, and file read parameters. Specifically, the method for determining fragment-sensitive files includes steps 9021B to 9022B:

[0168] Step 9021B: Compare the file size with the second preset file size threshold, compare the file read size with the preset file read size threshold, compare the internal fragmented data with the preset second fragmentation threshold, and compare the file read parameters with the preset read parameters.

[0169] Step 9022B: If the file size is greater than or equal to the second preset file size threshold, the file read size is greater than or equal to the preset file read size threshold, the internal fragmented data is greater than or equal to the preset second fragmentation threshold, and the file read parameters are greater than or equal to the preset read parameters, then the file type of the file to be processed is determined to be a fragile sensitive file.

[0170] In this embodiment, if the file to be processed meets one or more of the following conditions, the file type of the file to be processed is determined to be a non-fragment-sensitive file: the file size is less than the second preset file size threshold, the file read size is less than the preset file read size threshold, the internal fragmented data is less than the preset second fragmentation threshold, and the file read parameters are less than the preset read parameters.

[0171] In some implementations, it can be queried whether the file to be processed carries a fragment-sensitive file marker. If the file to be processed carries a fragment-sensitive file marker, the file type is determined to be a fragment-sensitive file. If the file to be processed does not carry a fragment-sensitive file marker, the fragment-sensitive file determination method shown in steps 9021B to 9022B above is executed to determine the file type of the file to be processed.

[0172] It should be noted that the embodiments of the present invention do not limit the specific values ​​of the preset first file size threshold, preset first fragmentation threshold, preset file write count threshold, preset second file size threshold, preset second fragmentation threshold, preset read parameters, and preset file read size threshold, and can be set according to specific application scenarios.

[0173] In some implementations, for the file to be processed, steps 9021A to 9022A and steps 9021B to 9022B described above can be executed in parallel to determine the file type of the file to be processed. Steps 9021A to 9022A and steps 9021B to 9022B can be executed in a preset execution order to determine the file type of the file to be processed. For example, the fragile file identification result of steps 9021A to 9022A can be executed first, followed by the fragment-sensitive file identification result of steps 9021B to 9022B. The file type of the file to be processed is determined based on the fragile file identification result and the fragment-sensitive file identification result. Alternatively, steps 9021B to 9022B can be executed first to obtain the fragment-sensitive file identification result, followed by the fragile file identification result of steps 9021A to 9022A. The file type of the file to be processed is determined based on the fragile file identification result and the fragment-sensitive file identification result. Among them, the fragile file identification result indicates whether the file to be processed is a fragile file, and the fragment-sensitive file identification result indicates whether the file to be processed is a fragment-sensitive file.

[0174] Step 903: If the file type is a preset target type, obtain the configuration parameters of each file segment in the file to be processed, and determine the target reserved area of ​​the file to be processed.

[0175] The target type includes at least one of fragile files and fragment-sensitive files.

[0176] In some implementations, if the file type is a preset target type, then reference can be made. Figure 6 The content of steps 4021 to 4023 will be understood and will not be elaborated here.

[0177] Step 904: Determine the segment order of each file segment based on the configuration parameters of each file segment.

[0178] This embodiment is referred to Figure 4 The content of step 403 in the previous section will be understood and will not be elaborated here.

[0179] Step 905: Determine the target location of each file segment in the target reserved area according to the segment order of each file segment.

[0180] This embodiment can be referred to. Figure 4 The content of step 404 in the previous section will be understood and will not be elaborated here.

[0181] Step 906: Write each file fragment to its corresponding target location in the target reserved area.

[0182] This embodiment can be referred to. Figure 4 We will understand step 405 in the text, and will not repeat the same content here.

[0183] Step 907: After step 902, if the file type is not the target type, the file to be processed to be written to the structured file system is written to the structured file system, or the file to be processed that has already been written to the structured file system is not processed.

[0184] In some implementations, when the file to be processed is a file already written in the structured file system, if the file type of the file to be processed is not a preset target type, then the file to be processed will not be internally fragmented.

[0185] In other implementations, when the file to be processed is a file to be written to the structured file system, if the file type of the file to be processed is not a preset target type, the file to be processed is written to the remaining space in the structured file system through a remote update mechanism, or the file to be processed is written to the target reserved area of ​​the file to be processed through a remote update mechanism, or the file to be processed is written to the remaining space in the structured file system through an in-situ update mechanism, or the file to be processed is written to the target reserved area of ​​the file to be processed through an in-situ update mechanism.

[0186] In this embodiment of the invention, the file type of the file to be processed is determined based on file attribute information. When the file type of the file to be processed is a preset target type, the target reserved area of ​​the file to be processed is determined for internal fragmentation processing. In this way, internal fragmentation processing can be performed on files to be processed that have internal fragmentation data less than a preset internal fragmentation threshold but require sequential reading, thereby reducing the IO consumption of such files during file reading and thus improving the performance of the storage device.

[0187] based on Figure 3 The illustrated embodiment, to further improve system performance and reduce the workload of the structured file system, firstly, the file type of the file to be processed can be determined according to the aforementioned methods for determining fragile files and fragmentation-sensitive files. When the file type of the file to be processed is a preset target type, it is determined whether to perform internal fragmentation processing based on the internal fragmentation data of the file to be processed. Through this two-round filtering based on file type and internal fragmentation data, the number of files requiring internal fragmentation processing is reduced, thereby reducing the workload of the structured file system. Figure 10 As shown, Figure 10 Another embodiment of the document processing method provided in this application may include the following steps:

[0188] Step 1001: Determine the file attribute information of the file to be processed.

[0189] File attribute information includes, but is not limited to, file size, internal fragmented data, number of file writes, file read size, and file read parameters. The file read parameters include the number of times the file to be processed will be read according to the target read method. The target read method includes sequential read.

[0190] In some implementations, the internal fragmented data of the file to be processed can be determined according to steps 4011 to 4014 above, which will not be elaborated here.

[0191] Step 1002: Determine the file type of the file to be processed based on the file attribute information.

[0192] The file types include, but are not limited to, fragile files, fragment-sensitive files, non-fragile files, non-fragment-sensitive files, and normal files. Normal files refer to files that are neither fragile nor fragment-sensitive. Fragile files are files whose storage addresses are not contiguous, and fragment-sensitive files are files that need to be read sequentially.

[0193] This embodiment can be referred to. Figure 9 Step 902 in the process determines the file type of the file to be processed, which will not be described in detail here.

[0194] Step 1003: If the file type is a preset target type and the internal fragmentation data of the file to be processed is greater than or equal to the preset internal fragmentation threshold, then obtain the configuration parameters of each file segment in the file to be processed and determine the target reserved area of ​​the file to be processed.

[0195] In some implementations, if the file type is a preset target type, the internal fragmentation data in the file attribute information of the file to be processed is compared with a preset internal fragmentation threshold. If the internal fragmentation data of the file to be processed is greater than or equal to the preset internal fragmentation threshold, the configuration parameters of each file segment in the file to be processed are obtained and the target reserved area of ​​the file to be processed is determined.

[0196] In some implementations, if the file type is a preset target type and the internal fragmentation data of the file to be processed is greater than or equal to a preset internal fragmentation threshold, then refer to Figure 6 The content of steps 4021 to 4023 will be understood and will not be elaborated here.

[0197] Step 1004: Determine the segment order of each file segment based on the configuration parameters of each file segment.

[0198] This embodiment can be referred to. Figure 4 We will understand step 403 in the previous section, and will not repeat the same content here.

[0199] Step 1005: Determine the target location of each file segment in the target reserved area according to the segment order of each file segment.

[0200] This embodiment can be referred to. Figure 4 We will understand step 404 in the previous section, and will not repeat the same content here.

[0201] Step 1006: Write each file fragment to its corresponding target location in the target reserved area.

[0202] This embodiment can be referred to. Figure 4 We will understand step 405 in the text, and will not repeat the same content here.

[0203] Step 1007: After step 1002, if the internal fragmentation data of the file to be processed is less than the preset internal fragmentation threshold, and / or the file type is not the target type, then the file to be processed to be written to the structured file system is written to the structured file system, or the file to be processed that has been written to the structured file system is not processed.

[0204] In some implementations, when the file to be processed is a file already written in the structured file system, if the internal fragmentation data of the file to be processed is less than a preset third fragmentation threshold, or if the file type of the file to be processed is not a preset target type, then the file to be processed will not be internally fragmented.

[0205] In other implementations, when the file to be processed is a file to be written to the structured file system, if the internal fragmentation data of the file to be processed is less than a preset internal fragmentation threshold, or the file type of the file to be processed is not a preset target type, then the file to be processed is written to the remaining space in the structured file system through an off-site update mechanism, or the file to be processed is written to the target reserved area of ​​the file to be processed through an off-site update mechanism, or the file to be processed is written to the remaining space in the structured file system through an in-situ update mechanism, or the file to be processed is written to the target reserved area of ​​the file to be processed through an in-situ update mechanism.

[0206] In this embodiment of the invention, the decision to perform internal fragmentation processing on the file to be processed is determined based on the file type and internal fragmentation data of the file to be processed. This avoids internal fragmentation processing on files to be processed that are only of the fragile type but do not require sequential reading, thereby reducing the workload of the structured file system and improving system stability.

[0207] based on Figure 3 The illustrated embodiment aims to improve system performance, enhance the processing efficiency of the structured file system during fragmentation, and reduce its workload. It retrieves files from a pre-established list of files to be processed and determines whether to perform internal fragmentation based on the internal fragmentation data of these files. This approach only requires determining whether to perform fragmentation based on the internal fragmentation data of the files in the list, thus improving the processing efficiency of the structured file system during fragmentation. Furthermore, by filtering based on both the list of files to be processed and their internal fragmentation data, the number of files to be processed is reduced, thereby lowering the workload of the structured file system.

[0208] like Figure 11 As shown, Figure 11 This is a schematic diagram of another embodiment of the document processing method provided in this application. Figure 11 Another embodiment of the document processing method provided in this application may include the following steps:

[0209] Step 1101: Traverse the pre-stored list of files to be processed, set the currently traversed file in the list as the file to be processed, and determine the internal fragmented data of the file to be processed.

[0210] In some implementations, the files stored in the list to be sorted are those carrying fragment-sensitive file tags and / or fragile file tags. The fragment-sensitive file tag indicates that the file type is fragment-sensitive, and the fragile file tag indicates that the file type is fragile.

[0211] Optionally, when the structured file system receives a file write request, it obtains the file attribute information of the file to be written corresponding to the file write request, determines the file type based on the file attribute information, and marks the file to be written as a fragment-sensitive file and / or a fragile file, and updates the list to be organized. Specifically, such as Figure 12 As shown, Figure 12 This is a schematic diagram of an embodiment of the method for updating a list to be organized provided in this application. The method for updating a list to be organized includes steps 1201 to 1205:

[0212] Step 1201: Respond to the file write request and obtain the file attribute information of the file to be written corresponding to the file write request.

[0213] File attribute information includes, but is not limited to, file size, internal fragmented data, number of file writes, file read size, and file read parameters. The file read parameters include the number of times the file to be written will be read according to the target read method. The target read method includes sequential read.

[0214] In some implementations, the internal fragmented data of the file to be written can be determined according to steps 4011 to 4014 above, which will not be elaborated here.

[0215] Step 1202: Determine the file type of the file to be written based on the file attribute information of the file to be written.

[0216] The file types include, but are not limited to, fragile files, fragment-sensitive files, non-fragile files, non-fragment-sensitive files, and normal files. Normal files refer to files that are neither fragile nor fragment-sensitive. Fragile files are files whose storage addresses are not contiguous, and fragment-sensitive files are files that need to be read sequentially.

[0217] This embodiment can be referred to. Figure 9 Step 902 in the process determines the file type of the file to be written, which will not be described in detail here.

[0218] Step 1203: If the file type of the file to be written is a preset target type, then the file to be written is marked.

[0219] In some implementations, if the file type to be written is a fragile file, then the file to be written is marked as a fragile file; if the file type to be written is a fragment-sensitive file, then the file to be written is marked as a fragment-sensitive file.

[0220] Step 1204: Write the marked file to be written to the target reserved area of ​​the structured file system, and add the file identifier of the marked file to be written to the pre-stored list to be organized.

[0221] In some implementations, when writing the marked file to be written to the target reserved area of ​​the structured file system, the file size, cumulative number of writes, and creation time of the file to be written can be referenced. Figure 6 The steps 4021 to 4023 in the process involve determining the target reserved area for the file to be written and writing the file to be written into the target reserved area, which will not be elaborated here.

[0222] In some implementations, when writing the marked file to be written to the target reserved area of ​​the structured file system, it is determined whether there is an existing target reserved area associated with the file to be written in the structured file system. If there is no existing target reserved area associated with the file to be written, then refer to... Figure 6 The steps 4021 to 4023 in the process determine the target reserved area of ​​the file to be written, which will not be described in detail here; if the file to be written has an existing target reserved area associated with it, then it is determined whether the file to be written carries a fragile file mark; if the file to be written does not carry a fragile file mark, then the file to be written is written to the remaining space of the structured file system.

[0223] For example, such as Figure 13 As shown, Figure 13 This is a schematic diagram of the file writing structure file system provided in an embodiment of the present invention, such as... Figure 13 As shown in Figure (a), when updating file segment A2 of file A, because file segments A1, A2, A3, and A4 in the regular storage area are discontinuous and out of order, file A is marked as a fragile file; when updating file segment A2 of file A, because file A has no associated existing target area, then according to... Figure 6 Steps 4021 to 4023 in the document determine the target reserved area for the file to be written, such as... Figure 13 As shown in Figure (b), the updated file segment A2 is written to the target reserved area Y00, and the file segment A2 in storage area 01 is erased; Figure 13As shown in Figure (c), when file A undergoes updates of file segments A3, A4, and A2 sequentially, the updated file segment A3 is written to the target reserved area Y01, the updated file segment A4 is written to the target reserved area Y02, and the next updated file segment A2 is written to the target reserved area Y03; and file segment A3 in storage area 10, file segment A4 in storage area 08, and file segment A2 in target reserved area Y00 are erased, as shown in Figure (c). Figure 13 As shown in Figure (d); Figure 13 As shown in Figure (e), when file segments A4, A2, and A3 of file A are updated sequentially again, the updated file segment A4 is written to the target reserved area Y04, the updated file segment A2 is written to the target reserved area Y05, and the updated file segment A3 is written to the target reserved area Y06. Figure 13 As shown, the file processing method provided by the present invention, after allocating a target reserved area to file A, the update behavior of file segments in subsequent files A will not affect the outside of the target reserved area, thereby alleviating the fragmentation problem of remaining space caused by file updates in the structured file system, that is, alleviating the file external fragmentation problem caused by file updates in the structured file system.

[0224] Step 1205: After step 1202, if the file type of the file to be written is not the preset target type, then the file to be written will be written to the structured file system.

[0225] In this embodiment, for a file to be written that carries a fragile file marker and / or a fragmentation-sensitive file marker, a continuous target reserved area is allocated to the file to be written, and the file to be written is written into the target reserved area, which can effectively alleviate the problem of external file fragmentation caused by updating and writing the file to be written.

[0226] In some implementation methods, see [reference] Figure 4 The content of step 406, which involves writing the file to be written to the structured file system, will not be elaborated here.

[0227] Step 1102: If the internal fragmentation data of the file to be processed is greater than or equal to the preset internal fragmentation threshold, then obtain the configuration parameters of each file segment in the file to be processed and determine the target reserved area of ​​the file to be processed.

[0228] In some implementations, if the internal fragmentation data of the file to be processed is greater than or equal to a preset internal fragmentation threshold, then refer to... Figure 6 The content of steps 4021 to 4023 will be understood and will not be elaborated here.

[0229] Step 1103: Determine the segment order of each file segment based on the configuration parameters of each file segment.

[0230] This embodiment can be referred to. Figure 4 We will understand step 403 in the previous section, and will not repeat the same content here.

[0231] Step 1104: Determine the target location of each file segment in the target reserved area according to the segment order of each file segment.

[0232] This embodiment can be referred to. Figure 4 We will understand step 404 in the previous section, and will not repeat the same content here.

[0233] Step 1105: Write each file fragment to its corresponding target location in the target reserved area.

[0234] This embodiment can be referred to. Figure 4 We will understand step 405 in the text, and will not repeat the same content here.

[0235] Step 1106: After step 1101, if the internal fragmented data of the file to be processed is less than the preset internal fragmentation threshold, then the file identifier of the file to be processed is removed from the pre-stored list to be sorted, and the next file in the pre-stored list to be sorted is traversed.

[0236] In this embodiment of the invention, it is only necessary to determine whether to perform fragmentation processing on the internal fragmented data of the files to be processed in the list to be organized, thereby improving the processing efficiency of the structured file system in the fragmentation process. Furthermore, the filtering is based on two conditions: the list to be organized and the internal fragmented data of the files to be processed, thereby reducing the number of files to be processed and achieving the effect of reducing the workload of the structured file system.

[0237] To better implement the file processing method provided in the embodiments of the present invention, based on the file processing method embodiments, the present invention provides a file processing apparatus, specifically, as follows: Figure 14 As shown, Figure 14 This is a schematic diagram of the structure of a document processing device provided in an embodiment of the present invention. The document processing device shown includes:

[0238] The acquisition module 1401 is used to acquire the configuration parameters of each file segment in the file to be processed and to determine the target reserved area of ​​the file to be processed.

[0239] The fragment sorting module 1402 is used to determine the fragment order of each file fragment according to the configuration parameters of each file fragment;

[0240] The fragment-sensitive file address mapping module 1403 is used to determine the target location of each file fragment in the target reserved area according to the fragment order of each file fragment;

[0241] The internal anti-fragmentation module 1404 is used to write each file fragment to its corresponding target location in the target reserved area.

[0242] In some implementations, the configuration parameters include the fragment encoding of the file segments; the fragment sorting module 1402 is used to determine whether there are first file segments with the same fragment encoding among the file segments; if there are first file segments with the same fragment encoding, the time difference between the write time of each first file segment and the current time is determined; according to the fragment encoding, the first file segment with the smallest time difference and the second file segment are sorted to obtain the fragment order; the second file segment is the file segment other than the first file segment among the file segments.

[0243] In some implementations, the fragment-sensitive file address mapping module 1403 is used to obtain the storage address of each target reserved storage area in the target reserved area; determine the storage order of each target reserved storage area according to the storage address of each target reserved storage area; and for each file fragment, determine the storage address of the target reserved storage area whose storage order is the same as the fragment order of the file fragment as the target location of the file fragment in the target reserved area.

[0244] In some implementations, the internal anti-fragmentation module 1404 is used to obtain the storage address of each file fragment; determine the offset position of each file fragment based on the storage address of each file fragment and the target position of the file fragment in the target reserved area; the offset position represents the position difference between the storage address of each file fragment and the target position of the file fragment in the target reserved area; for target file fragments with offset positions greater than preset offset positions, the target file fragment is written to the target position of the target file fragment in the target reserved area.

[0245] In some implementations, the acquisition module 1401 is used to determine the target reserved area of ​​the file to be processed based on the file size, cumulative number of writes, and creation time of the file to be processed.

[0246] In some implementations, the acquisition module 1401 is used to determine the modification frequency of the file to be processed within the interval based on the interval between the creation time of the file to be processed and the current time, and the cumulative number of writes to the file to be processed; determine the number of reserved storage areas for the file to be processed based on the modification frequency and the file size of the file to be processed; and determine the target reserved area for the file to be processed based on the number of reserved storage areas for the file to be processed, wherein the number of target reserved storage areas is greater than or equal to the number of reserved storage areas for the file to be processed.

[0247] In some embodiments, the document processing apparatus further includes;

[0248] The first judgment module 1405 is used to determine the internal fragmentation data of the file to be processed; if the internal fragmentation data of the file to be processed is greater than or equal to the preset internal fragmentation threshold, then the configuration parameters of each file segment in the file to be processed are obtained, and the target reserved area of ​​the file to be processed is determined.

[0249] In some implementations, the first judgment module 1405 is used to obtain the storage address of the file segments of the file to be processed; determine the discrete data of the file to be processed based on the storage address of each file segment; determine the void data of the file to be processed based on the state difference information between each file segment and the adjacent file segments of that file segment; and determine the internal fragmented data of the file to be processed based on the discrete data and void data of the file to be processed.

[0250] In some implementations, the first judgment module 1405 is used to compare the state information of each file segment with the state information of its neighboring file segments; determine the number of neighboring file segments whose state information is inconsistent with that of the file segment as the local hole data of the file segment; and obtain the hole degree of the file to be processed based on the local hole degree of each file segment.

[0251] In some implementations, the first judgment module 1405 is used to determine the segment order of each file segment in the file to be processed based on the configuration parameters of each file segment in the file to be processed; determine the storage order of each file segment based on the storage address of each file segment in the file to be processed; and determine the discrete data of the file to be processed based on the difference information between the storage order of each file segment and the segment order.

[0252] In some embodiments, the document processing apparatus further includes:

[0253] The second judgment module 1406 is used to determine the file attribute information of the file to be processed; based on the file attribute information, determine the file type of the file to be processed; if the file type is the target type, then execute the process of obtaining the configuration parameters of each file segment in the file to be processed and determining the target reserved area of ​​the file to be processed; the target type includes at least one of fragile files and fragment-sensitive files.

[0254] In some implementations, file attribute information includes file size, internal fragmented data, and number of file writes. The second judgment module 1406 includes:

[0255] The fragile file determination unit is used to compare the file size with a preset first file size threshold, compare the internal fragmented data with a preset first fragmentation threshold, and compare the number of file writes with a preset file write count threshold. If the file size is greater than or equal to the preset first file size threshold, the internal fragmented data is greater than or equal to the preset first fragmentation threshold, and the number of file writes is greater than or equal to the preset file write count threshold, then the file type of the file to be processed is determined to be a fragile file.

[0256] In some implementations, file attribute information includes file read size, file read parameters, file size, and internal fragmentation data; the second judgment module 1406 includes:

[0257] The fragmented sensitive file determination unit is used to compare the file size with a preset second file size threshold, compare the file read size with a preset file read size threshold, compare the internal fragmented data with a preset second fragmentation threshold, and compare the file read parameters with preset read parameters. The file read parameters include the number of times the file to be processed is read according to the target read method. If the file size is greater than or equal to the second preset file size threshold, the file read size is greater than or equal to the preset file read size threshold, the internal fragmented data is greater than or equal to the preset second fragmentation threshold, and the file read parameters are greater than or equal to the preset read parameters, then the file type of the file to be processed is determined to be a fragile sensitive file.

[0258] In some embodiments, the document processing apparatus further includes:

[0259] The third judgment module 1407 is used to determine the file attribute information of the file to be processed; the file attribute information includes file size and internal fragmentation data; based on the file attribute information, the file type of the file to be processed is determined; the target type includes at least one of fragile files and fragment-sensitive files; if the file type is the target type, and the internal fragmentation data of the file to be processed is greater than or equal to the preset internal fragmentation threshold, then the configuration parameters of each file segment in the file to be processed are obtained, and the target reserved area of ​​the file to be processed is determined.

[0260] The file processing apparatus provided in this embodiment of the invention determines the fragment order of each file fragment by configuring file fragment parameters, and determines the target position of each file fragment in the target reserved area according to the fragment order of each file fragment. In this way, it can ensure that each file fragment written in the target reserved area is continuous, thereby alleviating the problem of internal file fragmentation.

[0261] This invention also provides a file processing device, such as... Figure 15 As shown, it illustrates a structural schematic diagram of the document processing device involved in an embodiment of the present invention, specifically:

[0262] The file processing device may include components such as a processor 1501 with one or more processing cores, a memory 1502 with one or more computer-readable storage media, a power supply 1503, and an input unit 1504. Those skilled in the art will understand that... Figure 15 The document processing device structure shown does not constitute a limitation on the document processing device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0263] The processor 1501 is the control center of the file processing device. It connects various parts of the device via various interfaces and lines, and performs various functions and processes data by running or executing software programs and / or modules stored in the memory 1502, and by calling data stored in the memory 1502, thereby providing overall monitoring of the file processing device. Optionally, the processor 1501 may include one or more processing cores; preferably, the processor 1501 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 1501.

[0264] The memory 1502 can be used to store software programs and modules. The processor 1501 executes various functional applications and data processing by running the software programs and modules stored in the memory 1502. The memory 1502 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the file processing device, etc. In addition, the memory 1502 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 1502 may also include a memory controller to provide the processor 1501 with access to the memory 1502.

[0265] The document processing device also includes a power supply 1503 that supplies power to the various components. Preferably, the power supply 1503 can be logically connected to the processor 1501 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 1503 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0266] The file processing device may also include an input unit 1504, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0267] Although not shown, the file processing device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 1501 in the file processing device loads the executable files corresponding to the processes of one or more applications into the memory 1502 according to the following instructions, and the processor 1501 runs the applications stored in the memory 1502 to realize various functions, as follows:

[0268] Obtain the configuration parameters for each file segment in the file to be processed, and determine the target reserved area for the file to be processed;

[0269] The order of each file segment is determined based on its configuration parameters.

[0270] Based on the order of the file segments, determine the target location of each file segment in the target reserved area;

[0271] Write each file fragment to its corresponding target location in the target reserved area.

[0272] As will be understood by those skilled in the art, all or part of the steps in the various methods of the above embodiments can be performed by instructions or by controlling related hardware by instructions. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0273] To this end, embodiments of the present invention provide a storage medium storing a plurality of instructions that can be loaded by a processor to execute steps in any of the file processing methods provided in the embodiments of the present invention. For example, the instructions can execute the following steps:

[0274] Obtain the configuration parameters for each file segment in the file to be processed, and determine the target reserved area for the file to be processed;

[0275] The order of each file segment is determined based on its configuration parameters.

[0276] Based on the order of the file segments, determine the target location of each file segment in the target reserved area;

[0277] Write each file fragment to its corresponding target location in the target reserved area.

[0278] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0279] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0280] Since the instructions stored in the storage medium can execute the steps of any of the file processing methods provided in the embodiments of the present invention, the beneficial effects that any of the file processing methods provided in the embodiments of the present invention can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0281] The foregoing has provided a detailed description of a file processing method, apparatus, device, and storage medium provided by embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A file processing method, characterized in that, The method includes: Determine the internal fragmentation data of the file to be processed; if the internal fragmentation data of the file to be processed is greater than or equal to a preset internal fragmentation threshold, then obtain the configuration parameters of each file segment in the file to be processed. The configuration parameters are used to characterize the order of each file segment in the file to be processed, and determine the target reserved area of ​​the file to be processed. The order of each file segment is determined based on its configuration parameters. Based on the order of the file segments, determine the target location of each file segment in the target reserved area; Each of the aforementioned file fragments is written into its corresponding target location within the target reserved area; Determining the segment order of each file segment based on its configuration parameters includes: Determine whether there exists a first file segment with the same segment encoding among the file segments; If there are first file segments with the same segment encoding, then determine the time difference between the write time of each first file segment and the current time; According to the segment encoding, the first file segment and the second file segment with the smallest time difference are sorted to obtain the segment order; the second file segment is the file segment other than the first file segment among the file segments.

2. The file processing method as described in claim 1, characterized in that, Determining the target location of each file segment in the target reserved area based on the segment order of each file segment includes: Obtain the storage address of each target reserved storage area in the target reserved area; The storage order of each target reserved storage area is determined based on its storage address. For each file segment, the storage address of the target reserved storage area whose storage order is the same as the segment order of the file segment is determined as the target location of the file segment in the target reserved storage area.

3. The file processing method as described in claim 1, characterized in that, The method of determining the target position of each file segment in the target reserved area based on the segment order of each file segment includes: Obtain the storage address of each of the file segments; The offset position of each file segment is determined based on the storage address of each file segment and the target position of the file segment in the target reserved area; the offset position represents the position difference between the storage address of each file segment and the target position of the file segment in the target reserved area. For a target file segment whose offset position is greater than a preset offset position, the target file segment is written to the target position in the target reserved area.

4. The file processing method as described in claim 1, characterized in that, The determination of the target reserved area for the file to be processed includes: The target reserved area for the file to be processed is determined based on the file size, cumulative number of writes, and creation time of the file to be processed.

5. The file processing method as described in claim 4, characterized in that, The step of determining the target reserved area for the file to be processed based on its file size, cumulative write count, and creation time includes: Based on the interval between the creation time of the file to be processed and the current time, and the cumulative number of writes to the file to be processed, the modification frequency of the file to be processed within the interval is determined. Based on the modification frequency and the file size of the file to be processed, the number of reserved storage areas for the file to be processed is determined; Based on the number of reserved storage areas for the files to be processed, a target reserved area for the files to be processed is determined, wherein the number of target reserved storage areas is greater than or equal to the number of reserved storage areas for the files to be processed.

6. The file processing method as described in claim 1, characterized in that, The determination of the internal fragmentation data of the file to be processed includes: Obtain the storage address of the file segment of the file to be processed; Based on the storage address of each file segment, determine the discrete data of the file to be processed; Based on the state difference information between each file segment and its adjacent file segments, the hole data of the file to be processed is determined; Based on the discrete data and void data of the file to be processed, the internal fragmented data of the file to be processed is determined.

7. The file processing method as described in claim 6, characterized in that, The step of determining the hole data of the file to be processed based on the state difference information between each file segment and its adjacent data blocks includes: For each file segment, the state information of that file segment is compared with the state information of its adjacent file segments; The number of adjacent file segments whose state information is inconsistent with that file segment is determined as the local hole data of that file segment; The degree of voidness of the file to be processed is obtained based on the degree of local voidness of each file segment.

8. The file processing method as described in claim 6, characterized in that, The step of determining the discrete data of the file to be processed based on the storage address of each file fragment includes: The order of each file segment is determined based on the configuration parameters of each file segment in the file to be processed; The storage order of each file segment is determined based on the storage address of each file segment in the file to be processed; Based on the storage order of each file segment and the difference information between the segment order, the discrete data of the file to be processed is determined.

9. The document processing method according to any one of claims 1 to 5, characterized in that, Before obtaining the configuration parameters of each file segment in the file to be processed and determining the target reserved area of ​​the file to be processed, the method includes: Determine the file attribute information of the file to be processed; Based on the file attribute information, determine the file type of the file to be processed; If the file type is the target type, then the configuration parameters of each file segment in the file to be processed are obtained, and the target reserved area of ​​the file to be processed is determined; the target type includes at least one of fragile files and fragment-sensitive files.

10. The document processing method as described in claim 9, characterized in that, The file attribute information includes file size, internal fragmented data, and number of file writes; Determining the file type of the file to be processed based on the file attribute information includes: The file size is compared with a preset first file size threshold, the internal fragmented data is compared with a preset first fragmentation threshold, and the number of file writes is compared with a preset file write count threshold. If the file size is greater than or equal to the preset first file size threshold, the internal fragmented data is greater than or equal to the preset first fragmentation threshold, and the number of file writes is greater than or equal to the preset number of file writes threshold, then the file type of the file to be processed is determined to be a fragile file.

11. The document processing method as described in claim 9, characterized in that, The file attribute information includes file read size, file read parameters, file size, and internal fragmentation data; Determining the file type of the file to be processed based on the file attribute information includes: The file size is compared with a preset second file size threshold, the file read size is compared with a preset file read size threshold, the internal fragmented data is compared with a preset second fragmentation threshold, and the file read parameters are compared with preset read parameters; the file read parameters include the number of times the file to be processed is read according to the target read method; If the file size is greater than or equal to the preset second file size threshold, the file read size is greater than or equal to the preset file read size threshold, the internal fragmented data is greater than or equal to the preset second fragmentation threshold, and the file read parameters are greater than or equal to the preset read parameters, then the file type of the file to be processed is determined to be a fragile sensitive file.

12. The document processing method according to any one of claims 1 to 5, characterized in that, Before obtaining the configuration parameters of each file segment in the file to be processed and determining the target reserved area of ​​the file to be processed, the method includes: Determine the file attribute information of the file to be processed; the file attribute information includes file size and internal fragmentation data. Based on the file attribute information, determine the file type of the file to be processed; If the file type is the target type, and the internal fragmentation data of the file to be processed is greater than or equal to the preset internal fragmentation threshold, then the configuration parameters of each file segment in the file to be processed are obtained, and the target reserved area of ​​the file to be processed is determined; the target type includes at least one of fragile files and fragment-sensitive files.

13. A document processing device, characterized in that, The device includes: The acquisition module is used to determine the internal fragmentation data of the file to be processed; if the internal fragmentation data of the file to be processed is greater than or equal to a preset internal fragmentation threshold, the configuration parameters of each file segment in the file to be processed are acquired, the configuration parameters are used to characterize the order of each file segment in the file to be processed, and the target reserved area of ​​the file to be processed is determined. The fragment sorting module is used to determine the fragment order of each file fragment according to the configuration parameters of each file fragment; The fragment-sensitive file address mapping module is used to determine the target location of each file fragment in the target reserved area according to the fragment order of each file fragment; An internal anti-fragmentation recycling module is used to write each of the file fragments into its corresponding target location in the target reserved area; Determining the segment order of each file segment based on its configuration parameters includes: Determine whether there exists a first file segment with the same segment encoding among the file segments; If there are first file segments with the same segment encoding, then determine the time difference between the write time of each first file segment and the current time; According to the segment encoding, the first file segment and the second file segment with the smallest time difference are sorted to obtain the segment order; the second file segment is the file segment other than the first file segment among the file segments.

14. A document processing device, characterized in that, The device includes a memory and a processor; the memory stores an application program, and the processor runs the application program within the memory to perform the operations described in any one of claims 1 to 12.

15. A storage medium, characterized in that, The storage medium stores a plurality of instructions adapted for loading by a processor to perform the steps of the method according to any one of claims 1 to 12.

Images