File processing method and device, electronic equipment and storage medium
Through the file processing method based on reference grouping information, different files are removed and file blocks are built, which solves the problems of high server resource consumption and low file processing efficiency caused by the large number of small files during game version update, and achieves more efficient file processing and transmit.
Patent Information
- Application Number
- CN202411017736.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-29
- Publication Date
- 2024-11-15
AI Technical Summary
When the game version is updated, the number of small files is high, the server resource overhead is high, the network transmission time is long, and the file processing efficiency is low. Especially when the file block exceeds the threshold and needs to be reorganized, computing resources are consumed too much.
By dividing multiple files into groups based on reference grouping information, removing different files, building a different file pool, and generating file blocks based on the different files and the original group, reducing the number of reorganized files and improving file processing efficiency.
When the game version is updated, only the different files are reorganized, which reduces the number of file reorganizations, reduces server overhead, improves file processing efficiency, and reduces network transmission and disk fragmentation problems.
Smart Images

Figure CN118963799A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of file processing technology, and in particular to a file processing method, device, electronic equipment, and storage medium. Background Art
[0002] With the development of computer technology, games are welcomed by a large number of users, and the game distribution and construction processing technologies of major game manufacturers are becoming increasingly mature. When the game version is updated, an installation package will be built, and the user will complete the game update after downloading the installation package. The number of files in the installation package is very large, and many resource files are usually small, such as the Asset Bundles file of the Unity engine or the Pak Files file of the Unreal Engine. The server will have a high resource overhead when managing a large number of small files, and it will bring additional network overhead when distributing and downloading games, increase transmission time and bandwidth consumption, and also cause problems such as disk fragmentation.
[0003] In order to solve the problems caused by small files, relevant technologies will merge multiple small files into file blocks of a certain size according to the set threshold. However, after a new version of the game is released, a large number of file changes will occur, and the existing file blocks are likely to exceed the set threshold. This requires reorganization and merging of multiple small files. When there are a large number of small files, more computing resources are consumed and the file processing efficiency is low. Summary of the invention
[0004] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention provides a file processing method, device, electronic device, and storage medium, which can improve the efficiency of file merging and improve the efficiency of file processing.
[0005] In a first aspect, an embodiment of the present invention provides a file processing method, comprising:
[0006] Dividing the plurality of second files into a plurality of second groups based on reference grouping information, wherein the reference grouping information is used to indicate the grouping of the plurality of first files, the first files are installation files of a first version of a target application, the second files are installation files of a second version of the target application, and the first version is a previous version of the second version;
[0007] removing difference files from each of the second groups, and constructing at least one third group based on the plurality of difference files, wherein the difference files are the second files that are changed or newly added after the target application is updated from the first version to the second version;
[0008] A file block is generated based on each of the second grouping and the third grouping.
[0009] According to some embodiments of the present invention, before removing the difference files from each of the second groups, the method further includes:
[0010] determining the first file corresponding to the second file in the first version as an associated file;
[0011] When the associated file does not exist in the second file, determining the second file as the difference file;
[0012] Alternatively, when the associated file exists in the second file, the second file that is different from the associated file is determined as the difference file.
[0013] According to some embodiments of the present invention, determining the second file that is different from the associated file as the difference file includes:
[0014] Generate second meta information of the second file, and obtain first meta information of each of the first files;
[0015] generating difference information based on the first meta information of the associated file and the second meta information of the second file;
[0016] When the difference information is used to indicate a file change, the corresponding second file is determined as the difference file.
[0017] According to some embodiments of the present invention, the first meta information includes a first Hash value, and the first Hash value is generated based on the file content of the first file. Before dividing the plurality of second files into a plurality of second groups based on the reference grouping information, the method further includes:
[0018] Determine a reuse group based on the first Hash value, wherein the reuse group includes a plurality of the first files having the same first Hash value;
[0019] Determine any of the first files in the multiplexing group as a multiplexing file, and write a content multiplexing range in the first meta information of each of the first files, wherein the content multiplexing range is used to indicate a recording range of the content of the first file in the multiplexing file;
[0020] constructing grouping meta information based on all the first meta information of the multiplexing group, and deleting the first files except the multiplexing file in the multiplexing group;
[0021] A plurality of first groups are constructed based on a plurality of the first files that are not classified into the multiplexing group, and the reference group information is generated based on the first groups and the multiplexing group.
[0022] According to some embodiments of the present invention, removing the difference files from each of the second groups includes:
[0023] When the second group corresponds to the multiplexing group of the first version, retaining the multiplexing file;
[0024] The second meta information of the difference file is removed from the grouping meta information.
[0025] According to some embodiments of the present invention, the step of constructing at least one third group based on the plurality of difference files comprises:
[0026] Obtaining a preset first threshold;
[0027] The plurality of difference files are divided into a plurality of third groups based on a preset rule, wherein a group size of each of the third groups is less than or equal to the first threshold.
[0028] According to some embodiments of the present invention, after removing the difference files from each of the second groups, the method further includes:
[0029] Obtaining a preset second threshold, wherein the second threshold is less than the first threshold;
[0030] Determine the second packet whose packet size is smaller than the second threshold as a packet to be reassembled;
[0031] reorganizing the plurality of to-be-reorganized packets into a plurality of fourth packets based on the preset rule, wherein the packet size of the fourth packets is greater than the second threshold, and the packet size of the fourth packets is less than or equal to the first threshold;
[0032] The file block is generated based on the fourth grouping.
[0033] According to some embodiments of the present invention, the preset rules include:
[0034] Determine a first size value and a target path of each target file, wherein when the preset rule is used to construct the third group, the target file is the difference file, or when the preset rule is used to construct the fourth group, the target file is the second file of the group to be reassembled;
[0035] Building a virtual file tree based on the superior-subordinate relationship of the plurality of target paths, and adding each of the target files to the corresponding leaf node, wherein each of the target paths corresponds to a leaf node of the virtual file tree;
[0036] A first candidate file is selected from the leaf node based on the first threshold, and a target group is constructed based on the first candidate file, wherein a group size of the target group is smaller than the first threshold, and the target group is the third group or the fourth group.
[0037] According to some embodiments of the present invention, selecting a first candidate file from the leaf node based on the first threshold, and constructing a target group based on the first candidate file includes:
[0038] When the first size value of the first candidate file is smaller than the first threshold, determining a difference between the first threshold and the first size value of the first candidate file as a second size value;
[0039] Determine the current leaf node as the first node, and determine the second node and the third node based on a preset search step, wherein the node distance between the second node and the first node is less than or equal to the search step, and the node distance between the third node and the first node is greater than the search step;
[0040] Matching a second candidate file at the first node based on the target matching condition, wherein the second candidate file is the largest of the target files whose first size value is less than or equal to the second size value;
[0041] When the second candidate file cannot be matched at the first node, selecting one of the second nodes to match the second candidate file;
[0042] When the second candidate file cannot be matched at any of the second nodes, any of the third nodes is selected to match the second candidate file;
[0043] When the second candidate file cannot be matched at any of the third nodes, constructing the target group based on the first candidate file;
[0044] When the second candidate file is successfully matched, the second size value is reduced based on the first size value of the second candidate file, and the next second candidate file is matched based on the updated second size value, until the updated second size value cannot match any of the target files, and the target group is constructed based on the first candidate file and all the second candidate files.
[0045] According to some embodiments of the present invention, when the preset rule is used to construct the fourth grouping, selecting the first candidate file from the leaf node based on the first threshold includes:
[0046] When at least two second candidate files are matched, determine a target count value of each second candidate file, wherein the target count value is used to indicate the number of versions of the second candidate file that have not been changed;
[0047] The second candidate file having the largest target count value is adopted.
[0048] In a second aspect, an embodiment of the present invention provides a file processing device, including:
[0049] a grouping module, configured to group the plurality of second files into a plurality of second groups based on reference grouping information, wherein the reference grouping information is used to indicate the grouping of the plurality of first files, the first files are installation files of a first version of a target application, the second files are installation files of a second version of the target application, and the first version is a previous version of the second version;
[0050] a reorganization module, configured to remove difference files from each of the second groups, and construct at least one third group based on the plurality of difference files, wherein the difference files are the second files that are changed or newly added after the target application is updated from the first version to the second version;
[0051] The first packaging module is configured to generate a file block based on the second group and the third group respectively.
[0052] In a third aspect, an embodiment of the present invention provides an electronic device comprising at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions executable by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the file processing method described in the first aspect above.
[0053] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the file processing method as described in the first aspect above.
[0054] According to the method of the embodiment of the present invention, multiple second files are divided into multiple second groups based on reference grouping information, second files that are changed when the version is updated are determined as difference files, difference files are removed from each second group, and are reorganized into third groups based on the difference files, and file blocks are constructed based on the second group and the third group. In this way, after obtaining the installation package of the new version, the grouping method of the previous version can be inherited, and only the difference files need to be reorganized, which reduces the number of files involved in the file reorganization, reduces the server overhead, and improves the file processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] Figure 1 It is a schematic diagram of the processing process of a file processing method according to an embodiment of the present invention;
[0056] Figure 2 is a flow chart of a file processing method provided by an embodiment of the present invention;
[0057] Figure 3 is a schematic diagram of file reuse provided by an embodiment of the present invention;
[0058] Figure 4 is a schematic diagram of a processing process of a preset rule provided by an embodiment of the present invention;
[0059] Figure 5 is a flowchart of a file processing method applied to a game scene provided by an embodiment of the present invention;
[0060] Figure 6 is a structural schematic diagram of a file processing device provided by an embodiment of the present invention;
[0061] Figure 7 It is a structural diagram of an electronic device provided by another embodiment of the present invention. DETAILED DESCRIPTION
[0062] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.
[0063] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.
[0064] In the description of the present invention, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.
[0065] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.
[0066] It should be noted that game manufacturers will release new versions of games according to operational needs during the game operation process. After the developers complete the development of the installation files, they will release the installation files to the server. The server will encapsulate the installation files into installation packages for the user to download to complete the game update. The number of files in the installation package is very large, and many resource files are usually small, such as the Asset Bundles file of the Unity engine or the Pak Files file of the Unreal Engine. The server will have a high resource overhead when managing a large number of small files, and it will bring additional network overhead when distributing and downloading games, increase transmission time and bandwidth consumption, and also cause problems such as disk fragmentation. In order to solve the problems caused by small files, related technologies will merge multiple small files into file blocks of a certain size according to a set threshold. However, after the new version of the game is released, a large number of file changes will be generated, and the existing file blocks are likely to exceed the set threshold. This requires the reorganization and merging of multiple small files. When the number of small files is large, more computing resources are consumed, and the file processing efficiency is low.
[0067] Therefore, an embodiment of the present invention provides a file processing method, which divides multiple second files into multiple second groups based on reference grouping information, determines the second files that change when the version is updated as differential files, removes the differential files from each second group, regroups into a third group based on the differential files, and completes the construction of file blocks based on the second group and the third group. In this way, after obtaining the installation package of the new version, the grouping method of the previous version can be inherited, and only the differential files need to be reorganized, which reduces the number of files involved in the file reorganization, reduces the server overhead, and improves the file processing efficiency.
[0068] In the present invention, a file processing method, device, electronic device, and computer-readable storage medium are provided, which are described in detail one by one in the following embodiments.
[0069] Reference Figure 1 , Figure 1 A schematic diagram of a processing process of a file processing method provided by an embodiment of the present invention is shown in FIG. Figure 1As shown, when the target application completes the release of the first installation package of the first version, the grouping of the multiple first files of the first installation package is saved as reference grouping information. When the target application is updated to the second version, the developer releases the second installation package to the server. The server divides the multiple second files of the second installation package into multiple second groups according to the reference grouping information, determines the second files that have changed compared to the first version after updating to the second version as differential files, takes the differential files out of the second group to build an active file pool, and maintains the grouping of the remaining second files and saves them to the inert file pool. The files in the inert file pool and the active file pool are isolated from each other, and the differential files in the active file pool are reorganized to obtain at least one third group, and file blocks are generated according to the second group. At the same time, file blocks are generated based on the third group. This reduces the reorganization operation of the inert file pool, reduces the number of file reorganizations, and improves the efficiency of file processing.
[0070] Reference Figure 2 , Figure 2 A flowchart of a file processing method provided by an embodiment of the present invention, wherein the file processing method includes but is not limited to the following steps:
[0071] S21, dividing the multiple second files into multiple second groups based on the reference grouping information, wherein the reference grouping information is used to indicate the grouping of the multiple first files, the first file is the installation file of the first version of the target application, the second file is the installation file of the second version of the target application, and the first version is the previous version of the second version.
[0072] Specifically, the installation files of different versions of the target application can be released in the form of resource packages, the installation files in the first resource package of the first version are first files, and the installation files in the second resource package of the second version are second files.
[0073] In actual applications, the first file and the second file can be different versions of the same file. The second file can be the modified first file, or it can be the first file that has not been modified after the new version is released. It is defined as the second file only because its version is the second version. Of course, the second file can also be an installation file newly added when the second version is released.
[0074] Exemplarily, taking a game as an example of the target application, after the first version of the game is released, the first installation package includes the first file 1 and the first file 2. During the development of the second version, the developer modified the first file 2 to obtain the second file 2, and added the second file 3. After the second version of the game is released, the second installation package includes the second file 1, the second file 2 and the second file 3, wherein the second file 1 is actually the first file 1, the only difference being that they belong to different versions.
[0075] It should be noted that the reference grouping information is the file grouping situation of the previous version. If the target application is released for the first time, the reference grouping information is empty, and there is no first version of this embodiment. In this case, each installation file can be directly determined as a difference file, and the grouping method of this embodiment is executed to complete file processing and save the corresponding reference grouping information. Therefore, this embodiment updates the reference grouping information once after each version is grouped, so that the grouping situation of the previous version can be directly applied for preliminary grouping when the next version is updated, thereby improving file processing efficiency.
[0076] In practical applications, the reference grouping information can be obtained by parsing the virtual release file system. The virtual release file system is a meta information summary of the version release file, and the reference grouping information is recorded in the meta information summary.
[0077] It should be noted that although a large number of files will change with each version update, there are also some files that will not change. After multiple version updates, continuously using the grouping of the previous version can gradually solidify some unchanged resource files in the same group, reduce the number of reorganized files, and improve the stability of the file group.
[0078] S22, removing difference files from each second group, and constructing at least one third group based on the multiple difference files, wherein the difference files are second files that are changed or newly added after the target application is updated from the first version to the second version.
[0079] Specifically, the second grouping is a set of files obtained by preliminary grouping based on the reference grouping information of the first version. The difference file is the file that has changed when the version is updated. The second file can be obtained by modifying the code based on the first file, or it can be a new file. The difference file of this embodiment is a file that the first version does not have or the content has changed.
[0080] In actual applications, the difference files do not only exist in the second group. Since the reference group information is obtained by grouping in the previous version based on the technical solution of this embodiment, the files involved in the second group are also limited to the files existing in the first version. If the second file is a new file, it cannot be classified into any second group based on the reference group information. In this embodiment, the new file is also determined as a difference file. Figure 1 As shown, if the difference file belongs to the second group, it is moved from the second group to the active file pool. If the difference file is a newly added file, it can be directly saved in the active file pool. This embodiment does not limit the difference file to be determined from the second group, but if there is a difference file in the second group, it will be removed.
[0081] In actual applications, after obtaining multiple difference files, an active file pool can be constructed based on the multiple difference files, and the remaining second files can be saved in the inert file pool in the form of a second group. The active file pool and the inert file pool are isolated from each other, so as to distinguish the unchanged second files and the difference files. Only the difference files can be reorganized to obtain the third group. The inert file pool can continue to use the existing second group, reducing a large number of file reorganizations and improving file processing efficiency.
[0082] In one implementation of the present invention, before removing the difference files from each second group, the method further includes but is not limited to the following steps:
[0083] The first file corresponding to the first version of the second file is determined as an associated file; when the second file has no associated file, the second file is determined as a difference file; or, when the second file has an associated file, the second file that is different from the associated file is determined as a difference file.
[0084] Specifically, the associated file is the first file corresponding to the second file in the first version. If the second file has an associated file and the content of the second file is different from the associated file, it can be determined that the second file is modified based on the first file, and the corresponding second file is determined as a difference file. If the content is the same, the second file has not been modified, but is just a representation of the first file in a different version, and is not determined as a difference file; if the second file does not have an associated file, the second file is a newly added file and is determined as a difference file for subsequent processing.
[0085] In actual applications, associated files can be identified by files with the same name under the same path. In the field of game development, the path of each resource file is relatively fixed. Therefore, if two files have the same path and file name, they can be determined to be two versions of the same file.
[0086] In practical applications, whether the second file is identical to the associated file can be determined based on content comparison or based on relevant file attributes. For example, if the file sizes are different, it can be determined that a change has occurred.
[0087] For example, in the first version of the game, there is file A under directory A, and in the second version of the game, there are file A and file B under directory A. Since the path and file name of file A are the same as those in the first version, the two files can be determined to be the second file and the associated file, and whether it is a difference file is determined based on whether the content of file A has changed; file B does not exist in the first version, so it is a new file and is determined to be a difference file.
[0088] In one implementation of the present invention, determining a second file that is different from the associated file as a difference file specifically includes but is not limited to the following steps:
[0089] Generate second meta information of the second file, obtain first meta information of each first file; generate difference information based on the first meta information of the associated file and the second meta information of the second file; when the difference information is used to indicate a file change, determine the corresponding second file as a difference file.
[0090] It should be noted that the first meta information and the second meta information have the same composition, the difference is that the first meta information belongs to the first file, and the second meta information belongs to the second file, and the first meta information and the second meta information can be generated based on the content of the corresponding file. The first meta information and the second meta information include the file name, file path, file size, modification time, file read and write attributes, and file content hash value. If it is the same file, the meta information generated in different versions is the same. Therefore, for the second file and the corresponding associated file, it is possible to determine whether the second file has changed through the difference detection of the first meta information and the second meta information. If the second file has changed, the second meta information will differ from the first meta information at least in the file size, modification time and file content hash value.
[0091] In practical applications, the difference information can be obtained based on a common difference detection algorithm, such as a common diff algorithm. The corresponding first meta-information and second meta-information are input into the diff algorithm to generate difference information. If there is no difference between the two, the difference information is empty. If there is a difference between the two, the difference information indicates the meta-information that has the difference, such as the file size and modification time mentioned above.
[0092] In an implementation of the present invention, the first meta information includes a first Hash value, and the first Hash value is generated based on the file content of the first file. Before dividing the plurality of second files into a plurality of second groups based on the reference grouping information, the method further includes but is not limited to the following steps:
[0093] A multiplexing group is determined based on a first hash value, wherein the multiplexing group includes multiple first files with the same first hash value; any first file in the multiplexing group is determined as a multiplexing file, and a content multiplexing range is written in the first meta information of each first file, wherein the content multiplexing range is used to indicate the recording range of the content of the first file in the multiplexing file; grouping meta information is constructed based on all the first meta information of the multiplexing group, and first files other than the multiplexing file are deleted in the multiplexing group; multiple first groups are constructed based on multiple first files that are not included in the multiplexing group, and reference group information is generated based on the first groups and the multiplexing group.
[0094] Specifically, the first hash value is a content hash generated based on the file content of the first file. The first hash value uniquely corresponds to the file content. When the first hash values of two files are the same, it can be determined that the contents of the two files are the same. This embodiment divides multiple first files with the same content into the same group based on the first hash value, and only retains one copy of the file. By increasing the content reuse range in the first meta information, the number of files is effectively reduced, the size of the installation package is reduced, and the network traffic overhead for downloading the installation package is reduced.
[0095] In actual applications, the first hash value is generated together with other metadata after the file is released. After the first metadata is generated for each first file, the first hash value can be matched in the same directory first. Files with the same content in the game field are usually saved in the same directory. Of course, the entire file system can also be searched based on the first hash value to determine multiple first files with the same first hash value, and a reuse group can be constructed based on the above first files.
[0096] It should be noted that after determining the reuse group, this embodiment retains only one copy of the file as the reuse file in the reuse group. Since the content is the same, any first file can be used as the reuse file, and the content reuse range is written in the first meta information of each first file to indicate the content reference of the reuse file. After constructing the reuse group, the grouping meta information is generated, and the first meta information of each first file is recorded through the grouping meta information. When accessing the reuse group, the first meta information of the corresponding first file can be obtained from the grouping meta information. The original first file can be restored through the content reference and the reuse file, thereby realizing resource reuse.
[0097] For example, Figure 3 As shown, taking file A, file B, file C and file D as examples, file A, file B and file C have the same first hash value, a reuse group is constructed based on the above three files, file A is retained as a file copy, and the number of content lines is used as the content reuse range. Since the content is the same, the content reuse range of file A, file B and file C for the same file is (m, n), that is, the code from the mth line to the nth line, m and n are positive numbers; after completing resource reuse, grouping meta information is generated, which records meta information A of file A, meta information B of file B and meta information C of file C.
[0098] It should be noted that after the first file without resource reuse generates the first group, the grouping of the first version has been determined, and the reference grouping information is generated so that the grouping can be inherited after the second version is released.
[0099] In one implementation of the present invention, removing the difference files from each second group specifically includes but is not limited to the following steps:
[0100] When the second group corresponds to the reuse group of the first version, the reuse file is retained; and the second meta information of the difference file is removed from the group meta information.
[0101] It should be noted that the second grouping is obtained based on the reference grouping information. This embodiment inherits the grouping situation of the previous version based on the reference grouping information, so at least one second grouping is obtained by inheriting the multiplexing group. If the second group is a multiplexing group and there is no difference file, it can be directly retained; if the second group includes difference files, since resource reuse is performed and the multiplexed files are retained, it is only necessary to delete all the second files related to the multiplexing group and delete the second meta-information of the difference files in the grouping meta-information. There is no need to repeat the multiplexing to improve the file processing efficiency.
[0102] For example, Figure 3 As shown, after updating to the second version, file C is a differential file, and file C is put into the active file pool to construct the third group, and the metadata C is deleted from the group metadata of the reuse group.
[0103] In an implementation of the present invention, at least one third group is constructed based on a plurality of difference files, specifically including but not limited to the following steps:
[0104] A preset first threshold is obtained; and a plurality of difference files are divided into a plurality of third groups based on a preset rule, wherein a group size of each third group is less than or equal to the first threshold.
[0105] Specifically, the first threshold is used to characterize the size of the file blocks that need to be merged for the resource package to be released. By setting the first threshold, the group size can be as close to the first threshold as possible when constructing the third group, thereby maximizing the grouping and packaging, thereby improving resource utilization.
[0106] Exemplarily, the first threshold takes a positive number S as an example, and is combined according to the file size of the difference files. If the file size Sa of a third file is smaller than S, another third file can be selected to construct a third group, and finally Σ(1, n)Fs≈S is satisfied, wherein n is the number of difference files in the third group, and Fs is the file size of the difference file. This embodiment limits the group size of the third group to be less than or equal to the first threshold, ensuring that the merging conditions of the resource packages are met.
[0107] In one implementation of the present invention, after removing the difference files from each second group, the method further includes but is not limited to the following steps:
[0108] A preset second threshold is obtained, wherein the second threshold is smaller than the first threshold; a second group whose group size is smaller than the second threshold is determined as a group to be reassembled; based on a preset rule, a plurality of groups to be reassembled are reassembled into a plurality of fourth groups, wherein a group size of the fourth group is larger than the second threshold, and a group size of the fourth group is smaller than or equal to the first threshold; and a file block is generated based on the fourth group.
[0109] It should be noted that, for the second group in the inert file pool, the files in the group have not changed during this update, so the file size is also the same as the first version. If the second group has more or larger difference files, after removing the difference files, the second group will be smaller, which is not conducive to space utilization. In this embodiment, a second threshold is set to represent the second group with a smaller group size. When the group size of the second group is less than the second threshold, the corresponding second group is determined as the group to be reorganized, and multiple fourth groups are reorganized so that the group size of the fourth group is close to the first threshold. The fourth groups are merged to obtain file blocks to improve space utilization.
[0110] In one implementation of the present invention, the preset rules include:
[0111] Determine a first size value and a target path for each target file, wherein when the preset rule is used to construct a third group, the target file is a differential file, or, when the preset rule is used to construct a fourth group, the target file is the second file of the group to be reorganized; construct a virtual file tree based on the superior-subordinate relationship of multiple target paths, and add each target file to a corresponding leaf node, wherein each target path corresponds to a leaf node of the virtual file tree; select a first candidate file from the leaf node based on a first threshold, and construct a target group based on the first candidate file, wherein the group size of the target group is less than the first threshold, and the target group is the third group or the fourth group.
[0112] It is worth noting that the present embodiment aims to reduce the number of files to be reorganized. After determining the group to be reorganized and the active file pool, the second file of the group to be reorganized is isolated from the third file of the active file pool, and all the second files are grouped based on preset rules. In addition, all the difference files are grouped based on the same preset rule. Therefore, when the preset rule is used to construct the third group, the difference file is used as the target file, and the second file of the group to be reorganized is not considered. When the preset rule is used to construct the fourth group, the difference file is not considered in the same way. It can further distinguish between inert files and active files, so that inert files can be accumulated continuously with the update of the version and distinguished from active files, thereby improving the stability of file grouping. The above-mentioned inert files are files that have not been changed during this version change, and the active files are difference files.
[0113] In actual applications, the target path is the longest path where the target file is located. The virtual file tree is the upper and lower levels of the target path to which the target file belongs. It is not necessary that each directory has a target file. Therefore, this embodiment does not limit the parent and child leaf nodes to be adjacent two-level nodes, as long as the upper and lower relationships can be reflected, that is, the parent node can be the first N levels of the child node, N is an integer greater than 1. Each path node is a leaf node, the root directory is the root node, and the target file is classified into the leaf node corresponding to the target path to which it belongs, and will not be classified into the upper level node of the target path.
[0114] Exemplarily, the constructed virtual file tree is as follows: Figure 4 As shown, path A is the root node, and its child nodes include path A-1 and path A-2. Path A-1 includes child nodes A-1-1 and A-1-2. The next level path A-2-1 of path A-2 does not include the target node, so it is not included in the virtual file tree. Path A-2-1-1 has the target file, and it is determined as the next level leaf node of path A-2. In addition, the target file of path A-1-1 is not recorded in path A-1, and each target file is only recorded in the longest path.
[0115] It should be noted that after determining the virtual path tree, multiple candidate files are preferentially selected from the same leaf node to construct a target group. Since they come from the same path, the target files in the target group have a strong correlation. In addition, during the grouping process, it is sufficient to ensure that the group size is close to the first threshold, which will not be repeated here.
[0116] In an implementation of the present invention, a first candidate file is selected from a leaf node based on a first threshold, and a target group is constructed based on the first candidate file, specifically including but not limited to the following steps:
[0117] When the first size value of the first candidate file is less than the first threshold, the difference between the first threshold and the first size value of the first candidate file is determined as the second size value; the current leaf node is determined as the first node, and the second node and the third node are determined based on the preset search step, wherein the node distance between the second node and the first node is less than or equal to the search step, and the node distance between the third node and the first node is greater than the search step; the second candidate file is matched at the first node based on the target matching condition, wherein the second candidate file is the largest of the target files whose first size value is less than or equal to the second size value; when the second candidate file cannot be matched at the first node, any second node is selected to match the second candidate file; when the second candidate file cannot be matched at any second node, any third node is selected to match the second candidate file; when the second candidate file cannot be matched at any third node, a target group is constructed based on the first candidate file; when the second candidate file is successfully matched, the second size value is reduced based on the first size value of the second candidate file, and the next second candidate file is matched based on the updated second size value, until the updated second size value cannot match any target file, and a target group is constructed based on the first candidate file and all the second candidate files.
[0118] It should be noted that after the first candidate files are obtained, if the first size value of the first candidate files is greater than or equal to the first threshold, the first candidate files can be grouped independently, which is beyond the scope of discussion of this embodiment.
[0119] Specifically, the first node is the current leaf node, the second node is a related node, and the third node is an irrelevant node. This embodiment determines the relevance of the leaf nodes by searching the step size. Since the virtual file tree is built based on the path, the closer the node distance is, the more likely multiple subdirectories are to belong to the same parent directory, and the stronger the correlation of the contents. If the node distance is farther, the two directories are likely to belong to different parent directories, and the correlation of the contents is weaker. This embodiment will give priority to merging files with stronger correlation, so that files with stronger correlation can be merged into the same file block.
[0120] For example, the search step size can be set according to actual needs, such as Figure 4 As shown, taking the search step size set to 2 as an example, taking the target path corresponding to the first node as path A-1-2 as an example, the node distance between the leaf node corresponding to path A-1-1 and the first node is 1, and the node distance between the leaf node corresponding to path A-1 and the first node is 2. The above two nodes are determined to be the second nodes, and the node distances between the remaining nodes and the first node are all greater than 2, and they are determined to be the third nodes.
[0121] In practical applications, when the first size value Sa of the first candidate file is less than the first threshold S, the second size value is determined to be (S-Sa), and the maximum value of the file size that satisfies Δ(S-Sa) is searched in the first node, so that the sum of the file sizes of the second candidate file and the first candidate file can be as close to S as possible. When the first node does not have the second candidate file, for example, the first size value of each target file of the first node is greater than (S-Sa), the same matching operation is performed from any second node. If the match fails, the match is performed from the next second node. If the second candidate file cannot be matched from any second node, the match is performed from the third node, and so on. The nodes are matched in sequence according to their relevance.
[0122] It should be noted that after a second candidate file is matched, the second size value is updated based on the first size value Sb of the second candidate file, that is, the second size value is updated to (S-Sa-Sb), and the above steps are repeated based on the second size value until a new second candidate file cannot be matched, and it can be determined that the group size of the target group has been as close to the first threshold S as possible within the optional range, the target group is grouped, and the corresponding file block is generated. Then a new first candidate file is selected from the remaining target files and the above rules are repeated until all files are combined.
[0123] Exemplarily, after the first candidate file is determined at the first node of path A-1-2, if each target file in the same path satisfies the first size value being less than S-Sa, the largest one is selected as the second candidate file. Taking the first size value of the first second candidate file as Sb as an example, the second size value is updated to S-Sa-Sb. If there is no target file meeting the condition at the first node, a search is performed from the second node of path A-1-1. If the search fails, a search is performed from the second node of path A-1. If further search fails, path A, path A-2 and path A-2-1-1 are tried in turn. If all the searches fail, a target group is constructed with the first candidate files and second candidate files already obtained, and the next first candidate file is selected.
[0124] It is worth noting that the grouping method for the difference file and the second file to be reorganized is the same, that is, a virtual file tree is constructed based on the difference file to complete the construction of the target group, and then a second virtual file tree is constructed based on the second file to be reorganized to complete the construction of the target group. Of course, this embodiment does not limit the operation order of the two file pools, and can ensure that the two file pools are isolated and execute the same grouping rules.
[0125] Through the technical solution of this embodiment, not only can the size of the file block be close to the first threshold S, but the principle of relevance priority can also be used to select directories, so that the target group can converge to its related target path as much as possible, thereby improving the file relevance within the file block.
[0126] In an implementation of the present invention, when the preset rule is used to construct the fourth grouping, selecting the first candidate file from the leaf node based on the first threshold specifically includes but is not limited to the following steps:
[0127] When at least two second candidate files are matched, a target count value of each second candidate file is determined, wherein the target count value is used to indicate the cumulative number of unchanged versions of the second candidate file; and the second candidate file with the largest target count value is adopted.
[0128] Specifically, the target count value is the number of versions of the second candidate file that have not changed. The target count value is a cumulative value. If one of the versions changes, the target count value is reset to zero. The larger the target count value, the less likely the second candidate file is to be updated with the version change, and the greater the file stability. Grouping multiple files with greater stability can reduce the probability of different files appearing in the group when subsequent versions are updated, making the corresponding grouping more stable and able to directly inherit the grouping of the previous version, thereby reducing the number of reorganized files.
[0129] In actual applications, solidified resource files usually do not change frequently. For example, system files and underlying logic files in the game field do not change frequently with version updates. If the above files are divided into the same target group, the probability of generating difference files and triggering reorganization is small. The higher the grouping stability, the number of groups to be reorganized can be reduced with version iteration, thereby improving file processing efficiency.
[0130] Exemplarily, the target count value of the second candidate file A is 3, and the target count value of the second candidate file B is 1. During reorganization, both the second candidate file A and the second candidate file B satisfy the second size value. Even if the first size value of the second candidate file B is greater than the first size value of the second candidate file A, the second candidate file A can be preferentially used for reorganization with the first candidate file, so that the stability of the fourth grouping is higher.
[0131] S23, generating a file block based on the second group and the third group respectively.
[0132] Specifically, the file block is a data block obtained by merging each group, merging each second file in the second group into a file package, or merging each difference file in the third group into a file package, and then compressing the file package to obtain the file block. In this embodiment, the second group is used to represent the unchanged inert files, and the third group is used to represent the changed active files. The two types of files are processed in isolation, and the files between the second group and the third group are isolated from each other. An independent file block is generated based on the second group, and an independent file block is synchronously generated based on the third group.
[0133] It should be noted that the second file in the second group may change when the version is updated, or it may remain consistent with the corresponding first file. Therefore, if all files have not changed, the second group can meet the merging condition of the first version. In fact, at least one second file has changed during the update process, which causes the second group to no longer meet the merging condition due to the existence of the difference file. For example, the group size exceeds the upper limit of the file block. Therefore, after the difference file is removed in this embodiment, the remaining second files in the second group must be smaller than the upper limit of the file block, and the processed second group can be directly used to generate the file block.
[0134] For example, Figure 1 As shown, the second group includes group 1, group 2 and group 3. After the difference files of each group are moved into the active file pool, group 1, group 2 and group 3 are classified into the inert file pool. The inert file pool is used to store files that have not changed after version updates. The active file pool and the inert file pool are isolated from each other. File block 1 is generated based on group 1, file block 2 is generated based on group 2, and file block 3 is generated based on group 3. Of course, the merged block can also be generated after reorganization according to the above embodiment, which will not be elaborated here; similarly, the third group obtained based on the active file pool includes group 4 and group 5, file block 4 is generated based on group 4, and file block 5 is generated based on group 5.
[0135] The following further describes the file processing method provided by the present invention by taking its application in a game scenario as an example. Figure 5 A flowchart of a file processing method applied to a game scene provided by an embodiment of the present invention is shown, which specifically includes the following steps:
[0136] S51, publishing a second version of the game resource package on the server, loading and parsing the virtual publishing file system published in the previous version to obtain reference grouping information, and dividing multiple resource files of the game resource package into multiple second groups based on the reference grouping information.
[0137] Among them, the resource file is the second file in the above embodiment.
[0138] S52, based on the published resources of the second version, generate second metadata for each second file, determine the associated files in the first version, generate difference information based on the diff algorithm, the first metadata and the second metadata, determine the changed and newly added files as active files based on the difference information, and determine the unchanged files as inert files.
[0139] The active file is the difference file in the above embodiment, and the inert file is the second file other than the difference file in the above embodiment.
[0140] S53, eliminating active files in the second group, building an active file pool based on the active files, determining the active files as target files, and executing step S55.
[0141] S54, determining the group size of the second group after eliminating the active file, and if it is smaller than the second threshold, performing reorganization, determining the inert file to be reorganized as the target file, and executing step S55.
[0142] S55, generating a virtual file tree according to the target path of the target file, randomly selecting a first candidate file from a leaf node of the virtual file tree, and determining related nodes and irrelevant nodes according to the search step.
[0143] Among them, the relevant node is the second node in the above embodiment, and the irrelevant node is the third node in the above embodiment.
[0144] S56, if the first size value of the first candidate file is less than the first threshold, the maximum value satisfying Δ(S-Sa) is preferentially searched in the current leaf node. If the search fails, the related nodes and irrelevant nodes are tried in turn. After the second candidate node is found, Δ(S-Sa-Sb) is applied and the search step is repeated until the second candidate node that meets the conditions cannot be found and the target group satisfies Σ(1, n)Fs≈S.
[0145] Among them, Δ(S-Sa) is the above-mentioned second size value, Δ(S-Sa-Sb) is the second size value updated based on the first size value of the second candidate node, and Σ(1, n)Fs is the group size of the target group.
[0146] S57, during the grouping process, if the first hash values recorded in the first meta information of multiple target files in the target group are the same, retain one reused file, record the first meta information of each target file into the grouping meta information, and write the reused content range into the first meta information.
[0147] S58: If the second group is a multiplexing group and includes a difference file, the difference file is moved into the active file pool to execute step S53, and the first meta information of the difference file is removed from the group meta information.
[0148] S59, generating file blocks based on the groups obtained in the above steps.
[0149] In the file processing method applied to the game launcher provided by the embodiment of the present invention, multiple second files are divided into multiple second groups based on reference grouping information, the second files that are changed during version update are determined as difference files, the difference files are removed from each second group, a virtual file tree is constructed based on the relevance of the file paths, the difference files are reorganized into a third group based on the virtual file tree, and files with the same content are reused to reduce redundant copies to reduce the package size. Finally, the construction of the file blocks is completed based on the second group and the third group, which effectively reduces the number of files involved in the file reorganization, reduces the server overhead, and improves the file processing efficiency.
[0150] Corresponding to the above method embodiment, the present invention also provides an embodiment of a file processing device, Figure 6 FIG. 2 is a schematic diagram showing the structure of a file processing device according to an embodiment of the present invention. Figure 6 As shown, the file processing device comprises:
[0151] A grouping module 601 is configured to group the plurality of second files into a plurality of second groups based on reference grouping information, wherein the reference grouping information is used to indicate the grouping of the plurality of first files, the first file is an installation file of a first version of a target application, the second file is an installation file of a second version of the target application, and the first version is a previous version of the second version;
[0152] The reorganization module 602 is configured to remove the difference files from each second group and construct at least one third group based on the multiple difference files, wherein the difference files are second files that are changed or newly added after the target application is updated from the first version to the second version;
[0153] The first packaging module 603 is configured to generate a file block based on the second grouping and the third grouping.
[0154] In an implementation of the embodiment of the present invention, the file processing device is further configured to:
[0155] A file association module is configured to determine the first file corresponding to the first version of the second file as an associated file;
[0156] A first difference identification module is configured to determine the second file as a difference file when there is no associated file for the second file;
[0157] The second difference identification module is configured to, when there is an associated file with the second file, determine the second file that is different from the associated file as a difference file.
[0158] In an implementation of the embodiment of the present invention, the second difference identification module is further configured as follows:
[0159] A meta information acquisition module, configured to generate second meta information of the second file and acquire first meta information of each first file;
[0160] A difference detection module is configured to generate difference information based on the first meta information of the associated file and the second meta information of the second file;
[0161] The third difference identification module is configured to determine the corresponding second file as a difference file when the difference information is used to indicate a file change.
[0162] In addition, in an implementation of the present invention, the first meta information includes a first Hash value, the first Hash value is generated based on the file content of the first file, and the file processing device is further configured as follows:
[0163] a multiplexing grouping module, configured to determine a multiplexing group based on the first Hash value, wherein the multiplexing group includes a plurality of first files having the same first Hash value;
[0164] A multiplexing preparation module is configured to determine any first file in the multiplexing group as a multiplexing file, and write a content multiplexing range in the first meta information of each first file, wherein the content multiplexing range is used to indicate a recording range of the content of the first file in the multiplexing file;
[0165] A file multiplexing module is configured to construct grouping meta information based on all first meta information of the multiplexing group, and delete first files other than the multiplexing file in the multiplexing group;
[0166] The reference group information generating module is configured to construct a plurality of first groups based on a plurality of first files that are not classified into the multiplexing group, and to generate reference group information based on the first groups and the multiplexing group.
[0167] In addition, in one implementation of the present invention, the reorganization module 602 is further configured to:
[0168] a multiplexing reservation module configured to reserve the multiplexing file when the second group corresponds to the multiplexing group of the first version;
[0169] The multiplexing information removing module is configured to remove the second meta information of the difference file from the grouping meta information.
[0170] In addition, in one implementation of the present invention, the reorganization module 602 is further configured to:
[0171] A first threshold value acquisition module is configured to acquire a preset first threshold value;
[0172] The first rule application module is configured to divide the plurality of difference files into a plurality of third groups based on a preset rule, wherein the group size of each third group is less than or equal to a first threshold.
[0173] In addition, in one implementation of the present invention, the file processing device is further configured as follows:
[0174] A second threshold value acquisition module is configured to acquire a preset second threshold value, wherein the second threshold value is smaller than the first threshold value;
[0175] a reassembly determination module, configured to determine a second packet whose packet size is smaller than a second threshold as a packet to be reassembled;
[0176] A second rule application module is configured to reorganize the plurality of to-be-reorganized packets into a plurality of fourth packets based on a preset rule, wherein a packet size of the fourth packets is greater than a second threshold, and a packet size of the fourth packets is less than or equal to a first threshold;
[0177] The second packaging module is configured to generate a file block based on the fourth grouping.
[0178] In addition, in one implementation of the present invention, the file processing device is further configured as follows:
[0179] A file information collection module is configured to determine a first size value and a target path of each target file, wherein when the preset rule is used to construct the third grouping, the target file is a difference file, or when the preset rule is used to construct the fourth grouping, the target file is the second file of the grouping to be reassembled;
[0180] A file tree construction module is configured to construct a virtual file tree based on the superior-subordinate relationship of multiple target paths, and add each target file to a corresponding leaf node, wherein each target path corresponds to a leaf node of the virtual file tree;
[0181] The first file selection module is configured to select a first candidate file from a leaf node based on a first threshold, and construct a target group based on the first candidate file, wherein the group size of the target group is less than the first threshold, and the target group is the third group or the fourth group.
[0182] In addition, in one implementation of the present invention, the first file selection module is further configured to:
[0183] A first difference determination module is configured to determine, when the first size value of the first candidate file is less than a first threshold value, a difference between the first threshold value and the first size value of the first candidate file as a second size value;
[0184] A node determination module is configured to determine a current leaf node as a first node, and determine a second node and a third node based on a preset search step, wherein a node distance between the second node and the first node is less than or equal to the search step, and a node distance between the third node and the first node is greater than the search step;
[0185] A first matching module is configured to match a second candidate file at the first node based on a target matching condition, wherein the second candidate file is the largest target file whose first size value is less than or equal to the second size value;
[0186] The second matching module is configured to select a second node to match the second candidate file when the second candidate file cannot be matched at the first node;
[0187] A third matching module is configured to select a third node to match the second candidate file when the second candidate file cannot be matched at any second node;
[0188] A first grouping module is configured to construct a target group based on the first candidate file when the second candidate file cannot be matched at any third node;
[0189] The second grouping module is configured to, when successfully matching a second candidate file, reduce the second size value based on the first size value of the second candidate file, continue to match the next second candidate file based on the updated second size value, until the updated second size value cannot match any target file, and construct a target group based on the first candidate file and all the second candidate files.
[0190] In addition, in an implementation of the present invention, when the preset rule is used to construct the fourth group, the first file selection module is further configured as follows:
[0191] A counting module is configured to determine a target count value of each second candidate file when at least two second candidate files are matched, wherein the target count value is used to indicate the number of cumulative unchanged versions of the second candidate file;
[0192] The file selection module is configured to adopt the second candidate file with the largest target count value.
[0193] In the file processing device provided by the embodiment of the present invention, multiple second files are divided into multiple second groups based on reference grouping information, second files that are changed when the version is updated are determined as difference files, difference files are removed from each second group, and are reorganized into third groups based on the difference files, and file blocks are constructed based on the second group and the third group. In this way, after obtaining the installation package of the new version, the grouping method of the previous version can be inherited, and only the difference files need to be reorganized, which reduces the number of files involved in the file reorganization, reduces the server overhead, and improves the file processing efficiency.
[0194] The above is a schematic scheme of a file processing device of this embodiment. It should be noted that the technical scheme of the file processing device and the technical scheme of the above-mentioned file processing method belong to the same concept, and the details not described in detail in the technical scheme of the file processing device can be referred to the description of the technical scheme of the above-mentioned file processing method.
[0195] like Figure 7 As shown, Figure 7 The structure block diagram of an electronic device 700 provided according to an embodiment of the present invention is shown. The components of the electronic device 700 include but are not limited to a memory 710 and a processor 720. The processor 720 is connected to the memory 710 via a bus 730, and the database 750 is used to store data.
[0196] The electronic device 700 also includes an access device 740 that enables the electronic device 700 to communicate via one or more networks 760. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 740 may include one or more of any type of network interface (e.g., a network interface card (NIC)) that is wired or wireless, such as an IEEE 702.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.
[0197] In one embodiment of the present invention, the above components of the electronic device 700 and Figure 7 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 7 The electronic device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of the present invention. Those skilled in the art can add or replace other components as needed.
[0198] The electronic device 700 may be any type of stationary or mobile electronic device, including a mobile computer or mobile electronic device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable electronic device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary electronic device such as a desktop computer or a PC. The electronic device 700 may also be a mobile or stationary server.
[0199] The processor 720 is used to execute computer executable instructions of the object processing method.
[0200] The above is a schematic scheme of an electronic device of this embodiment. It should be noted that the technical scheme of the electronic device and the technical scheme of the object processing method described above are of the same concept, and the details not described in detail in the technical scheme of the electronic device can be referred to the description of the technical scheme of the object processing method described above.
[0201] An embodiment of the present invention further provides a storage medium, which is a computer-readable storage medium and stores a computer program. When the computer program is executed by a processor, the above-mentioned file processing method is implemented.
[0202] As a non-transient computer-readable storage medium, the memory can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage devices. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and are implemented to be located in one place, or may also be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.
[0203] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media generally include computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.
[0204] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above-mentioned implementation mode. Technical personnel familiar with the field can also make various equivalent deformations or substitutions under the shared conditions without violating the spirit of the present invention. These equivalent deformations or substitutions are all included in the scope defined by the claims of the present invention.
Claims
1. A file processing method, characterized in that: include: Dividing the plurality of second files into a plurality of second groups based on reference grouping information, wherein the reference grouping information is used to indicate the grouping of the plurality of first files, the first files are installation files of a first version of a target application, the second files are installation files of a second version of the target application, and the first version is a previous version of the second version; removing difference files from each of the second groups, and constructing at least one third group based on the plurality of difference files, wherein the difference files are the second files that are changed or newly added after the target application is updated from the first version to the second version; A file block is generated based on each of the second grouping and the third grouping.
2. The method according to claim 1, characterized in that Before removing the difference files from each of the second groups, the method further includes: determining the first file corresponding to the second file in the first version as an associated file; When the associated file does not exist in the second file, determining the second file as the difference file; Alternatively, when the associated file exists in the second file, the second file that is different from the associated file is determined as the difference file.
3. The method according to claim 2, characterized in that The step of determining the second file which is different from the associated file as the difference file includes: Generate second meta information of the second file, and obtain first meta information of each of the first files; generating difference information based on the first meta information of the associated file and the second meta information of the second file; When the difference information is used to indicate a file change, the corresponding second file is determined as the difference file.
4. The method according to claim 3, characterized in that The first meta information includes a first Hash value, and the first Hash value is generated based on the file content of the first file. Before dividing the plurality of second files into a plurality of second groups based on the reference grouping information, the method further includes: Determine a reuse group based on the first Hash value, wherein the reuse group includes a plurality of the first files having the same first Hash value; Determine any of the first files in the multiplexing group as a multiplexing file, and write a content multiplexing range into the first meta information of each of the first files, wherein the content multiplexing range is used to indicate a recording range of the content of the first file in the multiplexing file; constructing grouping meta information based on all the first meta information of the multiplexing group, and deleting the first files except the multiplexing file in the multiplexing group; A plurality of first groups are constructed based on a plurality of the first files that are not classified into the multiplexing group, and the reference group information is generated based on the first groups and the multiplexing group.
5. The method according to claim 4, characterized in that The removing the difference files from each of the second groups comprises: When the second group corresponds to the multiplexing group of the first version, retaining the multiplexing file; The second meta information of the difference file is removed from the grouping meta information.
6. The method according to claim 1, characterized in that The constructing at least one third group based on the plurality of difference files comprises: Obtaining a preset first threshold; The plurality of difference files are divided into a plurality of third groups based on a preset rule, wherein a group size of each of the third groups is less than or equal to the first threshold.
7. The method according to claim 6, characterized in that After removing the difference files from each of the second groups, the method further includes: Obtaining a preset second threshold, wherein the second threshold is less than the first threshold; Determine the second packet whose packet size is smaller than the second threshold as a packet to be reassembled; reorganizing the plurality of to-be-reorganized packets into a plurality of fourth packets based on the preset rule, wherein the packet size of the fourth packets is greater than the second threshold, and the packet size of the fourth packets is less than or equal to the first threshold; The file block is generated based on the fourth grouping.
8. The method according to claim 7, characterized in that The preset rules include: Determine a first size value and a target path of each target file, wherein when the preset rule is used to construct the third group, the target file is the difference file, or when the preset rule is used to construct the fourth group, the target file is the second file of the group to be reassembled; Building a virtual file tree based on the superior-subordinate relationship of the plurality of target paths, and adding each of the target files to the corresponding leaf node, wherein each of the target paths corresponds to a leaf node of the virtual file tree; A first candidate file is selected from the leaf node based on the first threshold, and a target group is constructed based on the first candidate file, wherein a group size of the target group is smaller than the first threshold, and the target group is the third group or the fourth group.
9. The method according to claim 8, characterized in that The selecting a first candidate file from the leaf node based on the first threshold, and constructing a target group based on the first candidate file, comprises: When the first size value of the first candidate file is smaller than the first threshold, determining a difference between the first threshold and the first size value of the first candidate file as a second size value; Determine the current leaf node as the first node, and determine the second node and the third node based on a preset search step, wherein the node distance between the second node and the first node is less than or equal to the search step, and the node distance between the third node and the first node is greater than the search step; Matching a second candidate file at the first node based on the target matching condition, wherein the second candidate file is the largest of the target files whose first size value is less than or equal to the second size value; When the second candidate file cannot be matched at the first node, selecting one of the second nodes to match the second candidate file; When the second candidate file cannot be matched at any of the second nodes, any of the third nodes is selected to match the second candidate file; When the second candidate file cannot be matched at any of the third nodes, constructing the target group based on the first candidate file; When the second candidate file is successfully matched, the second size value is reduced based on the first size value of the second candidate file, and the next second candidate file is matched based on the updated second size value, until the updated second size value cannot match any of the target files, and the target group is constructed based on the first candidate file and all the second candidate files.
10. The method according to claim 9, characterized in that When the preset rule is used to construct the fourth grouping, the selecting the first candidate file from the leaf node based on the first threshold comprises: When at least two second candidate files are matched, determine a target count value of each second candidate file, wherein the target count value is used to indicate the number of versions of the second candidate file that have not been changed; The second candidate file having the largest target count value is adopted.
11. A file processing device, characterized in that: include: a grouping module, configured to group the plurality of second files into a plurality of second groups based on reference grouping information, wherein the reference grouping information is used to indicate the grouping of the plurality of first files, the first files are installation files of a first version of a target application, the second files are installation files of a second version of the target application, and the first version is a previous version of the second version; a reorganization module, configured to remove difference files from each of the second groups, and construct at least one third group based on the plurality of difference files, wherein the difference files are the second files that are changed or newly added after the target application is updated from the first version to the second version; The first packaging module is configured to generate a file block based on the second group and the third group respectively.
12. An electronic device, characterized in that: It includes at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the file processing method as described in any one of claims 1 to 10.
13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the file processing method according to any one of claims 1 to 10.
