A method and related device for data management
By dividing files into differential files based on conditions and performing parallel calculations, the method addresses the inefficiency of single-thread serial processing in data updates, enhancing update efficiency through parallel processing.
Patent Information
- Application Number
- CN202010588837.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-06-24
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2040-06-24
AI Technical Summary
Existing data update methods using single-thread serial processing are time-consuming, especially with large data packets, affecting update efficiency.
A data management method that involves traversing files in different data packets to determine differences, dividing files into multiple differential files based on conditions like thresholds, file formats, or sizes, and performing parallel differential calculations to generate a patch package for efficient updates.
The method ensures efficient data update by parallel processing, reducing time consumption and ensuring normal multi-threaded operation, thereby improving update efficiency.
Smart Images

Figure CN111580851B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a data management method and related devices. Background Art
[0002] With the development of Internet technology, more and more applications appear in people's lives, and applications often need to be updated. How to improve the update efficiency of applications has become a difficult problem.
[0003] Generally, the old version data package and the new version data package can be compared for updating, that is, executing the differential process. The differential process is divided into traversing the files in the old and new version packages, decompressing the difference files in the new version package, merging the difference files in the new version package, generating a patch package, assembling the differential package, and then providing updates to the terminal application based on the differential package.
[0004] However, the above method uses single-thread serial processing in the process of obtaining differential packets in the server, which is time-consuming; and the time consumption will be greatly increased when the data packet is large, affecting the data update efficiency. Summary of the invention
[0005] In view of this, the present application provides a method for data management, which can effectively avoid the situation where the update time is very long when the data packet is large, and improve the efficiency of data update.
[0006] A first aspect of the present application provides a data management method, which can be applied to a system or program including a data management function in a terminal device, and specifically comprises: acquiring a first data packet and a second data packet, wherein the first data packet and the second data packet correspond to different versions of a target application;
[0007] Traversing the participating files contained in the first data packet and the second data packet to determine a difference file, wherein the difference file includes a directory difference item or a content difference item in the first data packet and the second data packet;
[0008] Dividing the participating files into at least two differential files based on a differential condition, wherein the differential condition includes at least one of the following: a preset threshold, a file format, and a file size, wherein the preset threshold is used to indicate the number of content difference items contained in the differential file, the file format corresponds to the differential file, and the file size is used to indicate the size of the differential file;
[0009] A parallel difference calculation is performed based on the difference file to obtain a patch package, where the patch package is used to instruct the terminal to perform a version update on the target application.
[0010] Optionally, in some possible implementations of the present application, if the differential condition is a preset threshold R, then dividing the participating files into at least two differential files based on the differential condition includes:
[0011] Determine the content difference item, the directory difference item or the same item in the difference file corresponding to the participating file, wherein the participating file includes N files included in the first data packet and M files included in the second data packet, where N and M are positive integers;
[0012] Dividing the content difference items based on the preset threshold R, the preset threshold is R, R<N and R<M, R is a positive integer;
[0013] Decompressing the divided content difference items to write at least two of the difference files;
[0014] The directory difference items or identical items after division are directly written into the differential file.
[0015] Optionally, in some possible implementations of the present application, if the difference condition is a file format, then dividing the participating files into at least two difference files based on the difference condition includes:
[0016] determining at least two file formats of the participating files;
[0017] The files in the first data packet and the files in the second data packet are divided based on the file format to obtain at least two differential files.
[0018] Optionally, in some possible implementations of the present application, the determining of at least two file formats of the participating files includes:
[0019] Determining the file types included in the participating files;
[0020] Determine multiple format information corresponding to the file type;
[0021] The file format is determined based on the format information.
[0022] Optionally, in some possible implementations of the present application, the method further includes:
[0023] Determining a format difference algorithm based on a file format corresponding to the difference file;
[0024] Parallel differential calculations are performed based on the format differential algorithm to update the patch package.
[0025] Optionally, in some possible implementations of the present application, the method further includes:
[0026] Generate a format catalog based on the file format;
[0027] Binding corresponding difference files according to the format directory to generate record information;
[0028] Patch format information is generated by combining the record information and the version information corresponding to the patch package, where the patch format information is used to instruct the terminal to identify the patch package.
[0029] Optionally, in some possible implementations of the present application, if the difference condition is a file size, then dividing the participating file into at least two difference files based on the difference condition includes:
[0030] Determine a partition threshold corresponding to the file size;
[0031] Based on the division threshold, files of corresponding sizes are extracted from the participating files to be divided into at least two of the differential files.
[0032] Optionally, in some possible implementations of the present application, if the file in the first data packet and the file in the second data packet correspond to the same file format, the difference condition is the preset threshold or the file size.
[0033] Optionally, in some possible implementations of the present application, dividing the participating file into at least two differential files based on a differential condition includes:
[0034] Obtain file parameters in the participating file, wherein the file parameters include the number or size of the file;
[0035] If the value indicated by the file parameter is greater than a file processing threshold, dividing the participating file into at least two threshold division files based on a first processing condition, the file processing threshold being set based on quantity or size, the first processing condition comprising at least one of the preset threshold, the file format or the file size;
[0036] Each of the threshold-divided files is divided into at least two differential files based on a second processing condition, wherein the second processing condition includes at least one of the preset threshold, the file format or the file size, and the first processing condition is different from the second processing condition.
[0037] Optionally, in some possible implementations of the present application, it is characterized in that the performing parallel differential calculation based on the differential file to obtain the patch package includes:
[0038] Determine a first temporary file and a second temporary file in the differential file, the first temporary file is used to indicate the differential file in the first data packet, and the second temporary file is used to indicate the differential file in the second data packet;
[0039] A differential calculation is performed based on the first temporary file and the second temporary file to obtain the patch package.
[0040] A second aspect of the present application provides a data management device, including: an acquisition unit, configured to acquire a first data packet and a second data packet, wherein the first data packet and the second data packet correspond to different versions of a target application;
[0041] A traversal unit, configured to traverse the participating files contained in the first data packet and the second data packet to determine a difference file, wherein the difference file includes a directory difference item or a content difference item in the first data packet and the second data packet;
[0042] a division unit, configured to divide the participating files into at least two differential files based on a differential condition, wherein the differential condition includes at least one of the following: a preset threshold, a file format, and a file size, wherein the preset threshold is used to indicate the number of the content difference items contained in the differential file, the file format corresponds to the differential file, and the file size is used to indicate the size of the differential file;
[0043] A management unit is used to perform parallel differential calculation based on the differential file to obtain a patch package, wherein the patch package is used to instruct the terminal to perform a version update on the target application.
[0044] Optionally, in some possible implementations of the present application, if the difference condition is a preset threshold R, the division unit is specifically used to determine the content difference item, the directory difference item, or the same item in the difference file corresponding to the participating file, the participating file including N files included in the first data packet and M files included in the second data packet, where N and M are positive integers;
[0045] The division unit is specifically used to divide the content difference items based on the preset threshold R, the preset threshold is R, R<N and R<M, and R is a positive integer;
[0046] The division unit is specifically used to decompress the content difference items after division to write into at least two of the difference files;
[0047] The division unit is specifically used to directly write the directory difference items or the same items after division into the differential file.
[0048] Optionally, in some possible implementations of the present application, if the difference condition is a file format, the division unit is specifically used to determine at least two file formats of the participating files;
[0049] The division unit is specifically configured to divide the files in the first data packet and the files in the second data packet based on the file format to obtain at least two differential files.
[0050] Optionally, in some possible implementations of the present application, the division unit is specifically used to determine the file types included in the participating files;
[0051] The division unit is specifically used to determine multiple format information corresponding to the file type;
[0052] The division unit is specifically configured to determine the file format based on the format information.
[0053] Optionally, in some possible implementations of the present application, the division unit is specifically used to determine a format difference algorithm based on a file format corresponding to the differential file;
[0054] The division unit is specifically used to perform parallel difference calculation based on the format difference algorithm to update the patch package.
[0055] Optionally, in some possible implementations of the present application, the division unit is specifically used to generate a format directory based on the file format;
[0056] The division unit is specifically used to bind the corresponding difference files according to the format directory to generate record information;
[0057] The division unit is specifically used to generate patch format information by combining the record information and the version information corresponding to the patch package, and the patch format information is used to instruct the terminal to identify the patch package.
[0058] Optionally, in some possible implementations of the present application, if the differential condition is a file size, the division unit is specifically used to determine a division threshold corresponding to the file size;
[0059] The division unit is specifically used to extract files of corresponding sizes from the participating files based on the division threshold, so as to divide them into at least two differential files.
[0060] Optionally, in some possible implementations of the present application, the division unit is specifically used to, if the file in the first data packet and the file in the second data packet correspond to the same file format, then the differential condition is the preset threshold or the file size.
[0061] Optionally, in some possible implementations of the present application, the partitioning unit is specifically used to obtain file parameters in the participating file, where the file parameters include the number or size of files;
[0062] The division unit is specifically configured to divide the participating file into at least two threshold division files based on a first processing condition if the value indicated by the file parameter is greater than a file processing threshold, the file processing threshold being set based on quantity or size, and the first processing condition comprising at least one of the preset threshold, the file format or the file size;
[0063] The division unit is specifically used to divide each of the threshold division files into at least two differential files based on a second processing condition, wherein the second processing condition includes at least one of the preset threshold, the file format or the file size, and the first processing condition is different from the second processing condition.
[0064] Optionally, in some possible implementations of the present application, it is characterized in that the management unit is specifically used to determine a first temporary file and a second temporary file in the differential file, the first temporary file is used to indicate the differential file in the first data packet, and the second temporary file is used to indicate the differential file in the second data packet;
[0065] The management unit is specifically used to perform differential calculation based on the first temporary file and the second temporary file to obtain the patch package.
[0066] A third aspect of the present application provides a data management method, comprising: obtaining a patch package sent by a server;
[0067] Determine at least two differential files indicated in the patch package, the differential files are obtained by the server dividing the files in the first data packet and the files in the second data packet based on a differential condition, the first data packet and the second data packet correspond to different versions of the target application, the differential condition includes at least one of the following: a preset threshold, a file format, and a file size, the preset threshold is used to indicate the number of the content difference items contained in the differential file, the file format corresponds to the differential file, and the file size is used to indicate the size of the differential file;
[0068] Performing a synthetic calculation based on the first data packet and the differential file to obtain the second data packet, wherein the first data packet is local version data;
[0069] A version update is performed based on the second data packet.
[0070] Optionally, in some possible implementations of the present application, performing synthetic calculation based on the first data packet and the differential file to obtain the second data packet includes:
[0071] Determining content difference items and directory difference items indicated in the differential file;
[0072] The content difference items and the directory difference items are processed in parallel based on the synthesis thread corresponding to each of the differential files, so as to update the first data packet to the second data packet.
[0073] A fourth aspect of the present application provides a data management device, including: an acquisition unit, configured to acquire a patch package sent by a server;
[0074] a determining unit, configured to determine at least two differential files indicated in the patch package, wherein the differential files are obtained by the server dividing the files in the first data packet and the files in the second data packet based on a differential condition, the first data packet and the second data packet correspond to different versions of the target application, and the differential condition includes at least one of the following: a preset threshold, a file format, and a file size, the preset threshold is used to indicate the number of content difference items contained in the differential file, the file format corresponds to the differential file, and the file size is used to indicate the size of the differential file;
[0075] A management unit, configured to perform synthesis calculation based on the first data packet and the differential file to obtain the second data packet, wherein the first data packet is local version data;
[0076] The management unit is further used to perform version update based on the second data packet.
[0077] Optionally, in some possible implementations of the present application, the management unit is specifically used to determine the content difference items and directory difference items indicated in the differential file;
[0078] The management unit is specifically configured to perform parallel processing on the content difference items and the directory difference items based on a synthesis thread corresponding to each of the differential files, so as to update the first data packet into the second data packet.
[0079] The fifth aspect of the present application provides a computer device, comprising: a memory, a processor and a bus system; the memory is used to store program code; the processor is used to execute the data management method described in the first aspect or any one of the first aspects, or the data management method described in the third aspect or any one of the third aspects according to the instructions in the program code.
[0080] In a sixth aspect, the present application provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on a computer, the computer executes the data management method described in the first aspect or any one of the first aspects, or the data management method described in the third aspect or any one of the third aspects.
[0081] In another aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes a computer instruction, and the computer instruction is stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the data management method provided in the first aspect or various optional implementations of the first aspect, or the data management method provided in the third aspect or various optional implementations of the third aspect.
[0082] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:
[0083] By obtaining a first data packet and a second data packet; then traversing the files in the first data packet and the files in the second data packet to determine a difference file, the difference file includes a directory difference item or a content difference item in the first data packet and the second data packet; and dividing the files in the first data packet and the files in the second data packet into at least two differential files based on at least one condition of a preset threshold, a file format or a file size, wherein the preset threshold is used to indicate the number of content difference items contained in the differential file, and each differential file corresponds to at least one file format; and then performing parallel differential calculation based on the differential file to obtain a patch package for instructing the terminal to perform a version update on the target application. Thus, the process of synthesizing the patch package by segments is realized. Since the process of synthesizing the patch package by segments is carried out in parallel, and the number of files that need to be decompressed and processed corresponding to each segment is the same, the normal multi-threaded parallel calculation is guaranteed, and the efficiency of data update is improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0084] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0085] Figure 1 A diagram of the network architecture that runs the data management system;
[0086] Figure 2 A data management process architecture diagram provided for an embodiment of the present application;
[0087] Figure 3 A flowchart of a data management method provided in an embodiment of the present application;
[0088] Figure 4 A schematic diagram of a data management scenario provided in an embodiment of the present application;
[0089] Figure 5 A schematic diagram of another data management scenario provided in an embodiment of the present application;
[0090] Figure 6 A schematic diagram of another data management scenario provided in an embodiment of the present application;
[0091] Figure 7 A flowchart of another data management method provided in an embodiment of the present application;
[0092] Figure 8 A schematic diagram of another data management scenario provided in an embodiment of the present application;
[0093] Fig. 9 A schematic diagram of another data management scenario provided in an embodiment of the present application;
[0094] Fig.10 A schematic diagram of another data management scenario provided in an embodiment of the present application;
[0095] Fig.11 A schematic diagram of another data management scenario provided in an embodiment of the present application;
[0096] Fig.12 A schematic diagram of another data management scenario provided in an embodiment of the present application;
[0097] Fig.13 A schematic diagram of another data management scenario provided in an embodiment of the present application;
[0098] Fig.14 A schematic diagram of the structure of a data management device provided in an embodiment of the present application;
[0099] Fig.15 A schematic diagram of the structure of another data management device provided in an embodiment of the present application;
[0100] Fig.16 A schematic diagram of the structure of a server provided in an embodiment of the present application;
[0101] Fig.17 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0102] The embodiment of the present application provides a data management method and related device, which can be applied to a system or program containing a data management function in a terminal device, by obtaining a first data packet and a second data packet; then traversing the files in the first data packet and the files in the second data packet to determine a difference file, the difference file includes a directory difference item or a content difference item in the first data packet and the second data packet; and dividing the files in the first data packet and the files in the second data packet into at least two difference files based on at least one condition of a preset threshold, a file format or a file size, wherein the preset threshold is used to indicate the number of content difference items contained in the difference file, and each difference file corresponds to at least one file format; and then performing parallel difference calculation based on the difference file to obtain a patch package for indicating the terminal to perform a version update on the target application. Thus, the process of synthesizing the patch package by segment is realized. Since the process of synthesizing the patch package by segment is carried out in parallel, and the number of files that need to be decompressed and processed corresponding to each segment is the same, the normal operation of multi-threaded parallel calculation is guaranteed, and the efficiency of data update is improved.
[0103] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "corresponding to" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0104] First, some terms that may appear in the embodiments of the present application are explained.
[0105] bsdiff / bspatch: an open source differential update algorithm, that is, the bsdiff algorithm is used on the server to generate a patch package, and the bspatch algorithm is used on the client to synthesize the old file and the patch package into a new file.
[0106] mergeBsdiff: An incremental update algorithm based on zip unpacking and bsdiff algorithm. It merges the difference files of the new and old versions and then performs bsdiff difference. It is applied to similar application platforms such as App Bao.
[0107] hdiffpatch: an open source differential update algorithm, that is, the hdiff algorithm is used on the server to generate a patch package, and the hpatch algorithm is used on the client to synthesize the old file and the patch package into a new file.
[0108] dexdiff / dexpatch: A deep differential synthesis algorithm for .dex format files.
[0109] Android application package (apk): used to distribute and install mobile applications and middleware. apk mainly contains files in .dex, .so, .png / .jpg, .json and other formats.
[0110] It should be understood that the data management method provided in the present application can be applied to a system or program including a data management function in a terminal device, such as an application platform. Specifically, the data management system can be run on Figure 1 In the network architecture shown in Figure 1 As shown in FIG. 1 , it is a network architecture diagram of the data management system. As can be seen from the figure, the data management system can provide data management with multiple information sources. The terminal establishes a connection with the server through the network. The server is responsible for detecting the version of the target application and obtaining the corresponding data packet. Then, the server generates a patch package according to the difference between the version data and sends the patch package to the terminal, so that the terminal can update the version of the local target application. It can be understood that Figure 1 A variety of terminal devices are shown in FIG. 1 . In actual scenarios, more or fewer types of terminal devices may participate in the data management process. The specific number and type depend on the actual scenario and are not limited here. In addition, Figure 1 One server is shown in the figure, but in actual scenarios, multiple servers may be involved, especially in scenarios where multiple content applications interact. The specific number of servers depends on the actual scenario.
[0111] In this embodiment, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, and this application does not limit this.
[0112] It should be noted that the data management method provided in this embodiment can also be performed offline, that is, without the participation of the server, and the terminal is connected to other terminals locally to perform data transmission and update between the terminals.
[0113] It can be understood that the above-mentioned data management system can run on a personal mobile terminal, for example: as an application such as an application platform, it can also run on a server, and it can also run on a third-party device to provide data management to obtain data management processing results of the information source; the specific data management system can be run in the above-mentioned device in the form of a program, and it can also be run as a system component in the above-mentioned device, and it can also be run as a cloud service program. The specific operation mode depends on the actual scenario and is not limited here.
[0114] With the development of Internet technology, more and more applications appear in people's lives, and applications often need to be updated. How to improve the update efficiency of applications has become a difficult problem.
[0115] Generally, the old version data package and the new version data package can be compared for updating, that is, executing the differential process. The differential process is divided into traversing the files in the old and new version packages, decompressing the difference files in the new version package, merging the difference files in the new version package, generating a patch package, assembling the differential package, and then providing updates to the terminal application based on the differential package.
[0116] However, the above method uses single-thread serial processing in the process of obtaining differential packets in the server, which is time-consuming; and the time consumption will be greatly increased when the data packet is large, affecting the data update efficiency.
[0117] In order to solve the above problems, this application proposes a data management method, which is applied to Figure 2 In the data management process framework shown in Figure 2 As shown, a process architecture diagram of data management provided in an embodiment of the present application is provided. First, the server side determines the old data packet according to the version of the current application of the terminal, and then determines the new data packet according to the latest version of the application. Then, the patch package is obtained based on the difference calculation between the new and old data packets, and the patch package is sent to the terminal, so that the terminal synthesizes an updated data packet based on the local data packet and the patch package, thereby realizing the version update of the application.
[0118] It can be understood that the method provided by the present application can be written as a program to serve as a processing logic in a hardware system, or as a data management device to implement the above processing logic in an integrated or external manner. As an implementation method, the data management device obtains a first data packet and a second data packet; then traverses the files in the first data packet and the files in the second data packet to determine a difference file, the difference file includes a directory difference item or a content difference item in the first data packet and the second data packet; and divides the files in the first data packet and the files in the second data packet into at least two difference files based on at least one condition of a preset threshold, a file format or a file size, wherein the preset threshold is used to indicate the number of content difference items contained in the difference file, and each difference file corresponds to at least one file format; and then performs parallel difference calculation based on the difference file to obtain a patch package for instructing the terminal to perform a version update on the target application. Thus, the process of synthesizing the patch package by segment is realized. Since the process of synthesizing the patch package by segment is carried out in parallel, and the number of files that need to be decompressed and processed corresponding to each segment is the same, the normal operation of multi-threaded parallel calculation is guaranteed, and the efficiency of data update is improved.
[0119] Combined with the above process architecture, the following will introduce the data management method in this application. Please refer to Figure 3 , Figure 3 A flowchart of a data management method provided in an embodiment of the present application can be applied to a server for application management. The embodiment of the present application at least includes the following steps:
[0120] 301. Obtain a first data packet and a second data packet.
[0121] In this embodiment, the first data packet and the second data packet correspond to different versions of the target application, and N and M are positive integers; wherein the first data packet is data under one version of the target application, and the second data packet is data under another version.
[0122] Specifically, the relationship between the first data packet and the second data packet can be the relationship between the old and new versions of the application, that is, the first data packet is the data corresponding to the old version of the application, and the second data packet is the data corresponding to the new version of the application; it can also be that the first data packet is the data corresponding to the new version of the application, and the second data packet is the data corresponding to the old version of the application, that is, the application restoration process.
[0123] It is understandable that the application versions corresponding to the first data packet and the second data packet may be one generation apart or multiple generations apart, and the specific difference depends on the actual scenario.
[0124] Optionally, the first data packet can be obtained through an update request sent by the terminal. That is, if the terminal needs to perform a version update on the target program, it sends a version update request to the server, and attaches the corresponding version identifier and the target version identifier. For example, if the user needs to update program 1 from version a to version b, a request identifier containing version a and version b is sent to the server, so that the server calls the corresponding first data packet (version a) and second data packet (version b) according to the request identifier.
[0125] 302. Traverse participating files including files in the first data package and files in the second data package to determine difference files.
[0126] In this embodiment, the participating files are a collection of files in the first data packet and files in the second data packet, and the difference file is determined based on the file directory difference or file content difference between the first data packet and the second data packet; wherein the file directory difference is the file included in the first data packet but not included in the second data packet, or the file not included in the first data packet but included in the second data packet; and the file content difference is that both the first data packet and the second data packet contain files corresponding to the same object, but the specific file contents are different.
[0127] It is understandable that the difference file can be stored in a temporary folder to facilitate the subsequent call of the difference process, and the temporary folder also contains the same files in the first data package and the second data package, that is, the differences between the files in the first data package and the files in the second data package are judged in turn and entered into the temporary folder.
[0128] Specifically, in the process of determining difference files based on mergeBsdiff, the identical and unilaterally existing files can be eliminated first, and then merged to obtain difference files; however, in the present application, the files are chosen to be input indiscriminately, thereby reducing file-level operations, which can effectively reduce the subsequent difference time.
[0129] 303. Divide the participating files into at least two differential files based on a differential condition.
[0130] In this embodiment, the differential condition includes at least one of a preset threshold, a file format, and a file size; that is, the differential condition can be to divide the differential files based on a preset threshold, or to divide the differential files based on a file format, or to divide the differential files based on a file size, or to divide the differential files based on a combination of a preset threshold and a file format, or a combination of multiple other conditions.
[0131] Next, the difference condition being the preset threshold value is described.
[0132] Specifically, when the differential condition is a preset threshold; the preset threshold is used to indicate the number of differential files contained in the differential file, that is, the differential file is divided into files under multiple threads for subsequent differential calculations; considering that the time consumption of each thread is mainly determined based on the number of differential files, the threads are divided based on the preset threshold; and in order to ensure that at least 2 threads appear, the value R of the preset threshold can be limited, that is, in the scenario where the first data packet contains N files and the second data packet contains M files, the value of R should satisfy R<N and R<M, and R is a positive integer.
[0133] It should be noted that since the differential file is the process data in the differential calculation process, it can be stored as temporary data, so it can also be called a differential temporary file, stored in a temporary folder, and deleted after the differential calculation is completed, so as not to occupy the system's storage resources and reduce the impact on the system during the data management process.
[0134] It is understandable that the number of difference files contained in the differential file is the same, but the total number of files contained in the differential file is not necessarily the same, that is, the differential file also includes the same files in the first data packet and the second data packet. For such files, the processing time in the thread will not be increased, so it can be not used as a technical condition for the preset threshold R.
[0135] Specifically, in the process of dividing the difference file into at least two differential files, since the difference file is determined based on the difference in the file directory or the difference in the file content, and different differences may be processed differently in the subsequent differential calculation process; therefore, for the content difference items, the content difference items can be divided based on the preset threshold R; and then the divided content difference items are decompressed to write into at least two differential files. By decompressing the files with only differences in the file content, the normal content comparison process is guaranteed.
[0136] In addition, for directory difference items, since they are unilateral differences, they can be directly replaced in subsequent differential calculations. Therefore, based on the file position relationship between the directory difference items and the content difference items, the directory difference items can be directly written into the differential file, thereby improving the efficiency of file processing.
[0137] Furthermore, to ensure that all files are processed indiscriminately, identical files are not included in the counting of the preset threshold R; and identical files are directly written into the differential file to ensure the integrity of the files.
[0138] 304. Perform parallel differential calculation based on the differential file to obtain a patch package.
[0139] In this embodiment, the patch package is used to instruct the terminal to update the version of the target application, and the patch package can also be called differential data. The differential calculation process can be based on the mergeBsdiff algorithm or other algorithms for calculating file differences, which are not limited here.
[0140] For ease of understanding, the data management method provided by this application is described below in conjunction with a specific scenario. Figure 4 As shown, it is a scenario schematic diagram of a method for data management provided by an embodiment of the present application. The figure shows the data composition of the first data packet (old version apk) and the second data packet (new version apk), wherein the first data packet includes files (A, B, C, D, E, G, H, I) and the second data packet includes files (A', B, D', E', F, G', H', I'), A and A' are files with file content differences, that is, the file contents are different; and B is the same file; C is a file based on file directory differences, that is, it exists in the first data packet but does not exist in the second data packet, as shown in the figure.
[0141] The process of obtaining differential data (patch package) by performing differential calculation on the first data packet (old version apk) and the second data packet (new version apk) in this scenario is as follows: Figure 5 As shown, it is a scenario diagram of another data management method provided by an embodiment of the present application. That is, for files that exist in both the new version apk and the old version apk, if the file contents are different, they are written to temporary files after decompression, such as A and A' in the figure, which become a and a' after decompression, and are written to old-temp-file (first temporary files 1, 2) for indicating the old version apk and new-temp-file (second temporary files 1, 2) for indicating the new version apk respectively; for files that exist in both the new and old version apk, if the file contents are the same, temporary files old-temp-file and new-temp-file are directly written, such as B in the figure; and for files that do not exist in the new version apk but exist in the old version apk, such as C in the figure, they are directly written to old-temp-file; for files that exist in the new version apk but do not exist in the old version apk, such as F in the figure, they are directly written to new-temp-file.
[0142] In addition, when the number of decompressed files written exceeds a certain number 3 (preset threshold R), the new and old temporary files are differentiated. The old version apk (A, B, C, D, E, G, H, I) is written into two temporary files: old-temp-file1 (a, B, C, d, e) and old-temp-file2 (g, h, i), and the new version apk (A', B, D', E', F, G', H', I') is written into two temporary files: new-temp-file1 (a', Bd', e') and new-temp-file2 (F, g', h', i'). Old-temp-file1 and new-temp-file1, old-temp-file2 and new-temp-file2 are used for differentiation to obtain diff file 1 (differential data 1) and diff file 2 (differential data 2). This process is carried out in parallel to obtain differential files.
[0143] In combination with the above embodiments, it can be seen that the process of synthesizing patch packages in segments can be realized by obtaining differential files in parallel. Since the process of synthesizing patch packages in segments is carried out in parallel and the number of files that need to be decompressed corresponding to each segment is the same, the normal multi-threaded parallel computing is guaranteed, that is, the efficiency of data updating is improved by re-combining.
[0144] Based on the above Figure 5 Based on the embodiment shown in FIG. 1 , some other embodiments are also included, that is, after the terminal receives the patch package, such as Figure 6 As shown, it is a scenario diagram of another data management method provided by an embodiment of the present application. That is, when synthesizing the new version apk (second data packet), the old-temp-file files (first temporary files 1, 2) and the diff files (patch packages) are subjected to the bspatch algorithm (synthesis algorithm) to obtain the new-temp-file files (second temporary files 1, 2). Then, the decompressed files (a', d', e', g', h', i') are obtained from the new-temp-file files, and finally the A'D'E'G'H'I' files are restored by the compression algorithm, thereby generating a new version apk (A'BD'E'FG'H'I').
[0145] Through the multi-threaded comparison process, the terminal can quickly perform multi-threaded parallel updates based on the local first data packet and the patch package sent by the server to obtain the second data packet, thereby improving the efficiency of application updates and reducing the time consumption of application version updates.
[0146] The above embodiment introduces the process of dividing based on the preset threshold. The following is an explanation of the difference condition as the file format. Figure 7 , Figure 7A flowchart of another data management method provided in an embodiment of the present application, the embodiment of the present application at least includes the following steps:
[0147] 701. Obtain a first data packet and a second data packet.
[0148] 702. Traverse participating files including files in the first data package and files in the second data package to determine difference files.
[0149] In this embodiment, step 701 and step 702 are Figure 3 Steps 301 and 302 of the illustrated embodiment are similar, and the description of related features can be referred to and will not be repeated here.
[0150] 703. Divide the participating files based on the file format to obtain at least two differential files.
[0151] In this embodiment, since the differential file is divided by format, in order to reduce the occupation of system resources, it can be deleted after the differential calculation is completed, and can be stored as a temporary file, so it can be used as a temporary format file. Considering that for common file formats (such as .dex, .so, .xml, .png, etc.), if the two file formats are different in the new and old versions of apk, then the two files must be different, the files can be segmented, merged and decompressed according to the file format, and then multi-threaded segmented differences can be performed, thereby effectively reducing the difference time.
[0152] The specific format type depends on the actual scenario and is not limited here.
[0153] In a possible scenario, if the files in the participating files correspond to the same file formats, the difference condition is set to a method of obtaining a difference file by dividing using a preset threshold, that is, Figure 3 The solutions of the illustrated embodiment will not be described in detail here.
[0154] In addition, the determination of the difference file is mainly used to indicate the way to write at least two differential files to the files in the first data packet and the files in the second data packet in the process of dividing the participating files based on the file format. That is, for the content difference items in the difference file, they are written into the differential file after decompression, because the difference in content needs to be decompressed to be fully obtained; and for the directory difference items in the difference file, they are directly written into the differential file, because the difference in directory can be achieved by direct file copying or deletion, and no content parsing is required; in addition, for the same files in the files in the first data packet and the files in the second data packet, they are also directly written into the differential file, in order to input the files in the first data packet and the files in the second data packet without distinction, that is, to reduce file-level operations and further reduce the time consumption of the processing process.
[0155] In another possible scenario, since the two file formats are different, the two files are not the same, so step 703 can be combined with step 702, that is, first divide the participating files according to the file format to obtain format folders corresponding to different file formats; then traverse the participating files to determine the difference file, as well as the directory difference items and content difference items in the difference file; then in the process of writing at least two differential files in the format folder, the directory difference items and content difference items are written into the differential file accordingly according to their distribution; specifically, the content difference items are written after decompression, and the directory difference items are written directly, thereby saving the time spent on file processing.
[0156] 704. Perform parallel differential calculation based on the differential file to obtain a patch package.
[0157] In this embodiment, the patch package is used to instruct the terminal to perform a version update on the target application.
[0158] Optionally, for the process of classifying by file format, different formats indicating the same type of data can be classified into one category, for example, png, jpg, .webp, etc., are all formats indicating pictures, so they can all be classified as .png format. The specific classification process can include first determining the file type corresponding to the difference file; then determining multiple format information corresponding to the file type; and then determining the file format based on the format information, thereby reducing the complexity of the file format and improving data processing efficiency.
[0159] Furthermore, in the process of performing differential calculations for temporary files of different formats, different differential algorithms for format settings can be set. That is, the format differential algorithm is determined based on the file format corresponding to the format temporary file; then, parallel differential calculations are performed based on the format differential algorithm to update the patch package, for example, the dexdiff / dexpatch algorithm is used for .dex format files, thereby reducing the size of the format file and further improving the efficiency of data processing.
[0160] Optionally, in order to facilitate the terminal side to read the patch package, specific format information can be set for the patch package. That is, a format directory is generated based on the file format; then the corresponding difference file is bound according to the format directory to generate record information; and then the patch format information is generated by combining the record information and the version information corresponding to the patch package. The patch format information is used to instruct the terminal to identify the patch package, thereby improving the terminal's recognition efficiency for the patch package and reducing the update time.
[0161] In order to facilitate the understanding of the above embodiment, a specific scenario is described below. Figure 8As shown, it is a scene schematic diagram of another data management method provided by an embodiment of the present application. The figure shows the data composition of the old version apk (first data packet) and the new version apk (second data packet); wherein the first data packet includes files (A, B, C, D, E, G, H, I) and the second data packet includes files (A', B, D', E', F, G', H', I'), A and A' are files with different file contents, that is, the file contents are different; and B is the same file; C is a file based on file directory differences, that is, it exists in the first data packet but does not exist in the second data packet, and the specific format type is shown in the mark in the figure.
[0162] For the process of obtaining the patch package through the old version apk (first data package) and the new version apk (second data package) in this scenario, Fig. 9 As shown, it is a scenario diagram of another data management method provided by an embodiment of the present application. First, traverse the files in the old version apk (first data packet) and the new version apk (second data packet), and write the files into the old version apk temporary file (first format temporary file 1, second format temporary file 1, third format temporary file 1 and fourth format temporary file 1) and the new version apk temporary file (first format temporary file 2, second format temporary file 2, third format temporary file 2 and fourth format temporary file 2) in sequence according to different formats, among which formats such as .png, jpg, .webp, etc. are all classified as temporary files in .png format; for files that exist in both the new version apk and the new version apk, if the file contents are different, they are decompressed and written to the temporary files, such as Fig. 9 A and A' in the file are decompressed into a and a' and then written into old-dex-temp-file and new-dex-temp-file; for files that exist in both the old and new versions of apk, if the file contents are the same, they are directly written into temporary files, such as B, which is directly written into old-dex-temp-file (first format temporary file 1) and new-dex-temp-file (first format temporary file 2); for files that do not exist on either side, they are directly written, such as C, which is directly written into old-dex-temp-file; further, when traversing to files of different formats, if the temporary file of that format does not exist, a new temporary file is created (with the file format as the prefix), such as old-xml-temp-file and new-xml-temp-file; and when any pair of temporary files is written, the algorithm corresponding to the specific format (such as dexdiff for dex format, hdiffpatch for hd format, bsdiff for bs format, etc.) is selected for differential analysis to obtain format differential data, and then the patch package is written after the differential analysis.
[0163] In this scenario, the update process on the terminal side may include: first obtaining the patch package sent by the server; then determining at least two differential files indicated in the patch package, the differential files being obtained by the server dividing the differential files based on the file format, and the differential files being obtained by comparing the first data packet with the second data packet; and performing a synthetic calculation based on the first data packet and the differential file to obtain the second data packet, the first data packet being the local version data; thereby performing a version update based on the second data packet.
[0164] Optionally, for the process of performing synthesis calculation based on the first data packet and the differential file to obtain the second data packet, due to the sequential nature of network downloads, the differential file downloaded first can be used for data synthesis, and the next differential file can be downloaded while the data is being synthesized, thereby realizing the process of downloading and synthesizing, and improving the version update efficiency of the terminal.
[0165] In a possible scenario, for the above terminal update process, such as Fig.10 As shown, it is a scene diagram of another data management method provided by an embodiment of the present application. That is, first download the differential file, and after downloading the PFH, start to check the header information, and exit directly if it does not meet the requirements; then after downloading the oldapk-file-record (first format temporary data 1), create a temporary file old-temp-file (first data packet) of the old version apk in a multi-threaded format, and generate a temporary file of a specific format from the file; if any diff-file (second temporary file) is downloaded, the new thread uses a synthesis algorithm to synthesize the old-temp-file and the diff-file, and after synthesis, the information of the newapk-file-record (first format temporary data 2) is used for asynchronous compression; until all diff-files have completed the synthesis and compression process, the generated file is the new version apk (second data packet).
[0166] Optionally, for the format information of the patch package in the above embodiment, please refer to Fig.11 , which is a schematic diagram of another data management method provided in an embodiment of the present application. The format information includes
[0167] PFH (patch package header information): contains information such as incremental algorithm version and local old version.
[0168] oldapk-file-record (old / new difference record): the file record involved in bsdiff difference in the old version of apk. Contains information such as offset, compression, compression method, length after compression, length before compression, etc.
[0169] newapk-file-record (new / old difference record): the file record involved in the difference in the new version of apk. Contains information such as offset, compression method, compression level, length after compression, length before compression, etc.
[0170] diff-record: Description of diff-files, including the number of diff-files, the offset, length, and difference algorithm of each diff-file.
[0171] diff-file: A file generated by decompressing files of different formats in segments and using a difference algorithm (the algorithm is not limited to mergeBsdiff).
[0172] It can be seen from the above embodiments that the time consumption of application update is reduced by reassembling differential participating files and segmented differences; and the size of differential packages is reduced by introducing multi-algorithm strategies; further, by modifying the patch package format, the time consumption of application data synthesis is reduced, thereby improving the efficiency of application updates.
[0173] Next, the difference condition is described as file size.
[0174] In this embodiment, the differential condition is the file size, that is, a partition threshold is set for each parallel processing thread, and the participating files are divided into multiple threads, that is, the partition threshold corresponding to the file size is first determined; then, based on the partition threshold, files of corresponding sizes are extracted from the participating files to be divided into at least two differential files. For example, if the partition threshold indicated by the file size is 10M, the participating files are written one by one into a temporary folder (differential file) of 10M in size, and when the files written in the temporary folder reach 10M, the next temporary folder is switched to be written.
[0175] Specifically, in the above-mentioned process of dividing according to file size, if a file is written into the first temporary folder and makes the size of the first temporary folder greater than the division threshold, it will be moved into the second temporary folder. The size of the first temporary folder may not reach the division threshold. At this time, the remaining files may not be written, thereby ensuring the order of writing of multiple temporary folders, avoiding file-level migration operations, and improving the accuracy of file writing.
[0176] It can be understood that the way files are written during file division according to file size is similar to the scenario when the preset threshold is used for division, that is, the content difference items are written into the differential file after decompression, the directory difference items are written directly into the differential file, and the same items are written directly into the differential file.
[0177] By dividing the participating files based on file size, the processing pressure of a single thread is reduced and the processing efficiency of multi-threaded parallelism is guaranteed.
[0178] In one possible scenario, the above-mentioned preset thresholds, file formats or file size division methods can be combined, that is, the differential condition can be a combination of preset thresholds and file formats; a combination of file formats and file sizes; a combination of preset thresholds and file sizes; a combination of preset thresholds, file formats and file sizes. The specific combination depends on the actual scenario.
[0179] In addition, considering that the combination process is a secondary division of the participating files, the division process of the above multiple differential conditions can be performed when the file parameters in the participating files meet certain conditions. For example, the file parameters in the participating files indicate that the file size is greater than 1GB. Specifically, the division process of the multiple differential conditions can include the following steps:
[0180] Get the file parameters in the participating files, including the number or size of the files;
[0181] If the value indicated by the file parameter is greater than the file processing threshold, the participating files are divided into at least two threshold division files based on a first processing condition, the file processing threshold is set based on quantity or size, and the first processing condition includes at least one of a preset threshold, a file format, or a file size;
[0182] Each threshold-divided file is divided into at least two differential files based on a second processing condition, the second processing condition includes at least one of a preset threshold, a file format or a file size, and the first processing condition is different from the second processing condition.
[0183] It can be understood that the value indicated by the file parameter is greater than the file processing threshold, which indicates the process of participating in the file size; for example, if the file parameter is 1000 files and the file processing threshold is 800, then the value indicated by the file parameter is greater than the file processing threshold, and a secondary division process is required.
[0184] Specifically, the secondary division process can be a combination of a preset threshold and a file format, that is, the first processing condition is the preset threshold, and the second processing condition is the file format; or the first processing condition is the file format, and the second processing condition is the preset threshold; it can also be a combination of file size and file format, that is, the first processing condition is the file size, and the second processing condition is the file format; or the first processing condition is the file format, and the second processing condition is the file size; it can also be a combination of a preset threshold and file size, that is, the first processing condition is the preset threshold, and the second processing condition is the file size; or the first processing condition is the file size, and the second processing condition is the preset threshold.
[0185] It should be noted that secondary division is not limited to only two divisions, that is, multiple division processes can be performed, for example, differential files can be divided according to preset thresholds, file formats and file sizes respectively. The specific order depends on the actual scenario and is not limited here.
[0186] By dividing the participating files into multiple differential conditions, the number of parallel processing threads is further increased, and the efficiency of file updating is improved.
[0187] The following describes the process of combining the above differential conditions in conjunction with specific scenarios.
[0188] In one possible scenario, the above Figure 3 The embodiments shown and Figure 7 The embodiments shown can be combined, that is, format division is performed based on the preset threshold division, such as Fig.12 As shown, a scenario schematic diagram of another method for data management provided by an embodiment of the present application. The figure shows a first data packet (A, B, C, D, E, F ...) and a second data packet (A`, B, C`, D`, E`, F` ...), that is, A, C, D, E, F are file content difference files, and A, B, C, D, E, F are divided into a first temporary file by setting a preset threshold R to 5, and A`, B, C`, D`, E`, F` are divided into a second temporary file, and further divided by the file format contained in the first temporary file and the second temporary file, so as to obtain a first format temporary file of the first data packet and a corresponding first format temporary file of the second data packet, and a temporary file under the second format, and then call the difference algorithm corresponding to the format to obtain a difference file, and refer to the above process for other data in the first data packet and the second data packet for processing, which will not be repeated here.
[0189] By further dividing temporary files based on the preset threshold division, the threads for file processing are increased, the size of the file is reduced, and the efficiency of file updating is improved.
[0190] In addition, in a possible scenario, the preset threshold value division can also be performed based on the format division, such as Fig.13As shown, a scenario schematic diagram of another method for data management provided by an embodiment of the present application. The figure shows a first data packet (A, B, C, D, E...) and a second data packet (A`, B, C`, D`, E`...), that is, A, C, D, E are file content difference files, the first data packet and the second data packet are divided by file format, and then a, B, c are divided into a first format temporary file 1, d, e are divided into a first format temporary file 2 by setting a preset threshold R to 2; and the corresponding temporary file in the second format, and then the difference algorithm corresponding to the format is called to obtain a difference file, and other data in the first data packet and the second data packet are processed according to the above process, which will not be repeated here.
[0191] In addition, the setting of the format algorithm and the setting of the patch package format can refer to and be combined with the description of the above embodiments, and will not be described in detail here.
[0192] By dividing temporary files according to preset thresholds based on format division, that is, by reassembling the participating files in the differential process and performing segmented differential, the number of file processing threads is increased; and by corresponding settings of different format algorithms, the size of the patch package is reduced; further, by modifying the patch package format, it is convenient for the terminal to parse the patch package, further improving the efficiency of file updates and reducing the time spent in the file update process.
[0193] In order to better implement the above solution of the embodiment of the present application, the following also provides related devices for implementing the above solution. Fig.14 , Fig.14 The data management device 1400 is a schematic diagram of a data management device provided in an embodiment of the present application. The data management device 1400 includes:
[0194] An acquiring unit 1401 is configured to acquire a first data packet and a second data packet, wherein the first data packet and the second data packet correspond to different versions of a target application;
[0195] A traversal unit 1402 is used to traverse the participating files contained in the first data packet and the second data packet to determine a difference file, wherein the difference file includes a directory difference item or a content difference item in the first data packet and the second data packet;
[0196] A division unit 1403 is used to divide the participating files into at least two differential files based on a differential condition, wherein the differential condition includes at least one of the following: a preset threshold, a file format, and a file size, wherein the preset threshold is used to indicate the number of the content difference items contained in the differential file, the file format corresponds to the differential file, and the file size is used to indicate the size of the differential file;
[0197] The management unit 1404 is used to perform parallel differential calculation based on the differential file to obtain a patch package, and the patch package is used to instruct the terminal to perform a version update on the target application.
[0198] Optionally, in some possible implementations of the present application, if the difference condition is a preset threshold R, the division unit 1403 is specifically used to determine the content difference item, the directory difference item, or the same item in the difference file corresponding to the participating file, the participating file including N files included in the first data packet and M files included in the second data packet, where N and M are positive integers;
[0199] The division unit 1403 is specifically configured to divide the content difference items based on the preset threshold R, where the preset threshold is R, R<N and R<M, and R is a positive integer;
[0200] The division unit 1403 is specifically configured to decompress the divided content difference items to write into at least two difference files;
[0201] The division unit 1403 is specifically configured to directly write the directory difference items or the same items after division into the differential file.
[0202] Optionally, in some possible implementations of the present application, if the difference condition is a file format, the division unit 1403 is specifically configured to determine at least two file formats of the participating files;
[0203] The division unit 1403 is specifically configured to divide the files in the first data packet and the files in the second data packet based on the file format to obtain at least two differential files.
[0204] Optionally, in some possible implementations of the present application, the division unit 1403 is specifically used to determine the file types included in the participating files;
[0205] The division unit 1403 is specifically used to determine multiple format information corresponding to the file type;
[0206] The dividing unit 1403 is specifically configured to determine the file format based on the format information.
[0207] Optionally, in some possible implementations of the present application, the dividing unit 1403 is specifically configured to determine a format difference algorithm based on a file format corresponding to the differential file;
[0208] The division unit 1403 is specifically configured to perform parallel difference calculation based on the format difference algorithm to update the patch package.
[0209] Optionally, in some possible implementations of the present application, the dividing unit 1403 is specifically configured to generate a format directory based on the file format;
[0210] The division unit 1403 is specifically used to bind the corresponding difference files according to the format directory to generate record information;
[0211] The dividing unit 1403 is specifically configured to generate patch format information in combination with the record information and the version information corresponding to the patch package, wherein the patch format information is used to instruct the terminal to identify the patch package.
[0212] Optionally, in some possible implementations of the present application, if the differential condition is a file size, the division unit 1403 is specifically configured to determine a division threshold corresponding to the file size;
[0213] The division unit 1403 is specifically configured to extract files of corresponding sizes from the participating files based on the division threshold, so as to divide the files into at least two differential files.
[0214] Optionally, in some possible implementations of the present application, the division unit 1403 is specifically used to, if the file in the first data packet and the file in the second data packet correspond to the same file format, then the differential condition is the preset threshold or the file size.
[0215] Optionally, in some possible implementations of the present application, the partitioning unit 1403 is specifically configured to obtain file parameters in the participating file, where the file parameters include the number or size of files;
[0216] The division unit 1403 is specifically configured to divide the participating file into at least two threshold division files based on a first processing condition if the value indicated by the file parameter is greater than a file processing threshold, the file processing threshold being set based on quantity or size, and the first processing condition comprising at least one of the preset threshold, the file format or the file size;
[0217] The division unit 1403 is specifically used to divide each of the threshold division files into at least two differential files based on a second processing condition, wherein the second processing condition includes at least one of the preset threshold, the file format or the file size, and the first processing condition is different from the second processing condition.
[0218] Optionally, in some possible implementations of the present application, it is characterized in that the management unit 1404 is specifically used to determine a first temporary file and a second temporary file in the differential file, the first temporary file is used to indicate the differential file in the first data packet, and the second temporary file is used to indicate the differential file in the second data packet;
[0219] The management unit 1404 is specifically configured to perform a differential calculation based on the first temporary file and the second temporary file to obtain the patch package.
[0220] By obtaining a first data packet and a second data packet; then traversing the files in the first data packet and the files in the second data packet to determine a difference file, the difference file includes a directory difference item or a content difference item in the first data packet and the second data packet; and dividing the files in the first data packet and the files in the second data packet into at least two differential files based on at least one condition of a preset threshold, a file format or a file size, wherein the preset threshold is used to indicate the number of content difference items contained in the differential file, and each differential file corresponds to at least one file format; and then performing parallel differential calculation based on the differential file to obtain a patch package for instructing the terminal to perform a version update on the target application. Thus, the process of synthesizing the patch package by segments is realized. Since the process of synthesizing the patch package by segments is carried out in parallel, and the number of files that need to be decompressed and processed corresponding to each segment is the same, the normal multi-threaded parallel calculation is guaranteed, and the efficiency of data update is improved.
[0221] The present application embodiment also provides a data management device 1500, such as Fig.15 FIG. 1 is a schematic diagram of a data management device provided in an embodiment of the present application, which specifically includes:
[0222] The acquisition unit 1501 is used to acquire the patch package sent by the server;
[0223] A determining unit 1502 is configured to determine at least two differential files indicated in the patch package, where the differential files are obtained by the server dividing a file in a first data packet and a file in a second data packet based on a differential condition, where the first data packet and the second data packet correspond to different versions of a target application, and where the differential condition includes at least one of the following: a preset threshold, a file format, and a file size, where the preset threshold is used to indicate the number of content difference items contained in the differential file, the file format corresponds to the differential file, and the file size is used to indicate the size of the differential file;
[0224] A management unit 1503 is configured to perform a synthesis calculation based on the first data packet and the differential file to obtain the second data packet, wherein the first data packet is local version data;
[0225] The management unit 1503 is further configured to perform version update based on the second data packet.
[0226] Optionally, in some possible implementations of the present application, the management unit 1503 is specifically configured to determine content difference items and directory difference items indicated in the differential file;
[0227] The management unit 1503 is specifically configured to perform parallel processing on the content difference items and the directory difference items based on the synthesis thread corresponding to each of the differential files, so as to update the first data packet into the second data packet.
[0228] The present application also provides a server. Fig.16 , Fig.16 is a schematic diagram of the structure of a server provided in an embodiment of the present application. The server 1600 can execute the above Fig.14 The steps of the data management device shown in the figure; the server 1600 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 1622 (for example, one or more processors) and memory 1632, one or more storage media 1630 (for example, one or more mass storage devices) storing application programs 1642 or data 1644. Among them, the memory 1632 and the storage medium 1630 can be short-term storage or permanent storage. The program stored in the storage medium 1630 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the server. Furthermore, the central processing unit 1622 can be configured to communicate with the storage medium 1630 to execute a series of instruction operations in the storage medium 1630 on the server 1600.
[0229] The server 1600 may also include one or more power supplies 1626, one or more wired or wireless network interfaces 1650, one or more input and output interfaces 16516, and / or one or more operating systems 1641, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
[0230] The steps performed by the data management device in the above embodiment can be based on the Fig.16 The server structure shown.
[0231] The present application also provides a terminal device, such as Fig.17 FIG. 2 is a schematic diagram of another terminal device provided in an embodiment of the present application. The terminal device can execute the above Fig.15 For the sake of convenience, the steps of the data management device shown only show the parts related to the embodiment of the present application. For specific technical details not disclosed, please refer to the method part of the embodiment of the present application. The terminal can be any terminal device including a mobile phone, a tablet computer, a personal digital assistant (PDA), a point of sales (POS), a car computer, etc., taking the terminal as a mobile phone as an example:
[0232] Fig.17 FIG. 1 is a block diagram showing a partial structure of a mobile phone related to a terminal provided in an embodiment of the present application. Fig.17 The mobile phone includes: a radio frequency (RF) circuit 1710, a memory 1720, an input unit 1730, a display unit 1740, a sensor 1750, an audio circuit 1760, a wireless fidelity (WiFi) module 1770, a processor 1780, and a power supply 1790. Those skilled in the art will understand that Fig.17 The mobile phone structure shown in the figure does not constitute a limitation on the mobile phone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.
[0233] Combine the following Fig.17 A detailed introduction to the various components of the mobile phone:
[0234] The RF circuit 1710 can be used for receiving and sending signals during the process of sending and receiving information or making calls. In particular, after receiving the downlink information of the base station, it is sent to the processor 1780 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit 1710 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, etc. In addition, the RF circuit 1710 can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communication can use any communication standard or protocol, including but not limited to the global system of mobile communication (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), long term evolution (LTE), email, short messaging service (SMS), etc.
[0235] The memory 1720 can be used to store software programs and modules. The processor 1780 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 1720. The memory 1720 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 1720 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.
[0236] The input unit 1730 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile phone. Specifically, the input unit 1730 may include a touch panel 1731 and other input devices 1732. The touch panel 1731, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 1731 or near the touch panel 1731 using any suitable object or accessory such as a finger, stylus, etc., and the air touch operation within a certain range on the touch panel 1731), and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel 1731 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact point coordinates, and then sends it to the processor 1780, and can receive and execute commands sent by the processor 1780. In addition, the touch panel 1731 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1731, the input unit 1730 can also include other input devices 1732. Specifically, the other input devices 1732 can include but are not limited to one or more of a physical keyboard, a function key (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, etc.
[0237] The display unit 1740 can be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 1740 may include a display panel 1741. Optionally, the display panel 1741 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch panel 1731 may cover the display panel 1741. When the touch panel 1731 detects a touch operation on or near it, it is transmitted to the processor 1780 to determine the type of touch event. Subsequently, the processor 1780 provides a corresponding visual output on the display panel 1741 according to the type of touch event. Although in Fig.17 In the embodiment, the touch panel 1731 and the display panel 1741 are used as two independent components to realize the input and output functions of the mobile phone, but in some embodiments, the touch panel 1731 and the display panel 1741 can be integrated to realize the input and output functions of the mobile phone.
[0238] The mobile phone may also include at least one sensor 1750, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 1741 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 1741 and / or the backlight when the mobile phone is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.
[0239] The audio circuit 1760, the speaker 1761, and the microphone 1762 can provide an audio interface between the user and the mobile phone. The audio circuit 1760 can transmit the received audio data to the speaker 1761 after converting the received audio data into an electrical signal, which is converted into a sound signal for output; on the other hand, the microphone 1762 converts the collected sound signal into an electrical signal, which is received by the audio circuit 1760 and converted into audio data, and then the audio data is output to the processor 1780 for processing, and then sent to another mobile phone through the RF circuit 1710, or the audio data is output to the memory 1720 for further processing.
[0240] WiFi is a short-range wireless transmission technology. The mobile phone can help users send and receive emails, browse web pages and access streaming media through the WiFi module 1770. It provides users with wireless broadband Internet access. Fig.17 A WiFi module 1770 is shown, but it is understandable that it is not an essential component of the mobile phone and can be omitted as needed without changing the essence of the invention.
[0241] The processor 1780 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. It executes various functions of the mobile phone and processes data by running or executing software programs and / or modules stored in the memory 1720 and calling data stored in the memory 1720. Optionally, the processor 1780 may include one or more processing units; optionally, the processor 1780 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1780.
[0242] The mobile phone also includes a power supply 1790 (such as a battery) for supplying power to various components. Optionally, the power supply can be logically connected to the processor 1780 through a power management system, thereby managing functions such as charging, discharging, and power consumption management through the power management system.
[0243] Although not shown, the mobile phone may also include a camera, a Bluetooth module, etc., which will not be described in detail here.
[0244] In the embodiment of the present application, the processor 1780 included in the terminal also has the function of executing each step of the above-mentioned page processing method.
[0245] The present application also provides a computer-readable storage medium in which data management instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the above-mentioned Figures 3 to 13 The illustrated embodiment describes the steps performed by the data management device in the method.
[0246] The present application also provides a computer program product including data management instructions, which, when executed on a computer, enables the computer to execute the above Figures 3 to 13 The illustrated embodiment describes the steps performed by the data management device in the method.
[0247] The present application also provides a data management system, which may include Fig.14 or Fig.15 The data management device in the described embodiments, or Fig.16 The server in the described embodiments, or Fig.17 The terminal device described.
[0248] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0249] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0250] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0251] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0252] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a data management device, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.
[0253] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A method for data management, characterized in that: include: Acquire a first data packet and a second data packet, wherein the first data packet and the second data packet correspond to different versions of a target application; Traversing participating files including files in the first data packet and files in the second data packet to determine a difference file, wherein the difference file includes directory difference items or content difference items in the first data packet and the second data packet; Dividing the participating files into at least two differential files based on a differential condition, wherein the differential condition includes at least one of the following: a preset threshold, a file format, and a file size, wherein the preset threshold is used to indicate the number of content difference items contained in the differential file, the file format corresponds to the differential file, and the file size is used to indicate the size of the differential file; Performing parallel differential calculation based on the differential file to obtain a patch package, wherein the patch package is used to instruct the terminal to perform a version update on the target application; The step of dividing the participating files into at least two differential files based on the differential condition comprises: Obtain file parameters in the participating file, wherein the file parameters include the number or size of the file; If the value indicated by the file parameter is greater than a file processing threshold, dividing the participating file into at least two threshold division files based on a first processing condition, the file processing threshold being set based on quantity or size, the first processing condition comprising at least one of the preset threshold, the file format or the file size; Each of the threshold-divided files is divided into at least two differential files based on a second processing condition, wherein the second processing condition includes at least one of the preset threshold, the file format or the file size, and the first processing condition is different from the second processing condition.
2. The method according to claim 1, characterized in that If the difference condition is a preset threshold R, dividing the participating files into at least two difference files based on the difference condition includes: Determine the content difference item, the directory difference item or the same item in the difference file corresponding to the participating file, wherein the participating file includes N files included in the first data packet and M files included in the second data packet, where N and M are positive integers; Dividing the content difference items based on the preset threshold R, the preset threshold is R, R<N and R<M, R is a positive integer; Decompressing the divided content difference items to write at least two of the difference files; The directory difference items or identical items after division are directly written into the differential file.
3. The method according to claim 1, characterized in that If the difference condition is a file format, dividing the participating files into at least two difference files based on the difference condition includes: determining at least two file formats of the participating files; The files in the first data packet and the files in the second data packet are divided based on the file format to obtain at least two differential files.
4. The method according to claim 3, characterized in that The determining of at least two file formats of the participating files includes: Determining the file types included in the participating files; Determine multiple format information corresponding to the file type; The file format is determined based on the format information.
5. The method according to claim 3, characterized in that: The method further comprises: Determining a format difference algorithm based on a file format corresponding to the difference file; Parallel differential calculations are performed based on the format differential algorithm to update the patch package.
6. The method according to claim 3, characterized in that The method further comprises: Generate a format catalog based on the file format; Binding corresponding difference files according to the format directory to generate record information; Patch format information is generated by combining the record information and the version information corresponding to the patch package, where the patch format information is used to instruct the terminal to identify the patch package.
7. The method according to claim 1, characterized in that If the difference condition is file size, dividing the participating files into at least two difference files based on the difference condition includes: Determine a partition threshold corresponding to the file size; Based on the division threshold, files of corresponding sizes are extracted from the participating files to be divided into at least two of the differential files.
8. The method according to claim 1, characterized in that If the file in the first data packet and the file in the second data packet correspond to the same file format, the difference condition is the preset threshold or the file size.
9. The method according to any one of claims 1 to 8, characterized in that: The performing parallel differential calculation based on the differential file to obtain a patch package includes: Determine a first temporary file and a second temporary file in the differential file, the first temporary file is used to indicate the differential file in the first data packet, and the second temporary file is used to indicate the differential file in the second data packet; A differential calculation is performed based on the first temporary file and the second temporary file to obtain the patch package.
10. A method of data management, characterized in that: include: Get the patch package sent by the server; Determine at least two differential files indicated in the patch package, the differential files are obtained by the server dividing the files in the first data packet and the files in the second data packet based on a differential condition, the first data packet and the second data packet correspond to different versions of the target application, the differential condition includes at least one of the following: a preset threshold, a file format, and a file size, the preset threshold is used to indicate the number of content difference items contained in the differential file, the file format corresponds to the differential file, and the file size is used to indicate the size of the differential file; Performing parallel synthesis calculation based on the first data packet and the differential file to obtain the second data packet, wherein the first data packet is local version data; Performing version update based on the second data packet; The differential file is obtained by dividing the file in the first data packet and the file in the second data packet by the server based on the differential condition, and includes: Obtain file parameters in the participating files, the file parameters including the number or size of files; the participating files include files in the first data packet and files in the second data packet; If the value indicated by the file parameter is greater than a file processing threshold, dividing the participating file into at least two threshold division files based on a first processing condition, the file processing threshold being set based on quantity or size, the first processing condition comprising at least one of the preset threshold, the file format or the file size; Each of the threshold-divided files is divided into at least two differential files based on a second processing condition, wherein the second processing condition includes at least one of the preset threshold, the file format or the file size, and the first processing condition is different from the second processing condition.
11. The method according to claim 10, characterized in that The performing parallel synthesis calculation based on the first data packet and the differential file to obtain the second data packet includes: Determining content difference items and directory difference items indicated in the differential file; The content difference items and the directory difference items are processed in parallel based on the synthesis thread corresponding to each of the differential files, so as to update the first data packet to the second data packet.
12. A data management device, characterized in that: include: An acquiring unit, configured to acquire a first data packet and a second data packet, wherein the first data packet and the second data packet correspond to different versions of a target application; a traversal unit, configured to traverse participating files including files in the first data packet and files in the second data packet to determine a difference file, wherein the difference file includes a directory difference item or a content difference item in the first data packet and the second data packet; a division unit, configured to divide the participating files into at least two differential files based on a differential condition, wherein the differential condition includes at least one of the following: a preset threshold, a file format, and a file size, wherein the preset threshold is used to indicate the number of the content difference items contained in the differential file, the file format corresponds to the differential file, and the file size is used to indicate the size of the differential file; A management unit, configured to perform parallel differential calculation based on the differential file to obtain a patch package, wherein the patch package is used to instruct the terminal to perform a version update on the target application; The division unit is specifically used for: Obtain file parameters in the participating file, wherein the file parameters include the number or size of the file; If the value indicated by the file parameter is greater than a file processing threshold, dividing the participating file into at least two threshold division files based on a first processing condition, the file processing threshold being set based on quantity or size, the first processing condition comprising at least one of the preset threshold, the file format or the file size; Each of the threshold-divided files is divided into at least two differential files based on a second processing condition, wherein the second processing condition includes at least one of the preset threshold, the file format or the file size, and the first processing condition is different from the second processing condition.
13. A computer device, characterized in that: The computer device comprises a processor and a memory: The memory is used to store program codes; the processor is used to execute the data management method described in any one of claims 1 to 9, or the data management method described in any one of claims 10 and 11 according to instructions in the program codes.
14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the data management method described in any one of claims 1 to 9, or the data management method described in any one of claims 10 and 11.
15. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the data management method described in any one of claims 1 to 9 above, or the data management method described in any one of claims 10 and 11.
