Data compression method and device

Abstract

The present application provides a data compression method and device, wherein the data compression method includes: obtaining initial data associated with a compression task; comparing the initial data with historical data associated with the compression task to obtain difference data; determining a target container corresponding to the difference data, and extracting container data associated with the target container from the initial data, wherein the container data includes the difference data; filling the container data into the target container according to a filling rule corresponding to the target container, and generating a compressed data packet associated with the compression task according to the filling result. The initial data is compressed, and the space occupation is reduced through data compression, thereby improving the subsequent processing efficiency of the initial data.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data compression method, a data compression device, a data decompression method, a data decompression device, a computing device, and a computer-readable storage medium. Background Art

[0002] With the development of Internet technology, the amount of data that enterprises need to process every day is increasing. In order to achieve higher data processing efficiency, data compression technology is usually used to compress the data to be processed. Among them, data compression refers to a technical method that reduces the amount of data to reduce storage space without losing useful information, improves its transmission, storage and processing efficiency, or reorganizes data according to a preset algorithm to reduce data redundancy and storage space.

[0003] In the prior art, when data is compressed, the compression effect is often not ideal, which affects the efficiency of subsequent data processing. Therefore, it is urgent to provide a solution to the above problem. Summary of the invention

[0004] In view of this, the embodiment of the present application provides a data compression method to solve the technical defects existing in the prior art. The embodiment of the present application also provides a data compression device, a data decompression method, a data decompression device, a computing device, and a computer-readable storage medium.

[0005] According to a first aspect of an embodiment of the present application, a data compression method is provided, which is applied to a server, and includes:

[0006] Get the initial data associated with the compression task;

[0007] Comparing the initial data with the historical data associated with the compression task to obtain difference data;

[0008] Determine a target container corresponding to the difference data, and extract container data associated with the target container from the initial data, wherein the container data includes the difference data;

[0009] The container data is filled into the target container according to the filling rule corresponding to the target container, and a compressed data packet associated with the compression task is generated according to the filling result.

[0010] According to a second aspect of an embodiment of the present application, a data compression device is provided, which is applied to a server, and includes:

[0011] A data acquisition module, configured to acquire initial data associated with the compression task;

[0012] A comparison module, configured to compare the initial data with the historical data associated with the compression task to obtain difference data;

[0013] an extraction module, configured to determine a target container corresponding to the difference data, and extract container data associated with the target container from the initial data, wherein the container data includes the difference data;

[0014] The generating module is configured to fill the container data into the target container according to the filling rule corresponding to the target container, and generate a compressed data packet associated with the compression task according to the filling result.

[0015] According to a third aspect of an embodiment of the present application, a data decompression method is provided, which is applied to a client and includes:

[0016] Get the compressed data packet associated with the decompression task;

[0017] extracting container data of the target container from the compressed data packet according to an extraction rule of the target container corresponding to the compressed data packet;

[0018] Determining historical data associated with the container data according to the attribute information of the container data;

[0019] Target data is generated based on the container data and the historical data.

[0020] According to a fourth aspect of an embodiment of the present application, a data decompression device is provided, which is applied to a client and includes:

[0021] A packet acquisition module, configured to acquire a compressed data packet associated with a decompression task;

[0022] a data extraction module configured to extract container data of the target container from the compressed data packet according to an extraction rule of the target container corresponding to the compressed data packet;

[0023] a data determination module configured to determine historical data associated with the container data according to attribute information of the container data;

[0024] The data generation module is configured to generate target data based on the container data and the historical data.

[0025] According to a fifth aspect of an embodiment of the present application, a computing device is provided, including:

[0026] Memory and processor;

[0027] The memory is used to store computer-executable instructions, and the processor implements the steps of the data compression method or the data decompression method when executing the computer-executable instructions.

[0028] According to a sixth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores computer-executable instructions, which, when executed by a processor, implement the steps of the data compression method or the data decompression method.

[0029] According to a seventh aspect of an embodiment of the present application, a chip is provided, which stores a computer program, and when the computer program is executed by the chip, the steps of the data compression method or the data decompression method are implemented.

[0030] The data compression method provided by the present application obtains the initial data associated with the compression task, and compares the initial data with the historical data associated with the compression task to obtain the difference data, that is, obtains the data that has changed compared with the historical data, and then determines the target container corresponding to the difference data, and extracts the container data associated with the target container from the initial data, wherein the container data contains the difference data, and realizes the extraction of data strongly related to the difference data in the initial data through the target container corresponding to the difference data. Then, according to the filling rule corresponding to the target container, the container data is filled into the target container, and a compressed data packet associated with the compression task is generated according to the filling result, so that the extracted data is assembled to generate a compressed data packet to be processed, so that the initial data is compressed, and the space occupation is reduced through data compression, and the subsequent processing efficiency of the initial data is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is a flow chart of a data compression method provided by an embodiment of the present application;

[0032] Figure 2 It is a schematic diagram of the statistical correlation change frequency in a data compression method provided in an embodiment of the present application;

[0033] Figure 3 is a schematic diagram of node sub-data in a data compression method provided in an embodiment of the present application;

[0034] Figure 4 is a schematic diagram of an execution process in a data compression method provided in an embodiment of the present application;

[0035] Figure 5 is a flow chart of a data decompression method provided by an embodiment of the present application;

[0036] Figure 6is a processing flow chart of a data compression method applied to a game scene provided by an embodiment of the present application;

[0037] Figure 7 is a structural schematic diagram of a data compression device provided by an embodiment of the present application;

[0038] Figure 8 It is a structural schematic diagram of a data decompression device provided by an embodiment of the present application;

[0039] Fig. 9 It is a structural block diagram of a computing device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0040] Many specific details are described in the following description to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present application, so the present application is not limited by the specific implementation disclosed below.

[0041] The terms used in one or more embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present application. The singular forms of "a", "said" and "the" used in one or more embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of the present application refers to and includes any or all possible combinations of one or more associated listed items.

[0042] It should be understood that, although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present application, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first.

[0043] In the present application, a data compression method is provided. The embodiments of the present application also provide a data compression device, a data decompression method, a data decompression device, a computing device, and a computer-readable storage medium. The embodiments are described in detail one by one in the following embodiments.

[0044] In online games, there is a large amount of traffic data that needs to be broadcast. For example, in massively multiplayer online role-playing games based on state synchronization, it is inevitable that a lot of data needs to be broadcast to each client in the same scene.

[0045] In actual applications, some game data can be trimmed according to the client's attention. When there are too many players in the same scene in the game, such as in the game attack and defense battle, involving two camps of 400 vs. 400 people (a total of 800 people) fighting in the same scene, a huge amount of state synchronization data will be generated (there will be N*N synchronization volume, N is the number of players on the scene). Some of this data can be trimmed according to the player's attention. For example, when releasing skills, if the two people are not directly attacking each other, or if the players are not from the same team, the combat skill data generated can be trimmed. However, some game data cannot be trimmed. For example, the data of player movement, because each player must see the position of other players on the scene, the direction they will move, and the actions they will make, so these data cannot be trimmed.

[0046] In the prior art, when the number of players in an area reaches a relatively high level, the broadcast can be down-converted and details can be discarded. Among them, down-converting refers to the reduction of the frequency of broadcasting. Usually, the broadcast synchronizes the character movement data with an accuracy of 16 frames per second. When the density of people reaches more than 50 people nearby, it is reduced to synchronizing the character movement data every 2 frames. This can indeed effectively reduce traffic, but the negative effects are also obvious. The characters will teleport, the movements will be discontinuous, and it will look like a freeze. Discarding details means that some variables that are not very important are no longer synchronized, such as turning off the walking, running, and jumping states during movement, and uniformly displaying them as running states, etc., but this will cause the movement of the characters to be very monotonous and distorted. If the player jumps on some complex terrain (such as a slope), the action and position will be severely distorted, which will cause the attribute side to fail to aim. Therefore, it is still necessary to try to ensure the integrity of the data and not to do too much down-conversion.

[0047] In addition, there are also scenarios in the prior art where the transmitted data is compressed. For example, a mobile data packet contains about 40 variables, and the changes in these variables are actually related to each other in a certain manner. However, the usual compression algorithms are based on a large number of null values ​​(values ​​of 0) or a large number of repeated data (such as string text) and compress the data by removing redundancy. This compression method does not support these data in the mobile data packet well. Because the variable lengths of variables in the mobile data are usually different, there is not much comparability between each other, and there will not be a large number of null values ​​in the mobile data, and there is no simple repetition pattern to follow, so the compression effect is not good.

[0048] Figure 1 A flowchart of a data compression method provided according to an embodiment of the present application is shown, which specifically includes the following steps:

[0049] Step 102: Acquire initial data associated with the compression task.

[0050] Specifically, a compression task refers to a task for compressing initial data. In practical applications, since the initial data needs to be transmitted, stored, and processed, directly performing these processes on the initial data requires a lot of computing costs and time costs. Therefore, by compressing the initial data, the storage space of the initial data can be reduced, and the efficiency of transmitting, storing, and processing it can be improved. Correspondingly, initial data refers to the data to be compressed. For example: game data, transaction data, user data, etc. For the convenience of description, this embodiment takes game data as an example for explanation. The compression of other data can refer to the same or corresponding description content of this embodiment, and this embodiment will not be elaborated in detail here.

[0051] Based on this, if it is determined that there is initial data with compression requirements, a compression task can be created to compress it. In specific implementation, the initial data can be obtained from a specific storage path or from other data platforms or databases, and there is no limitation here.

[0052] For example: obtain the attribute data D1 to be transmitted that is associated with the compression task in the game scene.

[0053] Step 104: Compare the initial data with the historical data associated with the compression task to obtain difference data.

[0054] Specifically, based on the above-mentioned acquisition of the initial data associated with the compression task, considering that there may be a part of the data in the initial data that has been processed (such as stored or transmitted, etc.), in order to avoid repeated processing of these data, the unprocessed data, that is, the difference data, can be first identified from the initial data, so as to compress the initial data based on the difference data. Among them, historical data refers to processed data of the same type as the initial data. In practical applications, since the difference between data created at adjacent time points is usually small, in order to further reduce the amount of data processing, the data at the previous time point corresponding to the initial data at adjacent time points is usually used as historical data. In addition, historical data at any other time can also be determined, without limitation here. Correspondingly, difference data refers to data that exists in the initial data that is different from the historical data.

[0055] Based on this, after obtaining the initial data, the initial data is compared with the historical data to obtain the difference data. In specific implementation, the data values ​​of the data with the same name in the initial data and the historical data can be compared. If they are inconsistent, it indicates that the data is difference data; if they are consistent, it indicates that the data is not difference data. In addition, the data in the initial data and the data in the historical data can be compared one by one according to the data arrangement order to obtain the difference data, which is not limited here.

[0056] Continuing with the above example, based on the acquisition of attribute data D1, the attribute data D1 contains 40 attribute variables, and the attribute data D2 transmitted by the server last time also contains 40 attribute variables. The variable values ​​of the attribute variables with the same name in the two are compared one by one, and it is determined that there are 5 attribute variables in the attribute data D1 whose variable values ​​are different from those of the attribute data D2. These 5 attribute variables are used as difference data.

[0057] Step 106: Determine a target container corresponding to the difference data, and extract container data associated with the target container from the initial data, wherein the container data includes the difference data.

[0058] Specifically, after obtaining the difference data, if the difference data is directly used as the compression result of the initial data, mask data is needed to mark the difference data so that the difference data can be restored to the initial data, considering that the amount of data in the initial data is large. However, in the case of a large amount of data, the mask data needs to occupy a large amount of storage space and / or transmission bandwidth. Therefore, in this embodiment, the corresponding relationship between the data in the initial data and each initial container is pre-established. After obtaining the difference data, the target container corresponding to the difference data in the initial container can be determined.

[0059] Among them, the initial container refers to a pre-established container for filling data. The initial container can be understood as a virtual object, which has corresponding container information, such as container identification, container serial number, container size and other information. The target container refers to the container in the initial container corresponding to the difference data. Container data refers to data that is allowed to be filled into the target container. In practical applications, each initial container corresponds to a part of the initial data. These parts of data are similar to each other. Therefore, the initial container also indicates a division of the initial data. Usually, the data that often changes together in the initial data are classified into one category corresponding to one initial container, and the data that does not have associated changes are classified into one category corresponding to another initial container. In addition, data with extremely low frequency of change can also be classified into one category and also correspond to one initial container.

[0060] Taking the initial data as movement data as an example, the movement data package contains 40 movement variables, and the change frequencies of these 40 movement variables are not the same. Among them, some movement variables usually change together, such as the horizontal coordinates of the target point that the character faces and runs, because the character turns during the running process, and the horizontal coordinates of the target point will also change; the longitudinal speed and the jump counter usually change together, because the character jumps and the longitudinal speed changes. There are also some movement variables that are mutually exclusive, such as the locked foot flag and the coordinate change. In addition, there are some movement variables that are obviously unrelated, such as the maximum running speed and the water suspension flag. In addition, there are some movement variables that change very rarely, for example: the locked movement flag will only be triggered when the character is fixed, which is much lower than the frequency of left and right changes when walking. Among them, movement variables that often change together can be called strongly associated variables, and vice versa, they are weakly associated variables and completely unrelated variables. Therefore, according to the strength of the correlation between the change frequencies of these movement variables, the movement variables can be divided into different types (strongly associated type, weakly associated type, and unrelated type), and different types of movement variables correspond to different initial containers.

[0061] Based on this, the target container corresponding to the difference data can be determined in the initial container according to the correspondence between the pre-established data information (such as data identification, data scenario, etc.) and the container information of the initial container (such as container identification, container name, etc.). Since there may be multiple difference data, and different difference data may correspond to different initial containers. Therefore, the number of target containers can also be one or more. On the basis of determining the target container, according to the above correspondence, all the data corresponding to the target container can be determined in the initial data, that is, the container data associated with the target container.

[0062] Furthermore, considering that the amount of data in the initial data and the number of initial containers may be relatively large, in order to avoid spending a lot of time in the process of determining the corresponding relationship between the two, the container set corresponding to the initial data as a whole can be searched in advance, and then the container data associated with the target container can be determined from the initial container. The specific implementation is as follows:

[0063] Determine a container set corresponding to the initial data according to the type information of the initial data; select a target container from the container set based on the data attribute information of the difference data; and extract the container data from the initial data according to the associated data information of the target container.

[0064] Specifically, type information refers to information used to indicate the data type of the initial data, and the type information may be information such as a type name or a type identifier. A container set refers to a set of initial containers corresponding to the initial data. Data attribute information refers to information related to the data, such as data name, data identifier, etc. Associated data information refers to information related to the data associated with the target container, and the associated data information may also be information such as a data name or a data identifier.

[0065] Based on this, the container set corresponding to the initial data can be determined according to the correspondence between the pre-defined type information and the container set and the type information of the initial data; and the target container corresponding to the difference data can be determined in the container set according to the correspondence between the pre-defined data attribute information and the initial container and the data attribute information of the difference data; and the container data can be extracted from the initial data according to the relationship between the associated data information of the target container and the initial data.

[0066] Using the above example, based on the difference data obtained above being 5 attribute variables, according to the data type of the attribute data D1: attribute type, the container set corresponding to the attribute type is determined to be C. The variable names of these 5 attribute variables are: d1, d3, d13, d22, d29. According to the correspondence between the variable names of these 5 attribute variables and the container identifier of the initial container, the target container corresponding to the 3 attribute variables (d1, d3, d29) in the difference data is determined from the container set C to be: target container c1, and the target container corresponding to the other 2 attribute variables (d13, d22) in the difference data is: target container c2. Then, according to the variable names associated with the target container c1: d1, d3, d5, d16 and d29, the 5 attribute variables with the variable names: d1, d3, d5, d16 and d29 are extracted from the attribute data D1 (these 5 attribute variables include the 3 attribute variables d1, d3, d29 in the difference data above). And according to the variable names associated with the target container c2: d7, d13, d22, d38, the four attribute variables with variable names: d7, d13, d22, d38 are extracted from the attribute data D1 (these four attribute variables include the two attribute variables d13 and d22 in the above difference data).

[0067] In summary, by first determining the container set corresponding to the initial data and then determining the target container corresponding to the difference data in the container set, it is avoided to screen the target container from a large number of initial containers, and the efficiency of determining the target container is improved.

[0068] Step 108: filling the target container with the container data according to the filling rule corresponding to the target container, and generating a compressed data packet associated with the compression task according to the filling result.

[0069] Specifically, based on the above extraction of container data associated with the target container, it is considered that if the container data is directly and arbitrarily filled into the target container, the container data will still occupy a lot of storage space. Therefore, the container data can be filled according to a pre-set rule (i.e., a filling rule), so as to further achieve the purpose of data compression. Among them, the filling rule refers to the rule for filling data into the initial container. The filling rule may include information such as filling data information, filling order, filling position, filling priority and / or filling strategy. Correspondingly, the filling result refers to the data generated after the container data is filled into the target container.

[0070] Based on this, the obtained container data is filled according to the filling rule corresponding to the target container, which can be understood as compressing the container data according to the filling rule. In specific implementation, the filling can be achieved by operations including splicing and / or cutting of data.

[0071] Continuing with the above example, based on the five attribute variables named d1, d3, d5, d16 and d29 extracted from the attribute data D1, these five attribute variables are trimmed according to the corresponding variable length (i.e., storage bit length), and the trimmed data is added to the target container c1 according to the preset filling order, and then the spliced ​​data in the target container c1 is encapsulated to generate a data compression package.

[0072] Considering that if the correspondence between the initial data and the initial container is set arbitrarily, the compression effect of the generated compressed data packet will not be obvious. Therefore, in order to enhance the compression effect of the initial data, the initial container can be created according to the different strengths of the correlation between the data. The specific implementation includes the following steps:

[0073] Step 1082: Acquire a set of periodic historical data within a preset period associated with the compression task.

[0074] Specifically, the preset period refers to a preset time period, which may be 5 hours, 1 day, or 3 days, etc. Correspondingly, the periodic historical data set refers to a set of historical data (i.e., periodic historical data) generated or obtained within the preset period. The periodic historical data refers to historical data within the preset period.

[0075] Based on this, the specific method of obtaining the periodic historical data set associated with the compression task is similar to the above-mentioned method of obtaining the initial data. Please refer to the above-mentioned specific implementation method of obtaining the initial data, which will not be repeated here.

[0076] For example: Get the historical attribute data set (periodic historical data set) within 1 day (preset period).

[0077] Step 1084: Compare the periodic historical data at adjacent time points in the periodic historical data set to obtain change information of the periodic historical sub-data in each periodic historical data.

[0078] Specifically, adjacent time points refer to the time points at which the periodic historical data are generated or obtained, which are adjacent within the preset period. For example, there are 3 periodic historical data within the preset period, and the times corresponding to these 3 periodic historical data are 2 o'clock, 4 o'clock, and 6 o'clock. Then, the periodic historical data corresponding to 2 o'clock and 4 o'clock are the periodic historical data of adjacent time points; the periodic historical data corresponding to 4 o'clock and 6 o'clock are the periodic historical data of adjacent time points.

[0079] Based on this, the periodic historical data of adjacent time points are compared, usually comparing the periodic historical data of the next adjacent time point with the periodic historical data of the previous time point, thereby obtaining the change information of the periodic historical sub-data in the periodic historical data of the next adjacent time point compared with the periodic historical sub-data in the periodic historical data of the previous time point. The periodic historical sub-data refers to the sub-data contained in the periodic historical data. For example, if the periodic historical data is composed of 5 variables, each variable can be called periodic historical sub-data.

[0080] Correspondingly, the change information is information related to data changes, and the change information may include: the data name or data identifier of the changed periodic historical sub-data, the number of changes, the data name or data identifier of the periodic historical sub-data that has not changed, etc. For example: among the 40 periodic historical sub-data contained in the periodic historical data corresponding to 6 o'clock, compared with the periodic historical data corresponding to 4 o'clock at the previous time point, 3 periodic historical sub-data (data identifiers are: dn4, dn20, dn23) have changed. Then the change information obtained for the 40 periodic historical sub-data corresponding to 6 o'clock is: the change frequency associated with dn4 and dn20 (i.e., the number of times they change together) is recorded as 1, the change frequency associated with dn4 and dn23 is recorded as 1, and the change frequency associated with dn20 and dn23 is recorded as 1.

[0081] It should be noted that, since the periodic historical data at the earliest time point in the preset period does not have the periodic historical data at the previous time point in the preset period, it is possible to obtain the historical data at the previous time point outside the preset period for comparison, or directly set the change information of the periodic historical sub-data in the periodic historical data at the earliest time point to null, which is not limited here.

[0082] Step 1086: Based on the change information, correlation statistics are performed on the periodic historical sub-data corresponding to the periodic historical data set to obtain correlation statistical information.

[0083] On the basis of obtaining the change information, correlation statistics are performed on the periodic historical sub-data based on the obtained change information. Specifically, the correlation statistics may be the number of changes of the periodic historical sub-data that have changed together in the periodic historical data within a preset period, and / or the number of changes of the periodic historical sub-data that have not changed together in the periodic historical data within a preset period, etc. In addition, the correlation statistics may also be the data identifier, change time and other information of the periodic historical sub-data that has been statistically changed. Therefore, the correlation statistical information may include the above-mentioned number of changes, data identifier, and / or change time and other information. The correlation statistical information may also be recorded in the form of a table or a chart.

[0084] Continuing with the above example, based on the above-mentioned acquisition of the historical attribute data set, the historical attribute data set includes 200 historical attribute data, and each historical attribute data includes 40 historical attribute sub-data. The historical attribute data of two adjacent time points in these 200 historical attribute data are compared to obtain the change information corresponding to each historical attribute data, among which the change information corresponding to the historical attribute data D2 is: the correlation change frequency of d4 (historical attribute sub-data) and d20 is 1, the correlation change frequency of d4 and d23 is 1, and the correlation change frequency of d20 and d23 is 1. The change information of these 200 historical attribute data is then counted, and the obtained correlation statistical information includes: the correlation change frequency of d1 and d20 is 110 times, the correlation change frequency of d2 and d23 is 180 times, ... the correlation change frequency of d20 and d23 is 20 times. The correlation statistical information can also be recorded in the form of tables or charts, as shown in the following example. Figure 2 shown.

[0085] Step 1088: performing subpackaging processing on the target historical data associated with the periodic historical data set according to the associated statistical information, and generating an initial container and a filling rule corresponding to the initial container based on the subpackaging processing result.

[0086] Specifically, the target historical data refers to the historical data to be processed by subpackaging. The association between the target historical data and the periodic historical data set can be that the data types of the two are the same, and / or the data structures of the two are the same. In specific implementation, the target historical data can also be any one of the periodic historical data, which is not limited here.

[0087] Accordingly, performing subpackaging processing on the target historical data refers to dividing the sub-data in the target historical data (ie, the target historical sub-data) into different data packets. Accordingly, the subpackaging processing result refers to the data packets obtained after dividing the target historical data.

[0088] Based on this, there are various ways to sub-packet the target historical data associated with the periodic historical data set according to the associated statistical information, for example, the associated change frequency in the associated statistical information is divided into frequency intervals, and the target historical sub-data corresponding to the associated change frequency in each frequency interval is divided into a data packet. Or the target historical sub-data corresponding to the same associated change frequency is divided into a data packet.

[0089] Furthermore, since different sub-packaging processing methods have different effects on compression results, in order to ensure that the sub-packaging processing results can reach the ideal compression target, the association weights between the target historical sub-data can be calculated first, and then sub-packaging is performed based on the association weights. The specific implementation includes the following steps:

[0090] Step 10882: Determine target historical data having the same data type as the periodic historical data in the periodic historical data set.

[0091] Specifically, the data type refers to the type to which the periodic historical data belongs. The data type may be an order type, a movement type, an attack type, a skin type, a scene type, etc., and is not limited here.

[0092] Step 10884: Calculate the association weights between the target historical sub-data in the target historical data based on the association statistical information.

[0093] Specifically, the association weight refers to data indicating the strength of the association relationship between the target historical sub-data. The association weight can be calculated by a preset weight formula. The preset weight formula can be calculated by dividing the number of times two variables change together by the sum of the number of times the two variables change, as shown below:

[0094] nWeight=nSameChange / (nAOcc+nBOcc);

[0095] The preset weight formula can also be shown as follows:

[0096] nWeight = nSameChange*min(nASize, nBSize)-min((nAOcc-nSameChange)*nBSize, (nBOcc-nSameChange)*nASize); where nSameChange: indicates the number of times the two target historical sub-data A and target historical sub-data B change together, nAOcc: indicates the number of times the target historical sub-data A changes; nBOcc: indicates the number of times the target historical sub-data B changes; nASize: indicates the size of the space occupied by the target historical sub-data A; nBSize: indicates the size of the space occupied by the target historical sub-data B. The meaning of this preset weight formula is to subtract the invalid communication volume from the effective communication volume, that is, the additional income generated when the two target historical sub-data change together minus the loss caused by the change of a single target historical sub-data while the other target historical sub-data does not change.

[0097] Step 10886: Based on the association weight, divide the target historical sub-data in the target historical data into at least one data packet.

[0098] Specifically, based on the above-mentioned association weights, the target historical sub-data with the same association weights can be divided into one data packet, and the target historical sub-data with the same association weights in the same preset interval can also be divided into one data packet, without limitation. In specific implementation, the target historical data can be divided into packets according to the association statistical information: the strongly related target historical sub-data are classified, that is, the strongly related target historical sub-data that often change together are divided into one data packet, the unrelated target historical sub-data are divided into one data packet, and the target historical sub-data with extremely low frequency changes are divided into one data packet.

[0099] Continuing with the above example, on the basis of obtaining the associated statistical information, the attribute variables with an associated change frequency greater than 150 times are divided into data packet A, and the attribute variables with an associated change frequency greater than 50 and less than 150 times are divided into data packet B, and the attribute variables with an associated change frequency less than or equal to 50 times are divided into data packet C, and further create the initial container c1 corresponding to data packet A and the filling rule r1 of the initial container c1, the initial container c corresponding to data packet B and the filling rule r2 corresponding to the initial container c2, and the initial container c3 corresponding to data packet C and the filling rule r3 corresponding to the initial container c3.

[0100] In summary, based on the correlation statistics of historical data within a period of time, the data is packaged according to the number of correlation changes in the correlation statistical information, and the corresponding initial containers and filling rules are created, so that data with similar numbers of correlation changes can be filled together, ensuring the rationality of creating the initial containers and filling rules.

[0101] Furthermore, since different subpackaging methods have different compression effects, in order to more clearly identify which subpackaging method is better and adopt a better subpackaging method for subpackaging, the embodiment of the present application can evaluate multiple subpackaging methods, and then determine the final subpackaging method according to the evaluation results. The specific implementation includes the following steps:

[0102] Step 10886-2: Connecting data nodes corresponding to target historical sub-data in the target historical data based on the association weights to generate at least one tree set;

[0103] Specifically, a data node is formed by abstracting each target historical sub-data into a node. Connecting these data nodes in one way may generate multiple connection subtrees (each connection subtree corresponds to a data packet), and these connection subtrees are combined into a tree set; therefore, if multiple connections are made to these data nodes, each connection method corresponds to a tree set. Among them, a tree set refers to a set consisting of at least one connection subtree.

[0104] Furthermore, considering that if multiple connection modes are randomly generated, there is no regularity in the generated connection modes, and many of them may be contrary to the goal of subpackaging, resulting in a waste of computing resources. Therefore, in order to save computing resources, the connection can be made based on the result of each connection based on the association weight. The specific implementation includes the following steps:

[0105] Based on the association weight and the i-th node connection relationship, the data nodes corresponding to the target historical sub-data in the target historical data are connected for the i-th time to generate the i-th initial tree set;

[0106] Determine whether the number of connected subtrees in the i-th initial tree set is greater than a preset threshold;

[0107] If it is greater than, it indicates that the number of connected subtrees is large and the next connection can be made, then the i-th initial tree set is added to the tree set group as an intermediate tree set; determine the target connection relationship corresponding to the data node in the i-th initial tree set, i is incremented by 1, and the target connection relationship is used as the i-th node connection relationship, and return to execute the i-th connection based on the association weight and the i-th node connection relationship, the data nodes corresponding to the target historical sub-data in the target historical data are connected for the i-th time, and the i-th initial tree set is generated;

[0108] If they are equal, it indicates that the number of connected subtrees is small and the next connection will not be performed, then the intermediate tree sets in the tree set group are combined into at least one tree set.

[0109] Specifically, the i-th node connection relationship refers to the connection relationship between the data nodes generated by the i-1-th connection before the i-th connection is made to the data nodes corresponding to the target historical sub-data. Before the first connection, the first node connection relationship is empty. The i-th initial tree set refers to the tree set composed of at least one connection subtree generated by the first connection between the data nodes corresponding to the target historical sub-data. Correspondingly, the preset threshold refers to a preset threshold value for the number of connection subtrees. When the number of connection subtrees reaches the preset threshold value, it indicates that the next connection can be stopped. The preset threshold value also represents the minimum threshold value for the number of acceptable data packets. The value of the preset threshold value can be a positive integer less than the number of nodes, such as 1 or 2, and is not limited here.

[0110] Based on this, each time a connection is made between data nodes, one edge or two edges can be connected, and there is no restriction here. When each connection is completed, at least one connection subtree can be generated based on the data nodes and the edges between these data nodes. In addition, since each time a connection is completed, there may be data nodes without any edges. In this case, the data node is treated as a connection subtree alone. Optionally, in this embodiment, an MST (Minimum Spanning Tree) minimum spanning tree algorithm can be used: such as Prim algorithm or Kruskal algorithm, etc., to connect data nodes according to the association weights between data nodes, and each connection step is regarded as a connection, and the connection subtree generated by each connection is used as the initial tree set corresponding to the current connection.

[0111] When the number of connected subtrees is greater than the preset threshold, it indicates that the number of connected subtrees is large and the next connection can be made. Then the i-th initial tree set is added to the tree set group as an intermediate tree set to retain the connection result corresponding to the i-th initial tree set. The connection relationship corresponding to the data node in the i-th initial tree set (i.e., the target connection relationship) is used as the connection basis for the next connection (i.e., the i+1 node connection relationship), and so on, and a new i-th initial tree set is continuously iterated. Each generated i-th initial tree set is added to the tree set group as an intermediate tree set for retention.

[0112] When the number of connected subtrees is greater than a preset threshold, it indicates that the number of connected subtrees is large and the next connection can be made, then the i-th initial tree set is added to the tree set group as an intermediate tree set, so as to retain the connection method of the i-th initial tree set. The connection relationship corresponding to the data node in the i-th initial tree set (i.e., the target connection relationship) is used as the connection basis for the next connection (i.e., the i+1-th node connection relationship), and so on, and a new i-th initial tree set is continuously iterated to generate. The new i-th initial tree set is added to the tree set group as an intermediate tree set for retention. It should be noted that before the intermediate tree set is added, the tree set group is empty.

[0113] When the number of connected subtrees is equal to a preset threshold, it indicates that the number of connected subtrees is small and no next connection is made, then the intermediate tree sets retained in the previous connection process are combined into at least one tree set.

[0114] For example, the target historical attribute data includes 40 target historical attribute sub-data, and a corresponding data node is created for each target historical attribute sub-data, thus generating 40 data nodes. The Kruskal algorithm is used to connect the first edge between the two data nodes with the largest association weight among the 40 data nodes as the first connection, and 39 connection subtrees are generated, and these 39 connection subtrees form the first initial tree set. If the result of judging whether 39 is greater than the preset threshold value 1 is greater than, the first initial tree set is added to the tree set group as an intermediate tree set. And the connection relationship between the data nodes in the first initial tree set is taken as the second node connection relationship. Based on the association weight, the Kruskal algorithm is used to connect the second edge of the 40 data nodes as the second connection on the basis of the relationship of the second node connection, and 38 connection subtrees are generated, and these 38 connection subtrees form the second initial tree set, and so on, until the number of connection subtrees is 1. The intermediate tree set added to the tree set group forms at least one tree set.

[0115] In summary, based on the association weight, the next connection is made on the basis of each connection, which ensures that the nodes on both sides of the edge with a large association weight are associated first. In this way, the connected nodes are divided into one data packet, which improves the rationality of the connection and saves computing resources.

[0116] Step 10886-4: Evaluate each tree set according to the associated statistical information to obtain evaluation data corresponding to each tree set.

[0117] Specifically, evaluating each tree set according to associated statistical information refers to evaluating the space occupancy of each tree set according to the associated statistical information, that is, evaluating whether the connected subtree divisions in the tree set can achieve a compression effect that significantly reduces the amount of data to be processed during data processing.

[0118] Furthermore, in order to accurately evaluate the quality of each tree set, in this embodiment, the target historical sub-data corresponding to the connected subtrees in each tree set can be spliced, and then the tree set can be evaluated according to the splicing result. For any tree set, the specific implementation includes the following steps:

[0119] According to the storage information of the node sub-data corresponding to the data nodes in each connection sub-tree in the tree set, the node sub-data corresponding to each connection sub-tree are spliced ​​to generate the spliced ​​sub-data corresponding to each connection sub-tree;

[0120] An evaluation calculation is performed according to the splicing storage information of the splicing sub-data and the associated statistical information to obtain evaluation data corresponding to the tree set.

[0121] Specifically, node sub-data refers to the target historical sub-data corresponding to the data node. Storage information refers to information related to the storage of node sub-data. The storage information can be information such as the storage bit length of the node sub-data, and / or the alignment length. Spliced ​​sub-data refers to data generated after splicing node sub-data. The number of spliced ​​sub-data can be one or more, which is not limited here. Spliced ​​storage information refers to information related to the storage of spliced ​​sub-data. The spliced ​​storage information can be information such as the storage bit length of the spliced ​​sub-data, and / or the alignment length. Evaluation data refers to the evaluation result obtained by evaluating and calculating the tree set. The evaluation data can be in the form of an evaluation score, and can also be in other forms such as an evaluation grade, which is not limited here.

[0122] In specific implementation, a preset evaluation function can be used to evaluate and calculate the tree set. The preset evaluation function first calculates the total size of each connected subtree multiplied by the number of changes in the node subdata with the most changes in the connected subtree, and then adds the mask length. Then the calculation results corresponding to each connected subtree are summed, which can be specifically shown as follows:

[0123]

[0124] Among them, EScore: represents the evaluation score, k represents the serial number of the current connection subtree, and n represents the number of connection subtrees; vChange: represents the number of changes in the node subdata corresponding to each data node in the connection subtree, mSize: represents the minimum space corresponding to the spliced ​​subdata in each connection subtree, and mLen: represents the mask length. Specifically, the mask length refers to the length of the mask corresponding to the connection subtree in the tree set. Specifically, each connection subtree corresponds to a mask bit, and the length of the mask is equal to the number of connection subtrees in the tree set. A mask bit of 1 indicates that there is a connection subtree corresponding to the mask bit, and a mask bit of 0 indicates that there is no connection subtree corresponding to the mask bit.

[0125] For example, based on the above-mentioned acquisition of at least one tree set, for any tree set, it is assumed that the tree set S includes 3 connected subtrees, wherein the data nodes in the connected subtree 1 correspond to 3 target historical attribute subdata, and the storage bit lengths of the two target historical attribute subdata are obtained, and the obtained storage bit lengths are 2 bits and 6 bits respectively. The two target historical attribute subdata in the connected subtree 1 are spliced ​​to generate spliced ​​subdata cd1; the data nodes in the connected subtree 2 correspond to 3 target historical attribute subdata, and the storage bit lengths of the three target historical attribute subdata are obtained, and the obtained storage bit lengths are 5 bits, 2 bits, and 3 bits respectively. The three target historical attribute subdata in the connected subtree 2 are spliced ​​to generate spliced ​​subdata cd2; and so on, the spliced ​​subdata cd3 corresponding to the connected subtree 3 is generated according to the 35 target historical attribute subdata corresponding to the data nodes in the connected subtree 3. Among them, the splicing sub-data cd1 occupies 1 byte of space (splicing storage information), the splicing sub-data cd2 occupies 2 bytes of space (splicing storage information), and the splicing sub-data cd3 occupies 15 bytes of space (splicing storage information). The above splicing storage information and the associated statistical information of the above 40 target historical attribute sub-data are evaluated and calculated according to the above preset evaluation function, and the evaluation score of the tree set S is 80 points.

[0126] In summary, the node sub-data corresponding to each connected subtree in the tree set are first spliced, and then the tree set is evaluated and calculated as a whole based on the splicing results and the statistical correlation frequency, which ensures the comprehensiveness of the evaluation and improves the accuracy of the evaluation.

[0127] Furthermore, considering that a simple splicing process of the node sub-data corresponding to each connection sub-tree cannot achieve an obvious data compression effect, therefore, in this embodiment, each node sub-data and various alignment lengths can be traversed to try multiple splicing methods, and the idea of ​​dynamic programming is adopted to obtain a splicing method that achieves an ideal data compression effect and perform splicing. The specific implementation includes the following steps:

[0128] Obtain the preset alignment length and the storage bit length of the node sub-data corresponding to each connection sub-tree;

[0129] The node sub-data corresponding to each connection sub-tree, the storage bit length and the preset alignment length are traversed, and the node sub-data corresponding to each connection sub-tree are bit-domain spliced ​​to generate spliced ​​sub-data corresponding to each connection sub-tree.

[0130] Specifically, the preset alignment length refers to the pre-set alignment length for data splicing, and the alignment length includes: 8 bits, 16 bits, 32 bits, 64 bits, etc. For example, if the preset alignment length is 8 bits, the node sub-data is spliced ​​with a length of 8 bits. The preset alignment length can be at least one. The storage bit length refers to the length of the valid data bits in the node sub-data. For example, if the valid data bit of the node sub-data A is 1 bit, the storage bit length of the node sub-data is 1. Bit field splicing refers to splicing the binary bits occupied by the node sub-data to form a complete byte.

[0131] In practical applications, because computers encounter "bit field" problems when storing or synchronizing data, some simple data are flag bits that indicate presence or absence, and only need to occupy 1 bit, such as: whether it is suspended in water (the smallest unit of computer storage is 1 byte = 8 bits). The length of some data is smaller than the smallest unit of computer representation, such as mobile state enumeration, which only needs 6 bits. These data can be pieced together through "bit fields" to save transmission bandwidth during transmission. However, computers have an "alignment" limitation. All spliced ​​data must be a complete 8-bit, 16-bit, 32-bit or 64-bit data. And a piece of data cannot cross alignment in the bit field state. If an alignment variable is not fully filled, the empty space behind it can only be left empty. For example Figure 3 After the second data byMoveFrameCount shown, there are 6 bits that are empty, and the third data starts after the 6 empty bits. Therefore, in order to further compress the space occupation, it is also necessary to adjust the order according to the bit length of each node sub-data, and try to splice as many node sub-data as possible under the alignment restriction.

[0132] Based on this, the node sub-data corresponding to each connection sub-tree, the storage bit length and the preset alignment length are traversed, and the node sub-data corresponding to each connection sub-data are spliced ​​in bit fields. The node sub-data is first converted into a bit field form, and then each node sub-data is traversed in turn, and the storage bit length of the node sub-data is enumerated and the preset alignment length is tried to be spliced ​​with the spliced ​​node sub-data. That is, each of these node sub-data is tried under the restriction of the preset alignment length. Various possible splicing methods are then selected to generate spliced ​​sub-data. In specific implementation, the dynamic programming idea can be adopted to calculate and realize the above splicing according to the state transfer equation.

[0133] Specifically, the state transfer equation is as follows:

[0134]

[0135] Among them, i represents the node sub-data, j represents the alignment length used by the last node sub-data, k represents the number of bits remaining in the field of the last alignment length, Align(j) represents the length of the jth alignment length, and len(i) represents the storage bit length of the i-th node sub-data.

[0136] The state transfer equation specifically traverses each node sub-data and traverses all preset alignment lengths. If the current preset alignment length is less than the storage bit length of the current node sub-data, the next preset alignment length is enumerated. If the current preset alignment length is greater than or equal to the storage bit length of the current node sub-data, the current node sub-data is attempted to be spliced ​​with the splicing result corresponding to the previous node sub-data, and the splicing method with the smallest splicing length is selected during the traversal process.

[0137] In addition, the splicing can also be achieved through the following steps: obtain the i-th node child data corresponding to the connected subtree, where i is a positive integer and i starts from 1; and obtain the j-th alignment length, and use the j-th alignment length as the alignment length of the i-th node child data, where j is a positive integer and j starts from 1; then determine whether the j-th alignment length is less than the storage bit length of the i-th node child data; if so, j is incremented by 1, and return to execute to obtain the j-th alignment length; if not, obtain the n-th alignment length, where n is a positive integer and n starts from 1, and determine whether the j-th alignment length is equal to the n-th alignment length;

[0138] If they are equal, the i-th node sub-data is spliced ​​with the splicing result containing the i-1-th node sub-data to obtain the i-th splicing result; if they are not equal, the difference between the j-th alignment length and the storage bit length of the i-th node sub-data (i.e., the number of remaining bits) is calculated;

[0139] If the difference is greater than or equal to 0, the i-th node sub-data is concatenated with the concatenation result of the i-1-th node sub-data; i is incremented by 1, and the above-mentioned process of obtaining the i-th node sub-data corresponding to the number of connections is returned; if the difference is less than 0, n is incremented by 1, and the process of obtaining the n-th alignment length is returned.

[0140] In summary, we will try to perform bit-field splicing on the node sub-data by traversal in order to obtain the best splicing effect, further compress the space occupancy and improve the degree of compression.

[0141] Step 10886-6: Based on the evaluation data, determine a target tree set corresponding to the target historical data in at least one tree set.

[0142] Specifically, based on the evaluation data obtained above, if the evaluation data is an evaluation score, the tree set with the highest evaluation score in at least one tree set can be determined as the target tree set according to the score. If the evaluation data is a level, the tree set with the highest evaluation level in at least one tree set can be determined as the target tree set according to the level. Each connected subtree in the target tree set indicates the division of the target historical subdata.

[0143] Step 10886-8: Encapsulate the node sub-data corresponding to the data nodes of each connection sub-tree in the target tree set in the target historical data respectively to generate at least one data packet.

[0144] Specifically, based on the above determination of the target tree set, the node sub-data divided according to each connection sub-tree is encapsulated according to a preset data protocol (such as UDP protocol, TCP protocol) to generate data packets corresponding to each connection sub-tree.

[0145] Step 10888: Generate an initial container corresponding to each data packet and a filling rule corresponding to the initial container according to the data attribute information of the target historical sub-data in each data packet.

[0146] Specifically, based on the generation of at least one data packet, a corresponding initial container and a filling rule corresponding to the initial container are created for each data packet, so that data associated with the initial container can be filled according to the filling rule.

[0147] Furthermore, considering that the created initial container is used to fill the corresponding container data, the clearer the filling rule is, the easier it is to fill. Therefore, in this embodiment, in order to increase the ease of filling, the generation of any initial container and the filling rule corresponding to any initial container is specifically implemented as follows:

[0148] Determining packet attribute information of the data packet according to data attribute information of the target historical sub-data in the data packet;

[0149] Creating an initial container corresponding to the data packet according to the packet attribute information;

[0150] Determine the filling position information of each target historical sub-data in the initial container according to the storage bit length and the splicing order of the target historical sub-data in the data packet;

[0151] Based on the filling position information, a filling rule corresponding to the initial container is created.

[0152] Specifically, packet attribute information refers to information related to the data packet, such as: data packet identifier, the amount of data contained in the data packet, etc. The splicing order refers to the order of the target historical sub-data in the data packet. The filling position information refers to information that can identify the position of the historical sub-data in the initial container. The filling position information may include the filling start position and filling end position corresponding to each historical sub-data in the initial container.

[0153] Based on this, the data packet identifier and data of the data packet are determined according to the data attribute information of the target historical sub-data in the data packet, and then the initial container corresponding to the data packet is created according to the data packet identifier and data quantity of the data packet. The initial container is an abstract concept, which only includes basic information such as the container identifier and the amount of data contained in the container.

[0154] In actual applications, the starting position of each target historical sub-data in the initial container can be determined based on the storage bit length and splicing order of the target historical sub-data in the data packet, and then each target historical sub-data and the corresponding filling starting position can be used as the filling rule of the initial container.

[0155] For example: according to the data quantity 3 of the target historical sub-attribute variable in data packet A, these three target historical attribute variables are d2, d23 and d30 respectively, and the packet attribute information of the data packet is determined to be A3. The container name of the corresponding initial container created according to the packet attribute information A3 is c-A3, and then the storage bit lengths of d2, d23 and d30 are obtained as 4 bits, 5 bits and 12 bits respectively, and the splicing order between them is d2, d30, d23, then the filling starting position of d2 is determined to be the 1st bit, the filling starting position of d3 is the 5th bit, and the filling starting position of d23 is the 17th bit. These three information are used as filling position information to create the filling rules corresponding to the initial container c-A3.

[0156] In summary, the initial container is created first, and then the filling position information is determined according to the attribute information of the target historical sub-data, and the filling rule corresponding to the initial container is created, which ensures the accuracy of filling and improves the efficiency of filling.

[0157] Based on the above generation of compressed data packets, considering that there may be multiple compressed data packets in the data transmission scenario, in order to ensure the integrity of the transmission, this embodiment encapsulates these compressed data packets into a compressed data total packet for transmission, and the specific implementation is as follows:

[0158] Determining at least two compressed data packets associated with the compression task;

[0159] Combining at least two compressed data packets according to a preset coding sequence and coding identifiers corresponding to at least two compressed data packets to generate a combined compressed data packet;

[0160] Generating a data packet mask corresponding to the combined compressed data packet according to a preset encoding sequence and the encoding identifier;

[0161] generating a compressed data packet based on the data packet mask and the combined compressed data packet;

[0162] The compressed data package is sent to the client.

[0163] Specifically, the preset coding sequence refers to the preset coding sequence of the compressed data packets. Correspondingly, the corresponding number of each compressed data packet in the preset coding is the coding identifier.

[0164] Based on this, combining at least two compressed data packets according to the preset coding sequence and the coding identifiers means combining the compressed data packets according to the arrangement order of their coding identifiers in the preset coding sequence.

[0165] For example, the initial data corresponds to 64 compressed data packets, and the 64 compressed data packets correspond to numbers 1-64. The order of 1-64 is the preset coding order. There are 10 compressed data packets associated with the compression task. These 10 compressed data packets are combined according to the preset coding order to generate a combined compressed data packet CB. Among them, the mask bit corresponding to the vacant data packet is 0, and the mask bit corresponding to these 10 compressed data packets is 1, then a 64-bit data packet mask is generated. Combine the compressed data packet CD and the 64-bit data packet mask to generate a compressed data total packet TD, and send the compressed data total packet TD to the client.

[0166] In summary, multiple compressed data packets are combined, and the combined result is combined with the corresponding data packet mask to generate a compressed data total packet for data transmission, which reduces the transmission bandwidth and increases the transmission efficiency.

[0167] It should be noted that because games need to be maintained and upgraded and requirements change, and game data will also change with requirements, if the frequency of associated changes in data is collected each time compression is performed and then the above data compression is performed, it will consume a lot of computing resources and time costs, and will also become an obstacle to daily modification requirements. Therefore, the compression method needs to be further automated through the use of tools to improve maintainability.

[0168] Therefore, the most original uncompressed data structure can be maintained, and then the initial data within the cycle can be automatically obtained through the tool. The frequency of changes in each variable can be counted in actual operation to generate a table. This table will be handed over to a tool, which will execute the above MST and dynamic programming algorithms to generate the "compressed version protocol" of the data packet solution and the corresponding compression and decompression function bodies.

[0169] The game server will generate the original protocol data structure before sending the protocol, and then hand it over to the supporting compression function generated by the tool. After comparing it with the last synchronization, find the changed variables, convert it into a compressed version of the protocol and send it. After the client receives the compressed version of the protocol, it will use the decompression function to combine the data received last time, reconvert and assemble it into the original protocol data structure (the variables that are not synchronized in the compression protocol are the variables that have been received before), and then hand this original protocol data structure over to the corresponding protocol processing function. The advantage of doing this is that you only need to maintain the old sending and receiving protocol functions and protocol data structures, decouple the optimization of the compression algorithm from daily maintenance, and hand it over to the tool for automated processing. The client generates the same program source code as the server for the entire solution and compiles it in. Because the frequency and relevance of variable synchronization will not change frequently during actual operation, you can use the tool to optimize it once in a while, and there is no need to make it real-time. In this way, the complexity of the entire process is not very high, and performance can be taken into consideration.

[0170] Specifically, Figure 4 As shown in (a), it can include the tool automatic optimization process and the original production process. In the original production process, after adding, deleting and modifying the protocol variables, the corresponding modifications are made manually, and the modifications are sent, the functions are received, and the received functions are compiled to complete the maintenance. In the tool automatic optimization process, after adding, deleting and modifying the protocol variables, data collection is run to obtain the historical protocol variables within the preset period, and the frequency analyzer is used to analyze and generate a frequency association table. Then, the sub-packet (data packet) solution generation tool is used to directly generate the code, and the protocol (sub-packet solution) / compression / decompression function is optimized through the code. And the optimized function is compiled to complete the maintenance.

[0171] Furthermore, Figure 4 The actual application process of the two methods in (a) is as follows Figure 4As shown in (b): After the broadcast is initiated, the broadcast data is assembled into the original protocol packet. When optimization is turned on, the difference (difference data) between the original protocol packet and the previous broadcast is compared, and compressed based on the difference (optimized protocol splicing sub-packets), and the generated compression result is sent instead of the original protocol data packet (i.e., the hijacking process). The original protocol data packet is transmitted over the network and received by the client. If the client sends an optimized protocol, then (decompression) the sub-packet is unpacked according to the mask, and the local existing data corresponding to the sub-packet is queried. The two are reassembled to obtain the original protocol packet format, and the original protocol packet format is used as the original protocol to be processed (i.e., the transfer process). If the client receives the original protocol data packet, that is, it is not an optimized protocol, then the original process is followed to directly process the original protocol data.

[0172] In summary, the data compression method provided by the embodiment of the present application obtains the initial data associated with the compression task; compares the initial data with the historical data associated with the compression task to obtain difference data; determines the target container corresponding to the difference data, and extracts the container data associated with the target container from the initial data, wherein the container data contains the difference data; fills the container data into the target container according to the filling rule corresponding to the target container, and generates a compressed data packet associated with the compression task according to the filling result. The initial data is compressed, the space occupied by the initial data is reduced, and the subsequent processing efficiency of the initial data is improved.

[0173] Corresponding to the above method embodiment, this specification also provides a data decompression method embodiment, Figure 5 A flowchart of a data decompression method provided by an embodiment of the present specification is shown, which is applied to a client and specifically includes the following steps:

[0174] Step 502: Obtain a compressed data packet associated with the decompression task.

[0175] Specifically, the decompression task refers to the task of decompressing a compressed data packet. The decompression task can be understood as the reverse process of the compression task in the above method embodiment.

[0176] Step 504: extracting container data of the target container from the compressed data packet according to an extraction rule of the target container corresponding to the compressed data packet.

[0177] Specifically, the extraction rule refers to the rule for extracting container data in the target container. The extraction rule is similar to the filling rule in the above method embodiment, and may also have extraction information related to the container data, such as extraction order, extraction data information, extraction name, extraction location information, extraction strategy and other information, which is not limited here.

[0178] Step 506: Determine historical data associated with the container data according to the data attribute information of the container data.

[0179] Specifically, according to the data type of the initial data to which the container data belongs, the historical data at the last time point corresponding to the initial data to which the container data belongs may be obtained.

[0180] Step 508: Generate target data based on the container data and the historical data.

[0181] Specifically, the target data can be understood as the initial data in the above method embodiment. The target data can be generated by removing the data corresponding to the container data from the historical data and then combining the removed historical data with the container data.

[0182] Optionally, considering that in the process of data transmission between the server and the server, the data transmitted is usually not a single compressed data packet, therefore, it is necessary to first parse the received compressed data packet to obtain at least one compressed data packet. In this embodiment, the specific implementation is as follows:

[0183] Receive the compressed data package sent by the server;

[0184] Parsing the compressed data total packet to obtain a data packet mask and a combined compressed data compressed packet;

[0185] Based on the data packet mask, at least one compressed data packet is extracted from the combined compressed data packet, wherein the compressed data packet is included in the at least one compressed data packet.

[0186] Specifically, parsing the compressed data packet can be understood as disassembling the data packet mask and the combined compressed data packet in the compressed data packet.

[0187] For example: receive the compressed data packet TD sent by the server, parse the compressed data packet, and obtain a 64-bit packet mask and a combined compressed data packet CD. According to the 64-bit packet mask, extract 10 compressed data packets with a mask bit of 1 in the combined data packet CD, and decompress the data in these 10 compressed data packets respectively to obtain the container data corresponding to each compressed data packet. Then, according to the data type T of the container data, obtain the historical data of the previous adjacent time point corresponding to the data type T, and then replace the data with the same data name in the historical data with the container data, so that the target data can be restored.

[0188] In summary, by receiving the compressed data package sent by the server and parsing the compressed data package, each compressed data package is obtained, thereby realizing the reception of compressed data packages based on the data compression package, and obtaining multiple data compression packages at one time by parsing the compressed data package.

[0189] In summary, the data decompression method provided by the embodiment of the present application achieves the restoration of the data in the compressed data packet by acquiring the compressed data packet associated with the decompression task; extracting the container data of the target container from the compressed data packet according to the extraction rule of the target container corresponding to the compressed data packet, and then determining the historical data associated with the container data according to the attribute information of the container data; and generating the target data based on the container data and the historical data. The target data is restored by merging the container data with the historical data.

[0190] The above is a schematic scheme of a data decompression method of this embodiment. It should be noted that the technical scheme of the data decompression method and the technical scheme of the above data compression method belong to the same concept, and the details of the technical scheme of the data decompression method that are not described in detail can be referred to the description of the technical scheme of the above data compression method.

[0191] The following combination Figure 6 Taking the application of the data compression method provided in this application to a game scene as an example, the data compression method is further described. Figure 6 A processing flow chart of a data compression method applied to a game scene provided by an embodiment of the present application is shown, which specifically includes the following steps:

[0192] Step 602: Acquire a set of periodic historical data within a preset period associated with the compression task.

[0193] Specifically, a set of periodic historical movement data within one week is obtained.

[0194] Step 604: Compare the periodic historical data at adjacent time points in the periodic historical data set to obtain change information of the periodic historical sub-data in each periodic historical data.

[0195] Specifically, the periodic historical motion data set includes 100 periodic historical motion data. The periodic historical motion data at adjacent time points in the 100 periodic historical motion data are compared, and the changes in each periodic historical motion sub-data at the later time point among the adjacent time points within this week are recorded.

[0196] Step 606: Perform correlation statistics on the periodic historical sub-data corresponding to the periodic historical data set based on the change information to obtain correlation statistical information.

[0197] Specifically, according to the recorded change information, the number of changes in 30 periodic mobile historical sub-data in 100 periodic historical mobile data is counted, and the associated change frequencies obtained include: the associated change frequency of periodic mobile historical sub-data md1 and md5 is 60 times, the associated change frequency of md2 and d33 is 30 times, ... the associated change frequency of md8 and md40 is 5 times. Furthermore, the individual change frequency of each periodic mobile historical sub-data can also be counted, for example: the individual change frequency of md1 is 80 times, the individual change frequency of md5 is 65 times,

[0198] Step 608: Determine target historical data having the same data type as the periodic historical data in the periodic historical data set.

[0199] Specifically, any one of the periodic historical movement data is determined as the target historical data.

[0200] Step 610: Calculate association weights between target historical sub-data in the target historical data according to the association statistical information.

[0201] Specifically, according to the above statistical association change frequency and the preset weight formula of the above method embodiment, the association weight between every two target historical movement sub-data is calculated, wherein the calculation process of md1 and md5 is as follows:

[0202] nWeight = 60*min(5kb, 8kb)-min((65-60)*8kb, (80-60)*5kb) = 260. 5kb is the space occupied by md1, and 8kb is the space occupied by md5.

[0203] Step 612: Based on the association weight, the data nodes corresponding to the target historical sub-data in the target historical data are connected to generate at least one tree set.

[0204] Specifically, the data nodes corresponding to the 30 target historical mobile sub-data can be connected multiple times based on the order of association weights from large to small, and the connection subtrees generated by each connection are regarded as a tree set. In addition, each time the next connection is made based on the previous connection. Then the first connection generates 29 connection subtrees, and these 29 connection sub-data are regarded as a tree set. The second connection is made based on these 29 connection subtrees to generate 28 connection subtrees, and these 28 connection subtrees are also regarded as a tree set, and so on. When all the data nodes are finally connected into a connection subtree, the connection is stopped. This connection process can be implemented using the MST algorithm, and 29 tree sets can be obtained in the end.

[0205] Step 614: Obtain the preset alignment length and the storage bit length of the node sub-data corresponding to each connection sub-tree.

[0206] Specifically, four preset alignment lengths are obtained: 8 bits, 16 bits, 32 bits, and 64 bits, and the storage bit length of each node sub-data is obtained, where the storage bit length of md1 is 5 bits, the storage bit length of md5 is 10 bits, and the storage bit length of md7 is 3 bits.

[0207] Step 616: traverse the node sub-data, storage bit length and preset alignment length corresponding to each connection sub-tree, perform bit domain splicing on the node sub-data corresponding to each connection sub-tree, and generate spliced ​​sub-data corresponding to each connection sub-tree.

[0208] Specifically, assuming that the connection subtree T1 includes 3 node sub-data, namely md1, md5 and md7. By executing the above state transition equation, traversing these 3 node sub-data, the preset alignment length, and the difference between each preset alignment length and the storage bit length, various possible connection methods are tried, and the best connection method is obtained among the connection methods for bit field splicing to generate the spliced ​​sub-data corresponding to the connection sub-data T1.

[0209] Step 618: Perform evaluation calculation based on the splicing storage information and associated statistical information of the splicing sub-data to obtain evaluation data corresponding to the tree set.

[0210] Specifically, for the 29 tree sets, each tree set is evaluated and calculated according to the preset evaluation function in the above method embodiment to obtain the evaluation score corresponding to each tree set. Assume that the evaluation score corresponding to the tree set TS1 is 75.

[0211] Step 620: Based on the evaluation data, determine a target tree set corresponding to the target historical data in at least one tree set.

[0212] Specifically, assuming that the evaluation score corresponding to the tree set TS1 is 75, which is the highest evaluation score, then among the 29 tree sets, it is determined that the target tree set corresponding to the target historical movement data is ST1.

[0213] Step 622: Encapsulate the node sub-data corresponding to the data nodes of each connection sub-tree in the target tree set in the target historical data respectively to generate at least one data packet.

[0214] Specifically, assuming that the tree set ST1 includes three connection subtrees, the node subdata in each connection subtree are encapsulated respectively to generate a data packet corresponding to each connection subtree.

[0215] Step 624: Generate an initial container corresponding to each data packet and a filling rule corresponding to the initial container according to the data attribute information of the target historical sub-data in each data packet.

[0216] Specifically, according to the data information of the target historical movement sub-data in the three data packets, an initial container corresponding to each data packet and a filling rule corresponding to each of the three initial containers are generated.

[0217] Step 626: Obtain initial data associated with the compression task.

[0218] Specifically, the mobile data IMD associated with the compression task is obtained.

[0219] Step 628: Compare the initial data with the historical data associated with the compression task to obtain differential data.

[0220] Specifically, the mobile data IMD is compared with the historical mobile data HMD associated with the compression task to obtain difference data between the two, and the difference data includes two mobile variables, namely md1 and md5.

[0221] Step 630: Determine the target container corresponding to the difference data, and extract container data associated with the target container from the initial data, wherein the container data includes the difference data.

[0222] Specifically, according to the correspondence between the data attribute information and the container information, it is determined that the target containers corresponding to md1 and md5 are both TC1. Then, the container mobile data associated with TC1 is extracted from the mobile data IMD.

[0223] Step 632: fill the target container with container data according to the filling rule corresponding to the target container, and generate a compressed data packet associated with the compression task according to the filling result.

[0224] Specifically, the container mobile data is filled into the target container according to the filling position information of the target container, and a compressed data packet corresponding to the mobile data IMD is generated.

[0225] Furthermore, in the case where there are multiple compressed data packets, the multiple compressed data packets can be used to generate a compressed data general packet and sent to the client to complete the transmission of the mobile data.

[0226] In summary, the data compression method provided by the embodiment of the present application obtains the initial data associated with the compression task, and compares the initial data with the historical data associated with the compression task to obtain the difference data, that is, obtains the data that has changed compared with the historical data, and then determines the target container corresponding to the difference data, and extracts the container data associated with the target container from the initial data, wherein the container data contains the difference data, and realizes the extraction of data strongly related to the difference data in the initial data through the target container corresponding to the difference data. Then, according to the filling rule corresponding to the target container, the container data is filled into the target container, and a compressed data packet associated with the compression task is generated according to the filling result, so that the extracted data is assembled to generate a compressed data packet to be processed, so that the initial data is compressed, and the space occupation is reduced through data compression, and the subsequent processing efficiency of the initial data is improved.

[0227] Corresponding to the above data compression method embodiment, the present application also provides a data compression device embodiment, Figure 7 FIG. 1 is a schematic diagram showing the structure of a data compression device provided by an embodiment of the present application. Figure 7 As shown, the device comprises:

[0228] The data acquisition module 702 is configured to acquire initial data associated with the compression task;

[0229] A comparison module 704 is configured to compare the initial data with the historical data associated with the compression task to obtain difference data;

[0230] An extraction module 706 is configured to determine a target container corresponding to the difference data, and extract container data associated with the target container from the initial data, wherein the container data includes the difference data;

[0231] The generating module 708 is configured to fill the container data into the target container according to the filling rule corresponding to the target container, and generate a compressed data packet associated with the compression task according to the filling result.

[0232] Optionally, the generation of the filling rule runs the following modules:

[0233] An acquisition set module, configured to acquire a set of period history data within a preset period associated with the compression task;

[0234] A data comparison module is configured to compare the periodic historical data at adjacent time points in the periodic historical data set to obtain change information of the periodic historical sub-data in each periodic historical data;

[0235] A statistical module, configured to perform correlation statistics on the periodic historical sub-data corresponding to the periodic historical data set based on the change information to obtain correlation statistical information;

[0236] The container generation module is configured to perform subpackaging processing on the target historical data associated with the periodic historical data set according to the associated statistical information, and generate an initial container and a filling rule corresponding to the initial container based on the subpackaging processing result.

[0237] Optionally, the generating container module includes:

[0238] A data determination submodule is configured to determine target historical data having the same data type as the periodic historical data in the periodic historical data set;

[0239] a weight calculation submodule, configured to calculate the association weights between the target historical sub-data in the target historical data according to the association statistical information;

[0240] a division submodule, configured to divide the target historical sub-data in the target historical data into at least one data packet based on the association weight;

[0241] The rule generation submodule is configured to generate an initial container corresponding to each data packet and a filling rule corresponding to the initial container according to data attribute information of the target historical sub-data in each data packet.

[0242] Optionally, the division submodule includes:

[0243] A tree set generation unit is configured to connect the data nodes corresponding to the target historical sub-data in the target historical data based on the association weight to generate at least one tree set;

[0244] An evaluation unit, configured to evaluate each tree set according to the associated statistical information to obtain evaluation data corresponding to each tree set;

[0245] a determination set unit, configured to determine, based on the evaluation data, a target tree set corresponding to the target historical data in at least one tree set;

[0246] The encapsulation unit is configured to respectively encapsulate the node sub-data corresponding to the data nodes of each connection sub-tree in the target tree set in the target historical data to generate at least one data packet.

[0247] Optionally, the division submodule is further configured to:

[0248] According to the storage information of the node sub-data corresponding to the data nodes in each connection sub-tree in the tree set, the node sub-data corresponding to each connection sub-tree are spliced ​​to generate the spliced ​​sub-data corresponding to each connection sub-tree;

[0249] An evaluation calculation is performed according to the splicing storage information of the splicing sub-data and the associated statistical information to obtain evaluation data corresponding to the tree set.

[0250] Optionally, the division submodule is further configured to:

[0251] Obtain the preset alignment length and the storage bit length of the node sub-data corresponding to each connection sub-tree;

[0252] The storage bit length and the preset alignment length are traversed, and the node sub-data corresponding to each connection sub-tree are spliced ​​in the bit domain to generate the spliced ​​sub-data corresponding to each connection sub-tree.

[0253] Optionally, the spanning tree set unit is further configured to:

[0254] Based on the association weight and the i-th node connection relationship, the data nodes corresponding to the target historical sub-data in the target historical data are connected for the i-th time to generate the i-th initial tree set;

[0255] Determine whether the number of connected subtrees in the i-th initial tree set is greater than a preset threshold;

[0256] If it is greater than, add the i-th initial tree set as an intermediate tree set to the tree set group; determine the target connection relationship corresponding to the data node in the i-th initial tree set, i is incremented by 1, use the target connection relationship as the i-th node connection relationship, and return to execute the i-th connection based on the association weight and the i-th node connection relationship, for the data nodes corresponding to the target historical sub-data in the target historical data, to generate the i-th initial tree set;

[0257] If they are equal, the intermediate tree sets in the tree set group are combined into at least one tree set.

[0258] Optionally, the generation rule submodule is further configured to:

[0259] Determining packet attribute information of the data packet according to data attribute information of the target historical sub-data in the data packet;

[0260] Creating an initial container corresponding to the data packet according to the packet attribute information;

[0261] Determine the filling position information of each target historical sub-data in the initial container according to the storage bit length and the splicing order of the target historical sub-data in the data packet;

[0262] Based on the filling position information, a filling rule corresponding to the initial container is created.

[0263] Optionally, the data compression device further includes:

[0264] Determining at least two compressed data packets associated with the compression task;

[0265] Combining at least two compressed data packets according to a preset coding sequence and coding identifiers corresponding to at least two compressed data packets to generate a combined compressed data packet;

[0266] Generating a data packet mask corresponding to the combined compressed data packet according to a preset encoding sequence and the encoding identifier;

[0267] generating a compressed data packet based on the data packet mask and the combined compressed data packet;

[0268] The compressed data package is sent to the client.

[0269] Optionally, the extraction module 706 includes:

[0270] A container set determination submodule is configured to determine a container set corresponding to the initial data according to the type information of the initial data;

[0271] A selection submodule, configured to select a target container from the container set based on data attribute information of the difference data;

[0272] The data extraction submodule is configured to extract the container data from the initial data according to the associated data information of the target container.

[0273] In summary, the data compression device provided in the embodiment of the present application obtains the initial data associated with the compression task, and compares the initial data with the historical data associated with the compression task to obtain the difference data, that is, obtains the data that has changed compared with the historical data, and then determines the target container corresponding to the difference data, and extracts the container data associated with the target container from the initial data, wherein the container data contains the difference data, and realizes the extraction of data strongly related to the difference data in the initial data through the target container corresponding to the difference data. Then, according to the filling rule corresponding to the target container, the container data is filled into the target container, and a compressed data packet associated with the compression task is generated according to the filling result, so that the extracted data is assembled to generate a compressed data packet to be processed, so that the initial data is compressed, and the space occupation is reduced through data compression, and the subsequent processing efficiency of the initial data is improved.

[0274] The above is a schematic scheme of a data compression device of the present embodiment. It should be noted that the technical scheme of the data compression device and the technical scheme of the data compression method described above belong to the same concept. For details not described in detail in the technical scheme of the data compression device, please refer to the description of the technical scheme of the data compression method described above. In addition, each component in the device embodiment should be understood as a functional module that must be established to implement each step of the program flow or each step of the method, and each functional module is not an actual functional division or separation definition. The device claim defined by such a group of functional modules should be understood as a functional module architecture that mainly implements the solution through a computer program recorded in the specification, and should not be understood as a physical device that mainly implements the solution through hardware.

[0275] Corresponding to the above data decompression method embodiment, the present application also provides a data decompression device embodiment, Figure 8 A schematic diagram of the structure of a data decompression device provided by an embodiment of the present application is shown, which is applied to a client, such as Figure 8 As shown, the device comprises:

[0276] The packet acquisition module 802 is configured to acquire a compressed data packet associated with the decompression task;

[0277] The data extraction module 804 is configured to extract the container data of the target container from the compressed data packet according to the extraction rule of the target container corresponding to the compressed data packet;

[0278] The data determination module 806 is configured to determine the historical data associated with the container data according to the data attribute information of the container data;

[0279] The data generating module 808 is configured to generate target data based on the container data and the historical data.

[0280] Optionally, the packet acquisition module 802 includes:

[0281] The receiving submodule is configured to receive the compressed data packet sent by the server;

[0282] A parsing submodule is configured to parse the compressed data packet to obtain a data packet mask and a combined compressed data packet;

[0283] The packet extraction submodule is configured to extract at least one compressed data packet from the combined compressed data packet based on the data packet mask, wherein the compressed data packet is included in the at least one compressed data packet.

[0284] In summary, the data decompression device provided in the embodiment of the present application achieves the restoration of the data in the compressed data packet by acquiring the compressed data packet associated with the decompression task; extracting the container data of the target container from the compressed data packet according to the extraction rule of the target container corresponding to the compressed data packet, and then determining the historical data associated with the container data according to the attribute information of the container data; and generating the target data based on the container data and the historical data. The target data is restored by merging the container data with the historical data.

[0285] The above is a schematic scheme of a data decompression device of the present embodiment. It should be noted that the technical scheme of the data decompression device and the technical scheme of the above-mentioned data decompression method belong to the same concept. For the details not described in detail in the technical scheme of the data decompression device, please refer to the description of the technical scheme of the above-mentioned data decompression method. In addition, each component in the device embodiment should be understood as a functional module that must be established to implement each step of the program flow or each step of the method, and each functional module is not an actual functional division or separation definition. The device claim defined by such a group of functional modules should be understood as a functional module architecture that mainly implements the solution through the computer program recorded in the specification, and should not be understood as a physical device that mainly implements the solution through hardware.

[0286] Fig. 9 The structure block diagram of a computing device 900 provided according to an embodiment of the present application is shown. The components of the computing device 900 include but are not limited to a memory 910 and a processor 920. The processor 920 is connected to the memory 910 via a bus 930, and the database 950 is used to store data.

[0287] The computing device 900 also includes an access device 940, which enables the computing device 900 to communicate via one or more networks 960. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 940 may include one or more of any type of network interface (e.g., a network interface card (NIC)) of wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0288] In one embodiment of the present application, the above components of the computing device 900 and Fig. 9 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Fig. 9 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of the present application. Those skilled in the art may add or replace other components as needed.

[0289] The computing device 900 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 900 may also be a mobile or stationary server.

[0290] The processor 920 is used to execute computer executable instructions of the above-mentioned data compression method or data decompression method.

[0291] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned data compression method or data decompression method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above-mentioned data compression method or data decompression method.

[0292] An embodiment of the present application further provides a computer-readable storage medium storing computer instructions, which are used for the above-mentioned data compression method or data decompression method when executed by a processor.

[0293] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the data compression method or the data decompression method described above belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the data compression method or the data decompression method described above.

[0294] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0295] An embodiment of the present application further provides a chip storing a computer program, which implements the steps of the above-mentioned data compression method or data decompression method when executed by the chip.

[0296] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0297] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0298] The preferred embodiments of the present application disclosed above are only used to help explain the present application. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the present application. The present application selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present application, so that those skilled in the art can understand and use the present application well. The present application is only limited by the claims and their full scope and equivalents.

Claims

1. A data compression method, characterized in that: Applied to the server, including: Get the initial data associated with the compression task; Comparing the initial data with the historical data associated with the compression task to obtain difference data; Determine a target container corresponding to the difference data, and extract container data associated with the target container from the initial data, wherein the container data includes the difference data; According to the filling rule corresponding to the target container, the container data is filled into the target container, and a compressed data packet associated with the compression task is generated according to the filling result, wherein the target container is determined from the initial container according to the difference data, and the initial container corresponding to each data packet is created according to the data attribute information of the target historical sub-data in each data packet, and the target historical sub-data in each data packet is obtained by packetizing the target historical sub-data based on the association weight between the target historical sub-data in the target historical data with the same data type of the periodic historical data in the periodic historical data set within a preset period associated with the compression task.

2. The data compression method according to claim 1, characterized in that: The generation of the filling rule includes: Comparing the periodic historical data at adjacent time points in the periodic historical data set to obtain change information of the periodic historical sub-data in each periodic historical data; Based on the change information, correlation statistics are performed on the periodic historical sub-data corresponding to the periodic historical data set to obtain correlation statistical information; The target historical data associated with the periodic historical data set is subpackaged according to the associated statistical information, and an initial container and a filling rule corresponding to the initial container are generated based on the subpackage processing result.

3. The data compression method according to claim 2, characterized in that: The step of performing subpackaging processing on the target historical data associated with the periodic historical data set according to the associated statistical information, and generating an initial container and a filling rule corresponding to the initial container based on the subpackaging processing result, includes: Calculate the association weights between the target historical sub-data in the target historical data according to the association statistical information; Based on the association weight, dividing the target historical sub-data in the target historical data into at least one data packet; According to the data attribute information of the target historical sub-data in each data packet, an initial container corresponding to each data packet and a filling rule corresponding to the initial container are generated.

4. The data compression method according to claim 3, characterized in that: The step of dividing the target historical sub-data in the target historical data into at least one data packet based on the association weight includes: Based on the association weights, data nodes corresponding to target historical sub-data in the target historical data are connected to generate at least one tree set; Evaluate each tree set according to the associated statistical information to obtain evaluation data corresponding to each tree set; Based on the evaluation data, determining a target tree set corresponding to the target historical data in at least one tree set; The node sub-data corresponding to the data nodes of each connection sub-tree in the target tree set in the target historical data are respectively encapsulated to generate at least one data packet.

5. The data compression method according to claim 4, characterized in that: Determining the evaluation data corresponding to any one of the at least one tree set includes: According to the storage information of the node sub-data corresponding to the data nodes in each connection sub-tree in the tree set, the node sub-data corresponding to each connection sub-tree are spliced ​​to generate the spliced ​​sub-data corresponding to each connection sub-tree; An evaluation calculation is performed according to the splicing storage information of the splicing sub-data and the associated statistical information to obtain evaluation data corresponding to the tree set.

6. The data compression method according to claim 5, characterized in that: The method of performing splicing processing on the node sub-data corresponding to each connection sub-tree according to the storage information of the node sub-data corresponding to the data node in each connection sub-tree in the tree set to generate the spliced ​​sub-data corresponding to each connection sub-tree includes: Obtain the preset alignment length and the storage bit length of the node sub-data corresponding to each connection sub-tree; The node sub-data corresponding to each connection sub-tree, the storage bit length and the preset alignment length are traversed, and the node sub-data corresponding to each connection sub-tree are bit-domain spliced ​​to generate spliced ​​sub-data corresponding to each connection sub-tree.

7. The data compression method according to claim 4, characterized in that: The step of connecting the data nodes corresponding to the target historical sub-data in the target historical data based on the association weight to generate at least one tree set includes: Based on the association weight and the i-th node connection relationship, the data nodes corresponding to the target historical sub-data in the target historical data are connected for the i-th time to generate the i-th initial tree set; Determine whether the number of connected subtrees in the i-th initial tree set is greater than a preset threshold; If it is greater than, add the i-th initial tree set as an intermediate tree set to the tree set group; determine the target connection relationship corresponding to the data node in the i-th initial tree set, i is incremented by 1, use the target connection relationship as the i-th node connection relationship, and return to execute the i-th connection based on the association weight and the i-th node connection relationship, for the data nodes corresponding to the target historical sub-data in the target historical data, to generate the i-th initial tree set; If they are equal, the intermediate tree sets in the tree set group are combined into at least one tree set.

8. The data compression method according to claim 3, characterized in that: The generation of any initial container and the filling rule corresponding to any initial container includes: Determining packet attribute information of the data packet according to data attribute information of the target historical sub-data in the data packet; Creating an initial container corresponding to the data packet according to the packet attribute information; Determine the filling position information of each target historical sub-data in the initial container according to the storage bit length and the splicing order of the target historical sub-data in the data packet; Based on the filling position information, a filling rule corresponding to the initial container is created.

9. The data compression method according to claim 1, characterized in that: After generating the compressed data packet associated with the compression task according to the filling result, the method further includes: Determining at least two compressed data packets associated with the compression task; Combining at least two compressed data packets according to a preset coding sequence and coding identifiers corresponding to at least two compressed data packets to generate a combined compressed data packet; Generating a data packet mask corresponding to the combined compressed data packet according to a preset encoding sequence and the encoding identifier; generating a compressed data packet based on the data packet mask and the combined compressed data packet; The compressed data package is sent to the client.

10. The data compression method according to claim 1, characterized in that: The determining the target container corresponding to the difference data and extracting the container data associated with the target container from the initial data includes: Determining, according to the type information of the initial data, a container set corresponding to the initial data; Selecting a target container from the container set based on data attribute information of the difference data; The container data is extracted from the initial data according to the associated data information of the target container.

11. A data decompression method, characterized in that: Applied to the client, including: Acquire a compressed data packet associated with the decompression task, wherein the compressed data packet is determined according to the data compression method according to claim 1; extracting container data of the target container from the compressed data packet according to an extraction rule of the target container corresponding to the compressed data packet; Determining historical data associated with the container data according to data attribute information of the container data; Target data is generated based on the container data and the historical data.

12. The data decompression method according to claim 11, characterized in that: The step of obtaining the compressed data packet associated with the decompression task includes: Receive the compressed data package sent by the server; Parsing the compressed data total packet to obtain a data packet mask and a combined compressed data compressed packet; Based on the data packet mask, at least one compressed data packet is extracted from the combined compressed data packet, wherein the compressed data packet is included in the at least one compressed data packet.

13. A data compression device, characterized in that: Applied to the server, including: A data acquisition module, configured to acquire initial data associated with the compression task; A comparison module, configured to compare the initial data with the historical data associated with the compression task to obtain difference data; an extraction module, configured to determine a target container corresponding to the difference data, and extract container data associated with the target container from the initial data, wherein the container data includes the difference data; A generation module is configured to fill the container data into the target container according to the filling rule corresponding to the target container, and generate a compressed data packet associated with the compression task according to the filling result, wherein the target container is determined from the initial container according to the difference data, and the initial container corresponding to each data packet is created according to the data attribute information of the target historical sub-data in each data packet, and the target historical sub-data in each data packet is obtained by packetizing the target historical sub-data based on the association weight between the target historical sub-data in the target historical data with the same data type of the periodic historical data in the periodic historical data set within a preset period associated with the compression task.

14. A data decompression device, characterized in that: Applied to the client, including: an acquisition module, configured to acquire a compressed data packet associated with a decompression task, wherein the compressed data packet is determined according to the data compression method according to claim 1; a data extraction module configured to extract container data of the target container from the compressed data packet according to an extraction rule of the target container corresponding to the compressed data packet; a data determination module configured to determine historical data associated with the container data according to data attribute information of the container data; The data generation module is configured to generate target data based on the container data and the historical data.

15. A computing device comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, characterized in that: When the processor executes the computer instructions, the steps of the method described in any one of claims 1-10 or 11-12 are implemented.

16. A computer-readable storage medium storing computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method described in any one of claims 1-10 or 11-12 are implemented.

17. A computer program product, characterized in that The method comprises computer instructions which, when executed by a processor, implement the steps of the method described in any one of claims 1-10 or 11-12.

Images