Data compression method and device, data decompression method and device, electronic equipment and readable storage medium

Abstract

The invention discloses a data compression method and device, a data decompression method and device, electronic equipment and a storage medium. The compression method comprises the following steps: sampling and analyzing program running data to obtain call stack text data which comprises call stack sampling texts corresponding to sampling time; constructing a coding table; according to the coding table, the stack text data is compressed, coded and called, and a coding result summary table is obtained; according to the sampling time, the byte offset of the starting byte of the coding result of the calling stack sampling text relative to the starting byte of the coding result summary table, and the byte length of the coding result of the calling stack sampling text, constructing a coding result index table; and generating a compressed file for calling the stack text data according to each table. According to the method, the byte starting position and the byte length of a coding result corresponding to sampling time in the coding process are recorded, a coding result index table is obtained, and then the coding result within any time range can be rapidly positioned and read according to the table.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data compression and decompression method, device, electronic device and computer-readable storage medium. Background Art

[0002] In software engineering, profiling refers to the process of collecting relevant information during program execution (such as the number of function calls and duration, memory allocation frequency and size, etc.). Analyzing the data collected by profiling can quickly locate the performance bottleneck of the program and provide a reference for program optimization.

[0003] Different Profile tools have different storage formats and corresponding parsing tools for the collected data. The parsing tools can convert the collected data into a readable call stack text format, and then generate visual charts (such as flame graphs) based on this. Since the sampling frequency of the Profile tool is at the millisecond granularity, the amount of call stack text data obtained by sampling for several minutes is usually large. Therefore, the call stack text data is usually directly compressed using a general compression algorithm before storage. When reading data based on data query requirements, first decompress the call stack text data, then traverse to find the required text content, and then generate a flame graph based on this.

[0004] However, in the above technology, since the call stack text is written in the order of sampling time, when it is necessary to read the sampling data within any time range, the required sampling data can only be located and read by traversing the file header in order. Therefore, random reading of the call stack text data is slow. In addition, the compression result of the call stack text data compressed by the general compression algorithm cannot support random reading. Summary of the invention

[0005] The present application provides a data compression and decompression method, device, electronic device and computer-readable storage medium to solve or at least partially solve the above problems. The details are as follows.

[0006] In a first aspect, the present application provides a data compression method, the method comprising:

[0007] Sampling program running data and parsing the sampling results to obtain call stack text data; the call stack text data includes call stack sampling text corresponding to each sampling time;

[0008] Constructing a coding table for the call stack text data;

[0009] According to the coding table, the call stack text data is compressed and encoded to obtain a coding result summary table of the call stack text data;

[0010] Constructing an encoding result index table of the call stack text data according to the sampling time, the byte offset of the encoding result starting byte of the call stack sampling text relative to the starting byte of the encoding result summary table, and the byte length of the encoding result of the call stack sampling text;

[0011] A compressed file of the call stack text data is generated according to the encoding table, the encoding result summary table and the encoding result index table.

[0012] In a second aspect, the present application provides a data decompression method, the method comprising:

[0013] Receive a data query instruction for a target call stack sampling text whose sampling time is within a target time range;

[0014] In response to the data query instruction, a coding table, a coding result summary table and a coding result index table of the call stack text data are obtained from a compressed file of the call stack text data; wherein the compressed file of the call stack text data is compressed by the data compression method described in the first aspect, and the call stack text data includes call stack sample texts corresponding to each sampling time;

[0015] Query the target byte offset and target byte length corresponding to the start and end time of the target time range from the encoding result index table; the target byte offset is the byte offset of the encoding result starting byte of the target call stack sample text relative to the starting byte of the encoding result summary table, and the target byte length is the byte length of the encoding result of the target call stack sample text;

[0016] Determining a target encoding result from the encoding result summary table according to the target byte offset and the target byte length;

[0017] According to the encoding table, the target encoding result is decoded to obtain the target call stack sample text.

[0018] In a third aspect, an embodiment of the present application further provides a data compression device, the device comprising:

[0019] A sampling and parsing module is used to sample program running data and parse the sampling results to obtain call stack text data; the call stack text data includes call stack sampling text corresponding to each sampling time;

[0020] A first construction module is used to construct a coding table for the call stack text data;

[0021] A compression encoding module, used for compressing and encoding the call stack text data according to the encoding table to obtain an encoding result summary table of the call stack text data;

[0022] A second construction module is used to construct an encoding result index table of the call stack text data according to the sampling time, the byte offset of the encoding result starting byte of the call stack sampling text relative to the starting byte of the encoding result summary table, and the byte length of the encoding result of the call stack sampling text;

[0023] A generation module is used to generate a compressed file of the call stack text data according to the encoding table, the encoding result summary table and the encoding result index table.

[0024] In a fourth aspect, an embodiment of the present application further provides a data decompression device, the device comprising:

[0025] A receiving module, used for receiving a data query instruction for a target call stack sampling text whose sampling time is within a target time range;

[0026] An acquisition module, configured to obtain, in response to the data query instruction, a coding table, a coding result summary table, and a coding result index table of the call stack text data from a compressed file of the call stack text data; wherein the compressed file of the call stack text data is compressed by the data compression method described in the first aspect, and the call stack text data includes call stack sample texts corresponding to each sampling time;

[0027] A query module, used for querying the target byte offset and target byte length corresponding to the start and end time of the target time range from the encoding result index table; the target byte offset is the byte offset of the encoding result starting byte of the target call stack sample text relative to the starting byte of the encoding result summary table, and the target byte length is the byte length of the encoding result of the target call stack sample text;

[0028] A determination module, used to determine a target encoding result from the encoding result summary table according to the target byte offset and the target byte length;

[0029] A decoding module is used to decode the target encoding result according to the encoding table to obtain the target call stack sampling text.

[0030] In a fifth aspect, an embodiment of the present application further provides an electronic device, including:

[0031] Processor; and

[0032] The memory is used to store a program. After the electronic device is powered on and the program is run by the processor, the method described in the first aspect or the second aspect is executed.

[0033] In a sixth aspect, an embodiment of the present application further provides a computer-readable storage medium storing a program, which is executed by a processor to perform the method described in the first aspect or the second aspect.

[0034] Compared with the prior art, this application has the following advantages:

[0035] In the data compression method provided in the embodiment of the present application, when encoding the call stack sampling text, the corresponding sampling time and the start and end byte positions of the encoding result in the encoding result summary table are recorded to obtain an encoding result index table organized in the sampling time sequence and the start and end positions of the encoding result bytes, and the table is placed in the final compressed file. In this way, when it is necessary to read the call stack sampling text of a certain time range, the byte range that needs to be read from the encoding result summary table can be quickly located through the encoding result index table, and then the required call stack sampling text can be restored in combination with the encoding table. By performing data compression through this data compression method, random access on the compressed file can be supported, so the entire file does not need to be decompressed when reading, which can greatly reduce the memory consumption when reading. In addition, it can also realize the rapid positioning and reading of the compressed results within any sampling time range, solving the problem that after using a general compression algorithm, it is impossible to quickly locate the data of any sampling time range in the compression result. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a flow chart of a data compression method provided in an embodiment of the present application;

[0037] Figure 2 is a flow chart of a data decompression method provided in an embodiment of the present application;

[0038] Figure 3 is a block diagram of a data compression device provided in an embodiment of the present application;

[0039] Figure 4 is a block diagram of a data decompression device provided in an embodiment of the present application;

[0040] Figure 5 It is a logical structure diagram of an electronic device for implementing data compression and / or decompression provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work belongs to the scope of protection of the present application.

[0042] The terms "a", "an", "the" and "said" are used in this specification to indicate the presence of one or more elements / components / etc.; the terms "including" and "having" are used to express an open-ended inclusion and mean that additional elements / components / etc. may exist in addition to the listed elements / components / etc.; the terms "first" and "second" etc. are used only as labels and are not intended to limit the quantity of their objects.

[0043] It should be understood that in the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" is merely a way to describe the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. "Including A, B and / or C" means including any one, any two, or any three of A, B, and C.

[0044] It should be understood that in the embodiments of the present application, "B corresponding to A", "B corresponding to A", "A corresponds to B", or "B corresponds to A" means that B is associated with A, and B can be determined according to A. Determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0045] Before describing the embodiments of the present application in detail, the prior art is first introduced and analyzed in more detail.

[0046] Before the present application was proposed, in order to solve the problem that the amount of call stack text data is usually large and not conducive to storage, a general compression algorithm is usually used to compress it before storage. Among them, common compression algorithms include DEFLATE, LZ series (LZ4, LZ77, LZ78, LZW, LZMA), Zstandard, etc. These compression algorithms are a type of lossless compression algorithm, which searches for repeated parts in the original data, records these parts through a designed data structure, namely a dictionary, and replaces the repeated parts in the original data with references to the dictionary, thereby achieving data compression. This type of algorithm is also called a "dictionary encoder."

[0047] Here, the DEFLATE algorithm is used as an example for explanation. The DEFLATE algorithm includes the following steps:

[0048] Match and replace repeated bytes with pointer references;

[0049] The byte is replaced with a shorter Huffman code, using the byte's frequency of occurrence as a weight.

[0050] In terms of data compression rate, general compression algorithms need to strike a balance between compression speed and compression ratio, and usually provide parameters for control. The step of "eliminating duplicate strings" in the DEFLATE algorithm is the part with the largest amount of calculation, so the Compression Level (from 0 to 9) parameter is provided to control the matching length and matching range of repeated bytes. The larger the Compression Level, the larger the matching length and matching range of the byte string, the higher the compression ratio, and the slower the compression speed; conversely, the faster the compression speed, the lower the compression ratio.

[0051] When the compressed call stack text data is read based on data query requirements, it is first decompressed, and then the call stack text data is traversed to search for the required text content.

[0052] In the prior art, although the method of directly compressing the call stack text data obtained after sampling using a general compression algorithm can bring about a good compression effect, it has the following disadvantages in terms of data reading:

[0053] (1) Random reading of call stack text data is slow: Since the call stack text is organized in the order of sampling time, this organization means that when you need to read the sampling results within any time range, you can only traverse the file header in sequence to locate and read the required results. This method has a time complexity of O(N), where N is the size of the call stack text data.

[0054] (2) Random reading of compression results is not supported: When compressing the call stack text data, compression algorithms such as the DEFLATE algorithm replace repeated strings within the matching range with references to the first occurrence (backward reference). Therefore, when the nth paragraph of text content needs to be read, the text content of paragraphs 1 to n-1 must be decompressed sequentially first. There is a lack of flexibility in random reading of compressed data, and it is not possible to meet the random reading requirements of the call stack text content within a certain period of time.

[0055] Based on this, the present application provides a data compression method and a data decompression method to solve or at least partially solve the above problems.

[0056] The execution subject of the data compression method and the data decompression method can be a terminal device or a server. The terminal device can be a desktop computer, a laptop computer, a tablet computer, a mobile phone, etc., or other electronic devices, which are not specifically limited in this application. The server is used to provide background services for the client of the application in the terminal device. For example, the server can be the background server of the above-mentioned application. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms, which are not specifically limited in this application.

[0057] like Figure 1 As shown, the data compression method includes the following steps S110 to S150.

[0058] Step S110: sampling the program running data and parsing the sampling results to obtain call stack text data; the call stack text data includes call stack sampling text corresponding to each sampling time.

[0059] An electronic device runs one or more programs through a central processing unit (CPU). In the data compression method provided in the present application, the program running data is first periodically sampled for the programs running on the CPU of the electronic device, and then the sampling results are parsed into a readable call stack text format to obtain the call stack text data to be compressed.

[0060] Among them, periodic sampling means collecting program running data from the CPU once at a fixed interval. Therefore, the program running data obtained by each sampling corresponds to a sampling time, and the program running data obtained by each sampling can be parsed into call stack sampling text. Each call stack sampling text also corresponds to a sampling time. The above call stack text data includes the call stack sampling text obtained by multiple sampling and parsing.

[0061] Step S120: construct a coding table for call stack text data.

[0062] In this step, a coding table of the call stack text data can be constructed according to a preset coding strategy. The coding table is used to characterize the correspondence between the original text of the call stack text data and the coding result, wherein the coding result of the call stack text data is related to the adopted coding strategy.

[0063] Optionally, the coding strategy based on which the coding table is constructed may be a coding strategy that does not require decoding any bit in [0, n-1] when decoding the nth bit, for example, may include a Huffman coding strategy, an arithmetic coding strategy, etc., wherein arithmetic coding and Huffman coding belong to entropy coding.

[0064] Step S130: compress and encode the call stack text data according to the encoding table to obtain an encoding result summary table of the call stack text data.

[0065] In this step, the call stack text data may be encoded according to the above encoding table, and compression of the call stack text data is achieved by encoding the call stack text data into shorter encoded data.

[0066] When the call stack text data is compressed and encoded, the encoding result of the call stack text data can be obtained, including the encoding result of the call stack sampling text corresponding to each sampling time. Therefore, the encoding result summary table is essentially a summary of the encoding results of multiple call stack sampling texts.

[0067] Step S140: construct an encoding result index table of the call stack text data according to the sampling time, the byte offset of the encoding result starting byte of the call stack sampling text relative to the starting byte of the encoding result summary table, and the byte length of the encoding result of the call stack sampling text.

[0068] Next, an encoding result index table may be constructed, which may be used when searching for data in the encoding result of the call stack text data.

[0069] The encoding result index table may include a field representing the sampling time, a field representing the byte offset of the encoding result starting byte of the call stack sampling text relative to the starting byte of the encoding result summary table, and a field representing the encoding result byte length of the call stack sampling text. Therefore, each row of data in the encoding result index table may represent the starting position and length of the call stack sampling text corresponding to a sampling time or sampling time interval in the encoding result. In this way, during the decompression process, the encoding result of the call stack sampling text corresponding to a certain sampling time or sampling time interval can be found through the encoding result index table, and then this part of the encoding result can be decompressed.

[0070] Step S150: Generate a compressed file of the call stack text data according to the encoding table, the encoding result summary table and the encoding result index table.

[0071] After obtaining the coding table, coding result summary table and coding result index table of the call stack text data, a compressed file including the coding table, coding result summary table and coding result index table can be generated.

[0072] In the data compression method provided in the embodiment of the present application, for the call stack text data obtained by sampling and parsing the program running data, a coding table of the call stack text data can be first constructed, wherein the call stack text data includes the call stack sampling text corresponding to each sampling time; then, according to the coding table, the call stack text data is compressed and encoded to obtain a coding result summary table; then, a byte offset including the sampling time, the starting byte of the coding result of the call stack sampling text relative to the starting byte of the coding result summary table, and the byte length of the coding result of the call stack sampling text is constructed; and then, according to the coding table, the coding result summary table and the coding result index table, a compressed file of the call stack text data is generated to realize the compression of the call stack text data.

[0073] In the data compression method, when encoding the call stack sampling text, the corresponding sampling time and the start and end byte positions of the encoding result in the encoding result summary table are recorded to obtain an encoding result index table organized according to the sampling time sequence and the start and end positions of the encoding result bytes, and the table is placed in the final compressed file. In this way, when it is necessary to read the call stack sampling text of a certain time range, the encoding result index table can be used to quickly locate the byte range that needs to be read from the encoding result summary table, and then the required call stack sampling text can be restored in combination with the encoding table. By performing data compression through the data compression method, random access on the compressed file can be supported, so the entire file does not need to be decompressed when reading, which can greatly reduce the memory consumption when reading. In addition, it can also realize the rapid positioning and reading of the compressed results within any sampling time range, solving the problem that after using a general compression algorithm, it is impossible to quickly locate the data of any sampling time range in the compression result.

[0074] In an optional implementation, in the above step S120, a coding table for the call stack text data can be constructed specifically through the following steps S121-S123.

[0075] Step S121: determining a plurality of unit data to be encoded in the call stack text data; each unit data has different data content;

[0076] Step S122: determining the encoding result corresponding to each unit data; wherein the encoding result corresponding to each unit data is unique;

[0077] Step S123: Generate a coding table including the correspondence between unit data and corresponding coding results.

[0078] In this implementation, the data content of each unit data to be encoded is different, that is, this implementation can adopt a lossless encoding strategy based on dictionary encoding to determine the encoding results of data fragments with non-repetitive content in the call stack text data, and form a coding dictionary including the correspondence between each data fragment and the corresponding coding result, that is, the above-mentioned coding table.

[0079] Among them, the encoding result corresponding to each unit data is unique, that is, the encoding strategy adopted follows the encoding uniqueness principle. The encoding uniqueness principle means that the same data (referring to unit data in the embodiment of the present application) has only one encoding result, and one encoding result only identifies the same data (referring to unit data in the embodiment of the present application).

[0080] In the embodiment of the present application, by adopting a coding strategy that follows the coding uniqueness principle, the coding results of repeated content in the call stack text data can be decoupled, so that there is no correlation between the coding results of the same content, and decompressing this part of the coding results does not require decoding the previous coding results. Therefore, for such a scheme, when decoding the nth byte, there is no need to first decode at least one byte in the [0, n-1] bytes. The coding strategy that follows the coding uniqueness principle is adopted for encoding, so that when decompressing the compressed file of the call stack text data, only the coding results within the search range can be decoded, without the need to additionally decode the coding results outside the search range. Therefore, this scheme can support random reading of compression results. This scheme is flexible in the random reading of compressed data, and can well meet the needs of random reading of call stack sampled text within a certain period of time.

[0081] In an embodiment of the present application, the call stack sampling text specifically includes a sampling time code line and a sampling data code line, wherein the sampling time code line includes the sampling time of the call stack sampling text, and the sampling data code line is the specific data obtained by sampling.

[0082] In the process of implementing this application, the inventor discovered that according to the principle of locality of computer program operation, when the CPU is running a program and reading instructions, it will focus on accessing a smaller continuous area in the memory for a period of time. Therefore, in the profile results, the sampled data code lines are also concentrated in a certain range, and a large number of code lines will be repeated at the bottom of the stack.

[0083] Based on the above findings, in an optional implementation, the above step S121 may further include the following steps S1211-S1212.

[0084] Step S1211: de-duplicate each sample data code line in the call stack text data;

[0085] Step S1212: Determine each code line of the sampled data remaining after deduplication as the first unit data.

[0086] In this embodiment, the encoding can be performed in units of code lines. According to the encoding strategy of dictionary encoding, the repeated sampled data code lines in the call stack text data are first removed, and then the remaining non-repeated sampled data code lines are determined as the first unit data to be encoded.

[0087] Taking into account the data characteristics of the call stack sampling text that a large number of repeated code lines will appear at the bottom of the stack, encoding in units of code lines can replace longer code lines with shorter encoding results compared to encoding in units of fixed-length or random-length strings, further improving the compression rate of the call stack text data, making the compressed file of the call stack text data smaller, and further reducing the storage space required for the compressed file of the call stack text data.

[0088] In summary, in this implementation, according to the characteristics of Profile data (the principle of locality of program execution), encoding is performed in units of code lines, which can make the encoding result shorter and save storage space.

[0089] In addition, the sampling time code line may specifically include a sampling time identifier (such as ##) and a sampling time. Accordingly, the above step S121 may further include the following step S1213.

[0090] Step S1213: Determine the sampling time identifier as the second unit data.

[0091] In the call stack text format, the time can be identified by an identifier. In this embodiment, the sampling time identifier can also be used as the unit data to be encoded, so that the sampling time identifier can be decompressed during decompression, so that the decompressed call stack text data also meets the readable call stack text format, ensuring the readability of the decompressed call stack text data.

[0092] In the embodiment of the present application, the sampling data code line (ie, the first unit data) and the sampling time identifier (ie, the second unit data) are both unit data to be encoded.

[0093] In addition, in an optional implementation manner, the above step S130 may specifically obtain the encoding result summary table of the call stack text data in the following manner, specifically including:

[0094] Step S131: Convert the sampling time into a sampling timestamp binary number.

[0095] First, the sampling time may be converted into a sampling timestamp, and then the sampling timestamp may be converted into a floating point number type (such as float64 type), so as to obtain a sampling timestamp binary number.

[0096] Step S132: According to the coding table, the sampling time identifier and the sampling data code line in the call stack text data are converted into corresponding coding results.

[0097] The coding table of the call stack text data stores the correspondence between each unit data in the call stack text data and the unit data coding result, wherein the unit data includes a sampling time identifier and a sampling data code line with non-repetitive content. Therefore, according to the coding table, each sampling time identifier in the call stack text data can be converted into a corresponding coding result, and each sampling data code line in the call stack text data can be converted into a corresponding coding result.

[0098] Step S133: for each call stack sample text, arrange them according to the encoding result byte length and encoding result of the sampling time identifier, the sampling timestamp binary number, and the encoding result byte length and encoding result of each sampling data code line to obtain the encoding result of the call stack sample text.

[0099] For each call stack sample text, its encoding result includes the following contents: the encoding result byte length and encoding result of the sampling time identifier, the sampling timestamp binary number, and the encoding result byte length and encoding result of each sampling data code line in the call stack sample text, wherein the encoding result byte length and encoding result of each sampling data code line are adjacent. The above contents are arranged to obtain the encoding result of the call stack sample text.

[0100] The organization format of the encoding result of the above call stack sampling text can be represented by the following Table 1.

[0101] Table 1

[0102]

[0103] Step S134: Arrange the encoding results of each call stack sample text in the order of sampling time to obtain a summary table of encoding results of the call stack text data.

[0104] In this step, the encoding results of each call stack sample text are arranged in ascending order according to the sampling time, that is, an encoding result summary table summarizing the encoding results of each call stack sample text is obtained.

[0105] In addition, in another implementation of the embodiments of the present application, optionally, step S140 can specifically implement the construction of the encoding result index table through the following steps S141-S142.

[0106] Step S141: merging the sampling times into a plurality of sampling time intervals according to a preset time interval; the sampling time interval includes at least two sampling times.

[0107] Step S142: construct an encoding result index table of call stack text data according to the sampling time interval, the byte offset of the encoding result starting byte of the first call stack sampling text within the sampling time interval relative to the starting byte of the encoding result summary table, and the byte length of the encoding result of the call stack sampling text within the sampling time interval.

[0108] In this implementation, considering that the sampling frequency of the Profile is usually in milliseconds, the data volume of the sampling results will be very large. For example, if sampling is performed once every 10 milliseconds, 99 sampling results will be generated every second. In this way, if an index item is established in the encoding result index table for each sampling time, the data volume of the encoding result index table will also be very large, which is not conducive to search. In the above case, the time complexity of searching the encoding result index table is O(N), where N is the size of the encoding result index table without merging the sampling time.

[0109] Therefore, in this embodiment, each sampling time can be first merged into multiple sampling time intervals according to a preset time interval, so that each call stack sample text within a sampling time interval is used as the basis for establishing an index item in the encoding result index table. For example, an index item can be established in the encoding result index table based on the call stack sample text within 1 second.

[0110] Then, store the sampling time interval, the byte offset of the starting byte of the encoded result of the first call stack sampling text within the sampling time interval relative to the starting byte of the encoded result summary table, and the byte length of the encoded result of the call stack sampling text within the sampling time interval as a row of data in the encoded result index table, thereby establishing an index entry. In this way, the time complexity of searching the encoded result index table can be reduced from O(N) to O(P), where P is the size of the encoded result index table in the case of merging sampling times, and P < N. Additionally, the more sampling times included in a sampling time interval, the greater the gap between P and N, that is, P << N, and correspondingly, the time complexity of searching the encoded result index table will be reduced more.

[0111] In addition, in another implementation manner of the embodiments of the present application, optionally, step S150 can specifically be implemented to generate a compressed file of call stack text data through the following steps S151 - S152.

[0112] Step S151: Serialize the encoding table, the encoded result summary table, and the encoded result index table.

[0113] In this step, an object serialization processing strategy can be adopted to organize the encoding table, the encoded result summary table, and the encoded result index table to obtain a serialized file.

[0114] Among them, the object serialization processing strategy includes serialization strategies such as msgpack, protobuf, json, etc., and the embodiments of the present application do not aim to limit it.

[0115] Step S152: Compress the serialized file obtained after serialization processing to obtain a compressed file of call stack text data.

[0116] In this step, on the serialized compression result, general compression algorithms such as snappy and gzip can be used for further compression, which can obtain a higher compression ratio and further save the storage space of the compressed file of call stack text data.

[0117] After experimentation, compared with the solution of directly using a general compression algorithm to compress call stack text data, the above - mentioned solution provided by the embodiments of the present application can increase the compression ratio by 70% - 80%. The following Table 2 shows the comparison of the results of compressing a 59MB call stack text data with a sampling frequency of 10ms for 5 minutes using protobuf as the serialization strategy and then using the snappy algorithm in step S151 (Solution A) and directly using the gzip algorithm (using the highest compression mode Compression Level = 9) (Solution B):

[0118] Table 2

[0119]

[0120] Of course, in other optional implementations, the coding table, the coding result summary table and the coding result index table may be serialized to directly obtain a compressed file of the call stack text data, that is, the serialized file is used as the final compressed file.

[0121] Corresponding to the data compression method provided in the embodiment of the present application, the embodiment of the present application also provides a data decompression method. Figure 2 As shown, the data decompression method includes the following steps S210 to S250.

[0122] Step S210: receiving a data query instruction for a target call stack sample text whose sampling time is within a target time range.

[0123] The electronic device can receive a data query instruction for querying a call stack sample text within a certain time range, and then read the call stack sample text to be queried, that is, the target call stack sample text, from a compressed file of the call stack text data through the following steps.

[0124] Step S220: In response to the data query instruction, obtain the encoding table, encoding result summary table and encoding result index table of the call stack text data from the compressed file of the call stack text data; wherein, the compressed file of the call stack text data is compressed by the data compression method as described above, and the call stack text data includes the call stack sampling text corresponding to each sampling time.

[0125] Since the compressed file of the call stack text data is generated based on the coding table, coding result summary table and coding result index table of the call stack text data, in this step, in response to the data query instruction, the coding table, coding result summary table and coding result index table can be obtained from the compressed file of the call stack text data.

[0126] Step S230: query the target byte offset and target byte length corresponding to the start and end times of the target time range from the encoding result index table; the target byte offset is the byte offset of the starting byte of the encoding result of the target call stack sampling text relative to the starting byte of the encoding result summary table, and the target byte length is the byte length of the encoding result of the target call stack sampling text.

[0127] Since the encoding result index table includes the sampling time, the byte offset of the encoding result starting byte of the call stack sampling text relative to the starting byte of the encoding result summary table, and the byte length of the encoding result of the call stack sampling text, the byte offset and byte length corresponding to the sampling time within the target time range can be determined in the encoding result index table according to the start and end time [start, end] of the target time range to be queried, that is, the target byte offset and target byte length. In this way, according to the target byte offset and target byte length, the byte range to be read can be located in the encoding result summary table.

[0128] The target byte offset may specifically include the byte offset offset1 corresponding to the start time start of the target time range, and the byte offset offset2 corresponding to the end time end of the target time range. Accordingly, the target byte length may specifically include the byte length length1 corresponding to the start time start of the target time range, and the byte length length2 corresponding to the end time end of the target time range.

[0129] Step S240: Determine the target encoding result from the encoding result summary table according to the target byte offset and the target byte length.

[0130] Among them, the byte range to be read in the encoding result summary table (that is, the byte range of the target encoding result) is [a, b], where a represents the starting byte of the byte range to be read, and b represents the ending byte of the byte range to be read. In the embodiment of the present application, a=offset1, b=offset2+length2. That is, the byte range of the target encoding result to be read in the encoding result summary table is [offset1, offset2+length2].

[0131] Step S250: Decode the target encoding result according to the encoding table to obtain the target call stack sample text.

[0132] In this step, according to the encoding table, the content of the byte range [offset1, offset2+length2] in the encoding result summary table is decoded, so that the target call stack sample text required for the query can be restored.

[0133] In the data decompression method provided in the embodiment of the present application, a coding result index table recording the byte starting position and byte length of the coding result corresponding to each sampling time in the coding process is obtained from the compressed file, and then the coding result within any sampling time range can be quickly located and read according to the coding result index table.

[0134] In an optional implementation, since the compressed file of the call stack text data can be obtained after serialization and compression processing in sequence, correspondingly, the above-mentioned step S220 can be specifically obtained through the following steps S221-S222 to obtain the encoding table, encoding result summary table and encoding result index table of the call stack text data from the compressed file of the call stack text data.

[0135] Step S221: decompressing the compressed file of the call stack text data to obtain a serialized file of the call stack text data;

[0136] Step S222: Deserialize the serialized file to obtain a coding table, a coding result summary table, and a coding result index table of the call stack text data.

[0137] In this implementation scheme, if the coding table, coding result summary table and coding result index table of the call stack text data have been serialized and compressed again after serialization, then when the compressed file is decompressed, it is first decompressed and then deserialized to obtain the coding table, coding result summary table and coding result index table.

[0138] In another optional implementation, the above step S230 can be specifically implemented by the following step S231.

[0139] Step S231: According to the start and end time of the target time range, a binary search is performed in the encoding result index table to obtain the target byte offset and target byte length corresponding to the start and end time.

[0140] Binary search is also called binary search, which is a more efficient search strategy. However, binary search requires that the linear table must adopt a sequential storage structure, and the elements in the table are arranged in order according to the keywords. In the embodiment of the present application, the encoding result index table can be arranged in the order of sampling time, so the binary search strategy can be adopted.

[0141] The principle of binary search is: first, assuming that the elements in the table are arranged in ascending order, compare the key recorded in the middle position of the table with the search key. If the two are equal, the search is successful; otherwise, use the middle position record to divide the table into two sub-tables, front and back. If the key recorded in the middle position is greater than the search key, further search the front sub-table, otherwise further search the back sub-table. Repeat the above process until a record that meets the conditions is found, making the search successful, or until the sub-table does not exist, at which time the search is unsuccessful.

[0142] Of course, in other optional embodiments, the encoding result index table may also be traversed, that is, searched sequentially, so as to find the target byte offset and target byte length corresponding to the start and end times of the target time range.

[0143] In an embodiment of the present application, the byte offset and byte length in the encoding result index table can be arranged in order of sampling time or sampling time interval, wherein the sampling time interval is formed by merging at least two sampling times according to a preset time interval. That is, in the encoding result index table, a row of index items can indicate data corresponding to one sampling time, or data corresponding to at least two sampling times.

[0144] Accordingly, the above step S231 can be specifically implemented through the following steps S2311-S2312.

[0145] Step S2311: According to the start time of the target time range or the sampling time interval to which the start time belongs, a binary search is performed in the encoding result index table to obtain the first target byte offset corresponding to the start time or the sampling time interval to which the start time belongs.

[0146] Step S2312: According to the deadline of the target time range or the sampling time interval to which the deadline belongs, a binary search is performed in the encoding result index table to obtain the second target byte offset and the second target byte length corresponding to the deadline or the sampling time interval to which the deadline belongs.

[0147] Among them, the first target byte offset offset1 indicates the starting position of the byte to be read from the encoding result summary table, and the sum of the second target byte offset offset2 and the second target byte length2 indicates the ending position of the byte to be read from the encoding result summary table.

[0148] Among them, when the index items of the encoding result index table do not incorporate the sampling time, the time complexity of traversing and searching the encoding result index table is O(N), where N is the size of the encoding result index table when the sampling time is not incorporated. In the same case, the time complexity of binary search of the encoding result index table is O(logN), and O(logN)<O(N).

[0149] In addition, when the index items in the encoding result index table are combined with the sampling time, the time complexity of traversing and searching the encoding result index table is O(P), where P is the size of the encoding result index table when the sampling time is combined. In the same case, the time complexity of binary search of the encoding result index table is O(logP), and O(logP)<O(P). Among them, O(logP)<O(logN).

[0150] In summary, the binary search method can reduce the time complexity of searching the encoding result index table.

[0151] Corresponding to the implementation method of binary search of the encoding result index table, in step S240, the target encoding result to be read can be determined from the encoding result summary table through the following steps S241-S242.

[0152] Step S241: Determine the byte search range according to the first target byte offset, the second target byte offset and the second target byte length.

[0153] Step S242: Search the encoding result summary table for a target encoding result that is within the byte search range.

[0154] The byte search range is [offset1, offset2+length2]. The encoding result within the byte range of [offset1, offset2+length2] can be found from the encoding result summary table, which is the target encoding result to be read.

[0155] In addition, when the byte offsets and byte lengths in the encoding result index table are arranged according to the sampling time interval, since each index item in the encoding result index table is composed of data corresponding to multiple sampling times, the sampling time interval to which the start and end times of the target time range belong can be first found, and these two sampling time intervals and the sampling time interval between the two cover the target time range. Therefore, the encoding result data corresponding to these two sampling time intervals and the sampling time interval between the two are decompressed to obtain multiple call stack sampling texts including the target call stack sampling text whose sampling time is within the target time range.

[0156] Therefore, in step S250, the target call stack sample text may be obtained in the following manner, including:

[0157] Step S251: Decode the target encoding result according to the encoding table to obtain the call stack sampling text within the sampling time interval that overlaps with the target time range.

[0158] Step S252: Determine the target call stack sampling text whose sampling time is within the target time range in the data obtained after decoding.

[0159] The read target encoding result is first decoded to obtain the call stack sampling texts within each sampling time interval where the target time range overlaps. However, the decoded data may include, in addition to the target call stack sampling text to be queried, call stack sampling texts whose sampling time is not within the target time range. Therefore, only the target call stack sampling texts whose sampling time is within the target time range can be extracted from the decoded data, thus obtaining the data to be queried.

[0160] In the embodiment of the present application, when the encoding result index table combines multiple sampling times as one index item:

[0161] Regarding the method of traversing and searching the encoding result index table, the time complexity of searching data in the embodiment of the present application is O(P+M), that is, O(P)+O(M). Among them, O(P) is the time complexity of traversing and searching the encoding result index table; P is the size of the encoding result index table; O(M) is the time complexity of sequentially searching for the required data in the decoded data after locating the byte search range and decoding; M is the time granularity used when constructing the encoding result index table. Taking the encoding result index table with a sampling frequency of 10ms and a construction granularity of seconds as an example, then M=1s / 10ms=100. Among them, M< <P<<N。

[0162] For the binary search encoding result index table, the time complexity of searching data in the embodiment of the present application is O(logP+M), that is, O(logP)+O(M). Among them, O(logP) is the time complexity of the binary search encoding result index table.

[0163] Among them, O(N)>O(P+M)>O(logP+M), O(N) is the time complexity of searching for data by traversing the call stack text data in the prior art; N is the size of the call stack text data.

[0164] The following is an example to illustrate the process of the above data compression and decompression method. It can be understood that the content in the following example is only for a more convenient and clear understanding of the embodiment of the present application, and does not constitute a limitation on the present application. In this example, some simple and necessary steps or data are not fully reflected, and the data involved in the example is only for the purpose of explaining the present application and does not mean that it is directly available.

[0165] This example uses the processing of three sampling results obtained when executing the following python source code pf2_loop_test.py as an example. The three call stack texts obtained after sampling and parsing are as follows:

[0166]

[0167]

[0168]

[0169] Among them, ##2023-03-0515:36:01.000, ##2023-03-0515:36:02.000 and ##2023-03-0515:36:03.000 are sampling time code lines, and ## is a sampling time identifier. The other contents in the above call stack text data are sampling data code lines.

[0170] In this example, the Huffman coding strategy is used to encode, compress and index the call stack text data obtained by Profile sampling and parsing.

[0171] The Huffman coding strategy encodes characters based on their frequency of occurrence. Therefore, we first extract and count the unit data to be encoded (including sampling time identifier and sampling data code line) in the call stack sampling text and the number of times they appear. The statistical results are as follows:

[0172]

[0173]

[0174] By constructing the corresponding Huffman tree, the following coding table can be generated:

[0175] Unit data Huffman coding (binary) ## 1 PyEval_EvalFrameEx 1000 _IO_file_write 000110 _IO_file_xsputn 100110 PyEval_EvalCodeEx 011110 pf2_loop_test.py: <module> :1 < / module> 111110 ＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝＝ 001110 / usr / lib / x86_64-linux-gnu / libc-2.28.so@0x7a75f 101110 PyEval_EvalCode 01010 PyRun_SimpleFileExFlags 11010 __libc_start_main 00010 _start 10010 __write 10000 _IO_do_write 01100 fputs 11100 PyFile_WriteString 00100 Py_Main 10100 / usr / bin / python2.7@0x120e2f 010110 PyRun_FileExFlags 110110 pf2_loop_test.py:c:4 100000 pf2_loop_test.py:b:7 000000

[0176] In this example, the original call stack text data is traversed, and each call stack sample text is replaced according to the encoding table, wherein the sampling time is converted into a timestamp and then stored using the float64 type.

[0177] Taking the first call stack sampling text as an example, the sampling time 2023-03-0515:36:01.000 is converted to the timestamp (seconds) 1678001761.000, and then stored using the float64 type (00100000010011000100011110110011000001), and the encoded byte length and encoding result are shown below.

[0178]

[0179] Organize the contents in the above table into the following format:

[0180]

[0181] The compression results of the three call stack sample texts are as follows (the bold part indicates the encoding length, the underlined part indicates the encoding content, and the italic + bold + underlined part indicates the timestamp):

[0182] Compression result of the first call stack sampling text: sampling time 2023-03-0515:36:01.000

[0183]

[0184] Repeat the same steps to get the compression results of the remaining two call stack sample texts:

[0185] Compression result of the second call stack sampling text: sampling time 2023-03-0515:36:02.000

[0186]

[0187] Compression result of the third call stack sampling text: sampling time 2023-03-0515:36:03.000

[0188]

[0189] The compression results of the above three call stack sample texts are the encoding result summary table.

[0190] According to the sampling timestamp, the starting byte offset and byte length of the corresponding call stack sampling text compression result are organized into the following encoding result index table:

[0191] Timestamp Offset starting byte offset Length byte length 1678001761 0 50 1678001762 50 54 1678001763 104 46

[0192] Among them, the starting byte offset and byte length of the timestamp within 1 second can be combined into one index item.

[0193] The above encoding table, encoding result summary table and encoding result index table are organized using object serialization. Taking protobuf as an example, the serialized data is as follows:

[0194]

[0195] In this way, the encoding table, encoding result summary table and encoding result index table are serialized into binary data through protobuf serialization.

[0196] Furthermore, the serialized files may be compressed using common compression algorithms such as snappy and gzip to obtain a higher compression rate.

[0197] At this point, a compressed file of the call stack text data is obtained through encoding compression.

[0198] Take the compressed file obtained in the above way as an example, and read the sampling result with the timestamp [start, end] = [1678001761, 1678001762] as an example:

[0199] The compressed file serialized in protobuf format is deserialized to obtain the encoding table, encoding result summary table and encoding result index table.

[0200] By performing a binary search on the encoding result index table, we obtain the compression result index information offset1=0, length1=50 of timestamp 1678001761, and the compression result index information offset2=50, length2=54 of timestamp 1678001762, and the byte search range is [offset1, offset2+length2]=[0,104].

[0201] Extract the binary codes in the range of [0:104] from the encoding result summary table, and replace the read binary codes with the corresponding original text according to the encoding table.

[0202] At this point, the original text of the call stack text data has been restored by decompression.

[0203] Corresponding to the data compression method provided in the embodiment of the present application, the embodiment of the present application also provides a data compression device 300. Figure 3 As shown, the device comprises:

[0204] The sampling and parsing module 301 is used to sample the program running data and parse the sampling results to obtain call stack text data; the call stack text data includes the call stack sampling text corresponding to each sampling time;

[0205] A first construction module 302 is used to construct a coding table for the call stack text data;

[0206] A compression encoding module 303 is used to compress and encode the call stack text data according to the encoding table to obtain an encoding result summary table of the call stack text data;

[0207] A second construction module 304 is used to construct an encoding result index table of the call stack text data according to the sampling time, the byte offset of the encoding result starting byte of the call stack sample text relative to the starting byte of the encoding result summary table, and the byte length of the encoding result of the call stack sample text;

[0208] The generation module 305 is used to generate a compressed file of the call stack text data according to the encoding table, the encoding result summary table and the encoding result index table.

[0209] Optionally, the first building module 302 includes:

[0210] A first determining unit is used to determine a plurality of unit data to be encoded in the call stack text data; each of the unit data has different data content;

[0211] A second determining unit is used to determine the encoding result corresponding to each of the unit data; wherein the encoding result corresponding to each of the unit data is unique;

[0212] A generating unit is used to generate a coding table including a correspondence between the unit data and the corresponding coding results.

[0213] Optionally, the call stack sampling text includes sampling time code lines and sampling data code lines;

[0214] The first determining unit is specifically configured to:

[0215] De-duplication of each of the sampled data code lines in the call stack text data;

[0216] Each code line of the sampled data remaining after deduplication is determined as the first unit data.

[0217] Optionally, the sampling time code line includes a sampling time identifier and a sampling time;

[0218] The first determining unit is further specifically configured to:

[0219] The sampling time identifier is determined as the second unit data.

[0220] Optionally, the compression encoding module 303 is specifically used for:

[0221] Convert the sampling time into a sampling timestamp binary number;

[0222] According to the coding table, converting the sampling time identifier and the sampling data code line in the call stack text data into corresponding coding results;

[0223] For each of the call stack sample texts, the encoding result byte length and encoding result of the sampling time identifier, the sampling timestamp binary number, and the encoding result byte length and encoding result of each of the sample data code lines are arranged to obtain the encoding result of the call stack sample text;

[0224] The encoding results of each of the call stack sample texts are arranged in the order of sampling time to obtain a summary table of encoding results of the call stack text data.

[0225] Optionally, the second building module 304 is specifically used to:

[0226] Combining the sampling times into a plurality of sampling time intervals according to a preset time interval; the sampling time interval includes at least two of the sampling times;

[0227] An encoding result index table of the call stack text data is constructed according to the sampling time interval, the byte offset of the starting byte of the encoding result of the first call stack sampling text within the sampling time interval relative to the starting byte of the encoding result summary table, and the byte length of the encoding result of the call stack sampling text within the sampling time interval.

[0228] Optionally, the generating module 305 is specifically used for:

[0229] Serializing the coding table, the coding result summary table, and the coding result index table;

[0230] The serialized file obtained after the serialization process is compressed to obtain a compressed file of the call stack text data.

[0231] Corresponding to the data decompression method provided in the embodiment of the present application, the embodiment of the present application also provides a data decompression device 400. Figure 4 As shown, the device comprises:

[0232] The receiving module 401 is used to receive a data query instruction for a target call stack sample text whose sampling time is within a target time range;

[0233] The acquisition module 402 is used to obtain the encoding table, encoding result summary table and encoding result index table of the call stack text data from the compressed file of the call stack text data in response to the data query instruction; wherein the compressed file of the call stack text data is compressed by the data compression method as described above, and the call stack text data includes the call stack sample text corresponding to each sampling time;

[0234] A query module 403 is used to query the target byte offset and target byte length corresponding to the start and end times of the target time range from the encoding result index table; the target byte offset is the byte offset of the encoding result starting byte of the target call stack sample text relative to the starting byte of the encoding result summary table, and the target byte length is the byte length of the encoding result of the target call stack sample text;

[0235] A determination module 404, configured to determine a target encoding result from the encoding result summary table according to the target byte offset and the target byte length;

[0236] The decoding module 405 is used to decode the target encoding result according to the encoding table to obtain the target call stack sample text.

[0237] Optionally, the query module 403 includes:

[0238] The query unit is used to perform a binary search in the encoding result index table according to the start and end times of the target time range to obtain the target byte offset and target byte length corresponding to the start and end times.

[0239] Optionally, the byte offsets and byte lengths in the encoding result index table are arranged in order of the sampling time or sampling time interval, and the sampling time interval is formed by merging at least two sampling times according to a preset time interval;

[0240] The query unit is specifically used for:

[0241] According to the start time of the target time range or the sampling time interval to which the start time belongs, a binary search is performed in the encoding result index table to obtain a first target byte offset corresponding to the start time or the sampling time interval to which the start time belongs;

[0242] According to the deadline of the target time range or the sampling time interval to which the deadline belongs, a binary search is performed in the encoding result index table to obtain a second target byte offset and a second target byte length corresponding to the deadline or the sampling time interval to which the deadline belongs.

[0243] Optionally, when the byte offset and the byte length in the encoding result index table are arranged according to the sampling time interval, the decoding module 405 is specifically used to:

[0244] Decoding the target encoding result according to the encoding table to obtain the call stack sampling text within the sampling time interval that overlaps with the target time range;

[0245] The target call stack sampling text whose sampling time is within the target time range is determined in the data obtained after decoding.

[0246] Optionally, the determining module 404 is specifically configured to:

[0247] Determine a byte search range according to the first target byte offset, the second target byte offset and the second target byte length;

[0248] The target encoding result within the byte search range is searched from the encoding result summary table.

[0249] Optionally, the compressed file of the call stack text data is obtained after serialization processing and compression processing in sequence;

[0250] The acquisition module 402 is specifically used for:

[0251] Decompressing the compressed file of the call stack text data to obtain a serialized file of the call stack text data;

[0252] The serialized file is deserialized to obtain a coding table, a coding result summary table and a coding result index table of the call stack text data.

[0253] Corresponding to the method provided in the embodiment of the present application, the embodiment of the present application also provides an electronic device for data compression and / or decompression. Figure 5 As shown, the electronic device includes: a processor 501; and a memory 502, which is used to store a program of a data compression and / or decompression method. After the device is powered on and the program of the data compression and / or decompression method is run by the processor, the following steps are performed:

[0254] Sampling program running data and parsing the sampling results to obtain call stack text data; the call stack text data includes call stack sampling text corresponding to each sampling time;

[0255] Constructing a coding table for the call stack text data;

[0256] According to the coding table, the call stack text data is compressed and encoded to obtain a coding result summary table of the call stack text data;

[0257] Constructing an encoding result index table of the call stack text data according to the sampling time, the byte offset of the encoding result starting byte of the call stack sampling text relative to the starting byte of the encoding result summary table, and the byte length of the encoding result of the call stack sampling text;

[0258] A compressed file of the call stack text data is generated according to the encoding table, the encoding result summary table and the encoding result index table.

[0259] or,

[0260] Receive a data query instruction for a target call stack sampling text whose sampling time is within a target time range;

[0261] In response to the data query instruction, a coding table, a coding result summary table and a coding result index table of the call stack text data are obtained from a compressed file of the call stack text data; wherein the compressed file of the call stack text data is compressed by the data compression method as described above, and the call stack text data includes call stack sample texts corresponding to each sampling time;

[0262] Query the target byte offset and target byte length corresponding to the start and end time of the target time range from the encoding result index table; the target byte offset is the byte offset of the encoding result starting byte of the target call stack sample text relative to the starting byte of the encoding result summary table, and the target byte length is the byte length of the encoding result of the target call stack sample text;

[0263] Determining a target encoding result from the encoding result summary table according to the target byte offset and the target byte length;

[0264] According to the encoding table, the target encoding result is decoded to obtain the target call stack sample text.

[0265] Corresponding to the method provided in the embodiment of the present application, the embodiment of the present application provides a computer-readable storage medium storing a program of a data compression and / or decompression method, the program is executed by a processor to perform the following steps:

[0266] Sampling program running data and parsing the sampling results to obtain call stack text data; the call stack text data includes call stack sampling text corresponding to each sampling time;

[0267] Constructing a coding table for the call stack text data;

[0268] According to the coding table, the call stack text data is compressed and encoded to obtain a coding result summary table of the call stack text data;

[0269] Constructing an encoding result index table of the call stack text data according to the sampling time, the byte offset of the encoding result starting byte of the call stack sampling text relative to the starting byte of the encoding result summary table, and the byte length of the encoding result of the call stack sampling text;

[0270] A compressed file of the call stack text data is generated according to the encoding table, the encoding result summary table and the encoding result index table.

[0271] or,

[0272] Receive a data query instruction for a target call stack sampling text whose sampling time is within a target time range;

[0273] In response to the data query instruction, a coding table, a coding result summary table and a coding result index table of the call stack text data are obtained from a compressed file of the call stack text data; wherein the compressed file of the call stack text data is compressed by the data compression method as described above, and the call stack text data includes call stack sample texts corresponding to each sampling time;

[0274] Query the target byte offset and target byte length corresponding to the start and end time of the target time range from the encoding result index table; the target byte offset is the byte offset of the encoding result starting byte of the target call stack sample text relative to the starting byte of the encoding result summary table, and the target byte length is the byte length of the encoding result of the target call stack sample text;

[0275] Determining a target encoding result from the encoding result summary table according to the target byte offset and the target byte length;

[0276] According to the encoding table, the target encoding result is decoded to obtain the target call stack sample text.

[0277] It should be noted that for the detailed description of the device, electronic device and computer-readable storage medium provided in the embodiments of the present application, reference can be made to the relevant description of the method in the embodiments of the present application, which will not be repeated here.

[0278] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0279] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0280] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0281] Although the present application is disclosed as above in the form of a preferred embodiment, it is not intended to limit the present application. Any technical personnel in this field may make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

[0282] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0283] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0284] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A data compression method, characterized in that: The method comprises: Sampling program running data and parsing the sampling results to obtain call stack text data; the call stack text data includes call stack sampling text corresponding to each sampling time; Constructing a coding table for the call stack text data; According to the coding table, the call stack text data is compressed and encoded to obtain a coding result summary table of the call stack text data; Constructing an encoding result index table of the call stack text data according to the sampling time, the byte offset of the encoding result starting byte of the call stack sampling text relative to the starting byte of the encoding result summary table, and the byte length of the encoding result of the call stack sampling text; A compressed file of the call stack text data is generated according to the encoding table, the encoding result summary table and the encoding result index table.

2. The method according to claim 1, characterized in that The step of constructing a coding table for the call stack text data includes: Determining a plurality of unit data to be encoded in the call stack text data; each of the unit data has different data content; Determine the encoding result corresponding to each of the unit data; wherein the encoding result corresponding to each of the unit data is unique; A coding table including a correspondence between the unit data and the corresponding coding results is generated.

3. The method according to claim 2, characterized in that The call stack sampling text includes sampling time code lines and sampling data code lines; The determining of a plurality of unit data to be encoded in the call stack text data comprises: De-duplication of each of the sampled data code lines in the call stack text data; Each code line of the sampled data remaining after deduplication is determined as the first unit data.

4. The method according to claim 3, characterized in that The sampling time code line includes a sampling time identifier and a sampling time; The determining of a plurality of unit data to be encoded in the call stack text data further includes: The sampling time identifier is determined as the second unit data.

5. The method according to claim 4, characterized in that The method of compressing and encoding the call stack text data according to the encoding table to obtain an encoding result summary table of the call stack text data includes: Convert the sampling time into a sampling timestamp binary number; According to the coding table, converting the sampling time identifier and the sampling data code line in the call stack text data into corresponding coding results; For each of the call stack sample texts, the encoding result byte length and encoding result of the sampling time identifier, the sampling timestamp binary number, and the encoding result byte length and encoding result of each of the sample data code lines are arranged to obtain the encoding result of the call stack sample text; The encoding results of each of the call stack sample texts are arranged in the order of sampling time to obtain a summary table of encoding results of the call stack text data.

6. The method according to claim 1, characterized in that The step of constructing the encoding result index table of the call stack text data according to the sampling time, the byte offset of the encoding result starting byte of the call stack sampling text relative to the starting byte of the encoding result summary table, and the byte length of the encoding result of the call stack sampling text comprises: Combining the sampling times into a plurality of sampling time intervals according to a preset time interval; the sampling time interval includes at least two of the sampling times; An encoding result index table of the call stack text data is constructed according to the sampling time interval, the byte offset of the starting byte of the encoding result of the first call stack sampling text within the sampling time interval relative to the starting byte of the encoding result summary table, and the byte length of the encoding result of the call stack sampling text within the sampling time interval.

7. The method according to claim 1, characterized in that The step of generating a compressed file of the call stack text data according to the coding table, the coding result summary table and the coding result index table comprises: Serializing the coding table, the coding result summary table, and the coding result index table; The serialized file obtained after the serialization process is compressed to obtain a compressed file of the call stack text data.

8. A data decompression method, characterized in that: The method comprises: Receive a data query instruction for a target call stack sampling text whose sampling time is within a target time range; In response to the data query instruction, a coding table, a coding result summary table and a coding result index table of the call stack text data are obtained from a compressed file of the call stack text data; wherein the compressed file of the call stack text data is compressed by the data compression method described in any one of claims 1 to 7, and the call stack text data includes call stack sample texts corresponding to each sampling time; Query the target byte offset and target byte length corresponding to the start and end time of the target time range from the encoding result index table; the target byte offset is the byte offset of the encoding result starting byte of the target call stack sample text relative to the starting byte of the encoding result summary table, and the target byte length is the byte length of the encoding result of the target call stack sample text; Determining a target encoding result from the encoding result summary table according to the target byte offset and the target byte length; According to the encoding table, the target encoding result is decoded to obtain the target call stack sample text.

9. The method according to claim 8, characterized in that The step of querying the target byte offset and the target byte length corresponding to the start and end times of the target time range from the encoding result index table includes: According to the start and end times of the target time range, a binary search is performed in the encoding result index table to obtain the target byte offset and target byte length corresponding to the start and end times.

10. The method according to claim 9, characterized in that The byte offsets and byte lengths in the encoding result index table are arranged in order according to the sampling time or sampling time interval, and the sampling time interval is formed by merging at least two sampling times according to a preset time interval; According to the start and end time of the target time range, a binary search is performed in the encoding result index table to obtain the target byte offset and target byte length corresponding to the start and end time, including: According to the start time of the target time range or the sampling time interval to which the start time belongs, a binary search is performed in the encoding result index table to obtain a first target byte offset corresponding to the start time or the sampling time interval to which the start time belongs; According to the deadline of the target time range or the sampling time interval to which the deadline belongs, a binary search is performed in the encoding result index table to obtain a second target byte offset and a second target byte length corresponding to the deadline or the sampling time interval to which the deadline belongs.

11. The method according to claim 10, characterized in that In a case where the byte offset and the byte length in the encoding result index table are arranged according to the sampling time interval, decoding the target encoding result according to the encoding table to obtain the target call stack sampling text includes: Decoding the target encoding result according to the encoding table to obtain the call stack sampling text within the sampling time interval that overlaps with the target time range; The target call stack sampling text whose sampling time is within the target time range is determined in the data obtained after decoding.

12. The method according to claim 10, characterized in that Determining the target encoding result from the encoding result summary table according to the target byte offset and the target byte length includes: Determine a byte search range according to the first target byte offset, the second target byte offset and the second target byte length; The target encoding result within the byte search range is searched from the encoding result summary table.

13. The method according to claim 8, characterized in that The compressed file of the call stack text data is obtained after serialization processing and compression processing in sequence; The method of obtaining the encoding table, encoding result summary table and encoding result index table of the call stack text data from the compressed file of the call stack text data includes: Decompressing the compressed file of the call stack text data to obtain a serialized file of the call stack text data; The serialized file is deserialized to obtain a coding table, a coding result summary table and a coding result index table of the call stack text data.

14. A data compression device, characterized in that: The device comprises: A sampling and parsing module is used to sample program running data and parse the sampling results to obtain call stack text data; the call stack text data includes call stack sampling text corresponding to each sampling time; A first construction module is used to construct a coding table for the call stack text data; A compression encoding module, used for compressing and encoding the call stack text data according to the encoding table to obtain an encoding result summary table of the call stack text data; A second construction module is used to construct an encoding result index table of the call stack text data according to the sampling time, the byte offset of the encoding result starting byte of the call stack sampling text relative to the starting byte of the encoding result summary table, and the byte length of the encoding result of the call stack sampling text; A generation module is used to generate a compressed file of the call stack text data according to the encoding table, the encoding result summary table and the encoding result index table.

15. A data decompression device, characterized in that: The device comprises: A receiving module, used for receiving a data query instruction for a target call stack sampling text whose sampling time is within a target time range; An acquisition module, configured to obtain, in response to the data query instruction, a coding table, a coding result summary table and a coding result index table of the call stack text data from a compressed file of the call stack text data; wherein the compressed file of the call stack text data is compressed by the data compression method described in any one of claims 1 to 7, and the call stack text data includes call stack sample texts corresponding to each sampling time; A query module, used for querying the target byte offset and target byte length corresponding to the start and end time of the target time range from the encoding result index table; the target byte offset is the byte offset of the encoding result starting byte of the target call stack sample text relative to the starting byte of the encoding result summary table, and the target byte length is the byte length of the encoding result of the target call stack sample text; A determination module, used to determine a target encoding result from the encoding result summary table according to the target byte offset and the target byte length; A decoding module is used to decode the target encoding result according to the encoding table to obtain the target call stack sampling text.

16. An electronic device, characterized in that: include: processor; as well as The memory is used to store a program. After the electronic device is powered on and the program is run by the processor, the method according to any one of claims 1 to 13 is executed.

17. A computer-readable storage medium, characterized in that: A program is stored, and the program is run by a processor to execute the method according to any one of claims 1 to 13.

Images