A byte array data parsing method, device, equipment and medium

By receiving and converting byte arrays in the ClickHouse database and directly parsing them into Java bitmap data structures, the problem of low query efficiency is solved, and a simplified data parsing process and resource saving is achieved.

CN118916362BActive Publication Date: 2024-12-13HANGZHOU YUNSHEN TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411402686.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-10-09
Publication Date
2024-12-13
Estimated Expiration
2044-10-09

AI Technical Summary

Technical Problem

When querying 64-bit bitmap data in the ClickHouse database, multiple encoding, decoding and segmentation operations are required, resulting in high computing resource occupation and low data query efficiency.

Method used

By receiving the target byte array, writing to the target byte buffer object, and performing small-endian to large-endian conversion according to the data type, it is directly parsed into the bitmap data structure in Java, avoiding multiple encoding and decoding operations.

Benefits of technology

The data query process is simplified, the consumption of computing resources is reduced, and the efficiency of data query is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118916362B_ABST
    Figure CN118916362B_ABST
Patent Text Reader

Abstract

The present application relates to the technical field of data query, and in particular to a data parsing method, device, equipment and medium for a byte array, the method comprising: receiving a target byte array from a given column-based database management system; the target byte array comprising a target data type, a target byte length, the number of target high-order elements and an intermediate byte array, writing the target byte array into a pre-constructed target byte buffer object, and parsing the target byte array in the target byte buffer object into a bitmap data structure according to the target data type and other contents corresponding to the target byte array in the target byte buffer object; the present invention can simplify the data parsing process in the data query process, save computing resources, and thus improve data query efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data query technology, and in particular to a data parsing method, device, equipment and medium for a byte array. Background Art

[0002] In the field of data processing and query, the 64-bit bitmap data structure is widely used in Java to process large-scale data sets as an efficient bitmap data structure. In order to facilitate data transmission and storage, the 64-bit bitmap data structure is often serialized into a byte array and stored in a column storage database (CK, ClickHouse). In the data query scenario of CK, the serialized byte array in CK needs to be parsed into a 64-bit bitmap object in Java to obtain the original data being queried.

[0003] In the prior art, when querying data in CK, it is necessary to read the corresponding byte array and provide the byte array to the Java backend system. In this process, it is necessary to use the String database to perform secondary encoding and decoding on the byte array, including operations such as remainder and segmentation, and then convert the decoded data into a 64-bit bitmap object to achieve the purpose of query. This query method occupies more computing resources and the parsing process is complicated, resulting in low data query efficiency. Summary of the invention

[0004] The object of the present invention is to provide a data parsing method, device, equipment and medium for a byte array, which can simplify the data parsing process in the data query process, save computing resources, and thus improve data query efficiency.

[0005] According to a first aspect of the present invention, there is provided a method for parsing data of a byte array, comprising the following steps:

[0006] A target byte array is received from a given column-based database management system; the target byte array includes a target data type, a target byte length, a target number of high-order elements, and an intermediate byte array; the intermediate byte array refers to any set of serialized data stored in the given column-based database management system.

[0007] The target byte array is written into a pre-constructed target byte buffer object in little endian order, and when the target data type is a first preset data type, a target byte length and the number of target high-order elements are obtained from the target byte buffer object.

[0008] An intermediate byte array is read from the target byte buffer object, and the read intermediate byte array is stored in a pre-constructed empty byte array; the length of the empty byte array corresponds to the target byte length.

[0009] The number of target high-order elements and the data in the empty byte array are stored in a preset new byte buffer object, so that the intermediate byte array stored in the empty byte array is converted into a byte array arranged in big-endian order through the new byte buffer object.

[0010] The content of the new byte buffer object is parsed to obtain a bitmap data structure.

[0011] Furthermore, the first preset data type refers to the data type of a target byte array in which the number of target high-order elements is not 0.

[0012] Furthermore, the method further comprises the following steps:

[0013] After the target byte array is written into the pre-constructed target byte buffer object in little endian order, the data of the target byte buffer object is sequentially read to obtain the first byte corresponding to the target byte buffer object.

[0014] When the first byte corresponding to the target byte buffer object is a first preset byte, it is determined that the target data type corresponding to the target byte array is the first preset data type.

[0015] Furthermore, the method further comprises the following steps:

[0016] When the target data type is a second preset data type, a number of valid bytes are obtained from the target byte buffer object; the valid bytes refer to byte arrays that are arranged in order and comply with data reading rules in the intermediate byte array in the target byte buffer object; the second preset data type refers to the data type of the target byte array whose number of target high-order elements is 0.

[0017] The valid bytes are read in sequence to obtain a plurality of long integer data.

[0018] The plurality of long integer data are sequentially stored in a pre-constructed bitmap data structure to implement parsing of the target byte array into a bitmap data structure; the initial state of the pre-constructed bitmap data structure is empty.

[0019] Further, the empty byte array is constructed by the following steps:

[0020] When the target byte length obtained from the target byte buffer object is the original byte length corresponding to the intermediate byte array obtained from the target byte buffer object, an empty byte array with a byte length equal to the target byte length is constructed.

[0021] When the target byte length obtained from the target byte buffer object is the byte length of the intermediate byte array obtained from the target byte buffer object after compression, the original byte length corresponding to the target byte length obtained from the target byte buffer object is calculated, and an empty byte array with a byte length equal to the original byte length is constructed.

[0022] Furthermore, when the first byte corresponding to the target byte buffer object is not the first preset byte and the second preset byte, the target byte array is parsed into an empty bitmap data structure.

[0023] According to a second aspect of the present invention, a byte array data parsing device is provided, the device comprising:

[0024] A receiving module is used to receive a target byte array from a given column-based database management system; the target byte array includes a target data type, a target byte length, the number of target high-order elements and an intermediate byte array; the intermediate byte array refers to any set of serialized data stored in the given column-based database management system.

[0025] The first acquisition module is used to write the target byte array into a pre-constructed target byte buffer object in little endian order, and when the target data type is a first preset data type, obtain a target byte length and the number of target high-order elements from the target byte buffer object.

[0026] A storage module is used to read an intermediate byte array from the target byte buffer object and store the read intermediate byte array into a pre-constructed empty byte array; the length of the empty byte array corresponds to the target byte length.

[0027] The first storage module is used to store the number of target high-order elements and the data in the empty byte array into a preset new byte buffer object, so as to convert the intermediate byte array stored in the empty byte array into a byte array arranged in big-endian order through the new byte buffer object.

[0028] The first parsing module is used to parse the content of the new byte buffer object to obtain a bitmap data structure.

[0029] Furthermore, the first preset data type refers to the data type of a target byte array in which the number of target high-order elements is not 0.

[0030] Furthermore, the first acquisition module also includes:

[0031] The reading module is used to sequentially read the data of the target byte buffer object after the target byte array is written into the pre-built target byte buffer object in little endian order, and obtain the first byte corresponding to the target byte buffer object.

[0032] The determination module is used to determine that the target data type corresponding to the target byte array is the first preset data type when the first byte corresponding to the target byte buffer object is the first preset byte.

[0033] Furthermore, the device further includes a second parsing module, and the second parsing module includes:

[0034] The second acquisition module is used to obtain a number of valid bytes from the target byte buffer object when the target data type is a second preset data type; the valid bytes refer to byte arrays that are arranged in order and comply with data reading rules in the intermediate byte array in the target byte buffer object.

[0035] The third acquisition module is used to read the plurality of valid bytes in sequence to obtain a plurality of long integer data.

[0036] The second storage module is used to store the plurality of long integer data in sequence into a pre-constructed bitmap data structure to realize parsing the target byte array into a bitmap data structure; the initial state of the pre-constructed bitmap data structure is empty.

[0037] Furthermore, the device further includes a first building module, and the first building module includes:

[0038] The second construction module is used to construct an empty byte array with a byte length equal to the target byte length when the target byte length obtained from the target byte buffer object is the original byte length corresponding to the intermediate byte array obtained from the target byte buffer object.

[0039] The third construction module is used to calculate the original byte length corresponding to the target byte length obtained from the target byte buffer object when the target byte length obtained from the target byte buffer object is the byte length of the intermediate byte array obtained from the target byte buffer object after compression processing, and construct an empty byte array with a byte length equal to the original byte length.

[0040] Furthermore, when the first byte corresponding to the byte buffer object is not the first preset byte and the second preset byte, the target byte array is parsed into an empty bitmap data structure.

[0041] According to a third aspect of the present invention, there is provided an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned byte array data parsing method when executing the computer program.

[0042] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium storing a computer program, wherein the computer program implements the above-mentioned byte array data parsing method when executed by a processor.

[0043] Compared with the prior art, the present invention has at least the following beneficial effects:

[0044] The present invention first receives a target byte array from a given column-based database management system, wherein the target byte array includes a target data type, a target byte length, the number of target high-order elements and an intermediate byte array. Since the data in the column-based database management system exists in the form of a byte array arranged in little endian order, it is necessary to convert it into an original bitmap data structure, write the target byte array into a target byte buffer object first, and then parse the target byte array into a bitmap data structure according to the target data type obtained from the target byte buffer object using a parsing method corresponding to the target data type. The present invention directly parses the serialized byte array in the column-based database management system into data of a bitmap data structure in Java through one conversion, and simultaneously solves the problem of inconsistency between the big endian data in the Java language and the little endian data algorithm in the CK, and directly parses the data from the byte array level without multiple encoding and decoding operations, effectively reducing the calculation process, and simplifying the data parsing process in the query process, thereby improving the data query efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0046] Figure 1 A flowchart of a method for parsing byte array data provided in Embodiment 1 of the present invention;

[0047] Figure 2 A flowchart of the steps of determining the target data type provided in the first embodiment of the present invention;

[0048] Figure 3 A flowchart of the steps of constructing an empty byte array provided in the first embodiment of the present invention;

[0049] Figure 4 Another flow chart of the byte array data parsing method provided in the first embodiment of the present invention;

[0050] Figure 5 A schematic diagram of the structure of a byte array data parsing device provided in Embodiment 2 of the present invention;

[0051] Figure 6 A partial structural diagram of a first acquisition module 200 provided in Embodiment 2 of the present invention;

[0052] Figure 7 A schematic diagram of the structure of a first building block provided in Embodiment 2 of the present invention;

[0053] Figure 8 A schematic diagram of the structure of a second analysis module provided in Embodiment 2 of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0055] Embodiment 1

[0056] like Figure 1 As shown, this embodiment provides a data parsing method for a byte array, comprising the following steps:

[0057] S100, receiving a target byte array from a given column-based database management system; the target byte array includes a target data type, a target byte length, a target number of high-order elements, and an intermediate byte array; the intermediate byte array refers to any set of serialized data stored in a given column-based database management system; it can be understood that: the target byte array is an array received including any intermediate byte array and some information of the intermediate byte array itself.

[0058] Specifically, the given column-based database management system refers to a system in which stored data exists in the form of byte arrays; in this embodiment, the given column-based database management system is the ClickHouse database.

[0059] S200, write the target byte array into a pre-constructed target byte buffer object in little endian order, and when the target data type is a first preset data type, obtain the target byte length and the number of target high-order elements from the target byte buffer object; it can be understood that: the target byte array arranged in little endian order still exists in little endian form after being written into the target byte buffer, and the target byte buffer includes the target data type, target byte length, the number of target high-order elements and the intermediate byte array in the received target byte array.

[0060] Specifically, the first preset data type refers to the data type of a target byte array in which the number of target high-order elements is not 0.

[0061] Furthermore, the first byte of the target byte array received from a given column-based database management system is used to represent the target data type. By representing the target data type with the first byte of the byte array, when the target byte array is received and read, the data type corresponding to the target byte array can be read first, and then the corresponding parsing method can be quickly selected to parse the target byte array.

[0062] When the first byte of the received target byte array is the first preset byte, it represents that the target data type is the first preset data type; when the first byte of the received target byte array is the second preset byte, it represents that the target data type is the second preset data type; technicians in this field set the first preset byte and the second preset byte according to actual needs, for example, setting the first preset byte to 1 and setting the second preset byte to 0.

[0063] Specifically, the target byte length refers to the byte length corresponding to the middle byte array in the target byte array.

[0064] Specifically, the target high-order element refers to an element in a high-order bucket obtained according to a high-order and low-order allocation rule of a bitmap data structure.

[0065] For better understanding, further explanation is given as follows. In the bitmap data structure, the 64-bit integer range is divided into high and low parts. The high part is usually used to represent a large range of integer values. The high part refers to 33-64 bits, and the low part refers to 1-32 bits. The number of target high-order elements refers to the number of elements in the high-order part. Since this number of elements can provide further information about the 64-bit bitmap data structure, such as the coverage of the data range and the distribution of the data, it plays a very important role in subsequent operations.

[0066] Furthermore, the target byte buffer object refers to a buffer that can store big-endian data and little-endian data. In a specific implementation, a byte buffer is usually used to perform data format conversion, for example, converting big-endian data to little-endian data, or converting little-endian data to big-endian data. In the present embodiment, the target byte buffer adopts a ByteBuffer buffer. This target byte buffer is only used for temporary storage of arrays in the present embodiment and is not used for data sorting type conversion.

[0067] As mentioned above, since the target byte array contains multiple array information, when querying data, only the intermediate byte array in the target byte array needs to be obtained, that is, the original byte array is converted into a new byte array. The target byte buffer object can more conveniently perform operations such as reading and identifying data on the byte array. That is, the target byte buffer object provides a more friendly way to operate the byte array, thereby ensuring the accuracy of the array conversion.

[0068] In a specific embodiment, Figure 2 As shown, the method further comprises the following steps:

[0069] S201, after the target byte array is written into a pre-constructed target byte buffer object in little endian order, sequentially read the data of the target byte buffer object to obtain the first byte corresponding to the target byte buffer object.

[0070] S202: When the first byte corresponding to the target byte buffer object is a first preset byte, determine that the target data type corresponding to the target byte array is the first preset data type.

[0071] As described above, by reading the first byte corresponding to the target byte buffer object, the target data type corresponding to the target byte array can be obtained first, so that different data parsing methods can be adopted for the target byte array according to different data types to maximize the parsing effect, thereby improving the efficiency of data query.

[0072] In a special case, when the first byte corresponding to the target byte buffer object is not the first preset byte and the second preset byte, the target byte array is parsed into an empty bitmap data structure.

[0073] S300, reading an intermediate byte array from the target byte buffer object, and storing the read intermediate byte array into a pre-constructed empty byte array.

[0074] Specifically, the length of the empty byte array corresponds to the target byte length.

[0075] In a specific embodiment, Figure 3As shown, the empty byte array is constructed by the following steps:

[0076] S301, when the target byte length obtained from the target byte buffer object is the original byte length corresponding to the intermediate byte array obtained from the target byte buffer object, construct an empty byte array with a byte length equal to the target byte length.

[0077] S302, when the target byte length obtained from the target byte buffer object is the byte length of the intermediate byte array obtained from the target byte buffer object after compression processing, calculate the original byte length corresponding to the target byte length obtained from the target byte buffer object, and construct an empty byte array with a byte length equal to the original byte length.

[0078] S400, storing the number of target high-order elements and the data in the empty byte array in a preset new byte buffer object, so as to convert the intermediate byte array stored in the empty byte array into a byte array arranged in big-endian order through the new byte buffer object.

[0079] Specifically, the big-endian and little-endian conversion method in the Java language is used to convert the little-endian data in the new byte buffer object into big-endian data to obtain a corresponding 64-bit bitmap data structure.

[0080] S500: Parse the content of the new byte buffer object to obtain a bitmap data structure.

[0081] As described above, when processing the byte array of the first preset data type, the target byte array is directly parsed at the byte array level, and the serialized byte array can be parsed into data of a bitmap data structure in Java through a little-endian to big-endian conversion. There is no need to perform multiple encoding and decoding operations, which reduces the object conversion and calculation process, thereby simplifying the data parsing process in the query process and improving data query efficiency.

[0082] In another embodiment, Figure 4 As shown, the method further comprises the following steps:

[0083] S001, when the target data type is a second preset data type, obtaining a number of valid bytes from the target byte buffer object.

[0084] Specifically, the valid bytes refer to the byte arrays in the intermediate byte array in the target byte buffer object that are arranged in order and comply with the data reading rules; the second preset data type refers to the data type of the target byte array whose number of target high-order elements is 0; for example, in this embodiment, the data stored in CK is long integer data, each data occupies 8 bytes, then the number of valid bytes is n times 8, n is the number of data elements, and n times 8 valid bytes are extracted as the obtained number of valid bytes.

[0085] S002, read the several valid bytes in sequence to obtain several long integer data; it can be understood that: read every eight bytes, each time read, read out a long integer data, and finally obtain n data after parsing.

[0086] S003, storing the plurality of long integer data in sequence into a pre-constructed bitmap data structure, so as to parse the target byte array into a bitmap data structure; an initial state of the pre-constructed bitmap data structure is empty.

[0087] As described above, for the byte array of the second preset data type, when reading data, no conversion of the byte buffer object is involved, and the read data is directly stored in a pre-built bitmap data structure, which makes the data parsing method simpler and is conducive to improving the efficiency of data query.

[0088] The present embodiment provides a data parsing method for a byte array. First, a target byte array is received from a given column-based database management system. The target byte array includes a target data type, a target byte length, a target number of high-order elements, and an intermediate byte array. Since the data in the column-based database management system exists in the form of a byte array arranged in little endian order, it is necessary to convert it into an original bitmap data structure. The target byte array is first written into a target byte buffer object. Then, according to the target data type obtained from the target byte buffer object, the target byte array is parsed into a bitmap data structure using a parsing method corresponding to the target data type. The present invention directly parses the serialized byte array in the column-based database management system into data of a bitmap data structure in Java through one conversion, and at the same time solves the problem of inconsistency between the big-endian data in the Java language and the little-endian data algorithm in the CK. The data is parsed directly from the byte array level without multiple encoding and decoding operations, which effectively reduces the calculation process and simplifies the data parsing process in the query process, thereby improving data query efficiency.

[0089] In another specific implementation scenario, the data of the bitmap data structure needs to be stored in a column-based database management system. Therefore, the present application also provides a method for storing data of the bitmap data structure, comprising the following steps:

[0090] P100, when the number of elements of the received bitmap data structure is greater than 32, the bitmap data structure is marked as a first preset data type, and the number of target high-order elements is obtained from the bitmap data structure; it can be understood that: the first preset data type refers to a target byte array whose number of target high-order elements is not 0, that is, when the number of elements of the bitmap data structure is not greater than 32, the number of target high-order elements is 0.

[0091] Specifically, the target high-order element refers to an element in a high-order bucket obtained according to the high-order and low-order allocation rules of the bitmap data structure.

[0092] For better understanding, further explanation is given as follows. In the bitmap data structure, the 64-bit integer range is divided into high and low parts. The high part is usually used to represent a large range of integer values. The high part refers to 33-64 bits, and the low part refers to 1-32 bits. The number of target high-order elements refers to the number of elements in the high-order part. Since this number of elements can provide further information about the 64-bit bitmap data structure, such as the coverage of the data range and the distribution of the data, it plays a very important role in subsequent data restoration operations.

[0093] P200, perform binary processing on the data of the bitmap data structure to obtain an initial byte array and an initial byte length corresponding to the bitmap data structure.

[0094] Specifically, the P200 step includes the following steps:

[0095] P201, converting the data of the bitmap data structure into binary byte data, and determining the binary byte data as an initial byte array corresponding to the bitmap data structure.

[0096] P202, obtain the byte length of the initial byte array, and determine the byte length of the initial byte array as the initial byte length corresponding to the bitmap data structure.

[0097] As mentioned above, by performing binary processing on the data of the bitmap data structure, it is converted into a byte array, thereby realizing the transformation of the data arrangement structure. Since the column-based database management system has the effect of efficiently storing data in the form of byte arrays, after the above-mentioned data object conversion, the structure of the data is kept consistent with the data structure stored in the column-based database management system, completing the preliminary processing of the data, which is conducive to realizing the rapid storage of large-scale data sets in the column-based database management system.

[0098] P300, converting the initial sequence type corresponding to the initial byte array into a target sequence type, and obtaining a target byte array; specifically comprising the following steps:

[0099] P301, obtain an initial sequence type corresponding to the initial byte array; the initial sequence type corresponding to the initial byte array is a big-endian byte sequence type.

[0100] P302, convert the initial byte array from a big-endian byte sequence type to a little-endian byte sequence type, and obtain a target byte array.

[0101] Specifically, a byte buffer object is used to convert the initial byte array from a big-endian byte sequence type to a little-endian byte sequence type, wherein the target byte buffer object refers to a buffer capable of storing big-endian data and little-endian data. In a specific implementation, a byte buffer is generally used for data format conversion, for example, converting big-endian data into little-endian data, or converting little-endian data into big-endian data; in this embodiment, the target byte buffer uses a ByteBuffer buffer.

[0102] As mentioned above, since the bitmap data structures in Java are all data arranged in big-endian order, and CK supports data arranged in little-endian order, the big-endian byte sequence type is converted into the little-endian byte sequence type through the byte buffer, so that the original bitmap data structure is converted into a data sequence that CK can read normally, which is conducive to fast data transmission and efficient storage.

[0103] P400, based on the initial byte length, perform byte compression processing on the initial byte array to obtain a target byte length corresponding to the compressed initial byte array.

[0104] Specifically, the P400 step also includes the following steps:

[0105] P401, encode the initial byte array corresponding to the bitmap data structure by using variable-length byte encoding to obtain a compressed encoding sequence.

[0106] Preferably, the variable-length byte encoding adopts varint encoding; varint encoding is a method of performing variable-length byte encoding on positive integers. For example, when an int-type integer occupies 4 bytes, if the value of the integer is less than 256, it can be stored in one byte of space, wasting 3 bytes of space. Therefore, using varint encoding can effectively save byte space of data, especially for integers with larger values, the compression effect is more obvious.

[0107] P402, obtain the first byte length corresponding to the compressed coding sequence; it can be understood that: the first byte length refers to the byte length of the coding sequence after compression.

[0108] P403, based on the variable-length byte encoding adopted, calculate the second byte length corresponding to the initial byte length; it can be understood that: the second byte length refers to the byte length that should be obtained when the variable-length byte encoding encodes the initial byte array. For example, when an int integer occupies 4 bytes, if the value of the integer is less than 256, the corresponding second byte length is 1 byte.

[0109] P404, when the first byte length is consistent with the second byte length, the first byte length is determined as the target byte length, otherwise, a prompt verification message is sent to the target user.

[0110] As described above, by compressing the initial byte array corresponding to the bitmap data structure, the storage space occupied by the byte data can be effectively reduced, thereby reducing the impact on data transmission and computing resources, which is conducive to quickly storing data in a column-based database management system while saving storage resources for column-based database management.

[0111] P500, splicing the first preset data type, target byte length, target number of high-order elements and target byte array into a new byte array according to a preset splicing order, so as to store the new byte array in a given column-based database management system.

[0112] Specifically, the preset splicing order refers to the order of sequentially splicing the first preset data type, the target byte length, the number of target high-order elements, and the target byte array.

[0113] As described above, by splicing multiple information into a new byte array, it is beneficial to achieve accurate restoration of data when querying data. In addition, the first preset data type is placed in the first byte of the target byte array, so that the data type of the target byte data is obtained first when reading data, so that different data restoration methods are used according to different data types to maximize data parsing efficiency.

[0114] Preferably, in another embodiment, the method further comprises the following steps:

[0115] P001, when the number of elements of the bitmap data structure is not greater than 32, obtain the occupied byte length of the bitmap data structure, and mark the bitmap data structure as a second preset data type; it can be understood that: the second preset data type refers to a target byte array whose number of target high-order elements is 0, that is, when the number of elements of the bitmap data structure is greater than 32, the number of target high-order elements is greater than 0.

[0116] P002, perform byte compression processing on the occupied byte length to obtain the key byte length.

[0117] Specifically, the manner of performing byte compression processing on the occupied byte length is consistent with the manner of performing byte compression processing on the initial byte array, which will not be described in detail here.

[0118] P003, converting the data of the bitmap data structure into data of a little-endian byte sequence type.

[0119] P004, concatenate the data of the second preset data type, key byte length and little endian byte sequence type into a new byte array, so as to store the new byte array in a given column-based database management system.

[0120] As mentioned above, when the bitmap data structure of a small data set is stored in a column-based database management system, there is no need to perform binary processing and serialized byte array conversion because the parsing will not fail due to the large amount of data when the data is restored. Instead, the data in the bitmap data structure can be directly read and written into a preset empty byte array in little-endian order, which reduces the data processing flow and facilitates the rapid storage of data.

[0121] In another specific implementation scenario, after the data is stored in a column-based database management system, it is necessary to query the required data from the column-based database management system according to various query requirements of the business. Therefore, the present application also provides a data type conversion method, comprising the following steps:

[0122] Z100, generating a target ANTLR4 expression according to a data query request; the data query request is a request including a number of query condition tags, the target ANTLR4 expression includes a number of intermediate expressions and a number of logical operation operators, and the intermediate expressions correspond to the query condition tags one by one.

[0123] Specifically, the query condition tags include APP category tags, location tags and device category tags; wherein the APP category tags include but are not limited to high-risk tags, low-risk tags, financial management tags, and fraud-related tags; the device category tags include but are not limited to simulator categories, device brands, and device models.

[0124] Specifically, the location tag may be one of a permanent residence, a home location, a workplace, and a latest location.

[0125] Specifically, the logical operation operator is any one of intersection “^”, union “#”, and difference “%”.

[0126] Z200, converting several intermediate expressions in the target ANTLR4 expression into several intermediate SQL fragments respectively, and each intermediate SQL fragment corresponds to an intermediate SQL tag.

[0127] Specifically, the SP00 step includes the following steps:

[0128] Z201, parse the target ANTLR4 expression to obtain several intermediate expressions; each of the intermediate expressions corresponds to at least one query condition sub-tag; each of the query condition tags includes at least one query condition sub-tag; it can be understood that: when the query condition tag is composed of several query condition sub-tags, the intermediate expression corresponding to the query condition tag is composed of sub-expressions corresponding to several query condition sub-tags.

[0129] For better explanation, an example is given below: when searching for a certain location and a certain device brand that includes a low-risk label APP and a financial label APP, "a certain location" and "a certain device brand" are each a query condition label, "low-risk label APP and financial label APP" is a query condition label, and "low-risk label APP" and "financial label APP" are both query condition sub-labels. Each query condition label corresponds to an intermediate expression, and each query condition sub-label corresponds to a sub-expression.

[0130] Z202, construct several temporary virtual tables; the temporary virtual tables correspond one-to-one to the intermediate expressions.

[0131] Specifically, a temporary virtual table is defined and constructed through the with clause in the SQL language.

[0132] For example: for the target ANTLR4 expression ((c0)^(c1#c2#c3#c4#c5)^(c6fa0#c6fa1))%((f0#f1)^(f2#f3)), the correspondence between the names of the generated temporary virtual tables and the intermediate expressions is: with_tmp_table_1 corresponds to c0, with_tmp_table_2 corresponds to c1#c2#c3#c4#c5, with_tmp_table_3 corresponds to c6fa0#c6fa1, with_tmp_table_4 corresponds to f0#f1, and with_tmp_table_5 corresponds to f2#f3.

[0133] Z203, store several intermediate expressions into several temporary virtual tables respectively, generate several intermediate SQL fragments, and determine the name of the temporary virtual table corresponding to each intermediate SQL fragment as the intermediate SQL label corresponding to the intermediate SQL fragment itself; it can be understood as: determining the table name corresponding to the with clause as the intermediate SQL label.

[0134] As described above, when processing complex expressions, several intermediate expressions after parsing are temporarily saved by building a temporary virtual table, and parsed in memory as a temporary memory table, which can improve the execution efficiency and readability of SQL statements, and is conducive to subsequent complex data calculations. Moreover, the temporary virtual table is only valid in the current query and can be destroyed after use, which is conducive to the rational use of memory resources.

[0135] Z300, according to the operation order of the logical operation operator corresponding to each intermediate expression in the target ANTLR4 expression, a plurality of intermediate SQL tags and a plurality of logical operation operators are recombined in a suffix expression manner to obtain a tag combination result.

[0136] Specifically, the Z300 step includes the following steps:

[0137] Z301, parse the target ANTLR4 expression to obtain the distribution order of several intermediate expressions and the operation order of several logical operation operators.

[0138] Z302, according to the distribution order of the several intermediate expressions, reorder the several intermediate SQL tags and the several logical operation operators to obtain a preprocessed SQL statement; the preprocessed SQL statement is a statement composed of prefix expressions; it can be understood as: sorting the several intermediate SQL tags according to the distribution order of the several intermediate expressions, and using the sorting result as the preprocessed SQL statement.

[0139] Z303, according to the operation order of the plurality of logical operation operators, convert the preprocessing SQL statement into an intermediate SQL statement combined with a suffix expression, and use the intermediate SQL statement as a label combination result.

[0140] For ease of understanding, an example is given below:

[0141] For example, in the target ANTLR4 expression mentioned above, the intermediate expressions are: (c0), (c1#c2#c3#c4#c5), (c6fa0#c6fa1), (f0#f1), (f2#f3); the operation order is: first perform the union operation in each intermediate expression, then calculate the intersection of the first three expressions and the intersection of the last two expressions in turn, and finally perform the difference operation on the two sets; after converting the target ANTLR4 expression into a suffix expression, the expression obtained is (c0) (c1#c2#c3#c4#c5) ^(c6fa0#c6fa1) ^ (f0#f1) (f2#f3) ^ %, that is, the intermediate SQL statement obtained is with_tmp_table_1with_tmp_table_2 ^ with_tmp_table_3 ^ with_tmp_table_4 with_tmp_table_5 ^ %.

[0142] As mentioned above, by converting the preprocessed SQL statement in the form of a prefix expression into an intermediate SQL statement in the form of a suffix expression, the computer does not need to consider the issue of operation priority during reading and calculation. It only needs to obtain the elements in sequence and then perform calculations. This is very convenient for computer data processing and is conducive to improving the efficiency of data query.

[0143] Z400, performs target operation function conversion on each logical operation operator in the label combination result, and obtains the data conversion result according to the combination order of the intermediate SQL labels in the label combination result; it can be understood as: combining the result after the target operation function conversion with several sorted intermediate SQL labels to obtain the data conversion result.

[0144] Specifically, the target operation function refers to the operation function corresponding to the logical operation operator in the SQL language environment; it can be understood that: each logical operation operator corresponds to a bitmap operation function of CK; for example, the operation function corresponding to the intersection is bitmapAnd, the operation function corresponding to the union is groupBitmapOrState or bitmapOr, and the operation function corresponding to the difference is bitmapAndNot.

[0145] Z500, generating a target SQL statement according to the plurality of intermediate SQL fragments and the data conversion result, so as to complete the data type conversion from the ANTLR4 expression to the SQL statement.

[0146] Specifically, the Z500 step includes the following steps:

[0147] Z501, obtain the intermediate sql fragment corresponding to each intermediate sql tag in the data conversion result; it can be understood that: the intermediate sql fragment corresponding to the intermediate sql tag refers to the sql statement in the temporary virtual table corresponding to the intermediate sql tag.

[0148] Z502, adding the intermediate sql fragments corresponding to each intermediate sql tag to the corresponding positions of the intermediate sql tag itself in the data conversion result, and deleting the intermediate sql tag itself to generate a target sql statement; it can be understood that: the target byte array is the byte array queried according to the converted target sql statement.

[0149] As mentioned above, by replacing the corresponding intermediate sql tags with several intermediate sql fragments and adding them to the data conversion results, it is shown that the target ANTLR4 expression has completed the complete conversion to the sql statement. Through this data type conversion method, any combination of any number of business query conditions can be converted from a complex expression to a bitmap calculation sql statement that can be executed by CK, thereby realizing fast and efficient query and processing of data, so as to meet multiple business query requirements at the same time.

[0150] Embodiment 2

[0151] like Figure 5 As shown, this embodiment 2 provides a byte array data parsing device, including:

[0152] The receiving module 100 is used to receive a target byte array from a given column-based database management system; the target byte array includes a target data type, a target byte length, a target number of high-order elements and an intermediate byte array; the intermediate byte array refers to any set of serialized data stored in a given column-based database management system; it can be understood that: the target byte array is an array received including any intermediate byte array and some information of the intermediate byte array itself.

[0153] Specifically, the given column-based database management system refers to a system in which stored data exists in the form of byte arrays; in this embodiment, the given column-based database management system is the ClickHouse database.

[0154] The first acquisition module 200 is used to write the target byte array into a pre-constructed target byte buffer object in little endian order, and when the target data type is a first preset data type, obtain the target byte length and the number of target high-order elements from the target byte buffer object; it can be understood that: the target byte array arranged in little endian order still exists in little endian form after being written into the target byte buffer, and the target byte buffer includes the target data type, target byte length, the number of target high-order elements and the intermediate byte array in the received target byte array.

[0155] Specifically, the first preset data type refers to the data type of a target byte array in which the number of target high-order elements is not 0.

[0156] Furthermore, the first byte of the target byte array received from a given column-based database management system is used to represent the target data type. By representing the target data type with the first byte of the byte array, when the target byte array is received and read, the data type corresponding to the target byte array can be read first, and then the corresponding parsing method can be quickly selected to parse the target byte array.

[0157] When the first byte of the received target byte array is the first preset byte, it represents that the target data type is the first preset data type; when the first byte of the received target byte array is the second preset byte, it represents that the target data type is the second preset data type; technicians in this field set the first preset byte and the second preset byte according to actual needs, for example, setting the first preset byte to 1 and setting the second preset byte to 0.

[0158] Specifically, the target byte length refers to the byte length corresponding to the middle byte array in the target byte array.

[0159] Specifically, the target high-order element refers to an element in a high-order bucket obtained according to a high-order and low-order allocation rule of a bitmap data structure.

[0160] For better understanding, further explanation is given as follows. In the bitmap data structure, the 64-bit integer range is divided into high and low parts. The high part is usually used to represent a large range of integer values. The high part refers to 33-64 bits, and the low part refers to 1-32 bits. The number of target high-order elements refers to the number of elements in the high-order part. Since this number of elements can provide further information about the 64-bit bitmap data structure, such as the coverage of the data range and the distribution of the data, it plays a very important role in subsequent operations.

[0161] Furthermore, the target byte buffer object refers to a buffer that can store big-endian data and little-endian data. In a specific implementation, a byte buffer is usually used to perform data format conversion, for example, converting big-endian data to little-endian data, or converting little-endian data to big-endian data. In the present embodiment, the target byte buffer adopts a ByteBuffer buffer. This target byte buffer is only used for temporary storage of arrays in the present embodiment and is not used for data sorting type conversion.

[0162] As mentioned above, since the target byte array contains multiple array information, when querying data, only the intermediate byte array in the target byte array needs to be obtained, that is, the original byte array is converted into a new byte array. The target byte buffer object can more conveniently perform operations such as reading and identifying data on the byte array. That is, the target byte buffer object provides a more friendly way to operate the byte array, thereby ensuring the accuracy of the array conversion.

[0163] In a specific embodiment, Figure 6 As shown, the first acquisition module also includes:

[0164] The reading module 201 is used to sequentially read the data of the target byte buffer object after the target byte array is written into the pre-built target byte buffer object in little endian order, and obtain the first byte corresponding to the target byte buffer object.

[0165] The determination module 202 is configured to determine that the target data type corresponding to the target byte array is the first preset data type when the first byte corresponding to the target byte buffer object is the first preset byte.

[0166] As described above, by reading the first byte corresponding to the target byte buffer object, the target data type corresponding to the target byte array can be obtained first, so that different data parsing methods can be adopted for the target byte array according to different data types to maximize the parsing effect, thereby improving the efficiency of data query.

[0167] In a special case, when the first byte corresponding to the target byte buffer object is not the first preset byte and the second preset byte, the target byte array is parsed into an empty bitmap data structure.

[0168] The storage module 300 is used to read the intermediate byte array from the target byte buffer object and store the read intermediate byte array into a pre-constructed empty byte array.

[0169] Specifically, the length of the empty byte array corresponds to the target byte length.

[0170] In a specific embodiment, Figure 7 As shown, the device further includes a first building module, and the first building module includes:

[0171] The second construction module 301 is used to construct an empty byte array with a byte length equal to the target byte length when the target byte length obtained from the target byte buffer object is the original byte length corresponding to the intermediate byte array obtained from the target byte buffer object.

[0172] The third construction module 302 is used to calculate the original byte length corresponding to the target byte length obtained from the target byte buffer object when the target byte length obtained from the target byte buffer object is the byte length of the intermediate byte array obtained from the target byte buffer object after compression processing, and to construct an empty byte array with a byte length equal to the original byte length.

[0173] The first storage module 400 is used to store the number of target high-order elements and the data in the empty byte array into a preset new byte buffer object, so as to convert the intermediate byte array stored in the empty byte array into a big-endian byte array through the new byte buffer object.

[0174] Specifically, the big-endian and little-endian conversion method in the Java language is used to convert the little-endian data in the new byte buffer object into big-endian data to obtain a corresponding 64-bit bitmap data structure.

[0175] The first parsing module 500 is used to parse the content of the new byte buffer object to obtain a bitmap data structure.

[0176] As described above, when processing the byte array of the first preset data type, the target byte array is directly parsed at the byte array level, and the serialized byte array can be parsed into data of a bitmap data structure in Java through a little-endian to big-endian conversion. There is no need to perform multiple encoding and decoding operations, which reduces the object conversion and calculation process, thereby simplifying the data parsing process in the query process and improving data query efficiency.

[0177] In another embodiment, Figure 8 As shown, the device further includes a second parsing module, and the second parsing module includes:

[0178] The second acquisition module 001 is used to acquire a number of valid bytes from the target byte buffer object when the target data type is a second preset data type.

[0179] Specifically, the valid bytes refer to the byte arrays in the intermediate byte array in the target byte buffer object that are arranged in order and comply with the data reading rules; the second preset data type refers to the data type of the target byte array whose number of target high-order elements is 0; for example, in this embodiment, the data stored in CK is long integer data, each data occupies 8 bytes, then the number of valid bytes is n times 8, n is the number of data elements, and n times 8 valid bytes are extracted as the obtained number of valid bytes.

[0180] The third acquisition module 002 is used to read the several valid bytes in sequence to obtain several long integer data; it can be understood that: reading is performed every eight bytes, and each time a long integer data is read, n data are finally obtained after parsing.

[0181] The second storage module 003 is used to store the plurality of long integer data in sequence into a pre-constructed bitmap data structure, so as to parse the target byte array into a bitmap data structure; the initial state of the pre-constructed bitmap data structure is empty.

[0182] As described above, for the byte array of the second preset data type, when reading data, no conversion of the byte buffer object is involved, and the read data is directly stored in a pre-built bitmap data structure, which makes the data parsing method simpler and is conducive to improving the efficiency of data query.

[0183] The present embodiment provides a data parsing device for a byte array. When the device is running, the device first receives a target byte array from a given column-based database management system. The target byte array includes a target data type, a target byte length, a target number of high-order elements, and an intermediate byte array. Since the data in the column-based database management system exists in the form of a byte array arranged in little endian order, it is necessary to convert it into an original bitmap data structure. The target byte array is first written into a target byte buffer object, and then, according to the target data type obtained from the target byte buffer object, the target byte array is parsed into a bitmap data structure using a parsing method corresponding to the target data type. The present invention directly parses the serialized byte array in the column-based database management system into data of a bitmap data structure in Java through one conversion, and at the same time solves the problem of inconsistency between the big-endian data in the Java language and the little-endian data algorithm in the CK, and directly parses the data at the byte array level without multiple encoding and decoding operations, effectively reducing the calculation process, and simplifying the data parsing process in the query process, thereby improving data query efficiency.

[0184] Embodiment 3

[0185] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented:

[0186] A target byte array is received from a given column-based database management system; the target byte array includes a target data type, a target byte length, a target number of high-order elements, and an intermediate byte array; the intermediate byte array refers to any set of serialized data stored in the given column-based database management system.

[0187] The target byte array is written into a pre-constructed target byte buffer object in little endian order, and when the target data type is a first preset data type, a target byte length and the number of target high-order elements are obtained from the target byte buffer object.

[0188] An intermediate byte array is read from the target byte buffer object, and the read intermediate byte array is stored in a pre-constructed empty byte array; the length of the empty byte array corresponds to the target byte length.

[0189] The number of target high-order elements and the data in the empty byte array are stored in a preset new byte buffer object, so that the intermediate byte array stored in the empty byte array is converted into a byte array arranged in big-endian order through the new byte buffer object.

[0190] The content of the new byte buffer object is parsed to obtain a bitmap data structure.

[0191] Embodiment 4

[0192] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0193] A target byte array is received from a given column-based database management system; the target byte array includes a target data type, a target byte length, a target number of high-order elements, and an intermediate byte array; the intermediate byte array refers to any set of serialized data stored in the given column-based database management system.

[0194] The target byte array is written into a pre-constructed target byte buffer object in little endian order, and when the target data type is a first preset data type, a target byte length and the number of target high-order elements are obtained from the target byte buffer object.

[0195] An intermediate byte array is read from the target byte buffer object, and the read intermediate byte array is stored in a pre-constructed empty byte array; the length of the empty byte array corresponds to the target byte length.

[0196] The number of target high-order elements and the data in the empty byte array are stored in a preset new byte buffer object, so that the intermediate byte array stored in the empty byte array is converted into a byte array arranged in big-endian order through the new byte buffer object.

[0197] The content of the new byte buffer object is parsed to obtain a bitmap data structure.

[0198] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0199] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0200] Although some specific embodiments of the present invention have been described in detail by way of example, it will be appreciated by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present invention. It will also be appreciated by those skilled in the art that various modifications may be made to the embodiments without departing from the scope and spirit of the present invention. The scope of the present invention is defined by the appended claims.

Claims

1. A data parsing method for a byte array, characterized in that: The following steps are involved: Receive a target byte array from a given column-based database management system; the target byte array includes a target data type, a target byte length, a target number of high-order elements, and an intermediate byte array; the intermediate byte array refers to any set of serialized data stored in the given column-based database management system; Writing the target byte array into a pre-constructed target byte buffer object in little endian order, and when the target data type is a first preset data type, obtaining a target byte length and the number of target high-order elements from the target byte buffer object; Reading an intermediate byte array from the target byte buffer object, and storing the read intermediate byte array into a pre-constructed empty byte array; the length of the empty byte array corresponds to the target byte length; The number of target high-order elements and the data in the empty byte array are stored in a preset new byte buffer object, so as to convert the intermediate byte array stored in the empty byte array into a byte array arranged in big-endian order through the new byte buffer object; The content of the new byte buffer object is parsed to obtain a bitmap data structure.

2. The data parsing method of byte array according to claim 1, characterized in that: The first preset data type refers to the data type of a target byte array in which the number of target high-order elements is not 0.

3. The data parsing method of byte array according to claim 2, characterized in that: The method further comprises the steps of: After the target byte array is written into the pre-constructed target byte buffer object in little endian order, the data of the target byte buffer object is sequentially read to obtain the first byte corresponding to the target byte buffer object; When the first byte corresponding to the target byte buffer object is a first preset byte, it is determined that the target data type corresponding to the target byte array is the first preset data type.

4. The data parsing method of byte array according to claim 2, characterized in that: The method further comprises the following steps: When the target data type is a second preset data type, a number of valid bytes are obtained from the target byte buffer object; the valid bytes refer to byte arrays in the middle byte array in the target byte buffer object that are arranged in order and comply with data reading rules; the second preset data type refers to the data type of the target byte array whose number of target high-order elements is 0; The valid bytes are read in sequence to obtain a plurality of long integer data; The plurality of long integer data are sequentially stored in a pre-constructed bitmap data structure to implement parsing of the target byte array into a bitmap data structure; the initial state of the pre-constructed bitmap data structure is empty.

5. The data parsing method of byte array according to claim 1, characterized in that: The empty byte array is constructed by the following steps: When the target byte length obtained from the target byte buffer object is the original byte length corresponding to the intermediate byte array obtained from the target byte buffer object, construct an empty byte array with a byte length equal to the target byte length; When the target byte length obtained from the target byte buffer object is the byte length of the intermediate byte array obtained from the target byte buffer object after compression, the original byte length corresponding to the target byte length obtained from the target byte buffer object is calculated, and an empty byte array with a byte length equal to the original byte length is constructed.

6. The data parsing method of byte array according to claim 3, characterized in that: When the first byte corresponding to the target byte buffer object is not the first preset byte and the second preset byte, the target byte array is parsed into an empty bitmap data structure.

7. A byte array data parsing device, characterized in that: The device comprises: A receiving module, used for receiving a target byte array from a given column-based database management system; the target byte array includes a target data type, a target byte length, a target number of high-order elements, and an intermediate byte array; the intermediate byte array refers to any set of serialized data stored in the given column-based database management system; A first acquisition module is used to write the target byte array into a pre-constructed target byte buffer object in little endian order, and when the target data type is a first preset data type, obtain a target byte length and the number of target high-order elements from the target byte buffer object; A storage module, used for reading an intermediate byte array from the target byte buffer object, and storing the read intermediate byte array into a pre-constructed empty byte array; the length of the empty byte array corresponds to the target byte length; A first storage module is used to store the number of target high-order elements and the data in the empty byte array into a preset new byte buffer object, so as to convert the intermediate byte array stored in the empty byte array into a byte array arranged in big-endian order through the new byte buffer object; The first parsing module is used to parse the content of the new byte buffer object to obtain a bitmap data structure.

8. The byte array data parsing device according to claim 7, characterized in that: The first preset data type refers to the data type of a target byte array in which the number of target high-order elements is not 0.

9. The byte array data parsing device according to claim 8, characterized in that: The first acquisition module also includes: A reading module, configured to sequentially read the data of the target byte buffer object after the target byte array is written into the pre-built target byte buffer object in little endian order, and obtain the first byte corresponding to the target byte buffer object; The determination module is used to determine that the target data type corresponding to the target byte array is the first preset data type when the first byte corresponding to the target byte buffer object is the first preset byte.

10. The byte array data parsing device according to claim 8, characterized in that: The device further includes a second parsing module, wherein the second parsing module includes: a second acquisition module, for acquiring a number of valid bytes from the target byte buffer object when the target data type is a second preset data type; the valid bytes refer to byte arrays in the intermediate byte array in the target byte buffer object that are arranged in order and comply with data reading rules; the second preset data type refers to the data type of the target byte array whose number of target high-order elements is 0; A third acquisition module is used to read the plurality of valid bytes in sequence to obtain a plurality of long integer data; The second storage module is used to store the plurality of long integer data in sequence into a pre-constructed bitmap data structure to realize parsing the target byte array into a bitmap data structure; the initial state of the pre-constructed bitmap data structure is empty.

11. The byte array data parsing device according to claim 7, characterized in that: The device further comprises a first building block, wherein the first building block comprises: A second construction module is used to construct an empty byte array whose byte length is the target byte length when the target byte length obtained from the target byte buffer object is the original byte length corresponding to the intermediate byte array obtained from the target byte buffer object; The third construction module is used to calculate the original byte length corresponding to the target byte length obtained from the target byte buffer object when the target byte length obtained from the target byte buffer object is the byte length of the intermediate byte array obtained from the target byte buffer object after compression processing, and construct an empty byte array with a byte length equal to the original byte length.

12. The byte array data parsing device according to claim 9, characterized in that: When the first byte corresponding to the byte buffer object is not the first preset byte and the second preset byte, the target byte array is parsed into an empty bitmap data structure.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for parsing byte array data as described in any one of claims 1 to 6 is implemented.

14. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for parsing byte array data as described in any one of claims 1 to 6 is implemented.

Citation Information

Patent Citations

  • A communication text data transmission method based on a TCP protocol

    CN109818930A

  • Byte order significant bit conversion method and device, equipment and medium

    CN117240842A