Data storage method, device, equipment and medium of bitmap data structure

Abstract

The present application relates to the technical field of data storage, and in particular to a data storage method, device, equipment and medium for a bitmap data structure. The method comprises: when the number of elements of a received bitmap data structure is greater than 32, marking the bitmap data structure as a first preset data type, obtaining the number of target high-order elements from the bitmap data structure, obtaining an initial byte array and an initial byte length after data processing of the bitmap data structure, obtaining a target byte array of a conversion sequence type according to the initial byte array, obtaining a target byte length after compressing the initial byte length, splicing the first preset data type, the target byte length, the number of target high-order elements and the target byte array into a new byte array for storage; the present invention can effectively reduce the data processing flow in the data storage process and reduce the utilization of computing resources, thereby realizing efficient and persistent storage of large-scale data sets.

Description

Technical Field

[0001] The present invention relates to the field of data storage technology, and in particular to a data storage method, device, equipment and medium for a bitmap data structure. Background Art

[0002] In the field of data storage, bitmap data structures are widely used in the processing of data sets, especially for efficient data filtering, that is, data collision operations in data sets. However, traditional 32-bit bitmap data structures have some limitations, and their bit number is not enough to represent large-scale data sets. For example, in the system used to maintain device serial numbers, there are currently more than 20 billion serial numbers stored, but technically the 32-bit bitmap data structure can only store 2 to the power of 32 minus 1, which is about 4.2 billion. The traditional method is to divide serial numbers greater than 4.2 billion or greater than 400 million by 4 and calculate the remainder, and then store them in segments. The calculation process is relatively redundant and complicated, and the storage of the entire data also occupies a large space during the calculation process, which is not conducive to the storage of large-scale data sets.

[0003] Column-based storage databases (CK, ClickHouse) are well-known for their high performance and low cost. Research has found that the underlying logic of CK supports 64-bit bitmap data structures, but there are currently no driver packages or related technical solutions on the market that implement data storage of 64-bit bitmap data structures on CK. Therefore, there is an urgent need for an efficient storage method that combines bitmap data structures with CK and is suitable for storing large-scale data sets. Summary of the invention

[0004] The purpose of the present invention is to provide a data storage method, device, equipment and medium for a bitmap data structure, which can convert data of a bitmap data structure into a byte array, effectively reduce the data processing flow during data storage and reduce the utilization of computing resources, thereby achieving efficient and persistent storage of large-scale data sets.

[0005] According to a first aspect of the present invention, a method for storing data of a bitmap data structure is provided, comprising the following steps:

[0006] When the number of elements of the received bitmap data structure is greater than 32, the bitmap data structure is marked as a first preset data type, and the number of target high-order elements is obtained from the bitmap data structure.

[0007] Binary processing is performed on the data of the bitmap data structure to obtain an initial byte array and an initial byte length corresponding to the bitmap data structure.

[0008] The initial sequence type corresponding to the initial byte array is converted into a target sequence type to obtain a target byte array; the converting the initial sequence type corresponding to the initial byte array into a target sequence type to obtain a target byte array comprises the following steps: obtaining the initial sequence type corresponding to the initial byte array; the initial sequence type corresponding to the initial byte array is a big endian byte sequence type; converting the initial byte array from a big endian byte sequence type to a little endian byte sequence type to obtain a target byte array.

[0009] Based on the initial byte length, byte compression processing is performed on the initial byte array to obtain a target byte length corresponding to the compressed initial byte array.

[0010] The first preset data type, target byte length, target number of high-order elements and target byte array are spliced ​​into a new byte array according to a preset splicing order, so as to store the new byte array in a given column-based database management system.

[0011] Furthermore, the target high-order element refers to an element in a high-order bucket obtained according to a high-order and low-order allocation rule of the bitmap data structure.

[0012] Furthermore, the binary processing of the data of the bitmap data structure to obtain the initial byte array and initial byte length corresponding to the bitmap data structure includes the following steps:

[0013] The data of the bitmap data structure is converted into binary byte data, and the binary byte data is determined as an initial byte array corresponding to the bitmap data structure.

[0014] The byte length of the initial byte array is obtained, and the byte length of the initial byte array is determined as the initial byte length corresponding to the bitmap data structure.

[0015] Further, the target byte length is obtained through the following steps:

[0016] The initial byte array corresponding to the bitmap data structure is encoded in a variable-length byte encoding manner to obtain a compressed encoding sequence.

[0017] Obtain a first byte length corresponding to the compressed encoding sequence.

[0018] Based on the adopted variable-length byte encoding, a second byte length corresponding to the initial byte length is calculated.

[0019] When the first byte length is consistent with the second byte length, the first byte length is determined as the target byte length; otherwise, a prompt verification message is sent to the target user.

[0020] Furthermore, the method further comprises the following steps:

[0021] When the number of elements of the bitmap data structure is not greater than 32, the occupied byte length of the bitmap data structure is obtained, and the bitmap data structure is marked as a second preset data type.

[0022] The occupied byte length is subjected to byte compression processing to obtain the key byte length.

[0023] The data of the bitmap data structure is converted into data of a little endian byte sequence type.

[0024] The data of the second preset data type, the key byte length and the little endian byte sequence type are concatenated into a new byte array, so as to store the new byte array in a given column-based database management system.

[0025] According to a second aspect of the present invention, there is provided a data storage device of a bitmap data structure, the device comprising:

[0026] The first acquisition module is used to mark the received bitmap data structure as a first preset data type when the number of elements in the received bitmap data structure is greater than 32, and to acquire the number of target high-order elements from the bitmap data structure.

[0027] The first processing module is used to perform binary processing on the data of the bitmap data structure to obtain an initial byte array and an initial byte length corresponding to the bitmap data structure.

[0028] The first conversion module is used to convert the initial sequence type corresponding to the initial byte array into a target sequence type to obtain a target byte array; the first conversion module is also used to obtain the initial sequence type corresponding to the initial byte array; the initial sequence type corresponding to the initial byte array is a big-endian byte sequence type; the initial byte array is converted from the big-endian byte sequence type to the little-endian byte sequence type to obtain a target byte array.

[0029] The first compression module is used to perform byte compression processing on the initial byte array based on the initial byte length, and obtain a target byte length corresponding to the compressed initial byte array.

[0030] The first splicing module is used to splice the first preset data type, target byte length, target number of high-order elements and target byte array into a new byte array according to a preset splicing order, so as to store the new byte array in a given column-based database management system.

[0031] Furthermore, the target high-order element refers to an element in a high-order bucket obtained according to a high-order and low-order allocation rule of the bitmap data structure.

[0032] Furthermore, the first processing module includes:

[0033] The first determining module is used to convert the data of the bitmap data structure into binary byte data, and determine the binary byte data as an initial byte array corresponding to the bitmap data structure.

[0034] The second determining module is used to obtain the byte length of the initial byte array, and determine the byte length of the initial byte array as the initial byte length corresponding to the bitmap data structure.

[0035] Furthermore, the first compression module includes:

[0036] The encoding module is used to encode the initial byte array corresponding to the bitmap data structure by using a variable-length byte encoding method to obtain a compressed encoding sequence.

[0037] The second acquisition module is used to acquire a first byte length corresponding to the compressed coding sequence.

[0038] A calculation module is used to calculate a second byte length corresponding to the initial byte length based on the adopted variable-length byte encoding.

[0039] The third determination module is used to determine the first byte length as the target byte length when the first byte length is consistent with the second byte length, and otherwise send a prompt verification message to the target user.

[0040] Furthermore, the device further includes a second processing module, and the second processing module includes:

[0041] A marking module is used to obtain the occupied byte length of the bitmap data structure when the number of elements of the bitmap data structure is not greater than 32, and mark the bitmap data structure as a second preset data type.

[0042] The second compression module is used to perform byte compression processing on the occupied byte length to obtain the key byte length.

[0043] The second conversion module is used to convert the data of the bitmap data structure into data of a little endian byte sequence type.

[0044] The second splicing module is used to splice the data of the second preset data type, key byte length and little endian byte sequence type into a new byte array to store the new byte array in a given column database management system.

[0045] According to a third aspect of the present invention, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned data storage method of the bitmap data structure when executing the computer program.

[0046] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium storing a computer program, wherein the computer program implements the above-mentioned data storage method of the bitmap data structure when executed by a processor.

[0047] Compared with the prior art, the present invention has at least the following beneficial effects:

[0048] The data storage method of the bitmap data structure described in the present invention performs binary processing on the data of the bitmap data structure when the number of elements of the received bitmap data structure is greater than 32, obtains the initial byte array and initial byte length corresponding to the bitmap data structure, converts the initial sequence type corresponding to the initial byte array into a target sequence type, obtains the target byte array, and through the transformation of the data structure and the order adjustment of the sequence, the data of the bitmap data structure can be quickly stored in a column-based database management system after processing, effectively reducing the data processing flow and reducing the utilization of computing resources during data storage, and then performs byte compression processing on the initial byte array, which is beneficial to reducing data storage resources and saving storage space of the column-based database management system, and finally splices the obtained first preset data type, target byte length, number of target high-order elements and target byte array into a new byte array, and stores the new byte array in a given column-based database management system, thereby realizing the combination of the bitmap data structure and the column-based database management system, thereby realizing efficient and persistent storage of large-scale data sets. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0050] Figure 1 A flowchart of a data storage method of a bitmap data structure provided in Embodiment 1 of the present invention;

[0051] Figure 2 A flowchart of step S200 provided in the first embodiment of the present invention;

[0052] Figure 3A flowchart of step S400 provided in the first embodiment of the present invention;

[0053] Figure 4 Another flow chart of the data storage method of the bitmap data structure provided in the first embodiment of the present invention;

[0054] Figure 5 A schematic diagram of the structure of a data storage device for a bitmap data structure provided in Embodiment 2 of the present invention;

[0055] Figure 6 A schematic diagram of the structure of the first processing module 200 provided in the second embodiment of the present invention;

[0056] Figure 7 A schematic diagram of the structure of a first compression module 400 provided in Embodiment 2 of the present invention;

[0057] Figure 8 This is a schematic diagram of the structure of the second processing module provided in Embodiment 2 of the present invention. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0059] Embodiment 1

[0060] like Figure 1 As shown, this embodiment 1 provides a data storage method of a bitmap data structure, including the following steps:

[0061] S100, when the number of elements of the received bitmap data structure is greater than 32, the bitmap data structure is marked as a first preset data type, and the number of target high-order elements is obtained from the bitmap data structure; it can be understood that: the first preset data type refers to a target byte array whose number of target high-order elements is not 0, that is, when the number of elements of the bitmap data structure is not greater than 32, the number of target high-order elements is 0.

[0062] Specifically, the target high-order element refers to an element in a high-order bucket obtained according to the high-order and low-order allocation rules of the bitmap data structure.

[0063] For better understanding, further explanation is given as follows. In the bitmap data structure, the 64-bit integer range is divided into high and low parts. The high part is usually used to represent a large range of integer values. The high part refers to 33-64 bits, and the low part refers to 1-32 bits. The number of target high-order elements refers to the number of elements in the high-order part. Since this number of elements can provide further information about the 64-bit bitmap data structure, such as the coverage of the data range and the distribution of the data, it plays a very important role in subsequent data restoration operations.

[0064] S200, performing binary processing on the data of the bitmap data structure to obtain an initial byte array and an initial byte length corresponding to the bitmap data structure.

[0065] Specifically, Figure 2 As shown, step S200 includes the following steps:

[0066] S201, converting data of the bitmap data structure into binary byte data, and determining the binary byte data as an initial byte array corresponding to the bitmap data structure.

[0067] S202, obtaining the byte length of the initial byte array, and determining the byte length of the initial byte array as the initial byte length corresponding to the bitmap data structure.

[0068] As mentioned above, by performing binary processing on the data of the bitmap data structure, it is converted into a byte array, thereby realizing the transformation of the data arrangement structure. Since the column-based database management system has the effect of efficiently storing data in the form of byte arrays, after the above-mentioned data object conversion, the structure of the data is kept consistent with the data structure stored in the column-based database management system, completing the preliminary processing of the data, which is conducive to realizing the rapid storage of large-scale data sets in the column-based database management system.

[0069] S300, converting the initial sequence type corresponding to the initial byte array into a target sequence type to obtain a target byte array; specifically comprising the following steps:

[0070] S301, obtaining an initial sequence type corresponding to the initial byte array; the initial sequence type corresponding to the initial byte array is a big-endian byte sequence type.

[0071] S302, converting the initial byte array from a big-endian byte sequence type to a little-endian byte sequence type, and acquiring a target byte array.

[0072] Specifically, a byte buffer object is used to convert the initial byte array from a big-endian byte sequence type to a little-endian byte sequence type, wherein the target byte buffer object refers to a buffer capable of storing big-endian data and little-endian data. In a specific implementation, a byte buffer is generally used for data format conversion, for example, converting big-endian data into little-endian data, or converting little-endian data into big-endian data; in this embodiment, the target byte buffer uses a ByteBuffer buffer.

[0073] As mentioned above, since the bitmap data structures in Java are all data arranged in big-endian order, and CK supports data arranged in little-endian order, the big-endian byte sequence type is converted into the little-endian byte sequence type through the byte buffer, so that the original bitmap data structure is converted into a data sequence that CK can read normally, which is conducive to fast data transmission and efficient storage.

[0074] S400: Based on the initial byte length, perform byte compression processing on the initial byte array to obtain a target byte length corresponding to the compressed initial byte array.

[0075] Specifically, Figure 3 As shown, step S400 also includes the following steps:

[0076] S401, encoding an initial byte array corresponding to the bitmap data structure by using a variable-length byte encoding method to obtain a compressed encoding sequence.

[0077] Preferably, the variable-length byte encoding adopts varint encoding; varint encoding is a method of performing variable-length byte encoding on positive integers. For example, when an int-type integer occupies 4 bytes, if the value of the integer is less than 256, it can be stored in one byte of space, wasting 3 bytes of space. Therefore, using varint encoding can effectively save byte space of data, especially for integers with larger values, the compression effect is more obvious.

[0078] S402, obtaining a first byte length corresponding to the compressed coding sequence; it can be understood that: the first byte length refers to the byte length of the coding sequence after compression.

[0079] S403, based on the adopted variable-length byte encoding, calculate the second byte length corresponding to the initial byte length; it can be understood that: the second byte length refers to the byte length that should be obtained when the variable-length byte encoding encodes the initial byte array, for example, when an int integer occupies 4 bytes, if the value of the integer is less than 256, then the corresponding second byte length is 1 byte.

[0080] S404: When the first byte length is consistent with the second byte length, the first byte length is determined as the target byte length; otherwise, a prompt verification message is sent to the target user.

[0081] As described above, by compressing the initial byte array corresponding to the bitmap data structure, the storage space occupied by the byte data can be effectively reduced, thereby reducing the impact on data transmission and computing resources, which is conducive to quickly storing data in a column-based database management system while saving storage resources for column-based database management.

[0082] S500, splicing the first preset data type, target byte length, target number of high-order elements and target byte array into a new byte array according to a preset splicing order, so as to store the new byte array in a given column-based database management system.

[0083] Specifically, the preset splicing order refers to the order of sequentially splicing the first preset data type, the target byte length, the number of target high-order elements, and the target byte array.

[0084] As described above, by splicing multiple information into a new byte array, it is beneficial to achieve accurate restoration of data when querying data. In addition, the first preset data type is placed in the first byte of the target byte array, so that the data type of the target byte data is obtained first when reading data, so that different data restoration methods are used according to different data types to maximize data parsing efficiency.

[0085] Preferably, in another embodiment, Figure 4 As shown, the method further comprises the following steps:

[0086] S001, when the number of elements of the bitmap data structure is not greater than 32, obtain the occupied byte length of the bitmap data structure, and mark the bitmap data structure as a second preset data type; it can be understood that: the second preset data type refers to a target byte array whose number of target high-order elements is 0, that is, when the number of elements of the bitmap data structure is greater than 32, the number of target high-order elements is greater than 0.

[0087] S002, performing byte compression processing on the occupied byte length to obtain the key byte length.

[0088] Specifically, the manner of performing byte compression processing on the occupied byte length is consistent with the manner of performing byte compression processing on the initial byte array, which will not be described in detail here.

[0089] S003, converting the data of the bitmap data structure into data of a little-endian byte sequence type.

[0090] S004, concatenating the data of the second preset data type, the key byte length and the little endian byte sequence type into a new byte array, so as to store the new byte array in a given column-based database management system.

[0091] As mentioned above, when the bitmap data structure of a small data set is stored in a column-based database management system, there is no need to perform binary processing and serialized byte array conversion because the parsing will not fail due to the large amount of data when the data is restored. Instead, the data in the bitmap data structure can be directly read and written into a preset empty byte array in little-endian order, which reduces the data processing flow and facilitates the rapid storage of data.

[0092] The data storage method of the bitmap data structure described in this embodiment, when the number of elements of the received bitmap data structure is greater than 32, the data of the bitmap data structure is binary processed to obtain the initial byte array and initial byte length corresponding to the bitmap data structure, the initial sequence type corresponding to the initial byte array is converted to the target sequence type, and the target byte array is obtained. Through the transformation of the data structure and the adjustment of the sequence order, the data of the bitmap data structure can be quickly stored in the column-based database management system after processing, effectively reducing the data processing flow and reducing the utilization of computing resources during data storage. Then, the initial byte array is byte compressed, which is beneficial to reducing data storage resources and saving storage space of the column-based database management system. Finally, the first preset data type, target byte length, number of target high-order elements and target byte array obtained are spliced ​​into a new byte array, and the new byte array is stored in a given column-based database management system, realizing the combination of the bitmap data structure and the column-based database management system, thereby realizing efficient and persistent storage of large-scale data sets.

[0093] In another specific implementation scenario, after the data is stored in a column-based database management system, when the required data needs to be queried from the column-based database management system according to various query requirements of the business, the present application also provides a data type conversion method, including the following steps:

[0094] P100, generating a target ANTLR4 expression according to a data query request; the data query request is a request including a number of query condition tags, the target ANTLR4 expression includes a number of intermediate expressions and a number of logical operation operators, and the intermediate expressions correspond to the query condition tags one by one.

[0095] Specifically, the query condition tags include APP category tags, location tags and device category tags; wherein the APP category tags include but are not limited to high-risk tags, low-risk tags, and financial management tags; the device category tags include but are not limited to simulator categories, device brands, and device models.

[0096] Specifically, the location tag may be one of a permanent residence, a home location, a workplace, and a latest location.

[0097] Specifically, the logical operation operator is any one of intersection “^”, union “#”, and difference “%”.

[0098] P200, converting several intermediate expressions in the target ANTLR4 expression into several intermediate SQL fragments respectively, and each intermediate SQL fragment corresponds to an intermediate SQL tag.

[0099] Specifically, the SP00 step includes the following steps:

[0100] P201, parse the target ANTLR4 expression to obtain several intermediate expressions; each of the intermediate expressions corresponds to at least one query condition sub-tag; each of the query condition tags includes at least one query condition sub-tag; it can be understood that: when the query condition tag is composed of several query condition sub-tags, the intermediate expression corresponding to the query condition tag is composed of sub-expressions corresponding to several query condition sub-tags.

[0101] For better explanation, an example is given below: when searching for a certain location and a certain device brand that includes a low-risk label APP and a financial label APP, "a certain location" and "a certain device brand" are each a query condition label, "low-risk label APP and financial label APP" is a query condition label, and "low-risk label APP" and "financial label APP" are both query condition sub-labels. Each query condition label corresponds to an intermediate expression, and each query condition sub-label corresponds to a sub-expression.

[0102] P202, construct several temporary virtual tables; the temporary virtual tables correspond one-to-one to the intermediate expressions.

[0103] Specifically, a temporary virtual table is defined and constructed through the with clause in the SQL language.

[0104] For example: for the target ANTLR4 expression ((c0)^(c1#c2#c3#c4#c5)^(c6fa0#c6fa1))%((f0#f1)^(f2#f3)), the correspondence between the names of the generated temporary virtual tables and the intermediate expressions is: with_tmp_table_1 corresponds to c0, with_tmp_table_2 corresponds to c1#c2#c3#c4#c5, with_tmp_table_3 corresponds to c6fa0#c6fa1, with_tmp_table_4 corresponds to f0#f1, and with_tmp_table_5 corresponds to f2#f3.

[0105] P203, store several intermediate expressions into several temporary virtual tables respectively, generate several intermediate SQL fragments, and determine the name of the temporary virtual table corresponding to each intermediate SQL fragment as the intermediate SQL label corresponding to the intermediate SQL fragment itself; it can be understood as: determining the table name corresponding to the with clause as the intermediate SQL label.

[0106] As described above, when processing complex expressions, several intermediate expressions after parsing are temporarily saved by building a temporary virtual table, and parsed in memory as a temporary memory table, which can improve the execution efficiency and readability of SQL statements, and is conducive to subsequent complex data calculations. Moreover, the temporary virtual table is only valid in the current query and can be destroyed after use, which is conducive to the rational use of memory resources.

[0107] P300, according to the operation order of the logical operation operator corresponding to each intermediate expression in the target ANTLR4 expression, a plurality of intermediate SQL tags and a plurality of logical operation operators are recombined in a suffix expression manner to obtain a tag combination result.

[0108] Specifically, the P300 step includes the following steps:

[0109] P301, parse the target ANTLR4 expression to obtain the distribution order of several intermediate expressions and the operation order of several logical operation operators.

[0110] P302, according to the distribution order of the several intermediate expressions, the several intermediate SQL tags and the several logical operation operators are reordered to obtain a preprocessed SQL statement; the preprocessed SQL statement is a statement composed of prefix expressions; it can be understood as: sorting the several intermediate SQL tags according to the distribution order of the several intermediate expressions, and using the sorting result as the preprocessed SQL statement.

[0111] P303, according to the operation order of the plurality of logical operation operators, convert the preprocessing SQL statement into an intermediate SQL statement combined with a suffix expression, and use the intermediate SQL statement as a label combination result.

[0112] For ease of understanding, an example is given below:

[0113] For example, in the target ANTLR4 expression mentioned above, the intermediate expressions are: (c0), (c1#c2#c3#c4#c5), (c6fa0#c6fa1), (f0#f1), (f2#f3); the operation order is: first perform the union operation in each intermediate expression, then calculate the intersection of the first three expressions and the intersection of the last two expressions in turn, and finally perform the difference operation on the two sets; after converting the target ANTLR4 expression into a suffix expression, the expression obtained is (c0) (c1#c2#c3#c4#c5) ^(c6fa0#c6fa1) ^ (f0#f1) (f2#f3) ^ %, that is, the intermediate SQL statement obtained is with_tmp_table_1with_tmp_table_2 ^ with_tmp_table_3 ^ with_tmp_table_4 with_tmp_table_5 ^ %.

[0114] As mentioned above, by converting the preprocessed SQL statement in the form of a prefix expression into an intermediate SQL statement in the form of a suffix expression, the computer does not need to consider the issue of operation priority during reading and calculation. It only needs to obtain the elements in sequence and then perform calculations. This is very convenient for computer data processing and is conducive to improving the efficiency of data query.

[0115] P400, performs target operation function conversion on each logical operation operator in the label combination result, and obtains the data conversion result according to the combination order of the intermediate SQL labels in the label combination result; it can be understood as: combining the result after the target operation function conversion with several sorted intermediate SQL labels to obtain the data conversion result.

[0116] Specifically, the target operation function refers to the operation function corresponding to the logical operation operator in the SQL language environment; it can be understood that: each logical operation operator corresponds to a bitmap operation function of CK; for example, the operation function corresponding to the intersection is bitmapAnd, the operation function corresponding to the union is groupBitmapOrState or bitmapOr, and the operation function corresponding to the difference is bitmapAndNot.

[0117] P500, generating a target SQL statement according to the plurality of intermediate SQL fragments and the data conversion result, so as to complete the data type conversion from the ANTLR4 expression to the SQL statement.

[0118] Specifically, the P500 step includes the following steps:

[0119] P501, obtain the intermediate sql fragment corresponding to each intermediate sql tag in the data conversion result; it can be understood that: the intermediate sql fragment corresponding to the intermediate sql tag refers to the sql statement in the temporary virtual table corresponding to the intermediate sql tag.

[0120] P502, adding the intermediate SQL fragments corresponding to each intermediate SQL tag to the corresponding position of the intermediate SQL tag itself in the data conversion result, and deleting the intermediate SQL tag itself to generate a target SQL statement.

[0121] As mentioned above, by replacing the corresponding intermediate sql tags with several intermediate sql fragments and adding them to the data conversion results, it is shown that the target ANTLR4 expression has completed the complete conversion to the sql statement. Through this data type conversion method, any combination of any number of business query conditions can be converted from a complex expression to a bitmap calculation sql statement that can be executed by CK, thereby realizing fast and efficient query and processing of data, so as to meet multiple business query requirements at the same time.

[0122] In another specific implementation scenario, after storing the data in a column-based database management system, when the required data is queried from the column-based database management system, the data needs to be parsed. Therefore, the present application also provides a byte array data parsing method, comprising the following steps:

[0123] Z100, receives a target byte array from a given column-based database management system; the target byte array includes a target data type, a target byte length, the number of target high-order elements and an intermediate byte array; the target data type includes a first preset data type and a second preset data type; it can be understood that: the target byte array is the byte array queried according to the converted target SQL statement.

[0124] Specifically, the given column-based database management system refers to a system in which stored data exists in the form of byte arrays; in this embodiment, the given column-based database management system is the ClickHouse database.

[0125] Specifically, the first preset data type refers to a target byte array whose number of target high-order elements is not 0, and the second preset data type refers to a target byte array whose number of target high-order elements is 0.

[0126] Specifically, the first byte of the target byte array received from a given column-based database management system is used to represent the target data type. By representing the target data type with the first byte of the byte array, when the target byte array is received and read, the data type corresponding to the target byte array can be read first, and then the corresponding parsing method can be quickly selected to parse the target byte array.

[0127] When the first byte of the received target byte array is the first preset byte, it represents that the target data type is the first preset data type; when the first byte of the received target byte array is the second preset byte, it represents that the target data type is the second preset data type; technicians in this field set the first preset byte and the second preset byte according to actual needs, for example, setting the first preset byte to 1 and setting the second preset byte to 0.

[0128] Specifically, the target byte length refers to the byte length corresponding to the middle byte array in the target byte array.

[0129] Specifically, the target high-order element refers to an element in a high-order bucket obtained according to a high-order and low-order allocation rule of a bitmap data structure.

[0130] For better understanding, further explanation is given as follows. In the bitmap data structure, the 64-bit integer range is divided into high and low parts. The high part is usually used to represent a large range of integer values. The high part refers to 33-64 bits, and the low part refers to 1-32 bits. The number of target high-order elements refers to the number of elements in the high-order part. Since this number of elements can provide further information about the 64-bit bitmap data structure, such as the coverage of the data range and the distribution of the data, it plays a very important role in subsequent operations.

[0131] Specifically, the intermediate byte array refers to any set of serialized data stored in a given column-based database management system; it can be understood that: the target byte array is an array received including any intermediate byte array and some information of the intermediate byte array itself.

[0132] Z200, writes the target byte array into a pre-constructed target byte buffer object; the intermediate byte array in the target byte array is arranged in little endian order in the target byte buffer object; it can be understood that: the target byte array arranged in little endian order still exists in little endian form after being written into the target byte buffer, and the target byte buffer includes the target data type, target byte length, number of target high-order elements and intermediate byte array in the received target byte array.

[0133] Specifically, the target byte buffer object refers to a buffer that can store big-endian data and little-endian data. In a specific implementation, a byte buffer is usually used to perform data format conversion, for example, converting big-endian data to little-endian data, or converting little-endian data to big-endian data. In this embodiment, the target byte buffer adopts a ByteBuffer buffer. This target byte buffer is only used for temporary storage of arrays in this embodiment and is not used for data sorting type conversion.

[0134] As mentioned above, since the target byte array contains multiple array information, when querying data, only the intermediate byte array in the target byte array needs to be obtained, that is, the original byte array is converted into a new byte array. The target byte buffer object can more conveniently perform operations such as reading and identifying data on the byte array. That is, the target byte buffer object provides a more friendly way to operate the byte array, thereby ensuring the accuracy of the array conversion.

[0135] Z300: Parse the target byte array in the target byte buffer object into a bitmap data structure according to the content of the target byte buffer object.

[0136] Specifically, the Z300 step includes the following steps:

[0137] Z310, sequentially read the data of the target byte buffer object to obtain the first byte corresponding to the target byte buffer object.

[0138] Z320, determine the target data type corresponding to the target byte array according to the first byte corresponding to the target byte buffer object.

[0139] As described above, by reading the first byte corresponding to the target byte buffer object, the target data type corresponding to the target byte array can be obtained first, so that different data parsing methods can be adopted for the target byte array according to different data types to maximize the parsing effect, thereby improving the efficiency of data query.

[0140] In a special case, when the first byte corresponding to the target byte buffer object is not the first preset byte and the second preset byte, the target byte array is parsed into an empty bitmap data structure.

[0141] In one embodiment, the Z300 step specifically includes the following steps:

[0142] Z301, when the target data type is a first preset data type, obtain the target byte length and the number of target high-order elements from the target byte buffer object.

[0143] Z303, construct an empty byte array corresponding to the target byte length obtained from the target byte buffer object.

[0144] Specifically, step Z303 also includes the following steps:

[0145] Z3031, when the target byte length obtained from the target byte buffer object is the original byte length corresponding to the intermediate byte array obtained from the target byte buffer object, construct an empty byte array with a byte length equal to the target byte length.

[0146] Z3032, when the target byte length obtained from the target byte buffer object is the byte length of the intermediate byte array obtained from the target byte buffer object after compression processing, calculate the original byte length corresponding to the target byte length obtained from the target byte buffer object, and construct an empty byte array with a byte length equal to the original byte length.

[0147] Z305, read the intermediate byte array arranged in little endian order from the target byte buffer object, and store the read intermediate byte array into the empty byte array.

[0148] Z307, stores the number of target high-order elements obtained from the target byte buffer object and the data in the empty byte array into a preset new byte buffer object, so as to convert the intermediate byte array stored in the empty byte array into a byte array arranged in big-endian order through the new byte buffer object.

[0149] Z309, parse the content of the new byte buffer object to obtain a bitmap data structure.

[0150] Specifically, the big-endian and little-endian conversion method in the Java language is used to convert the little-endian data in the new byte buffer object into big-endian data to obtain a corresponding 64-bit bitmap data structure.

[0151] As described above, when processing the byte array of the first preset data type, the target byte array is directly parsed at the byte array level, and the serialized byte array can be parsed into data of a bitmap data structure in Java through a little-endian to big-endian conversion. There is no need to perform multiple encoding and decoding operations, which reduces the object conversion and calculation process, thereby simplifying the data parsing process in the query process and improving data query efficiency.

[0152] In another embodiment, the Z300 step further comprises the following steps:

[0153] Z302: When the target data type is a second preset data type, obtain a number of valid bytes from the target byte buffer object.

[0154] Specifically, the valid bytes refer to the byte arrays in the intermediate byte array in the target byte buffer object that are arranged in order and comply with the data reading rules; for example, in this embodiment, the data stored in CK is long integer data, each data occupies 8 bytes, then the number of valid bytes is n times 8, n is the number of data elements, and n times 8 valid bytes are extracted as the obtained number of valid bytes.

[0155] Z304, read the several valid bytes in sequence to obtain several long integer data; it can be understood that: read every eight bytes, each time read, a long integer data is read out, and finally after parsing, n data are obtained.

[0156] Z306, storing the plurality of long integer data in sequence into a pre-constructed bitmap data structure to implement parsing of the target byte array into a bitmap data structure; the initial state of the pre-constructed bitmap data structure is empty.

[0157] As described above, for the byte array of the second preset data type, when reading data, no conversion of the byte buffer object is involved, and the read data is directly stored in a pre-built bitmap data structure, which makes the data parsing method simpler and is conducive to improving the efficiency of data query.

[0158] Embodiment 2

[0159] like Figure 5 As shown, this embodiment 2 provides a data storage device of a bitmap data structure, including:

[0160] The first acquisition module 100 is used to mark the bitmap data structure as a first preset data type and obtain the number of target high-order elements from the bitmap data structure when the number of elements of the received bitmap data structure is greater than 32; it can be understood that: the first preset data type refers to a target byte array whose number of target high-order elements is not 0, that is, when the number of elements of the bitmap data structure is not greater than 32, the number of target high-order elements is 0.

[0161] Specifically, the target high-order element refers to an element in a high-order bucket obtained according to the high-order and low-order allocation rules of the bitmap data structure.

[0162] For better understanding, further explanation is given as follows. In the bitmap data structure, the 64-bit integer range is divided into high and low parts. The high part is usually used to represent a large range of integer values. The high part refers to 33-64 bits, and the low part refers to 1-32 bits. The number of target high-order elements refers to the number of elements in the high-order part. Since this number of elements can provide further information about the 64-bit bitmap data structure, such as the coverage of the data range and the distribution of the data, it plays a very important role in subsequent data restoration operations.

[0163] The first processing module 200 is used to perform binary processing on the data of the bitmap data structure to obtain an initial byte array and an initial byte length corresponding to the bitmap data structure.

[0164] Specifically, Figure 6 As shown, the first processing module 200 includes:

[0165] The first determining module 201 is used to convert the data of the bitmap data structure into binary byte data, and determine the binary byte data as an initial byte array corresponding to the bitmap data structure.

[0166] The second determining module 202 is used to obtain the byte length of the initial byte array, and determine the byte length of the initial byte array as the initial byte length corresponding to the bitmap data structure.

[0167] As mentioned above, by performing binary processing on the data of the bitmap data structure, it is converted into a byte array, thereby realizing the transformation of the data arrangement structure. Since the column-based database management system has the effect of efficiently storing data in the form of byte arrays, after the above-mentioned data object conversion, the structure of the data is kept consistent with the data structure stored in the column-based database management system, completing the preliminary processing of the data, which is conducive to realizing the rapid storage of large-scale data sets in the column-based database management system.

[0168] The first conversion module 300 is used to convert the initial sequence type corresponding to the initial byte array into a target sequence type to obtain a target byte array; the first conversion module is also used to obtain the initial sequence type corresponding to the initial byte array; the initial sequence type corresponding to the initial byte array is a big-endian byte sequence type; the initial byte array is converted from a big-endian byte sequence type to a little-endian byte sequence type to obtain a target byte array.

[0169] Specifically, a byte buffer object is used to convert the initial byte array from a big-endian byte sequence type to a little-endian byte sequence type, wherein the target byte buffer object refers to a buffer capable of storing big-endian data and little-endian data. In a specific implementation, a byte buffer is generally used for data format conversion, for example, converting big-endian data into little-endian data, or converting little-endian data into big-endian data; in this embodiment, the target byte buffer uses a ByteBuffer buffer.

[0170] As mentioned above, since the bitmap data structures in Java are all data arranged in big-endian order, and CK supports data arranged in little-endian order, the big-endian byte sequence type is converted into the little-endian byte sequence type through the byte buffer, so that the original bitmap data structure is converted into a data sequence that CK can read normally, which is conducive to fast data transmission and efficient storage.

[0171] The first compression module 400 is used to perform byte compression processing on the initial byte array based on the initial byte length, and obtain a target byte length corresponding to the compressed initial byte array.

[0172] Specifically, Figure 7 As shown, the first compression module 400 includes:

[0173] The encoding module 401 is used to encode the initial byte array corresponding to the bitmap data structure in a variable-length byte encoding manner to obtain a compressed encoding sequence.

[0174] Preferably, the variable-length byte encoding adopts varint encoding; varint encoding is a method of performing variable-length byte encoding on positive integers. For example, when an int-type integer occupies 4 bytes, if the value of the integer is less than 256, it can be stored in one byte of space, wasting 3 bytes of space. Therefore, using varint encoding can effectively save byte space of data, especially for integers with larger values, the compression effect is more obvious.

[0175] The second acquisition module 402 is used to acquire the first byte length corresponding to the compressed coding sequence; it can be understood that: the first byte length refers to the byte length of the coding sequence after compression.

[0176] The calculation module 403 is used to calculate the second byte length corresponding to the initial byte length based on the adopted variable-length byte encoding; it can be understood that: the second byte length refers to the byte length that should be obtained when the variable-length byte encoding encodes the initial byte array. For example, when an int integer occupies 4 bytes, if the value of the integer is less than 256, the corresponding second byte length is 1 byte.

[0177] The third determination module 404 is configured to determine the first byte length as a target byte length when the first byte length is consistent with the second byte length, and otherwise send a prompt verification message to a target user.

[0178] As described above, by compressing the initial byte array corresponding to the bitmap data structure, the storage space occupied by the byte data can be effectively reduced, thereby reducing the impact on data transmission and computing resources, which is conducive to quickly storing data in a column-based database management system while saving storage resources for column-based database management.

[0179] The first splicing module 500 is used to splice the first preset data type, target byte length, target number of high-order elements and target byte array into a new byte array according to a preset splicing order, so as to store the new byte array in a given column-based database management system.

[0180] Specifically, the preset splicing order refers to the order of sequentially splicing the first preset data type, the target byte length, the number of target high-order elements, and the target byte array.

[0181] As described above, by splicing multiple information into a new byte array, it is beneficial to achieve accurate restoration of data when querying data. In addition, the first preset data type is placed in the first byte of the target byte array, so that the data type of the target byte data is obtained first when reading data, so that different data restoration methods are used according to different data types to maximize data parsing efficiency.

[0182] Preferably, Figure 8 As shown, the device further includes a second processing module, and the second processing module includes:

[0183] The marking module 001 is used to obtain the occupied byte length of the bitmap data structure and mark the bitmap data structure as a second preset data type when the number of elements of the bitmap data structure is not greater than 32; it can be understood that: the second preset data type refers to a target byte array whose number of target high-order elements is 0, that is, when the number of elements of the bitmap data structure is greater than 32, the number of target high-order elements is greater than 0.

[0184] The second compression module 002 is used to perform byte compression processing on the occupied byte length to obtain the key byte length.

[0185] Specifically, the manner of performing byte compression processing on the occupied byte length is consistent with the manner of performing byte compression processing on the initial byte array, which will not be described in detail here.

[0186] The second conversion module 003 is used to convert the data of the bitmap data structure into data of a little endian byte sequence type.

[0187] The second splicing module 004 is used to splice the data of the second preset data type, key byte length and little endian byte sequence type into a new byte array, so as to store the new byte array in a given column-based database management system.

[0188] As mentioned above, when the bitmap data structure of a small data set is stored in a column-based database management system, there is no need to perform binary processing and serialized byte array conversion because the parsing will not fail due to the large amount of data when the data is restored. Instead, the data in the bitmap data structure can be directly read and written into a preset empty byte array in little-endian order, which reduces the data processing flow and facilitates the rapid storage of data.

[0189] When the data storage device of the bitmap data structure described in this embodiment is running, when the number of elements of the received bitmap data structure is greater than 32, binary processing is performed on the data of the bitmap data structure to obtain the initial byte array and initial byte length corresponding to the bitmap data structure, and the initial sequence type corresponding to the initial byte array is converted into the target sequence type to obtain the target byte array. Through the transformation of the data structure and the adjustment of the sequence order, the data of the bitmap data structure can be quickly stored in the column-based database management system after processing, effectively reducing the data processing flow and reducing the utilization of computing resources during data storage. Then, byte compression processing is performed on the initial byte array, which is beneficial to reducing data storage resources and saving storage space of the column-based database management system. Finally, the first preset data type, target byte length, target number of high-order elements and target byte array obtained are spliced ​​into a new byte array, and the new byte array is stored in a given column-based database management system, realizing the combination of the bitmap data structure and the column-based database management system, thereby realizing efficient and persistent storage of large-scale data sets.

[0190] Embodiment 3

[0191] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented:

[0192] When the number of elements of the received bitmap data structure is greater than 32, the bitmap data structure is marked as a first preset data type, and the number of target high-order elements is obtained from the bitmap data structure.

[0193] Binary processing is performed on the data of the bitmap data structure to obtain an initial byte array and an initial byte length corresponding to the bitmap data structure.

[0194] The initial sequence type corresponding to the initial byte array is converted into a target sequence type to obtain a target byte array; the first conversion module is also used to obtain the initial sequence type corresponding to the initial byte array; the initial sequence type corresponding to the initial byte array is a big-endian byte sequence type; the initial byte array is converted from a big-endian byte sequence type to a little-endian byte sequence type to obtain a target byte array.

[0195] Based on the initial byte length, byte compression processing is performed on the initial byte array to obtain a target byte length corresponding to the compressed initial byte array.

[0196] The first preset data type, target byte length, target number of high-order elements and target byte array are spliced ​​into a new byte array according to a preset splicing order, so as to store the new byte array in a given column-based database management system.

[0197] Embodiment 4

[0198] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0199] When the number of elements of the received bitmap data structure is greater than 32, the bitmap data structure is marked as a first preset data type, and the number of target high-order elements is obtained from the bitmap data structure.

[0200] Binary processing is performed on the data of the bitmap data structure to obtain an initial byte array and an initial byte length corresponding to the bitmap data structure.

[0201] The initial sequence type corresponding to the initial byte array is converted into a target sequence type to obtain a target byte array; the first conversion module is also used to obtain the initial sequence type corresponding to the initial byte array; the initial sequence type corresponding to the initial byte array is a big-endian byte sequence type; the initial byte array is converted from a big-endian byte sequence type to a little-endian byte sequence type to obtain a target byte array.

[0202] Based on the initial byte length, byte compression processing is performed on the initial byte array to obtain a target byte length corresponding to the compressed initial byte array.

[0203] The first preset data type, target byte length, target number of high-order elements and target byte array are spliced ​​into a new byte array according to a preset splicing order, so as to store the new byte array in a given column-based database management system.

[0204] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0205] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0206] Although some specific embodiments of the present invention have been described in detail by way of example, it will be appreciated by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present invention. It will also be appreciated by those skilled in the art that various modifications may be made to the embodiments without departing from the scope and spirit of the present invention. The scope of the present invention is defined by the appended claims.

Claims

1. A data storage method of a bitmap data structure, characterized in that: The following steps are involved: When the number of elements of the received bitmap data structure is greater than 32, marking the bitmap data structure as a first preset data type, and obtaining the number of target high-order elements from the bitmap data structure; Performing binary processing on the data of the bitmap data structure to obtain an initial byte array and an initial byte length corresponding to the bitmap data structure; The initial sequence type corresponding to the initial byte array is converted into a target sequence type to obtain a target byte array; the converting the initial sequence type corresponding to the initial byte array into a target sequence type to obtain a target byte array comprises the following steps: obtaining the initial sequence type corresponding to the initial byte array; the initial sequence type corresponding to the initial byte array is a big endian byte sequence type; converting the initial byte array from the big endian byte sequence type to the little endian byte sequence type to obtain a target byte array; Based on the initial byte length, byte compression processing is performed on the initial byte array to obtain a target byte length corresponding to the compressed initial byte array; The first preset data type, target byte length, target number of high-order elements and target byte array are spliced ​​into a new byte array according to a preset splicing order, so as to store the new byte array in a given column-based database management system.

2. The data storage method of the bitmap data structure according to claim 1, characterized in that: The target high-order element refers to an element in a high-order bucket obtained according to the high-order and low-order allocation rules of the bitmap data structure.

3. The data storage method of the bitmap data structure according to claim 1, characterized in that: The binary processing of the data of the bitmap data structure to obtain the initial byte array and initial byte length corresponding to the bitmap data structure comprises the following steps: Converting the data of the bitmap data structure into binary byte data, and determining the binary byte data as an initial byte array corresponding to the bitmap data structure; The byte length of the initial byte array is obtained, and the byte length of the initial byte array is determined as the initial byte length corresponding to the bitmap data structure.

4. The data storage method of the bitmap data structure according to claim 1, characterized in that: Get the target byte length by following the steps below: Encoding the initial byte array corresponding to the bitmap data structure in a variable-length byte encoding manner to obtain a compressed encoding sequence; Obtaining a first byte length corresponding to the compressed encoding sequence; Based on the adopted variable-length byte encoding, calculating a second byte length corresponding to the initial byte length; When the first byte length is consistent with the second byte length, the first byte length is determined as the target byte length; otherwise, a prompt verification message is sent to the target user.

5. The data storage method of the bitmap data structure according to claim 1, characterized in that: The following steps are also included: When the number of elements of the bitmap data structure is not greater than 32, obtaining the occupied byte length of the bitmap data structure, and marking the bitmap data structure as a second preset data type; Perform byte compression processing on the occupied byte length to obtain the key byte length; Converting the data of the bitmap data structure into data of a little endian byte sequence type; The data of the second preset data type, the key byte length and the little endian byte sequence type are concatenated into a new byte array, so as to store the new byte array in a given column-based database management system.

6. A data storage device of a bitmap data structure, characterized in that: The device comprises: A first acquisition module, configured to mark the received bitmap data structure as a first preset data type and acquire the number of target high-order elements from the bitmap data structure when the number of elements of the received bitmap data structure is greater than 32; A first processing module, configured to perform binary processing on the data of the bitmap data structure to obtain an initial byte array and an initial byte length corresponding to the bitmap data structure; A first conversion module is used to convert the initial sequence type corresponding to the initial byte array into a target sequence type to obtain a target byte array; the first conversion module is also used to obtain the initial sequence type corresponding to the initial byte array; the initial sequence type corresponding to the initial byte array is a big endian byte sequence type; the initial byte array is converted from the big endian byte sequence type to the little endian byte sequence type to obtain a target byte array; A first compression module, configured to perform byte compression processing on the initial byte array based on the initial byte length, and obtain a target byte length corresponding to the compressed initial byte array; The first splicing module is used to splice the first preset data type, target byte length, target number of high-order elements and target byte array into a new byte array according to a preset splicing order, so as to store the new byte array in a given column-based database management system.

7. The data storage device of the bitmap data structure according to claim 6, characterized in that: The target high-order element refers to an element in a high-order bucket obtained according to the high-order and low-order allocation rules of the bitmap data structure.

8. The data storage device of the bitmap data structure according to claim 6, characterized in that: The first processing module comprises: A first determining module, configured to convert the data of the bitmap data structure into binary byte data, and determine the binary byte data as an initial byte array corresponding to the bitmap data structure; The second determining module is used to obtain the byte length of the initial byte array, and determine the byte length of the initial byte array as the initial byte length corresponding to the bitmap data structure.

9. The data storage device of the bitmap data structure according to claim 6, characterized in that: The first compression module comprises: An encoding module, used for encoding the initial byte array corresponding to the bitmap data structure by using a variable-length byte encoding method to obtain a compressed encoding sequence; A second acquisition module, used to acquire a first byte length corresponding to the compressed coding sequence; A calculation module, used for calculating a second byte length corresponding to the initial byte length based on the adopted variable-length byte encoding; The third determination module is used to determine the first byte length as the target byte length when the first byte length is consistent with the second byte length, and otherwise send a prompt verification message to the target user.

10. The data storage device of the bitmap data structure according to claim 6, characterized in that: The device further includes a second processing module, wherein the second processing module includes: a marking module, configured to obtain the occupied byte length of the bitmap data structure when the number of elements of the bitmap data structure is not greater than 32, and mark the bitmap data structure as a second preset data type; A second compression module is used to perform byte compression processing on the occupied byte length to obtain the key byte length; A second conversion module, used for converting the data of the bitmap data structure into data of a little endian byte sequence type; The second splicing module is used to splice the data of the second preset data type, key byte length and little endian byte sequence type into a new byte array to store the new byte array in a given column database management system.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the data storage method of the bitmap data structure according to any one of claims 1 to 5 is implemented.

12. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the data storage method of the bitmap data structure according to any one of claims 1 to 5 is implemented.

Images