Query method, device and equipment of user data and storage medium

Abstract

The application discloses a user data query method and device, equipment and storage medium, and belongs to the field of data processing. The method comprises the following steps: receiving a query request, wherein the query request comprises calculation logic information; determining a target bucket identifier in a target storage space according to a target bitmap identifier indicated by the calculation logic information, wherein the target storage space stores bitmap identifiers, bucket identifiers and bucket contents having a corresponding relationship, the bitmap identifier is used for representing the classification of user data, and the bucket identifier and the bucket content are used for restoring the user data; processing the bucket content corresponding to the target bucket identifier in the target storage space according to a logical operation indicated by the calculation logic information, obtaining a query result and feeding back the query result. According to the embodiment of the application, the query efficiency of the user data can be improved.

Description

Technical Field

[0001] This application belongs to the field of data processing, and in particular relates to a method, apparatus, device and storage medium for querying user data. Background Technology

[0002] With the continuous development of electronic information technology, more and more user data involved in business operations can be managed through electronic information technology. To facilitate the processing of user data, it can be divided according to user characteristics and managed accordingly. User data can be stored using a bitmap method based on the user's identifier. However, in scenarios where user data is queried, it is necessary to traverse all data stored in the bitmap method. Furthermore, as business operations continue to upgrade, the dimensions and volume of user data are constantly expanding, resulting in long query times and low query efficiency. Summary of the Invention

[0003] This application provides a method, apparatus, device, and storage medium for querying user data, which can improve the efficiency of user data query.

[0004] In a first aspect, embodiments of this application provide a method for querying user data, comprising: receiving a query request, the query request including computational logic information; determining a target bucket identifier in a target storage space according to a target bitmap identifier indicated by the computational logic information, the target storage space storing a bitmap identifier, a bucket identifier, and bucket content with a corresponding relationship, the bitmap identifier being used to characterize the classification of user data, and the bucket identifier and bucket content being used to reconstruct the user data; and processing the bucket content in the target storage space corresponding to the target bucket identifier according to the logical operation indicated by the computational logic information to obtain a query result and provide feedback.

[0005] Secondly, embodiments of this application provide a user data query device, comprising: a receiving module for receiving a query request, the query request including computational logic information; a logic processing module for determining a target bucket identifier in a target storage space according to a target bitmap identifier indicated by the computational logic information, the target storage space storing a bitmap identifier, a bucket identifier, and bucket content with a corresponding relationship, the bitmap identifier being used to characterize the classification of user data, and the bucket identifier and bucket content being used to reconstruct the user data; and a sending module for processing the bucket content in the target storage space corresponding to the target bucket identifier according to the logical operation indicated by the computational logic information to obtain a query result; and a sending module for feeding back the query result.

[0006] Thirdly, embodiments of this application provide an electronic device, including: a processor and a memory storing computer program instructions; the processor executes the computer program instructions to implement the user data query method of the first aspect.

[0007] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the user data query method of the first aspect.

[0008] This application provides a user data query method, apparatus, device, and storage medium. It pre-stores corresponding bitmap identifiers, bucket identifiers, and bucket contents in a target storage space. The bitmap identifier represents the classification of user data, and the bucket identifiers and bucket contents can be used to reconstruct the user data. Based on the target bitmap identifier indicated by the computational logic information in the query request, a target bucket identifier is determined in the target storage space. According to the logical operations indicated by the computational logic information, the bucket contents corresponding to the target bucket identifier are processed to obtain the query result corresponding to the query request. In the above user data query process, user data is stored in buckets using bitmap identifiers, bucket identifiers, and bucket contents. Each query can directly obtain the bucket identifier and bucket contents associated with the current query and perform the data operations required for the logical operations indicated by the query request, without needing to traverse all data. This shortens the time required for user data queries and improves query efficiency. Attached Figure Description

[0009] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A flowchart illustrating a user data query method provided in one embodiment of this application;

[0011] Figure 2 A schematic diagram illustrating an example of bucketed storage corresponding to some of the data in Table 2, provided in an embodiment of this application;

[0012] Figure 3 A flowchart illustrating a user data query method provided in another embodiment of this application;

[0013] Figure 4 A flowchart illustrating a user data query method provided in yet another embodiment of this application;

[0014] Figure 5 A schematic diagram illustrating an example of various methods for identifying storage buckets and their contents as provided in embodiments of this application;

[0015] Figure 6 A logical diagram illustrating an example of the application architecture for the user data query method provided in this application embodiment;

[0016] Figure 7 This is a schematic diagram of the structure of a user data query device provided in an embodiment of this application;

[0017] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0018] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples. It should be noted that the acquisition, storage, use, and processing of information and data in the embodiments of this application are all authorized by users or relevant organizations and comply with the relevant provisions of national laws and regulations.

[0019] With the continuous development of electronic information technology, more and more user data involved in business operations can be managed through electronic information technology. To facilitate the processing of user data, it can be divided according to user characteristics and managed accordingly. User data can be stored using a bitmap method based on user identifiers. However, in scenarios where user data is queried, it is necessary to traverse all data stored in the bitmap method. Furthermore, as business operations continue to upgrade, the dimensions and volume of user data are constantly expanding, resulting in long query times and low query efficiency. Even using efficient compressed bitmaps (i.e., RoaringBitmap), querying user data still requires traversing all user data, especially in scenarios with large user data volumes and multiple user classifications, where query time will be even longer and query efficiency will be even lower.

[0020] This application provides a method, apparatus, device, and storage medium for querying user data. It pre-stores corresponding bitmap identifiers, bucket identifiers, and bucket contents in a target storage space. The bitmap identifier represents the classification of user data, and the bucket identifiers and bucket contents can be used to reconstruct the user data. Based on a query request, the bucket identifier corresponding to the bitmap identifier indicated by the query request can be obtained from the target storage space. According to the logical operations indicated by the query request, the bucket contents corresponding to the obtained bucket identifier are processed to obtain the query result corresponding to the query request. In the above query process, user data is stored in buckets using bitmap identifiers, bucket identifiers, and bucket contents. Each query can directly obtain the bucket identifier and bucket contents associated with the current query and perform the data operations required for the logical operations indicated by the query request, without traversing all data. This shortens the time required for data input / output (IO), thereby reducing the time required for user data query and improving query efficiency.

[0021] The following describes the query method, apparatus, device, and storage medium for user data provided in this application.

[0022] The first aspect of this application provides a method for querying user data, applicable to user selection scenarios under multiple category labels, such as real-time user selection scenarios with a large number of users. This user data query method can be executed by a user data query device or equipment, and is not limited thereto. Figure 1 A flowchart illustrating a user data query method provided in one embodiment of this application is shown below. Figure 1 As shown, the method for querying user data may include steps S101 to S103.

[0023] In step S101, a query request is received.

[0024] A query request is used to retrieve user data. The query request includes computational logic information. This computational logic information can indicate logical operations and the categories of user data associated with those operations. The computational logic information can be considered as the implementation of query conditions. For example, a query request might indicate a query for user data of users aged 20 to 40 in city A1. The logical operations indicated by the computational logic information in the query request could include finding the intersection of user data under the city A1 category and user data under the age-20 to 40 category. Correspondingly, the categories of user data associated with the logical operations indicated by the computational logic information in the query request include the city A1 category and the 20-40 age category.

[0025] In step S102, the target bucket identifier is determined in the target storage space according to the target bitmap identifier indicated by the computational logic information.

[0026] The target storage space stores corresponding bitmap identifiers, bucket identifiers, and bucket contents. Bitmap identifiers represent the classification of user data and can be set according to the classification labels of the user data. For example, user data is classified according to gender, age, etc. For gender, the classification labels can include "male" and "female," corresponding to bitmap identifiers "man" and "woman." For age, the classification labels can include "0-20 years old," "21-30 years old," "31-40 years old," "41-50 years old," and "51 years and older," corresponding to bitmap identifiers "age_0_20," "age_21_30," "age_31_40," "age_41_50," and "age_51." Bucket identifiers and bucket contents are used to reconstruct the user data. Each piece of user data can be divided into two parts: one part forms the bucket identifier, and the other part forms the bucket contents. The bucket identifiers and bucket contents for the same piece of user data have a corresponding relationship. In some examples, user data may include high N bits and low M bits. The bucket identifier is derived from the high N bits of the user data, and the bucket content is derived from the low M bits. N and M are both positive integers, and their specific values ​​can be set according to the scenario, requirements, experience, etc. The values ​​of N and M determine the range of data that the target storage space can store. For example, if N = M = 16, then the target storage space can store 32 bits of data, that is, the range of data that the target storage space can store is 0 to 4294967295. It should be noted that if the number of bits in the user data is less than N+M bits, zeros can be padded at the high bits of the user data to obtain user data with a number of bits sufficient for N+M bits, thus obtaining the bucket identifier and bucket content based on the user data. For example, if N=M=16, user data can include user identifiers. If the user identifier is c00000704004, the prefix c00 can be removed, and 000704004 can be converted into 32-bit binary data "00000000000010101011111000000100". The high 16 bits "0000000000001010" can be converted into decimal data "10" as the bucket identifier, and the low 16 bits "1011111000000100" can be converted into decimal data "48644" as the bucket content. If the bucket identifiers and bucket content with corresponding relationships are obtained, the user identifier can be restored.

[0027] In some examples, the corresponding bitmap identifiers, bucket identifiers, and bucket contents can be stored based on the correspondence between bitmap identifiers and bucket identifiers, and the correspondence between bitmap identifiers, bucket identifiers, and bucket contents. In the correspondence between bitmap identifiers and bucket identifiers, the bitmap identifier and bucket identifier are treated as two objects, describing the correspondence between these two objects. In the correspondence between bitmap identifiers, bucket identifiers, and bucket contents, the bitmap identifier and bucket identifier can be treated as one object, and the bucket contents as another object, describing the correspondence between these two objects. Specifically, the target storage space stores pairs of keys and values. In the key-value pairs, if the key includes a bitmap identifier, the value includes the bucket identifier corresponding to the key; if the key includes both a bitmap identifier and a bucket identifier, the value includes the bucket contents corresponding to the key.

[0028] For example, user data can be shown in Table 1 below:

[0029] Table 1

[0030] c00000000001 male 21 0 1 c00000010001 male 46 0 1001 c00000704004 male 55 10 48644 c00003050003 male 33 46 35347 c00004010002 male 42 61 12306 c00032016002 male 23 488 34434 c00000004001 female 35 0 4001 c00001010001 female 19 15 26961 c00003050002 female 22 46 35346 c00032010002 female 57 488 28434

[0031] Table 1 shows the user identifier, user's gender, user's age, and the bucket identifier and bucket content derived from the user identifier. If the user identifier is categorized by gender and age, the keys and values ​​stored in the target storage space can be as shown in Table 2 below:

[0032] Table 2

[0033]

[0034]

[0035] In this table, "man" and "woman" are bitmap identifiers, representing the user's gender as "male" and "female," respectively; "age_0_20," "age_21_30," "age_31_40," "age_41_50," and "age_51" are bitmap identifiers, representing the user's age as "0 to 20 years old," "21 to 30 years old," "31 to 40 years old," "41 to 50 years old," and "51 years old and above," respectively. Keys including bitmap identifiers and bucket identifiers are represented in Table 2 as "bitmap identifier: bucket identifier," such as "man:0," where "man" is the bitmap identifier and "0" is the bucket identifier. Other keys in Table 2 that include bitmap identifiers and bucket identifiers are not listed here. Figure 2 This is a schematic diagram illustrating an example of bucketed storage corresponding to some of the data in Table 2, as provided in an embodiment of this application. Figure 2As shown, the bin identifiers corresponding to males include 488, 61, 46, 10, and 0. The bin content corresponding to male bin identifier 488 includes 34434, the bin content corresponding to male bin identifier 61 includes 12306, the bin content corresponding to male bin identifier 46 includes 35347, the bin content corresponding to male bin identifier 10 includes 48644, and the bin content corresponding to male bin identifier 0 includes 1 and 1001. The bin identifiers corresponding to females include 488, 46, 15, and 0. The bin content corresponding to female bin identifier 488 includes 28434, the bin content corresponding to female bin identifier 46 includes 35346, the bin content corresponding to female bin identifier 15 includes 26961, and the bin content corresponding to female bin identifier 0 includes 4001.

[0036] The bitmap identifier associated with the logical operation indicated by the computational logic information is the target bitmap identifier indicated by the computational logic information. Based on the correspondence between the target bitmap identifier and the bitmap identifiers and bucket identifiers in the target storage space, the bucket identifier corresponding to the target bitmap identifier can be determined as the target bucket identifier. For example, if the computational logic information in the query request indicates the intersection of user data for users aged 21 to 30 and user data for users who are male, and if the data stored in the target storage space is as shown in Table 2, then the corresponding target bitmap identifiers include "man" and "age_21_30", and the target bucket identifiers corresponding to the target bitmap identifiers may include "0", "10", "46", "61", and "488".

[0037] In step S103, the contents of the bucket corresponding to the target bucket identifier in the target storage space are processed according to the logical operation indicated by the calculation logic information to obtain the query result and feedback.

[0038] The processing of the content corresponding to the target bucket identifier differs depending on the logical operation performed. That is, the processing of the content corresponding to the target bucket identifier is related to the logical operation indicated by the computational logic information. Processing the content corresponding to the target bucket identifier yields at least a portion of the bucket content corresponding to the target bucket identifier. Based on the target bucket identifier and the processed portion of the bucket content, the user data can be reconstructed. Query results can be generated based on the reconstructed user data and sent back to the initiator of the query request. The query results may include the processed bucket content and the user data reconstructed from the target bucket identifier. The query results may also include, but are not limited to, the quantity of reconstructed user data and other information related to the reconstructed user data.

[0039] In this embodiment, bitmap identifiers, bucket identifiers, and bucket contents with corresponding relationships can be pre-stored in the target storage space. The bitmap identifier is used to characterize the classification of user data, and the bucket identifier and bucket contents can be used to reconstruct the user data. Based on the target bitmap identifier indicated by the computational logic information in the query request, a target bucket identifier is determined in the target storage space. According to the logical operations indicated by the computational logic information, the bucket contents corresponding to the target bucket identifier are processed to obtain the query result corresponding to the query request. In the above user data query process, user data is stored in buckets using bitmap identifiers, bucket identifiers, and bucket contents. Each query can directly obtain the bucket identifier and bucket contents associated with the current query and perform the data operations required for the logical operations indicated by the query request, without needing to traverse all data. This shortens the time required for user data queries and improves query efficiency.

[0040] In some embodiments, complex logical operations indicated by computational logic information can be broken down into binary logical operations. Based on the binary logical operations and the target bucket identifier associated with the binary logical operations, the data operations that need to be performed on the bucket content can be determined so as to obtain query results through data operations. Figure 3 A flowchart illustrating a user data query method provided in another embodiment of this application. Figure 3 and Figure 1 The difference is that, Figure 1 Step S103 can be further refined as follows: Figure 3 Steps S1031 to S1034 in the process.

[0041] In step S1031, the logical operation indicated by the computational logic information is split into binary logical operations.

[0042] In some cases, the logical operations indicated by the computational logic information are relatively complex, with more than three associated target bitmap identifiers. In such cases, to facilitate processing, the logical operations indicated by the computational logic information can be broken down into binary logical operations, with each binary logical operation associated with two target bitmap identifiers, making processing easier.

[0043] In step S1032, the target bucket identifier associated with the binary logic operation is obtained.

[0044] There are two target bitmap identifiers associated with the binary logic operation. The target bucket identifiers corresponding to these two target bitmap identifiers are the target bucket identifiers associated with the binary logic operation.

[0045] In step S1033, the data operation is determined based on the binary logic operation and the target bucket identifier associated with the binary logic operation.

[0046] Different binary logical operations may correspond to different data operations performed on the content of the bucket corresponding to the target bucket identifier. In some examples, the data operation can be determined based on whether the binary logical operation and the target bucket identifier associated with the binary logical operation are the same. If the target bucket identifiers associated with the binary logical operation are the same, the data operation performed on the content of the bucket corresponding to the same target bucket identifier may include the binary logical operation; if the target bucket identifiers associated with the binary logical operation are different, the data operation performed on the content of the bucket corresponding to different target bucket identifiers may include additional operations, which may include, but are not limited to, copying, negating, and discarding operations.

[0047] In step S1034, data operations are performed on the contents of the buckets that correspond to the target bucket identifier to obtain query results and provide feedback.

[0048] By performing the data operations determined in step S1033 on the bucket content that corresponds to the target bucket identifier, at least a portion of the bucket content can be obtained. The user data can be restored using the obtained bucket content and the target bucket identifier that corresponds to the obtained bucket content, and the query results can be obtained based on the restored user data.

[0049] In some examples, to further improve the speed of user data retrieval, data operations can be performed in parallel on the contents of the buckets corresponding to the target bucket identifiers associated with multiple binary logical operations, provided that the logical operations indicated by the computational logic information include multiple binary logical operations. It should be noted that the binary logical operations corresponding to the parallelizable data operations are independent of each other. If one binary logical operation requires the result of another binary logical operation, the data operations corresponding to these two binary logical operations cannot be executed in parallel.

[0050] To facilitate understanding of the above content, a specific example is provided here. User data is shown in Table 1 above, and the storage method of user data in the target storage space is shown in Table 2 above. If the logical operation indicated by the calculation logic information in the query request is to find user data for users who are male and aged 0 to 20 or 31 to 40, then this logical operation can be broken down into two binary logical operations. The first binary logical operation is to find the union of user data for users aged 0 to 20 and user data for users aged 31 to 40. The second binary logical operation is to find the intersection of the result of the first binary logical operation and user data for users who are male.

[0051] The first binary logic operation process may include steps a1 to a3.

[0052] In step a1, the target bitmap identifier "age_0_20" corresponding to ages 0 to 20 and the target bitmap identifier "age_31_40" corresponding to ages 31 to 40 are obtained. The target bucket identifier "age_0_20" is "15", and the target bucket identifier "age_31_40" is "0" and "46".

[0053] In step a2, the first binary logical operation is a union operation; for the same target bucket identifier, all bucket contents corresponding to the same target bucket identifier can be retrieved, that is, data processing for union operation is performed; for different target bucket identifiers, the bucket contents corresponding to different target bucket identifiers can be copied, that is, data processing for copy operation is performed.

[0054] In step a3, the target bucket identifiers are not the same in this example. According to the data processing determined in step a2, the bucket content "26961" corresponding to "age_0_20:15", the bucket content "4001" corresponding to "age_31_40:0", and the bucket content "35347" corresponding to "age_31_40:46" are copied in parallel. The result of the first binary logic operation includes bucket identifier "15" and bucket content "26961", bucket identifier "0" and bucket content "4001", and bucket identifier "46" and bucket content "35347".

[0055] The second binary logic operation process may include steps b1 to b3.

[0056] In step b1, the target bitmap identifier "man" for male is obtained. The target bucket identifiers corresponding to the target bitmap identifier "man" are "0", "10", "46", "61" and "488".

[0057] In step b2, the operator of the second binary logic operation is the intersection operation; for a target bucket identifier that is the same as the bucket identifier in the result of the first binary logic operation, the bucket content corresponding to the same target bitmap identifier can be used for the intersection operation; for a target bucket identifier that is different from the bucket identifier in the result of the first binary logic operation, the bucket content corresponding to the target bucket identifier is discarded.

[0058] In step b3, the same target bucket identifiers in this example include "0" and "46". According to the data processing determined in step b2, the bucket content "4001" corresponding to "age_31_40:0" and the bucket content "1" and "1001" corresponding to "man:0" have no intersection in the result of the first binary logic operation. The intersection of the bucket content "35347" corresponding to "age_31_40:46" and the bucket content "35347" corresponding to "man:46" is "35347". The buckets corresponding to the different target bitmap identifiers "10", "61" and "488" in this example are discarded. The result of the second binary logic operation includes the bucket identifier "46" and the bucket content "35347".

[0059] Based on the result of the second binary logical operation, the query result can be determined to include the user identifier "c00003050003" corresponding to the bucket identifier "46" and the bucket content "35347". The query result may also include the number of user data found. In this example, the number of user data found in the query result is 1.

[0060] In some embodiments, the bitmap identifier, bucket identifier, and bucket content with corresponding relationships in the target storage space can be updated based on the query results obtained from each query request. Figure 4 A flowchart illustrating a user data query method provided in another embodiment of this application. Figure 4 and Figure 1 The difference is that, Figure 4 The method for querying user data shown may also include steps S104 and S105.

[0061] In step S104, a new bitmap identifier is generated based on the calculation logic information, and a correspondence is established between the bucket identifier and bucket content corresponding to the query result and the new bitmap identifier.

[0062] In step S105, the new bitmap identifier for establishing the correspondence, the bucket identifier corresponding to the query result, and the bucket content are stored in the target storage space.

[0063] After obtaining the query results, a new bitmap identifier can be generated based on the computational logic information in the query request corresponding to the query results. This new bitmap identifier represents the computational logic information. A correspondence is established between the bucket identifier and bucket content corresponding to the query results and this new bitmap identifier, and stored in the target storage space. If a subsequent query request containing the computational logic information is received, the bucket identifier and bucket content corresponding to the new bitmap identifier can be directly retrieved, thus quickly obtaining the query results and further improving the speed and efficiency of user data retrieval. For example, in the above example, if the computational logic information indicates a search for user data of male users aged 0 to 20 or 31 to 40, a new bitmap identifier "Crowd0001" can be generated. Based on the bucket identifier and bucket content corresponding to the bitmap identifier "Crowd0001", the user data obtained can be reconstructed to include user data of male users aged 0 to 20 or 31 to 40.

[0064] In some embodiments, bucket identifiers and bucket contents can be stored in buckets. Based on the data volume of the bucket identifier, the data volume of the bucket contents, and the pre-defined bucket data volume, a corresponding storage method is used to store the bucket identifiers and bucket contents, thereby reducing the space occupied by the bucket identifiers and bucket contents in the target storage space. For example, Figure 5 A schematic diagram illustrating an example of various methods for identifying storage buckets and defining bucket contents provided in embodiments of this application, such as... Figure 5 As shown, the D2 data corresponding to the D1 data can be stored in compressed form, bitmap form, and array form; Figure 5 The data D2 corresponding to data number 65534 in D1 is stored in compressed form. (0,10) means that data numbers 0 to 10 are stored in compressed form. Similarly, (30,6000) means that data numbers 30 to 6000 are stored in compressed form. Figure 5 The data D2, which corresponds to data number 14672 in the D1 data, is stored in bitmap format. Figure 5 The data D2, corresponding to data point 2 in data D1, is stored as an array. If data D1 includes a bitmap identifier, data D2 may include a bucket identifier; if data D1 includes a bucket identifier, data D2 may include the bucket content.

[0065] In the target storage space, if the number of bucket identifiers corresponding to a bitmap identifier does not exceed the preset bucket data size, the bucket identifiers corresponding to the bitmap identifier are stored in array form; if the number of bucket identifiers corresponding to a bitmap identifier exceeds the preset bucket data size, the bucket identifiers corresponding to the bitmap identifier are stored in bitmap form. The preset bucket data size can be determined based on the single bucket capacity. Under the condition of single bucket capacity, if the number of bucket identifiers does not exceed the preset bucket data size, the space occupied by storing bucket identifiers in array form is less than the space occupied by storing bucket identifiers in bitmap form; if the number of bucket identifiers exceeds the preset bucket data size, the space occupied by storing bucket identifiers in array form is greater than the space occupied by storing bucket identifiers in bitmap form.

[0066] Similarly, in the target storage space, if the number of contents corresponding to the bitmap identifier and the bucket identifier does not exceed the preset bucket data size, the contents corresponding to the bitmap identifier and the bucket identifier are stored in array form; if the number of contents corresponding to the bitmap identifier and the bucket identifier exceeds the preset bucket data size, the contents corresponding to the bitmap identifier and the bucket identifier are stored in bitmap form. The preset bucket data size corresponding to the bucket content can be the same as the preset bucket data size corresponding to the bucket identifier. Under the condition of single bucket capacity, if the number of contents in the bucket does not exceed the preset bucket data size, the space occupied by storing the contents in array form is less than the space occupied by storing the contents in bitmap form; if the number of contents in the bucket exceeds the preset bucket data size, the space occupied by storing the contents in array form is greater than the space occupied by storing the contents in bitmap form.

[0067] For example, if the capacity of a single bucket is 8KB (i.e., kilobytes), then the preset bucket data volume can be 4096. When the number of bucket identifiers or bucket contents is less than or equal to 4096, the bucket identifiers or bucket contents can be stored in an array format as short type variables, with each bucket storing 4096 * 2 bytes (i.e., bytes) of data; when the number of bucket identifiers or bucket contents is greater than 4096, the bucket identifiers or bucket contents can be stored in a bitmap format, with each bucket storing 65536 * 1 bit (i.e., bits) of data.

[0068] In some examples, when bucket identifiers are stored in bitmap format, if there are consecutive bucket identifiers and the number of consecutive bucket identifiers exceeds the compression standard, the consecutive bucket identifiers are stored in a compressed manner. Similarly, when bucket content is stored in bitmap format, if there are consecutive bucket contents and the number of consecutive bucket contents exceeds the compression standard, the consecutive bucket contents are stored in a compressed manner. The compression standard is the minimum amount of data stored in compressed form that is less than the amount of data stored in uncompressed form. Bucket identifiers or bucket contents must be consecutive to be stored in compressed form. Furthermore, to minimize the space occupied by data storage, when the number of consecutive bucket identifiers or consecutive bucket contents exceeds the compression standard, compressed storage is used. In this case, the space occupied by compressed storage of bucket identifiers or bucket contents is less than the space occupied by uncompressed storage. Correspondingly, if the number of consecutive bucket identifiers or bucket contents stored in bitmap form does not exceed the compression standard, then the space occupied by storing bucket identifiers or bucket contents in an uncompressed manner is less than the space occupied by storing bucket identifiers or bucket contents in a compressed manner.

[0069] By considering the relationship between the number of bucket identifiers, the amount of bucket content, and the preset bucket data volume, a storage method is selected that reduces the space required to store bucket identifiers and bucket content. This minimizes the space occupied by bucket identifiers and bucket content that represent user data, especially in scenarios with very large user data volumes, such as hundreds of millions of user data. This can significantly reduce the space occupied by user data storage under each bitmap identifier. For example, user data under a single bitmap identifier for 1 billion users would require hundreds of megabytes of space to store without the storage method of bucket identifiers and bucket content described in this embodiment. However, with the storage method of bucket identifiers and bucket content described in this embodiment, the space occupied by user data under a single bitmap identifier for 1 billion users can be reduced by more than 73%, significantly reducing the space occupied by user data storage.

[0070] In the above embodiments, the target storage space may include an in-memory database and / or a cache space. The in-memory database may store the full set of corresponding bitmap identifiers, bucket identifiers, and bucket contents. The cache space may store a portion of the corresponding bitmap identifiers, bucket identifiers, and bucket contents. In some examples, where the target storage space includes a cache space, the cache space stores corresponding bitmap identifiers, bucket identifiers, and bucket contents that meet predetermined conditions. The predetermined conditions can be set according to scenarios, requirements, experience, etc. For example, the predetermined conditions may include the cache space storing a specified bitmap identifier and the bucket identifiers and bucket contents that correspond to the specified bitmap identifier. Another example is that the predetermined conditions may include the cache space storing the L most frequently queried bitmap identifiers and the bucket identifiers and bucket contents that correspond to the L most frequently queried bitmap identifiers, where L is a positive integer; storing the frequently queried bitmap identifiers and their corresponding bucket identifiers and bucket contents in the cache space can further improve the query speed and efficiency of user data.

[0071] The following example illustrates the logical architecture of the user data query method provided in this application embodiment. Figure 6 A logical diagram illustrating an example of the application architecture for the user data query method provided in this application embodiment, such as... Figure 6 As shown, the methods for querying user data may involve offline database 21, in-memory database 22, and memory 23.

[0072] Various types of user data can be retrieved from offline database 21, and bitmap identifiers, bucket identifiers, and bucket content with corresponding relationships can be generated based on the user data. For example, such as... Figure 6 As shown, user data such as user basic information table 211 and user transaction data table 212 can be obtained to generate a tag-bitmap data table 213, which includes bitmap identifiers, bucket identifiers, and bucket contents with corresponding relationships. The offline database 21 can synchronize the bitmap identifiers, bucket identifiers, and bucket contents with corresponding relationships to the in-memory database 22.

[0073] The in-memory database 22 stores bitmap identifiers, bucket identifiers, and bucket contents with corresponding relationships. The correspondence between bitmap identifiers, bucket identifiers, and bucket contents in the in-memory database 22 can be reflected by the correspondence between bitmap identifiers and bucket identifiers 221, and the correspondence between bitmap identifiers, bucket identifiers, and bucket contents 222.

[0074] Memory 23 may have a request parsing function 231, a bitmap logic operation function 232, and a cache space 233. The cache space 233 stores bitmap identifiers, bucket identifiers, and bucket contents with corresponding relationships, obtained from the memory database 22. Memory 23 can receive query requests, parse the query requests using the request parsing function 231, and pass the parsed logic operations to the bitmap logic operation function 232. The bitmap logic operation function 232, based on the logic operations, obtains the bucket identifiers and bucket contents required for the logic operations from the memory database 22 or the cache space 233, performs the logic operations, and obtains the query results. The bitmap logic operation function 232 can feed back the query results to the request parsing function 231, which then feeds back the query results to the querying party. The request parsing function 231 can synchronize the new bitmap identifier corresponding to the query request to the memory database 22. The bitmap logic operation function 232 can also synchronize the query results to the memory database 22. The memory database 22 can store the new bitmap identifiers and the bucket identifiers and bucket contents corresponding to the query results.

[0075] In this embodiment, user data can be stored separately according to bucket identifiers and bucket contents. By leveraging the high responsiveness of memory storage and the convenient data sharding and bitwise operations provided by bucket identifiers and bucket contents, efficient and accurate real-time querying and selection of multiple types of users can be achieved in scenarios with massive user numbers, thereby improving the effectiveness of refined management and analysis of user data.

[0076] A second aspect of this application provides a device for querying user data. Figure 7 This is a schematic diagram of the structure of a user data query device provided in an embodiment of this application, as shown below. Figure 7 As shown, the user data query device 300 may include a receiving module 301, a logic processing module 302, and a sending module 303.

[0077] The receiving module 301 can be used to receive query requests. The query request includes computational logic information.

[0078] The logic processing module 302 can be used to determine the target bucket identifier in the target storage space according to the target bitmap identifier indicated by the computational logic information. The target storage space stores bitmap identifiers, bucket identifiers, and bucket contents with corresponding relationships. The bitmap identifier is used to represent the classification of user data, and the bucket identifier and bucket contents are used to reconstruct the user data. It can also be used to process the bucket contents in the target storage space corresponding to the target bucket identifier according to the logical operation indicated by the computational logic information to obtain the query result.

[0079] The sending module 303 can be used to provide feedback on the query results.

[0080] In this embodiment, bitmap identifiers, bucket identifiers, and bucket contents with corresponding relationships can be pre-stored in the target storage space. The bitmap identifier is used to characterize the classification of user data, and the bucket identifier and bucket contents can be used to reconstruct the user data. Based on the target bitmap identifier indicated by the computational logic information in the query request, a target bucket identifier is determined in the target storage space. According to the logical operations indicated by the computational logic information, the bucket contents corresponding to the target bucket identifier are processed to obtain the query result corresponding to the query request. In the above user data query process, user data is stored in buckets using bitmap identifiers, bucket identifiers, and bucket contents. Each query can directly obtain the bucket identifier and bucket contents associated with the current query and perform the data operations required for the logical operations indicated by the query request, without needing to traverse all data. This shortens the time required for user data queries and improves query efficiency.

[0081] In some embodiments, the logic processing module 302 may be specifically used to: split the logic operation indicated by the computation logic information into binary logic operations; obtain the target bucket identifier associated with the binary logic operation; determine the data operation based on the binary logic operation and the target bucket identifier associated with the binary logic operation; perform data operations on the bucket content that corresponds to the target bucket identifier, obtain the query result and provide feedback.

[0082] In some examples, the logic processing module 302 may specifically be used to: determine data operations based on binary logic operations and whether the target bucket identifier associated with the binary logic operations is the same.

[0083] In some examples, the logic processing module 302 may specifically be used to: perform data operations in parallel on the bucket contents corresponding to the target bucket identifier associated with the multiple binary logic operations when the logic operation indicated by the computation logic information includes multiple binary logic operations.

[0084] In some embodiments, the logic processing module 302 can also be used to: generate a new bitmap identifier based on the calculation logic information, and establish a correspondence between the bucket identifier and bucket content corresponding to the query result and the new bitmap identifier; and store the new bitmap identifier with the established correspondence, the bucket identifier and bucket content corresponding to the query result into the target storage space.

[0085] In some examples, the target storage space stores pairs of keys and values. In the key-value pairs, if the key includes a bitmap identifier, the value includes a bucket identifier corresponding to the key; if the key includes both a bitmap identifier and a bucket identifier, the value includes the contents of the bucket corresponding to the key.

[0086] In some examples, the target storage space includes an in-memory database and / or a cache space. When the target storage space includes a cache space, the cache space stores corresponding bitmap identifiers, bucket identifiers, and bucket contents that meet predetermined conditions.

[0087] In some embodiments, user data includes high N bits of data and low M bits of data. The bucket identifier is obtained based on the high N bits of user data, and the bucket content is obtained based on the low M bits of user data, where N and M are both positive integers.

[0088] In the target storage space, if the number of bucket identifiers corresponding to the bitmap identifier does not exceed the preset bucket data volume, the bucket identifiers corresponding to the bitmap identifier are stored in array form; if the number of bucket identifiers corresponding to the bitmap identifier exceeds the preset bucket data volume, the bucket identifiers corresponding to the bitmap identifier are stored in bitmap form.

[0089] In the target storage space, if the number of contents in the bucket corresponding to the bitmap identifier and the bucket identifier does not exceed the preset bucket data volume, the contents in the bucket corresponding to the bitmap identifier and the bucket identifier are stored in array form; if the number of contents in the bucket corresponding to the bitmap identifier and the bucket identifier exceeds the preset bucket data volume, the contents in the bucket corresponding to the bitmap identifier and the bucket identifier are stored in bitmap form.

[0090] In some examples, if consecutive bucket identifiers exist in a bitmap-based storage format and the number of consecutive bucket identifiers exceeds the compression standard, the consecutive bucket identifiers are stored using compression. Similarly, if consecutive bucket content exists in a bitmap-based storage format and the number of consecutive bucket content exceeds the compression standard, the consecutive bucket content is stored using compression.

[0091] A third aspect of this application also provides an electronic device. Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 8 As shown, the electronic device 400 includes a memory 401, a processor 402, and a computer program stored in the memory 401 and executable on the processor 402.

[0092] In some examples, the processor 402 described above may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits that may be configured to implement the embodiments of this application.

[0093] Memory 401 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the user data query method according to embodiments of this application.

[0094] The processor 402 runs a computer program corresponding to the executable program code by reading the executable program code stored in the memory 401, so as to implement the user data query method in the above embodiment.

[0095] In some examples, electronic device 400 may also include a communication interface 403 and a bus 404. For example, Figure 8 As shown, the memory 401, processor 402, and communication interface 403 are connected through bus 404 and complete communication with each other.

[0096] The communication interface 403 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application. Input devices and / or output devices can also be connected through the communication interface 403.

[0097] Bus 404 includes hardware, software, or both, that couples components of electronic device 400 together. For example, and not limitingly, bus 404 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-E) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 404 may include one or more buses. Although specific buses are described and illustrated in the embodiments of this application, this application considers any suitable bus or interconnection.

[0098] A fourth aspect of this application also provides a computer-readable storage medium storing computer program instructions. When these computer program instructions are executed by a processor, they can implement the user data query method described in the above embodiments and achieve the same technical effect. To avoid repetition, further details are omitted here. The aforementioned computer-readable storage medium may include non-transitory computer-readable storage media, such as read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, etc., and is not limited thereto.

[0099] This application provides a computer program product. When the instructions in the computer program product are executed by the processor of an electronic device, the electronic device performs the user data query method described in the above embodiment and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0100] It should be clarified that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. For the device embodiments, equipment embodiments, computer-readable storage medium embodiments, and computer program product embodiments, the relevant parts can be referred to the description section of the method embodiments. This application is not limited to the specific steps and structures described above and shown in the figures. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application. Furthermore, for the sake of brevity, detailed descriptions of known methods and techniques are omitted here.

[0101] The aspects of this application have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0102] Those skilled in the art will understand that the above embodiments are exemplary and not restrictive. Different technical features appearing in different embodiments can be combined to achieve beneficial effects. Based on a study of the drawings, specification, and claims, those skilled in the art should be able to understand and implement other variations of the disclosed embodiments. In the claims, the term "comprising" does not exclude other means or steps; the quantifier "a" does not exclude a plurality; the terms "first" and "second" are used to identify names and not to indicate any particular order. No reference numerals in the claims should be construed as limiting the scope of protection. The functionality of multiple parts appearing in the claims can be implemented by a single hardware or software module. The appearance of certain technical features in different dependent claims does not mean that these technical features cannot be combined to achieve beneficial effects.

Claims

1. A method for querying user data, characterized in that, include: Receive a query request, the query request including computational logic information; Based on the target bitmap identifier indicated by the computational logic information, a target bucket identifier is determined in the target storage space. The target storage space stores bitmap identifiers, bucket identifiers, and bucket contents with corresponding relationships. The bitmap identifier is used to characterize the classification of user data, and the bucket identifier and bucket contents are used to reconstruct the user data. The user data includes high N bits of data and low M bits of data. The bucket identifier is obtained based on the high N bits of the user data, and the bucket contents are obtained based on the low M bits of the user data. N and M are both positive integers. Based on the logical operations indicated by the computational logic information, the contents of the bucket corresponding to the target bucket identifier in the target storage space are processed to obtain the query results and feedback. Specifically, logical operations are broken down into binary logical operations. When the binary logical operation operator is a union operation, for the same target bucket identifier, all bucket contents corresponding to the same target bucket identifier are taken; for different target bucket identifiers, the bucket contents corresponding to different target bucket identifiers are copied. When the binary logical operation operator is an intersection operation, for the same target bucket identifier, the bucket contents corresponding to the same target bitmap identifier are taken and the intersection operation is performed; for different target bucket identifiers, the bucket contents corresponding to the target bucket identifier are discarded.

2. The method according to claim 1, characterized in that, The step of processing the bucket content corresponding to the target bucket identifier in the target storage space according to the logical operation indicated by the computational logic information, obtaining the query result and feeding it back, includes: The logical operation indicated by the computational logic information is broken down into binary logical operations; Obtain the target bucket identifier associated with the binary logical operation; The data operation is determined based on the binary logic operation and the target bucket identifier associated with the binary logic operation; The data operation is performed on the bucket content that corresponds to the target bucket identifier to obtain the query results and provide feedback.

3. The method according to claim 2, characterized in that, The step of determining the data operation based on the binary logical operation and the target bucket identifier associated with the binary logical operation includes: The data operation is determined based on the binary logical operation and whether the target bucket identifier associated with the binary logical operation is the same.

4. The method according to claim 2, characterized in that, The data operation performed on the bucket content corresponding to the target bucket identifier includes: When the logical operation indicated by the computational logic information includes multiple binary logical operations, the data operation is performed in parallel on the bucket content corresponding to the target bucket identifier associated with the multiple binary logical operations.

5. The method according to claim 1, characterized in that, Also includes: A new bitmap identifier is generated based on the computational logic information, and a correspondence is established between the bucket identifier and bucket content corresponding to the query result and the new bitmap identifier. The new bitmap identifier that establishes the corresponding relationship, the bucket identifier corresponding to the query result, and the bucket content are stored in the target storage space.

6. The method according to claim 1, characterized in that, The target storage space stores pairs of keys and values; In a key-value pair, if the key includes a bitmap identifier, the value includes a bucket identifier corresponding to the key; if the key includes both a bitmap identifier and a bucket identifier, the value includes the contents of the bucket corresponding to the key.

7. The method according to claim 1, characterized in that, The target storage space includes an in-memory database and / or cache space; When the target storage space includes a cache space, the cache space stores bitmap identifiers, bucket identifiers, and bucket contents that meet predetermined conditions and have corresponding relationships.

8. The method according to claim 1, characterized in that, In the target storage space, If the number of bucket identifiers corresponding to the bitmap identifier does not exceed the preset bucket data size, the bucket identifiers corresponding to the bitmap identifier are stored in array form. If the number of bucket identifiers corresponding to a bitmap identifier exceeds the preset bucket data size, the bucket identifiers corresponding to the bitmap identifier are stored in bitmap format. If the number of contents in the bucket corresponding to the bitmap identifier and the bucket identifier does not exceed the preset bucket data volume, the contents in the bucket corresponding to the bitmap identifier and the bucket identifier are stored in array form. If the number of contents in the bucket corresponding to the bitmap identifier and the bucket identifier exceeds the preset bucket data volume, the contents in the bucket corresponding to the bitmap identifier and the bucket identifier are stored in bitmap form.

9. The method according to claim 8, characterized in that, In the bucket identifiers stored in bitmap format, if there are consecutive bucket identifiers and the number of consecutive bucket identifiers exceeds the compression standard, the consecutive bucket identifiers are stored using compression. In bucketed content stored in bitmap format, if there are consecutive buckets and the number of consecutive buckets exceeds the compression standard, the consecutive buckets are stored using compression.

10. A user data query device, characterized in that, include: A receiving module is used to receive query requests, wherein the query requests include computational logic information; The logic processing module is used to determine the target bucket identifier in the target storage space according to the target bitmap identifier indicated by the calculation logic information. The target storage space stores bitmap identifiers, bucket identifiers and bucket contents with corresponding relationships. The bitmap identifier is used to characterize the classification of user data. The bucket identifier and bucket contents are used to reconstruct the user data. The user data includes high N bits of data and low M bits of data. The bucket identifier is obtained based on the high N bits of the user data, and the bucket contents are obtained based on the low M bits of the user data. N and M are both positive integers. And, to process the contents of the bucket corresponding to the target bucket identifier in the target storage space according to the logical operation indicated by the computational logic information, so as to obtain the query result; The sending module is used to return the query results; Specifically, logical operations are broken down into binary logical operations. When the binary logical operation operator is a union operation, for the same target bucket identifier, all bucket contents corresponding to the same target bucket identifier are taken; for different target bucket identifiers, the bucket contents corresponding to different target bucket identifiers are copied. When the binary logical operation operator is an intersection operation, for the same target bucket identifier, the bucket contents corresponding to the same target bitmap identifier are taken and the intersection operation is performed; for different target bucket identifiers, the bucket contents corresponding to the target bucket identifier are discarded.

11. An electronic device, characterized in that, include: Processor and memory storing computer program instructions; When the processor executes the computer program instructions, it implements the user data query method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the user data query method as described in any one of claims 1 to 9.

Images