Label data storage method and system

By locating user IDs and tag IDs in memory through bit operations, the problem of frequent changes in Schema and low write performance under high concurrency in the prior art is solved, and efficient tag data storage and query is realized.

CN112800062BActive Publication Date: 2024-11-22BEIJING DUWO NETWORK TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110127920.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-01-29
Publication Date
2024-11-22
Estimated Expiration
2041-01-29

Smart Images

  • Figure CN112800062B_ABST
    Figure CN112800062B_ABST
Patent Text Reader

Abstract

The present invention discloses a tag data storage method and system, which divides a user identifier to be stored by the number of users in a user block and takes the integer and the remainder, respectively obtaining a first target user block and a first user offset corresponding to the user identifier to be stored, divides the tag identifier to be stored by 64 and takes the integer to obtain a first tag offset of the tag identifier to be stored, locates the first tag value in the first target user block according to the first user offset and the first tag offset, divides the tag identifier to be stored by 64 and takes the remainder to obtain a first tag position number in the binary bit corresponding to the first tag value of the tag identifier to be stored, sets the binary value corresponding to the first tag position number to 1, and completes adding a tag to be stored to the user to be stored. The present invention performs customized storage for users and tags, does not need to set a separate Schema, and the read and write operation process is realized by directly operating the memory, so it can be regularly persisted to the disk to improve the write performance.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of Internet technology, and more specifically, to a tag data storage method and system. Background Art

[0002] With the widespread popularization of Internet technology, more and more companies will identify online users through some specific tag data, analyze users' online behavior, and then provide targeted services to users.

[0003] In the prior art, when the tag data management platform stores the tag data, it often needs to rely on multiple basic software to implement it. For example, there are solutions based on relational database storage + caching, solutions based on document database storage + caching, and so on. Solutions based on relational database storage + caching often require a fixed schema. Schema stands for mode, which represents a column of a table in the database. It is often determined at the beginning of table construction. However, when the tag categories shown in the column change greatly, the schema needs to be changed frequently. The solution based on document database storage + caching solves the problem of frequent schema changes. However, the solution of document database storage + caching needs to synchronously operate the disk while operating the memory. Therefore, when the number of users and the corresponding number of tags to be stored is large, the writing time is basically measured in hours. For example, a whitelist tag of 700 million people may take 8 hours, so the solution of document database storage + caching takes a lot of time. Summary of the invention

[0004] In view of this, the present invention discloses a tag data storage method and system to realize customized storage for users and tags. It does not need to set a separate schema, and can also add the tag identifier to the user identifier based on a given user identifier and a tag identifier. In addition, the read and write operation process is implemented by directly operating the memory, so it can be persisted to the disk regularly, thereby greatly improving the write performance and improving the write efficiency.

[0005] A tag data storage method, comprising:

[0006] Acquire the tag data to be stored, wherein the tag data to be stored includes: a user identifier to be stored and a tag identifier to be stored;

[0007] Divide the user identifier to be stored by the number of users in the user block and round it up to an integer to obtain a target user block corresponding to the user identifier to be stored, recorded as a first target user block;

[0008] Divide the to-be-stored user identifier by the number of users in the user block and take the remainder, to obtain a user offset of the to-be-stored user identifier in the first target user block, recorded as a first user offset;

[0009] The label identifier to be stored is divided by 64 and rounded to an integer to obtain a label offset of the label identifier to be stored, which is recorded as a first label offset;

[0010] According to the first user offset and the first label offset, a label value is located in the first target user block, recorded as a first label value;

[0011] The tag identifier to be stored is divided by 64 and the remainder is obtained to obtain a tag position number of the tag identifier to be stored in the binary bit corresponding to the first tag value, which is recorded as a first tag position number;

[0012] The binary value corresponding to the first tag position number is set to 1, and the tag to be stored corresponding to the tag identifier to be stored is added to the user to be stored corresponding to the user identifier to be stored.

[0013] Optionally, the first target user block is: Tags[BASE_SIZE][TAG_BLOCK_RANGE]uint64;

[0014] Among them, Tags represents a tag set, BASE_SIZE represents the number of users in the user block, TAG_BLOCK_RANGE represents the number of elements of uint64, uint64 is an unsigned 64-bit integer, and the value is obtained by bit operation, and each bit represents a tag.

[0015] Optionally, the first user offset is: the subscript of the first dimension of the user identifier to be stored in the memory storage structure;

[0016] The memory storage structure is as follows:

[0017] Data[N]Tags[BASE_SIZE][TAG_BLOCK_RANGE]uint64;

[0018] Wherein, Data represents the variable name, N represents the number of one-dimensional array elements, Tags represents the tag set, BASE_SIZE represents the number of users in the user block, TAG_BLOCK_RANGE represents the number of uint64 elements, uint64 is an unsigned 64-bit integer, and the value is obtained by bit operation, and each bit represents a tag.

[0019] Optionally, the first tag offset is: the subscript of the second dimension of the tag identifier to be stored in the memory storage structure.

[0020] Optionally, also include:

[0021] Acquire the tag data to be queried, wherein the tag data to be queried includes: a user identifier to be queried and a tag identifier to be queried;

[0022] Divide the to-be-queried user identifier by the number of users in the user block and round it up to an integer, to obtain a target user block corresponding to the to-be-queried user identifier, recorded as a second target user block;

[0023] Divide the to-be-queried user identifier by the number of users in the user block and take the remainder, to obtain a user offset of the to-be-queried user identifier in the second target user block, recorded as a second user offset;

[0024] The tag identifier to be queried is divided by 64 and rounded to an integer to obtain a tag offset of the tag identifier to be queried, which is recorded as a second tag offset;

[0025] According to the second user offset and the second label offset, a label value is located in the second target user block and recorded as a second label value;

[0026] The tag identifier to be queried is divided by 64 and the remainder is obtained to obtain the tag position number of the tag identifier to be queried in the binary bit corresponding to the second tag value, which is recorded as the second tag position number;

[0027] Determine whether the second tag position number is 1 or 0;

[0028] If the second tag position number is 1, it is determined that the user to be queried corresponding to the user to be queried identifier has the tag to be queried corresponding to the tag to be queried identifier;

[0029] If the second tag position number is 0, it is determined that the user to be queried corresponding to the user to be queried identifier does not have the tag to be queried corresponding to the tag to be queried identifier.

[0030] A label data storage system, comprising:

[0031] A first acquisition unit is used to acquire the tag data to be stored, wherein the tag data to be stored includes: a user identifier to be stored and a tag identifier to be stored;

[0032] A first user block calculation unit, configured to divide the to-be-stored user identifier by the number of users in the user block and round it up to an integer, to obtain a target user block corresponding to the to-be-stored user identifier, recorded as a first target user block;

[0033] A first user offset calculation unit is used to divide the to-be-stored user identifier by the number of users in the user block and obtain a modulus to obtain a user offset of the to-be-stored user identifier in the first target user block, recorded as a first user offset;

[0034] A first label offset calculation unit is used to divide the label identifier to be stored by 64 and round it up to obtain a label offset of the label identifier to be stored, which is recorded as a first label offset;

[0035] A first label value calculation unit, configured to locate and obtain a label value in the first target user block according to the first user offset and the first label offset, recorded as a first label value;

[0036] A first label position number calculation unit, configured to obtain a label position number of the label identifier to be stored in a binary bit corresponding to the first label value by dividing the label identifier to be stored by 64 and taking a modulus, and record the result as a first label position number;

[0037] The storage unit is used to set the binary value corresponding to the first tag position number to 1, and complete adding the tag to be stored corresponding to the tag identifier to be stored to the user to be stored corresponding to the user identifier to be stored.

[0038] Optionally, the first target user block is: Tags[BASE_SIZE][TAG_BLOCK_RANGE]uint64;

[0039] Among them, Tags represents a tag set, BASE_SIZE represents the number of users in the user block, TAG_BLOCK_RANGE represents the number of elements of uint64, uint64 is an unsigned 64-bit integer, and the value is obtained by bit operation, and each bit represents a tag.

[0040] Optionally, the first user offset is: the subscript of the first dimension of the user identifier to be stored in the memory storage structure;

[0041] The memory storage structure is as follows:

[0042] Data[N]Tags[BASE_SIZE][TAG_BLOCK_RANGE]uint64;

[0043] Wherein, Data represents the variable name, N represents the number of one-dimensional array elements, Tags represents the tag set, BASE_SIZE represents the number of users in the user block, TAG_BLOCK_RANGE represents the number of uint64 elements, uint64 is an unsigned 64-bit integer, and the value is obtained by bit operation, and each bit represents a tag.

[0044] Optionally, the first tag offset is: the subscript of the second dimension of the tag identifier to be stored in the memory storage structure.

[0045] Optionally, also include:

[0046] A second acquisition unit is used to acquire the tag data to be queried, wherein the tag data to be queried includes: a user identifier to be queried and a tag identifier to be queried;

[0047] A second user block calculation unit is used to divide the to-be-queried user identifier by the number of users in the user block and round it up to an integer to obtain a target user block corresponding to the to-be-queried user identifier, recorded as a second target user block;

[0048] A second user offset calculation unit is used to divide the to-be-queried user identifier by the number of users in the user block and obtain a modulus to obtain a user offset of the to-be-queried user identifier in the second target user block, recorded as a second user offset;

[0049] A second label offset calculation unit is used to divide the label identifier to be queried by 64 and round it up to obtain a label offset of the label identifier to be queried, which is recorded as a second label offset;

[0050] A second label value calculation unit, configured to locate and obtain a label value in the second target user block according to the second user offset and the second label offset, recorded as a second label value;

[0051] A second label position number calculation unit, configured to obtain a label position number of the label identifier to be queried in a binary bit corresponding to the second label value by dividing the label identifier to be queried by 64 and taking a modulus, and record the result as a second label position number;

[0052] A determination unit, configured to determine whether the second tag position number is 1 or 0;

[0053] A first determining unit, configured to determine, if the second tag position number is 1, that the user to be queried corresponding to the user to be queried identifier has the tag to be queried corresponding to the tag to be queried identifier;

[0054] The second determining unit is configured to determine, if the second tag position number is 0, that the user to be queried corresponding to the user to be queried identifier does not have the tag to be queried corresponding to the tag to be queried identifier.

[0055] From the above technical solution, it can be known that the present invention discloses a tag data storage method and system, by dividing the user identifier to be stored by the number of users in the user block and taking the integer, obtaining the first target user block corresponding to the user identifier to be stored, dividing the user identifier to be stored by the number of users in the user block and taking the remainder, obtaining the first user offset of the user identifier to be stored in the first target user block, dividing the tag identifier to be stored by 64 and taking the integer, obtaining the first tag offset of the tag identifier to be stored, locating the first tag value in the first target user block according to the first user offset and the first tag offset, dividing the tag identifier to be stored by 64 and taking the remainder, obtaining the first tag position number of the tag identifier to be stored in the binary bit corresponding to the first tag value, setting the binary value corresponding to the first tag position number to 1, and completing the addition of the tag identifier to be stored corresponding to the tag identifier to be stored to the user to be stored corresponding to the user identifier to be stored. The present invention performs customized storage for users and tags, does not need to set the Schema separately, and can also add the tag identifier to the user identifier according to a given user identifier and a tag identifier. In addition, the read and write operation process is realized by directly operating the memory, so it can be regularly persisted to the disk, thereby greatly improving the write performance and writing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the disclosed drawings without paying creative work.

[0057] Figure 1 A flow chart of a tag data storage method disclosed in an embodiment of the present invention;

[0058] Figure 2 A schematic diagram of a cluster solution for storing tag data disclosed in an embodiment of the present invention;

[0059] Figure 3 A flow chart of a tag data query method disclosed in an embodiment of the present invention;

[0060] Figure 4 A structural schematic diagram of a label data storage system disclosed in an embodiment of the present invention;

[0061] Figure 5 The present invention is a schematic diagram of the structure of a tag data query module in a tag data storage system disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] The embodiment of the present invention discloses a tag data storage method and system to realize customized storage for users and tags. It does not need to set a schema separately, and can also add the tag identifier to the user identifier based on a given user identifier and a tag identifier. In addition, the read and write operation process is realized by directly operating the memory, so it can be persisted to the disk regularly, thereby greatly improving the write performance and improving the write efficiency.

[0064] See also Figure 1 , a flow chart of a tag data storage method disclosed in an embodiment of the present invention, the method comprising:

[0065] Step S101, obtaining tag data to be stored;

[0066] The tag data to be stored includes: a user identifier to be stored and a tag identifier to be stored.

[0067] It should be noted that the user identifier to be stored and the tag identifier to be stored in this embodiment are both integers not less than 0, for example, 64-bit data, represented by uint64. uint64 is an unsigned 64-bit integer, which uses bit operation to obtain values, each bit represents a tag, and 64 tags can be stored.

[0068] Step S102: Divide the user identifier to be stored by the number of users in the user block and round up to an integer to obtain a target user block corresponding to the user identifier to be stored, which is recorded as a first target user block;

[0069] It should be noted that the storage structure in this embodiment is as follows:

[0070] The memory part is a one-dimensional array, the elements of the one-dimensional array are a two-dimensional array, and the two-dimensional array can be called a user block or a block.

[0071] The memory storage structure is expressed as:

[0072] Data[N]Tags[BASE_SIZE][TAG_BLOCK_RANGE]uint64;

[0073] Where Data represents the variable name, N represents the number of one-dimensional array elements, BASE_SIZE represents the number of users in the user block. In actual applications, the value of BASE_SIZE depends on actual needs. For example, BASE_SIZE=100000. TAG_BLOCK_RANGE represents the number of uint64 elements. [TAG_BLOCK_RANGE]uint64 represents the total number of tags that can be stored. The value of TAG_BLOCK_RANGE depends on actual needs. For example, TAG_BLOCK_RANGE=3. [3]uint64 represents the total number of tags that can be stored.

[0074] The disk part can be simply described as follows: one user block corresponds to one user block file on the disk. The user data in the user block will be regularly written to the corresponding user block file on the disk through another thread. When the system restarts, multiple user block files will be read concurrently by multiple threads and recorded in the Data[N]Tags[BASE_SIZE][TAG_BLOCK_RANGE]uint64 structure.

[0075] The target user block corresponding to the user identifier to be stored obtained in this embodiment is the subscript of the one-dimensional array of the user identifier to be stored in the Data[N]Tags[BASE_SIZE][TAG_BLOCK_RANGE]uint64 structure. The corresponding element is obtained according to the subscript, that is, the first target user block is:

[0076] Tags[BASE_SIZE][TAG_BLOCK_RANGE]uint64.

[0077] The formula corresponding to step S102 can be expressed as:

[0078] uidIdx = uid / BASE_SIZE;

[0079] Wherein, uidIdx represents the first target user block, and uid represents the user identifier to be stored.

[0080] Step S103: Divide the user identifier to be stored by the number of users in the user block and take the remainder to obtain a user offset of the user identifier to be stored in the first target user block, recorded as a first user offset;

[0081] The calculated first user offset is the subscript of the first dimension of the user identifier to be stored in the Tags[BASE_SIZE][TAG_BLOCK_RANGE]uint64 structure.

[0082] The calculation formula corresponding to step S103 is:

[0083] uidPos = uid%BASE_SIZE;

[0084] Wherein, uidPos represents the first user offset.

[0085] Step S104: Divide the tag identifier to be stored by 64 and round it to an integer to obtain a tag offset of the tag identifier to be stored, which is recorded as a first tag offset;

[0086] The tag identifier to be stored is divided by 64 and rounded to obtain the first tag offset of the tag identifier to be stored. The first tag offset is also the subscript of the second dimension of the tag identifier to be stored in the Tags[BASE_SIZE][TAG_BLOCK_RANGE]uint64 structure.

[0087] The calculation formula corresponding to step S104 is:

[0088] tagIdx = tagId / 64;

[0089] Wherein, tagIdx represents the first tag offset, and tagId represents the tag identifier to be stored.

[0090] Step S105: According to the first user offset and the first label offset, locate in the first target user block to obtain a label value, which is recorded as a first label value;

[0091] The label value is the element value in uint64.

[0092] Step S106: Divide the tag identifier to be stored by 64 and obtain the modulus thereof to obtain the tag position number of the tag identifier to be stored in the binary bit corresponding to the first tag value, which is recorded as the first tag position number;

[0093] The label position number ranges from 0 to 63.

[0094] The calculation formula corresponding to step S106 is:

[0095] tagPos = tagId%64;

[0096] Where tagPos represents the first tag value.

[0097] Step S107: Set the binary value corresponding to the first tag position number to 1, and complete adding the tag to be stored corresponding to the tag identifier to be stored to the user to be stored corresponding to the user identifier to be stored.

[0098] In summary, the label data storage method disclosed in the present invention obtains the first target user block corresponding to the user identifier to be stored by dividing the user identifier to be stored by the number of users in the user block and taking the remainder, obtains the first user offset of the user identifier to be stored in the first target user block, divides the label identifier to be stored by 64 and takes the remainder to obtain the first label offset of the label identifier to be stored, locates the first label value in the first target user block according to the first user offset and the first label offset, divides the label identifier to be stored by 64 and takes the remainder to obtain the first label position number of the label identifier to be stored in the binary bit corresponding to the first label value, sets the binary value corresponding to the first label position number to 1, and completes adding the label identifier to be stored corresponding to the label identifier to be stored to the user to be stored corresponding to the user identifier to be stored. The present invention performs customized storage for users and labels, does not need to set the schema separately, and can also add the label identifier to the user identifier according to a given user identifier and a label identifier. In addition, the read and write operation process is realized by directly operating the memory, so it can be regularly persisted to the disk, thereby greatly improving the write performance and writing efficiency.

[0099] In addition, in the present invention, the memory and disk are both within one system process, and memory and disk related operations can ensure the atomicity of the operations by setting reasonable process locks and block locks, without considering the problem of consistent updates of multiple external storages and caches.

[0100] It should be noted that in this embodiment, [TAG_BLOCK_RANGE]uint64 tags can be stored simultaneously and can be recycled. In order to support more tags, this patent designs a cluster solution by customizing the user tag system Proxy, which is distinguished by range. Taking TAG_BLOCK_RANGE=3 as an example, according to the tag range [0,64*3), [64*3,64*3*2) in a progressive manner, it is routed to instances of different ranges to support a wider range of tag storage.

[0101] See also Figure 2 The schematic diagram of the cluster solution for tag data storage shown in FIG. 1 shows a block storage structure in which the memory is used to store user blocks, such as Figure 2 User block 1, user block 2, ..., user block N in the disk is used to store user block files, such as Figure 2The user block file 1, user block file 2, ..., user block N shown in the figure, a user block in the memory corresponds to a user block file on the disk, the stateless proxy layer is Proxy, Proxy receives an interface request carrying a label range, Proxy selects an instance according to the label range, and routes to the instance corresponding to the label range. After receiving the interface request, the instance calculates the target user block and user offset according to the user identifier, calculates the label offset according to the label identifier, and then calculates the label position number based on the target user block, user offset and note offset. If a label is added for the user, the binary (bit) value corresponding to the label position number is set to 1; if the label is queried, the label query result is determined based on whether the binary value corresponding to the label position number is 1 or 0, and the label query result is returned to the Proxy, where, Figure 2 The tag metadata in the storage tag is the semantics of the tag, which is a text description. For example, the tag 18 represents "whitelist".

[0102] Figure 1 The tag data storage shown in the embodiment shown shows the process of adding tags for users. Figure 1 On the basis of the illustrated embodiment, it is also possible to determine whether the user identification corresponds to a certain tag identification.

[0103] Therefore, in order to further optimize the above embodiment, Figure 1 Based on the embodiment shown, see Figure 3 , a flow chart of a tag data query method disclosed in an embodiment of the present invention, the method comprising:

[0104] Step S201, obtaining tag data to be queried;

[0105] The tag data to be queried includes: a user identifier to be queried and a tag identifier to be queried.

[0106] It should be noted that the user identifier to be queried and the tag identifier to be queried in this embodiment are both integers not less than 0, for example, 64-bit data, represented by uint64. uint64 is an unsigned 64-bit integer, which uses bit operation to obtain values. Each bit represents a tag, and 64 tags can be stored.

[0107] Step S202: Divide the to-be-queried user identifier by the number of users in the user block and round up to an integer to obtain a target user block corresponding to the to-be-queried user identifier, recorded as a second target user block;

[0108] The target user block corresponding to the user identifier to be queried obtained in this embodiment is the subscript of the one-dimensional array of the user identifier to be queried in the Data[N]Tags[BASE_SIZE][TAG_BLOCK_RANGE]uint64 structure. The corresponding element is obtained according to the subscript, that is, the second target user block is:

[0109] Tags[BASE_SIZE][TAG_BLOCK_RANGE]uint64.

[0110] The formula corresponding to step S202 can be expressed as:

[0111] uidIdx = uid / BASE_SIZE;

[0112] Wherein, uidIdx represents the second target user block, and uid represents the user identifier to be queried.

[0113] Step S203: divide the to-be-queried user identifier by the number of users in the user block and obtain the remainder, thereby obtaining a user offset of the to-be-queried user identifier in the second target user block, which is recorded as a second user offset;

[0114] The calculated user offset is the subscript of the second dimension of the user identifier to be queried in the Tags[BASE_SIZE][TAG_BLOCK_RANGE]uint64 structure.

[0115] The calculation formula corresponding to step S103 is:

[0116] uidPos = uid%BASE_SIZE;

[0117] Wherein, uidPos represents the second user offset.

[0118] Step S204: Divide the tag identifier to be queried by 64 and round it to an integer to obtain a tag offset of the tag identifier to be queried, which is recorded as a second tag offset;

[0119] The tag identifier to be queried is divided by 64 and rounded to an integer to obtain the second tag offset of the tag identifier to be queried, and the second tag offset is also the subscript of the second dimension of the tag identifier to be queried in the Tags[BASE_SIZE][TAG_BLOCK_RANGE]uint64 structure.

[0120] The calculation formula corresponding to step S204 is:

[0121] tagIdx = tagId / 64;

[0122] Wherein, tagIdx represents the second tag offset, and tagId represents the tag identifier to be queried.

[0123] Step S205: According to the second user offset and the second label offset, locate and obtain a label value in the second target user block, which is recorded as a second label value;

[0124] The label value is the element value in uint64.

[0125] Step S206: Divide the tag identifier to be queried by 64 and obtain the modulus thereof to obtain the tag position number of the tag identifier to be queried in the binary bit corresponding to the second tag value, which is recorded as the second tag position number;

[0126] The label position number ranges from 0 to 63.

[0127] The calculation formula corresponding to step S206 is:

[0128] tagPos = tagId%64;

[0129] Wherein, tagPos represents the second tag value.

[0130] Step S207, determining whether the second label position number is 1 or 0, if the second label position number is 1, executing step S208, if the second label position number is 0, executing step S209;

[0131] Step S208: Determine whether the user to be queried corresponding to the user to be queried identifier has the tag to be queried corresponding to the tag to be queried identifier.

[0132] Step S209: Determine that the user to be queried corresponding to the user to be queried identifier does not have the tag to be queried corresponding to the tag to be queried identifier.

[0133] In summary, the present invention obtains the second target user block corresponding to the user identifier to be stored by dividing the user identifier to be stored by the number of users in the user block and taking the remainder, obtains the second user offset of the user identifier to be stored in the second target user block by dividing the user identifier to be stored by the number of users in the user block and taking the remainder, obtains the second label offset of the label identifier to be stored by dividing the label identifier to be stored by 64 and taking the remainder, locates the second label value in the second target user block according to the second user offset and the second label offset, obtains the second label position number of the label identifier to be stored in the binary bit corresponding to the second label value by dividing the label identifier to be stored by 64 and taking the remainder, and determines whether the user to be queried corresponding to the user identifier to be queried has the label to be queried corresponding to the label identifier to be queried by judging whether the second label position number is 1 or 0. And, when the second label position number is 1, it is determined that the user to be queried has the label to be queried. Therefore, the present invention can judge whether the user identifier has the label identifier according to a given user identifier and a label identifier.

[0134] Corresponding to the above method embodiment, the present invention also discloses a label data storage system.

[0135] See also Figure 4 , a schematic diagram of a structure of a label data storage system disclosed in an embodiment of the present invention, the system comprising:

[0136] The first acquisition unit 301 is used to acquire the tag data to be stored, wherein the tag data to be stored includes: a user identifier to be stored and a tag identifier to be stored;

[0137] It should be noted that the user identifier to be stored and the tag identifier to be stored in this embodiment are both integers not less than 0, for example, 64-bit data, represented by uint64. uint64 is an unsigned 64-bit integer, which uses bit operation to obtain values, each bit represents a tag, and 64 tags can be stored.

[0138] A first user block calculation unit 302 is configured to divide the user identifier to be stored by the number of users in the user block and round it up to an integer to obtain a target user block corresponding to the user identifier to be stored, which is recorded as a first target user block;

[0139] It should be noted that the storage structure in this embodiment is as follows:

[0140] The memory part is a one-dimensional array, the elements of the one-dimensional array are a two-dimensional array, and the two-dimensional array can be called a user block or a block.

[0141] The memory storage structure is expressed as:

[0142] Data[N]Tags[BASE_SIZE][TAG_BLOCK_RANGE]uint64;

[0143] Where Data represents the variable name, N represents the number of one-dimensional array elements, BASE_SIZE represents the number of users in the user block. In actual applications, the value of BASE_SIZE depends on actual needs. For example, BASE_SIZE=100000. TAG_BLOCK_RANGE represents the number of uint64 elements. [TAG_BLOCK_RANGE]uint64 represents the total number of tags that can be stored. The value of TAG_BLOCK_RANGE depends on actual needs. For example, TAG_BLOCK_RANGE=3. [3]uint64 represents the total number of tags that can be stored.

[0144] The disk part can be simply described as follows: one user block corresponds to one user block file on the disk. The user data in the user block will be regularly written to the corresponding user block file on the disk through another thread. When the system restarts, multiple user block files will be read concurrently by multiple threads and recorded in the Data[N]Tags[BASE_SIZE][TAG_BLOCK_RANGE]uint64 structure.

[0145] The target user block corresponding to the user identifier to be stored obtained in this embodiment is the subscript of the one-dimensional array of the user identifier to be stored in the Data[N]Tags[BASE_SIZE][TAG_BLOCK_RANGE]uint64 structure. The corresponding element is obtained according to the subscript, that is, the first target user block is:

[0146] Tags[BASE_SIZE][TAG_BLOCK_RANGE]uint64.

[0147] A first user offset calculation unit 303 is configured to divide the to-be-stored user identifier by the number of users in the user block and obtain a modulus to obtain a user offset of the to-be-stored user identifier in the first target user block, recorded as a first user offset;

[0148] The calculated first user offset is the subscript of the first dimension of the user identifier to be stored in the Tags[BASE_SIZE][TAG_BLOCK_RANGE]uint64 structure.

[0149] A first label offset calculation unit 304 is used to divide the label identifier to be stored by 64 and round it up to obtain a label offset of the label identifier to be stored, which is recorded as a first label offset;

[0150] The tag identifier to be stored is divided by 64 and rounded to obtain the first tag offset of the tag identifier to be stored. The first tag offset is also the subscript of the second dimension of the tag identifier to be stored in the Tags[BASE_SIZE][TAG_BLOCK_RANGE]uint64 structure.

[0151] A first label value calculation unit 305 is used to locate and obtain a label value in the first target user block according to the first user offset and the first label offset, which is recorded as a first label value;

[0152] The label value is the element value in uint64.

[0153] A first label position number calculation unit 306 is used to obtain a label position number of the label identifier to be stored in the binary bit corresponding to the first label value by dividing the label identifier to be stored by 64 and taking a modulus, which is recorded as a first label position number;

[0154] The label position number ranges from 0 to 63.

[0155] The storage unit 307 is used to set the binary value corresponding to the first tag position number to 1, and complete adding the tag to be stored corresponding to the tag identifier to be stored to the user to be stored corresponding to the user identifier to be stored.

[0156] In summary, the label data storage system disclosed in the present invention obtains the first target user block corresponding to the user identifier to be stored by dividing the user identifier to be stored by the number of users in the user block and taking the remainder, obtains the first user offset of the user identifier to be stored in the first target user block by dividing the label identifier to be stored by 64 and taking the remainder, obtains the first label offset of the label identifier to be stored by dividing the label identifier to be stored by 64, locates the first label value in the first target user block according to the first user offset and the first label offset, obtains the first label position number of the label identifier to be stored in the binary bit corresponding to the first label value by dividing the label identifier to be stored by 64 and taking the remainder, sets the binary value corresponding to the first label position number to 1, and completes adding the label identifier to be stored corresponding to the label identifier to be stored to the user to be stored corresponding to the user identifier to be stored. The present invention performs customized storage for users and labels, does not need to set the schema separately, and can also add the label identifier to the user identifier according to a given user identifier and a label identifier. In addition, the read and write operation process is realized by directly operating the memory, so it can be regularly persisted to the disk, thereby greatly improving the write performance and writing efficiency.

[0157] In addition, in the present invention, the memory and disk are both within one system process, and memory and disk related operations can ensure the atomicity of the operations by setting reasonable process locks and block locks, without considering the problem of consistent updates of multiple external storages and caches.

[0158] The tag data storage shown in the above embodiment shows the process of adding tags for users. Figure 4 On the basis of the illustrated embodiment, it is also possible to determine whether the user identification corresponds to a certain tag identification.

[0159] Therefore, to further optimize the above embodiment, see Figure 5 , a structural diagram of a tag data query module in a tag data storage system disclosed in an embodiment of the present invention, the tag data query module includes:

[0160] The second acquisition unit 401 is used to acquire the tag data to be queried, wherein the tag data to be queried includes: a user identifier to be queried and a tag identifier to be queried;

[0161] A second user block calculation unit 402 is configured to divide the to-be-queried user identifier by the number of users in the user block and round it up to an integer to obtain a target user block corresponding to the to-be-queried user identifier, recorded as a second target user block;

[0162] The target user block corresponding to the user identifier to be queried obtained in this embodiment is the subscript of the one-dimensional array of the user identifier to be queried in the Data[N]Tags[BASE_SIZE][TAG_BLOCK_RANGE]uint64 structure. The corresponding element is obtained according to the subscript, that is, the second target user block is:

[0163] Tags[BASE_SIZE][TAG_BLOCK_RANGE]uint64.

[0164] A second user offset calculation unit 403 is configured to divide the to-be-queried user identifier by the number of users in the user block and obtain a modulus to obtain a user offset of the to-be-queried user identifier in the second target user block, recorded as a second user offset;

[0165] A second label offset calculation unit 404 is used to divide the label identifier to be queried by 64 and round it up to obtain a label offset of the label identifier to be queried, which is recorded as a second label offset;

[0166] A second label value calculation unit 405 is used to locate and obtain a label value in the second target user block according to the second user offset and the second label offset, which is recorded as a second label value;

[0167] A second label position number calculation unit 406 is used to obtain a label position number of the label identifier to be queried in the binary bit corresponding to the second label value by dividing the label identifier to be queried by 64 and taking a modulus, which is recorded as a second label position number;

[0168] A determination unit 407, configured to determine whether the second tag position number is 1 or 0;

[0169] A first determining unit 408 is configured to determine, if the second tag position number is 1, that the user to be queried corresponding to the user to be queried identifier has the tag to be queried corresponding to the tag to be queried identifier;

[0170] The second determining unit 409 is configured to determine, if the second tag position number is 0, that the user to be queried corresponding to the user to be queried identifier does not have the tag to be queried corresponding to the tag to be queried identifier.

[0171] In summary, the present invention obtains the second target user block corresponding to the user identifier to be stored by dividing the user identifier to be stored by the number of users in the user block and taking the remainder, obtains the second user offset of the user identifier to be stored in the second target user block by dividing the user identifier to be stored by the number of users in the user block and taking the remainder, obtains the second label offset of the label identifier to be stored by dividing the label identifier to be stored by 64 and taking the remainder, locates the second label value in the second target user block according to the second user offset and the second label offset, obtains the second label position number of the label identifier to be stored in the binary bit corresponding to the second label value by dividing the label identifier to be stored by 64 and taking the remainder, and determines whether the user to be queried corresponding to the user identifier to be queried has the label to be queried corresponding to the label identifier to be queried by judging whether the second label position number is 1 or 0. And, when the second label position number is 1, it is determined that the user to be queried has the label to be queried. Therefore, the present invention can judge whether the user identifier has the label identifier according to a given user identifier and a label identifier.

[0172] It should be noted that for the specific working principles of each component in the system embodiment, please refer to the corresponding part of the method embodiment, which will not be repeated here.

[0173] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0174] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0175] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A tag data storage method, characterized in that: include: The proxy in the stateless proxy layer receives an interface request carrying a label range, selects an instance according to the label range, and routes to the instance corresponding to the label range. After receiving the interface request, the instance obtains the label data to be stored, and the label data to be stored includes: a user identifier to be stored and a label identifier to be stored; The user identifier to be stored is divided by the number of users in the user block and rounded to an integer to obtain a target user block corresponding to the user identifier to be stored, which is recorded as a first target user block; wherein the memory storage structure is a one-dimensional array, the elements of the one-dimensional array are a two-dimensional array, and the two-dimensional array is a user block; the memory storage structure is expressed as: Data[N]Tags[BASE_SIZE][TAG_BLOCK_RANGE]uint64; Where Data represents the variable name, N represents the number of elements in a one-dimensional array, Tags represents a tag set, BASE_SIZE represents the number of users in a user block, TAG_BLOCK_RANGE represents the number of elements in uint64, and [TAG_BLOCK_RANGE]uint64 represents the total number of tags that can be stored at present; uint64 is an unsigned 64-bit integer that uses bitwise operation to obtain values, and each bit represents a tag; uidIdx = uid / BASE_SIZE; Wherein, uidIdx represents the first target user block, and uid represents the user identifier to be stored; The user identifier to be stored is divided by the number of users in the user block and the remainder is obtained to obtain the user offset of the user identifier to be stored in the first target user block, which is recorded as the first user offset; the first user offset is the subscript of the first dimension of the user identifier to be stored in the Tags[BASE_SIZE][TAG_BLOCK_RANGE]uint64 structure; The tag identifier to be stored is divided by 64 and rounded to an integer to obtain the tag offset of the tag identifier to be stored, which is recorded as the first tag offset; the first tag offset is the subscript of the second dimension of the tag identifier to be stored in the Tags[BASE_SIZE][TAG_BLOCK_RANGE]uint64 structure; According to the first user offset and the first label offset, a label value is located in the first target user block, recorded as a first label value; the first label value is an element value in the Tags[BASE_SIZE][TAG_BLOCK_RANGE]uint64 structure; The tag identifier to be stored is divided by 64 and the remainder is obtained to obtain a tag position number of the tag identifier to be stored in the binary bit corresponding to the first tag value, which is recorded as a first tag position number; The binary value corresponding to the first tag position number is set to 1, and the tag to be stored corresponding to the tag identifier to be stored is added to the user to be stored corresponding to the user identifier to be stored.

2. The tag data storage method according to claim 1, characterized in that: Also includes: Acquire the tag data to be queried, wherein the tag data to be queried includes: a user identifier to be queried and a tag identifier to be queried; Divide the to-be-queried user identifier by the number of users in the user block and round it up to an integer, to obtain a target user block corresponding to the to-be-queried user identifier, recorded as a second target user block; Divide the to-be-queried user identifier by the number of users in the user block and take the remainder, to obtain a user offset of the to-be-queried user identifier in the second target user block, recorded as a second user offset; The tag identifier to be queried is divided by 64 and rounded to an integer to obtain a tag offset of the tag identifier to be queried, which is recorded as a second tag offset; According to the second user offset and the second label offset, a label value is located in the second target user block and recorded as a second label value; The tag identifier to be queried is divided by 64 and the remainder is obtained to obtain the tag position number of the tag identifier to be queried in the binary bit corresponding to the second tag value, which is recorded as the second tag position number; Determine whether the second tag position number is 1 or 0; If the second tag position number is 1, it is determined that the user to be queried corresponding to the user to be queried identifier has the tag to be queried corresponding to the tag to be queried identifier; If the second tag position number is 0, it is determined that the user to be queried corresponding to the user to be queried identifier does not have the tag to be queried corresponding to the tag to be queried identifier.

3. A tag data storage system, characterized in that: include: A first acquisition unit is configured to receive an interface request carrying a label range at a proxy in the stateless proxy layer, select an instance according to the label range, and route to the instance corresponding to the label range. After receiving the interface request, the instance acquires the label data to be stored, wherein the label data to be stored includes: a user identifier to be stored and a label identifier to be stored; The first user block calculation unit is used to divide the user identifier to be stored by the number of users in the user block and round it up to obtain a target user block corresponding to the user identifier to be stored, which is recorded as a first target user block; wherein the memory storage structure is a one-dimensional array, the elements of the one-dimensional array are a two-dimensional array, and the two-dimensional array is a user block; the memory storage structure is expressed as: Data[N]Tags[BASE_SIZE][TAG_BLOCK_RANGE]uint64; Where Data represents the variable name, N represents the number of elements in a one-dimensional array, Tags represents a tag set, BASE_SIZE represents the number of users in a user block, TAG_BLOCK_RANGE represents the number of elements in uint64, and [TAG_BLOCK_RANGE]uint64 represents the total number of tags that can be stored at present; uint64 is an unsigned 64-bit integer that uses bitwise operation to obtain values, and each bit represents a tag; uidIdx = uid / BASE_SIZE; Wherein, uidIdx represents the first target user block, and uid represents the user identifier to be stored; A first user offset calculation unit is used to divide the user identifier to be stored by the number of users in the user block and obtain the remainder, so as to obtain a user offset of the user identifier to be stored in the first target user block, which is recorded as a first user offset; the first user offset is the subscript of the first dimension of the user identifier to be stored in the Tags[BASE_SIZE][TAG_BLOCK_RANGE]uint64 structure; A first label offset calculation unit is used to divide the label identifier to be stored by 64 and round it to obtain a label offset of the label identifier to be stored, which is recorded as a first label offset; the first label offset is the subscript of the second dimension of the label identifier to be stored in the Tags[BASE_SIZE][TAG_BLOCK_RANGE]uint64 structure; A first label value calculation unit is used to locate a label value in the first target user block according to the first user offset and the first label offset, and record it as a first label value; the first label value is an element value in the Tags[BASE_SIZE][TAG_BLOCK_RANGE]uint64 structure; A first label position number calculation unit, configured to obtain a label position number of the label identifier to be stored in a binary bit corresponding to the first label value by dividing the label identifier to be stored by 64 and taking a modulus, and record the result as a first label position number; The storage unit is used to set the binary value corresponding to the first tag position number to 1, and complete adding the tag to be stored corresponding to the tag identifier to be stored to the user to be stored corresponding to the user identifier to be stored.

4. The tag data storage system according to claim 3, characterized in that: Also includes: A second acquisition unit is used to acquire the tag data to be queried, wherein the tag data to be queried includes: a user identifier to be queried and a tag identifier to be queried; A second user block calculation unit is used to divide the to-be-queried user identifier by the number of users in the user block and round it up to an integer to obtain a target user block corresponding to the to-be-queried user identifier, recorded as a second target user block; A second user offset calculation unit is used to divide the to-be-queried user identifier by the number of users in the user block and obtain a modulus to obtain a user offset of the to-be-queried user identifier in the second target user block, recorded as a second user offset; A second label offset calculation unit is used to divide the label identifier to be queried by 64 and round it up to obtain a label offset of the label identifier to be queried, which is recorded as a second label offset; A second label value calculation unit, configured to locate and obtain a label value in the second target user block according to the second user offset and the second label offset, recorded as a second label value; A second label position number calculation unit, configured to obtain a label position number of the label identifier to be queried in a binary bit corresponding to the second label value by dividing the label identifier to be queried by 64 and taking a modulus, and record the result as a second label position number; A determination unit, configured to determine whether the second tag position number is 1 or 0; A first determining unit, configured to determine, if the second tag position number is 1, that the user to be queried corresponding to the user to be queried identifier has the tag to be queried corresponding to the tag to be queried identifier; The second determining unit is configured to determine, if the second tag position number is 0, that the user to be queried corresponding to the user to be queried identifier does not have the tag to be queried corresponding to the tag to be queried identifier.