Target object positioning method and device

By generating highly abstract statements and using bitmap technology, the real-time and accuracy of massive data retrieval in big data systems are solved, interactive query and the satisfaction of complex business needs are achieved, and the positioning process is simplified.

CN119179709APending Publication Date: 2024-12-24SHANGHAI BILIBILI TECH CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411419208.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-11
Publication Date
2024-12-24

AI Technical Summary

Technical Problem

Existing big data systems are difficult to achieve real-time and accurate interactive query in massive data retrieval, and the positioning scheme is difficult to use and adjust, which cannot meet complex business needs.

Method used

By obtaining positioning rules, a highly abstract first language type statement is generated, and a second language type statement for database query is generated based on label metadata. The bitmap is used for data storage and operation to achieve rapid positioning of the target object and Dynamic adjustment.

Benefits of technology

It realizes the output of billions of positioning data in a time-lapse manner in the big data system, simplifies the writing of query statements, supports rich label interchange and difference operations, improves the ease of use and flexibility, and is suitable for interactive queries.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119179709A_ABST
    Figure CN119179709A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a target object positioning method, and the method comprises the steps: obtaining a positioning rule which comprises one or more target labels matched with a target object, and a target label value corresponding to each target label; generating a first language type statement according to the positioning rule; label metadata is obtained, the label metadata is determined according to user data in a preset database, the user data comprises label values of all users under one or more labels, and the one or more labels comprise the one or more target labels; generating a second language type statement for database query according to the first language type statement and the tag metadata; and determining the target object according to the second language type statement and the preset database. According to the technical scheme, the complex query statement of the bottom layer can be automatically generated, the positioning result is returned, real-time interactive query is achieved, and usability is high.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the field of big data technology, and in particular, to a method, apparatus, computer device, computer-readable storage medium, and computer program product for locating a target object. Background Art

[0002] Big data systems can be used to store, process, and analyze massive amounts of data. For example, specific data subsets can be located from massive amounts of data for analysis based on business needs. Since business needs are complex and flexible, big data systems need to complete the retrieval of massive amounts of data in the shortest possible time to obtain real-time and accurate positioning results.

[0003] However, existing positioning solutions are difficult to use and adjust, and cannot meet users' needs for interactive queries.

[0004] It should be noted that the above content is not necessarily prior art, nor is it intended to limit the scope of patent protection of this application. Summary of the invention

[0005] The embodiments of the present application provide a target object positioning method, apparatus, computer equipment, computer-readable storage medium, and computer program product to solve or alleviate one or more of the technical problems raised above.

[0006] One aspect of an embodiment of the present application provides a method for locating a target object, the method comprising: Acquire a positioning rule, wherein the positioning rule includes one or more target tags matching the target object, and a target tag value corresponding to each target tag; generating a first language type sentence according to the positioning rule; Acquire tag metadata, where the tag metadata is determined according to user data in a preset database, where the user data includes tag values ​​of each user under one or more tags, where the one or more tags include the one or more target tags; generating a second language type sentence for database query according to the first language type sentence and the tag metadata; The target object is determined according to the second language type sentence and the preset database.

[0007] Optionally, the tag metadata includes: the name and version information of the tag; Correspondingly, generating a target SQL statement according to the first language type statement and the tag metadata includes: Determining the one or more target tags and a target tag value of each target tag according to the first language type sentence; Determining, according to the tag metadata, names and version information corresponding to each of the one or more target tags; The second language type statement is generated based on preset rules according to the name, version information, and target tag value corresponding to each of the one or more target tags.

[0008] Optionally, the preset database stores the user data in the form of a bitmap: Acquire the user data, and determine a tag value set for each tag, where the tag value set includes one or more tag values; Establishing a corresponding bitmap for each label value in the label value set of each label, the bitmap being used to represent users having the corresponding label value; The bitmap corresponding to each tag value is stored in the preset database.

[0009] Optionally, a corresponding bitmap is established for each label value in the label value set of each label, including: Acquire a user code set, the user code set including a user code of each user in the user data; Based on a preset rule, the user code set is divided into a plurality of user code slices, each user code slice including one or more user codes; For each user code slice: a corresponding bitmap is established for each label value in the label value set of each label, and the bitmap is used to represent the user code of the user with the corresponding label value in the user code slice.

[0010] Optionally, based on a preset rule, the user code set is divided into a plurality of user code segments, including: Obtaining a value of a predetermined area in each user code in the user code set; A modulo operation is performed on the value of the predetermined area, and the user code set is divided into the plurality of user code fragments according to the result of the modulo operation.

[0011] Optionally, the second language type statement is used to execute the positioning rule; Correspondingly, determining the target object according to the second language type sentence and the preset database includes: By running the second language type statement, a bit map corresponding to the target label value in the positioning rule is obtained, and a bit operation is performed on the obtained bit map to obtain a target bit map; The target bitmap is used to represent the target object.

[0012] Optionally, the target object positioning method further includes: Converting the target bitmap into user data corresponding to the target object in a preset manner; The user data corresponding to the target object is synchronized to a preset data warehouse.

[0013] Another aspect of an embodiment of the present application provides a target object positioning device, the device comprising: A first acquisition module is used to acquire a positioning rule, wherein the positioning rule includes one or more target tags matching the target object and a target tag value corresponding to each target tag; A first generating module, used for generating a first language type sentence according to the positioning rule; A second acquisition module is used to acquire tag metadata, where the tag metadata is determined according to user data in a preset database, where the user data includes tag values ​​of each user under one or more tags, where the one or more tags include the one or more target tags; A second generating module, configured to generate a second language type statement for database query according to the first language type statement and the tag metadata; A determination module is used to determine the target object according to the second language type sentence and the preset database.

[0014] Another aspect of an embodiment of the present application provides a computer device, including: at least one processor; and a memory communicatively coupled to the at least one processor; Wherein: the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as described above.

[0015] Another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method described above is implemented.

[0016] Another aspect of an embodiment of the present application provides a computer program product, including a computer program, which implements the method described above when executed by a processor.

[0017] The above technical solution adopted in the embodiment of the present application may have the following advantages: By obtaining the positioning rules, the target label and target label value that match the target object are determined. A highly abstract first language type statement is generated based on the positioning rules. The label metadata is obtained, and the label metadata is determined based on the user data in the preset database. The label metadata includes the underlying information of each label in the user data. Based on the first language type statement and the label metadata, an underlying second language type statement for database query is generated. By running the second language type statement, the target object can be located in the user data of the preset database. It can be seen that the embodiment of the present application only needs to provide basic positioning rules to automatically generate the underlying complex query statements, and execute and return the positioning results, wherein the positioning rules can be adjusted in real time and dynamically, thereby realizing interactive query, which is highly easy to use. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0019] Figure 1 The flowchart of the target object positioning method according to the first embodiment of the present application is schematically shown; Figure 2 A flowchart of storing user data in a preset database according to Embodiment 1 of the present application is schematically shown; Figure 3 The bit map and positioning process according to the first embodiment of the present application are schematically shown; Figure 4 Schematically shows Figure 2 Flow chart of sub-steps of step S202; Figure 5 The slicing process diagram according to the first embodiment of the present application is schematically shown; Figure 6 Schematically shows Figure 4 Flow chart of sub-steps of step S402; Figure 7 Schematically shows Figure 1 Flow chart of sub-steps of step S106; Figure 8 Another positioning process according to the first embodiment of the present application is schematically shown; Fig. 9 The statement conversion process according to the first embodiment of the present application is schematically shown; Fig.10 The data synchronization process between the preset database and the data warehouse according to the first embodiment of the present application is schematically shown; Fig.11A block diagram schematically shows a target object positioning device according to Embodiment 2 of the present application; and Fig.12 The hardware architecture diagram of the computer device according to the third embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0021] It should be noted that the descriptions involving "first", "second", etc. in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0022] In the description of the present application, it should be understood that the numerical labels before the steps do not indicate the order in which the steps are executed, but are only used to facilitate the description of the present application and to distinguish each step, and therefore should not be understood as a limitation on the present application.

[0023] First, the following terms are explained: DMP (Data Management Platform): Data management platform, which can be used as a user portrait platform.

[0024] Tag: A tag used to mark certain attributes of data, such as gender, region, age, etc.

[0025] mid: User unique identifier, also known as user code. mid can be an integer of any length and is used to uniquely identify a user. For example, a 64-bit integer can be used as mid.

[0026] Positioning: Based on the tag, a specific data subset that meets certain rules is located from the massive data, such as "male, age>24, Beijing". The specific data subset is the positioning data, which contains all the data that meets the rules.

[0027] bitmap: A data structure that identifies the existence of an integer N by whether the value of the bit at position N is 0 or 1.

[0028] Clickhouse: A column-based OLAP (Online Analytical Processing) database that supports efficient bitmap storage and calculation.

[0029] DSL (Domain Specific Language): A domain-specific language, which is a computer language designed for a specific domain in a computer system. It has the characteristics of highly abstracting domain knowledge and being simple and easy to use.

[0030] Secondly, in order to facilitate those skilled in the art to understand the technical solutions provided in the embodiments of the present application, the relevant technologies are described below: Big data systems can be used to store, process, and analyze massive amounts of data. For example, specific data subsets can be located from massive amounts of data for analysis based on business needs. Since business needs are complex and flexible, big data systems need to complete the retrieval of massive amounts of data in the shortest possible time to obtain real-time and accurate positioning results.

[0031] However, related positioning solutions have problems such as poor timeliness, low accuracy, only being able to estimate the amount of data that meets the positioning rules, unintuitive data structure, complex query statements, difficulty in use, difficulty in adjustment, and no support for real-time interactive query.

[0032] To this end, the embodiment of the present application provides a technical solution for target object positioning. In this technical solution: (1) Bitmap-based target object positioning outputs billions of positioning data (including accurate data volume and specific data) in seconds; (2) It is simple and easy to use, avoiding the need to write complex SQL statements, and is suitable for interactive queries; (3) It supports a rich variety of tags, at least hundreds of them, and different tags can flexibly perform intersection and difference operations; (4) Multiple tags are not coupled and can be updated and stored independently; (5) It is seamlessly linked to the big data platform infrastructure, supporting the import of data from hive tables and the output of positioning data to hive tables. See below for details.

[0033] The technical solutions of the present application are described below through multiple embodiments. It should be noted that these embodiments can be implemented in a variety of different forms and should not be interpreted as being limited to the embodiments described here.

[0034] Embodiment 1 Figure 1 The flowchart of the target object positioning method according to the first embodiment of the present application is schematically shown.

[0035] like Figure 1 As shown, the target object positioning method may include steps S100 to S108, wherein: Step S100: acquiring a positioning rule, wherein the positioning rule includes one or more target tags matching the target object, and a target tag value corresponding to each target tag.

[0036] Step S102: Generate a first language type sentence according to the positioning rule.

[0037] Step S104, obtaining tag metadata, wherein the tag metadata is determined according to user data in a preset database, wherein the user data includes tag values ​​of each user under one or more tags, and the one or more tags include the one or more target tags.

[0038] Step S106: Generate a second language type sentence for database query according to the first language type sentence and the tag metadata.

[0039] Step S108: determining the target object according to the second language type sentence and the preset database.

[0040] The target object positioning method provided in this embodiment: by obtaining positioning rules, the target label and target label value that match the target object are determined. A highly abstract first language type statement is generated based on the positioning rules. Label metadata is obtained, and the label metadata is determined based on the user data in a preset database. The label metadata includes the underlying information of each label in the user data. Based on the first language type statement and the label metadata, an underlying second language type statement for database query is generated. By running the second language type statement, the target object can be located in the user data of the preset database. It can be seen that the embodiment of the present application only needs to provide basic positioning rules to automatically generate the underlying complex query statements, and execute and return the positioning results, wherein the positioning rules can be adjusted in real time and dynamically, thereby realizing interactive query, which is highly easy to use.

[0041] The following combination Figure 1 , each step in steps S100~S108 and other optional steps are explained in detail.

[0042] The positioning scheme provided in the embodiment of the present application is used to locate the target object in the mass data. The target object can be a single data or a data set. The data can be user data, sales data, inventory data or other types of data. Accordingly, the target object can be one or more users, one or more sales data, one or more inventory records, etc. that meet the positioning rules. The positioning method provided in the embodiment of the present application will be exemplarily introduced below using user data as an example.

[0043] Step S100 , obtaining a positioning rule, wherein the positioning rule includes one or more target tags matching the target object, and a target tag value corresponding to each target tag.

[0044] Positioning rules can be used to filter out target objects from user data. User data can include tag values ​​of each user under one or more tags. For example, user data 1 is "mid1, gender female, age > 30, Beijing area", and user data 2 is "mid2, gender male, age > 20, Shanghai area". Among them: mid is the user's unique identifier (user code); gender, age, and area are tags of different dimensions; female, male, > 30, > 20, Beijing, and Shanghai are tag values. It can be seen that user data can include tags of multiple dimensions, each tag can have one or more tag values, and the tag value of a single user under a single tag is unique. Therefore, the positioning rule can specify tags and tag values ​​to locate target objects that meet the conditions. That is, the positioning rule can include one or more target tags that match the target object, and the target tag value corresponding to each target tag. Exemplarily, the positioning rule can be gender female, then user data 1 meets the positioning rule, user data 2 does not meet the positioning rule, and the target object includes user data 1 but does not include user data 2. The positioning rule can also be gender male and Shanghai, and the target object includes user data 2 but does not include user data 1. It should be noted that the above example only schematically lists three dimensions of labels, and two label values ​​under each dimension. In actual applications, user data may include labels of hundreds of dimensions, hundreds of millions of label values, and the total data volume may reach PB (Petabyte, quadrillion bytes) level.

[0045] Step S102 , generating a first language type sentence according to the positioning rule.

[0046] The first language type sentence may be a DSL sentence. For example, the positioning rule may be "gender male, age>30, Beijing and Shanghai area". The corresponding first language type sentence may be expressed as follows: @gender='male'&@age>= 30&@geo IN ('Beijing','Shanghai') As can be seen, DSL statements are more concise, and can highly abstract positioning services, improving usability. Users only need to specify basic query conditions, that is, specify specific positioning rules without paying attention to underlying details, avoiding writing complex statements, reducing positioning difficulty, and reducing usage barriers.

[0047] Step S104, obtain tag metadata, the tag metadata is determined according to user data in a preset database, the user data includes tag values ​​of each user under one or more tags, and the one or more tags include the one or more target tags.

[0048] User data can be stored in a preset database. To further improve positioning efficiency, the preset database can use a column storage database to meet large-scale data analysis needs and achieve faster data retrieval. For example: Clickhouse, Vertica, MonetDB, etc.

[0049] In an optional embodiment, the preset database may store user data in the form of a bitmap, such as Figure 2 As shown, the storing step may include: Step S200: Acquire the user data and determine a tag value set for each tag, where the tag value set includes one or more tag values.

[0050] Step S202: Create a corresponding bitmap for each tag value in the tag value set of each tag, where the bitmap is used to represent users with corresponding tag values.

[0051] Step S204: storing the bitmap corresponding to each tag value in the preset database.

[0052] In actual applications, user data can include hundreds of dimensions of tags, hundreds of millions of tag values, and the total data volume can reach the PB (Petabyte) level. Tags of different dimensions have different update cycles and storage formats, and the multi-dimensional wide table format involves the coupling of tags, which is used to store user data, resulting in reduced query efficiency and increased update overhead. To further improve positioning efficiency, user data can be stored in the form of bitmaps.

[0053] A bitmap is a data structure that can be used to represent a set of positive integers. The bottom layer is a series of continuous binary bits. The user code (mid) can be a positive integer of any length, numbered from 1 and gradually increasing. Therefore, a bitmap can be used to represent a set of a series of mids. Each bit can correspond to a user code (mid). For example, a bitmap can have 1000 bits to represent users mid1 to mid1000. The first bit represents mid1, the second bit represents mid2, and so on, the 1000th bit represents mid1000. The value of each bit is 0 or 1. If the value of the bit is 0, it means that the user mid corresponding to the bit is not in the set. If the value of the bit is 1, it means that the user mid corresponding to the bit is in the bitmap. For example, a bitmap is represented as: [0, 1, 1, 0, 1], where the 2nd, 3rd, and 5th bits are 1, then the integer set represented by the bitmap is {2, 3, 5}. The bitmap is a collection of bits that supports bit operations such as intersection, union, and difference, which can greatly improve computing efficiency.

[0054] As mentioned above, a bitmap can be created for each tag value to represent all users with the tag value. For example, a tag value set for each tag in the user data can be obtained. A corresponding bitmap is created for each tag value in the tag value set of each tag and stored in a preset database. Figure 3 As shown, for example, the bitmap of the label value "male" can be expressed as [0, 1, 1, 0, 1, 0, 1, 1, 0, 1], which means that the gender of users with mid of 2, 3, 5, 7, 8, 9, 11 is male. The bitmap of the label value ">30" can be expressed as [1, 1, 0, 0, 1, 1, 0, 1, 0, 1], which means that the age of users with mid of 1, 2, 5, 6, 7, 9, 11 is > 30. The bitmap of the label value "Beijing area" can be expressed as [0, 1, 1, 0, 1, 1, 0, 1, 0, 0], which means that the users with mid of 2, 3, 5, 6, 7, 9 are in Beijing area.

[0055] In this embodiment, corresponding bitmaps are established for different tag values. Multiple tags are not coupled and can be updated and stored independently. It also supports adding tags of new dimensions and new tag values ​​at any time, effectively optimizing storage and improving positioning performance, and enhancing flexibility and scalability to quickly adapt to new business needs.

[0056] In an optional embodiment, if Figure 4 As shown, step S202 may include: Step S400: Acquire a user code set, where the user code set includes the user code of each user in the user data.

[0057] Step S402: based on a preset rule, the user code set is divided into a plurality of user code slices, each user code slice including one or more user codes.

[0058] Step S404, for each user code slice: establish a corresponding bitmap for each tag value in the tag value set of each tag, the bitmap being used to represent the user code of the user with the corresponding tag value in the user code slice.

[0059] In actual applications, when there is too much user data (too many mids), there may be hundreds of millions of users with a certain tag value, and the bitmap corresponding to the tag value needs to support the storage and calculation of hundreds of millions of bits. A bitmap containing hundreds of millions of people has a size of hundreds of MB (megabytes), which will lead to a serious decline in computing efficiency. To improve computing efficiency, the bitmap can be stored in shards. For example, Figure 5 As shown, a user code set may be obtained first, that is, a user code of each user in the user data. The user code set may be divided into a plurality of user code slices according to preset rules, such as uniform slicing, uneven slicing, hash slicing, etc., and each slice includes one or more user codes. For each user code slice, a corresponding bitmap may be established for each label value in the slice, and the bitmap is used to represent the user code of the user with the corresponding label value in the user code slice, that is, the user data set with the label value.

[0060] In this embodiment, the bitmap is stored in slices, and there is no intersection between the slices, which can improve data processing and maintenance efficiency and ensure data integrity.

[0061] In an optional embodiment, correspondingly, Figure 6 As shown, step S402 may include: Step S600, obtaining a value of a predetermined area in each user code in the user code set.

[0062] Step S602: performing a modulo operation on the value of the predetermined area, and dividing the user code set into the plurality of user code fragments according to the result of the modulo operation.

[0063] For example, each mid may be a 64-bit integer. The predetermined area range may be the high 48 bits (17 to 64 bits) of each mid. The value of the high 48 bits of each mid is modulo N, where N is the number of slices. User codes with the same modulus are assigned to the same user code slice.

[0064] The sharding method of this embodiment can use the characteristics of Clickhouse database to store bitmaps, and put the lower 16 consecutive mids in the same shard as much as possible. The more consecutive bits in the bitmap, the better the compression algorithm can be applied to reduce the volume and further improve the computing efficiency. Compared with the solution of directly taking the modulus of the number of shards N, this optimized sharding method can save 30% of storage space and more than double the computing speed.

[0065] The above embodiments describe that user data is stored in a preset database (such as Clickhouse) in the form of a bitmap. The positioning process will be further described below.

[0066] Tag metadata may include the underlying information of each tag in the user data, such as the latest version of each tag, the name of the underlying partition, etc. Tag metadata may be stored in a preset data warehouse, such as hive, mysql, etc. Tag metadata may be determined based on user data stored in a preset database, that is, the information and update status of tags in user data in the preset database may be synchronized to the data warehouse.

[0067] Step S106 , generating a second language type statement for database query according to the first language type statement and the tag metadata.

[0068] In practical applications, the syntax and semantics of DSL statements can be parsed by a compiler, which includes at least a target label and a target label value. Combined with the label metadata corresponding to the target label, the DSL statement can be converted into a second language type statement, such as an SQL statement, for database query and to improve execution efficiency.

[0069] The conversion process is described below as an example.

[0070] In an optional embodiment, the tag metadata may include the name and version information of the tag. Figure 7 As shown, step S106 may include: Step S700: determining the one or more target tags and a target tag value of each target tag according to the first language type sentence.

[0071] Step S702: Determine the name and version information corresponding to each of the one or more target tags according to the tag metadata.

[0072] Step S704: Generate the second language type statement based on preset rules according to the name, version information, and target tag value corresponding to each of the one or more target tags.

[0073] In practical applications, such as Fig. 9 As shown, the target tag and target tag value can be determined according to the DSL statement, and the version information and underlying partition name corresponding to the target tag can be determined according to the tag metadata. According to the name, version information, and target tag value corresponding to each target tag, and according to the syntax rules of SQL, an SQL statement is generated. Exemplarily, the DSL statement: @gender='male' & @age>=30 & @geo IN ('Beijing', 'Shanghai') can be converted into the following SQL statement: SELECT groupBitmapAnd(bm) FROM (SELECT groupBitmapAndState(bm) FROM (SELECT groupBitmapOrState(bm) FROM index WHERE (tag_name = 'gender') AND version='20240201' AND (tag_value='male') UNION ALL SELECT groupBitmapOrState(bm) FROM index WHERE (tag_name = 'age') AND version='20240228' AND (tag_value>='30') ) UNION ALL SELECT groupBitmapOrState(bm) FROM index_table WHERE (tag_name = 'geography') AND version='20240227' AND (tag_value IN ('Beijing', 'Shanghai')) ) In this embodiment, a simplified DSL language is designed. Only by specifying the basic query conditions (location rules), the system can automatically generate the underlying complex SQL without paying attention to the underlying details; and the underlying layer can also upgrade and optimize the system at any time without affecting the query.

[0074] Step S108 , determine the target object according to the second language type statement and the preset database.

[0075] In actual applications, executing SQL statements on a preset database can return positioning results, including the number of users that meet the positioning rules and specific user data (target objects). An exemplary solution is provided below.

[0076] In an optional embodiment, the second language type statement is used to execute the positioning rule, and step S108 may include: by running the second language type statement, obtaining a bit map corresponding to the target label value in the positioning rule, and performing a bit operation on the obtained bit map to obtain a target bit map; wherein the target bit map is used to represent the target object.

[0077] like Figure 3 As shown in the figure, the second language type statement is used to query the target object of "male, >30, Beijing area". The user code corresponding to the bit with a value of 1 in the bitmap of each target label value represents the user with the target label value. According to the positioning rule, it is necessary to perform an intersection operation (AND) on the three bitmaps of male, >30, and Beijing area to obtain the final bitmap result, that is, the target bitmap [0, 1, 0, 0, 1, 0, 1, 0, 0], which means that users with mid values ​​of 2, 5, 7, and 9 are the target objects. For example Figure 8 As shown in the figure, there are three label values ​​A, B, and C. The positioning rule can be the target object that satisfies A and B but does not include C. The positioning process is as follows: first locate the bitmaps of the three label values ​​A, B, and C, and then perform the bitwise operation of A AND B AND NOT C on the three bitmaps to obtain the final bitmap.

[0078] In this embodiment, a bitmap is used to establish an inverted index for tags, and efficient positioning of the target object is achieved through real-time intersection and difference operations of the bitmap, thereby ensuring the timeliness and accuracy of positioning.

[0079] The detailed list of the user's original tags can be stored in the data warehouse, and the data format can be a hive table or a mysql table. Therefore, the embodiment of the present application provides a solution for writing the user data conversion bitmap in the data warehouse to the preset database Clickhouse. Conversely, the positioning result (target object) also needs to be rewritten back to the hive table of the data warehouse. Therefore, an exemplary solution is provided below.

[0080] In an optional embodiment, the target object positioning method may further include: converting the target bitmap into user data corresponding to the target object in a preset manner; and synchronizing the user data corresponding to the target object into a preset data warehouse.

[0081] In this embodiment, if Fig.10As shown, the user data corresponding to the target object can be synchronized to the data warehouse through dedicated tools, realizing the seamless integration of the bitmap index system and the big data platform, and supporting the import and export from the data warehouse to the bitmap index system and from the bitmap index system to the data warehouse.

[0082] Embodiment 2 Fig.11 The block diagram of the target object positioning device according to the second embodiment of the present application is schematically shown. The device can be divided into one or more program modules, one or more program modules are stored in a storage medium, and are executed by one or more processors to complete the embodiment of the present application. The program module referred to in the embodiment of the present application refers to a series of computer program instruction segments that can complete specific functions. The following description will specifically introduce the functions of each program module in this embodiment. Fig.11 As shown, the apparatus 1000 may include: a first acquisition module 1100, a first generation module 1200, a second acquisition module 1300, a second generation module 1400 and a determination module 1500, wherein: A first acquisition module 1100 is used to acquire a positioning rule, where the positioning rule includes one or more target tags matching the target object, and a target tag value corresponding to each target tag; A first generating module 1200, configured to generate a first language type sentence according to the positioning rule; A second acquisition module 1300 is used to acquire tag metadata, where the tag metadata is determined according to user data in a preset database, where the user data includes tag values ​​of each user under one or more tags, where the one or more tags include the one or more target tags; A second generating module 1400 is used to generate a second language type statement for database query according to the first language type statement and the tag metadata; The determination module 1500 is configured to determine the target object according to the second language type sentence and the preset database.

[0083] As an optional embodiment, the tag metadata includes: the name and version information of the tag; Correspondingly, generating a target SQL statement according to the first language type statement and the tag metadata includes: Determining the one or more target tags and a target tag value of each target tag according to the first language type sentence; Determining, according to the tag metadata, names and version information corresponding to each of the one or more target tags; The second language type statement is generated based on preset rules according to the name, version information, and target tag value corresponding to each of the one or more target tags.

[0084] As an optional embodiment, the preset database stores the user data in the form of a bitmap: Acquire the user data, and determine a tag value set for each tag, where the tag value set includes one or more tag values; Establishing a corresponding bitmap for each label value in the label value set of each label, the bitmap being used to represent users having the corresponding label value; The bitmap corresponding to each tag value is stored in the preset database.

[0085] As an optional embodiment, a corresponding bitmap is established for each label value in the label value set of each label, including: Acquire a user code set, the user code set including a user code of each user in the user data; Based on a preset rule, the user code set is divided into a plurality of user code slices, each user code slice including one or more user codes; For each user code slice: a corresponding bitmap is established for each label value in the label value set of each label, and the bitmap is used to represent the user code of the user with the corresponding label value in the user code slice.

[0086] As an optional embodiment, based on a preset rule, the user code set is divided into a plurality of user code fragments, including: Obtaining a value of a predetermined area in each user code in the user code set; A modulo operation is performed on the value of the predetermined area, and the user code set is divided into the plurality of user code fragments according to the result of the modulo operation.

[0087] As an optional embodiment, the second language type statement is used to execute the positioning rule; Correspondingly, determining the target object according to the second language type sentence and the preset database includes: By running the second language type statement, a bit map corresponding to the target label value in the positioning rule is obtained, and a bit operation is performed on the obtained bit map to obtain a target bit map; The target bitmap is used to represent the target object.

[0088] As an optional embodiment, the device 1000 is further used for: Converting the target bitmap into user data corresponding to the target object in a preset manner; The user data corresponding to the target object is synchronized to a preset data warehouse.

[0089] Embodiment 3 Fig.12 The schematic diagram of the hardware architecture of a computer device 10000 suitable for implementing the target object positioning method according to the third embodiment of the present application is shown. In some embodiments, the computer device 10000 may be a terminal device such as a smart phone, a wearable device, a tablet computer, a personal computer, a vehicle terminal, a game console, a virtual device, a workbench, a digital assistant, a set-top box, a robot, etc. In other embodiments, the computer device 10000 may be a rack server, a blade server, a tower server, or a cabinet server (including an independent server, or a server cluster composed of multiple servers), etc. Fig.12 As shown, the computer device 10000 includes but is not limited to: a memory 10010, a processor 10020, and a network interface 10030 that can communicate with each other through a system bus. Among them: The memory 10010 includes at least one type of computer-readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (such as SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10010 can be an internal storage module of the computer device 10000, such as a hard disk or memory of the computer device 10000. In other embodiments, the memory 10010 can also be an external storage device of the computer device 10000, such as a plug-in hard disk equipped on the computer device 10000, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 10010 can also include both the internal storage module of the computer device 10000 and its external storage device. In this embodiment, the memory 10010 is generally used to store an operating system and various application software installed in the computer device 10000, such as program code of a target object positioning method, etc. In addition, the memory 10010 can also be used to temporarily store various data that have been output or will be output.

[0090] In some embodiments, the processor 10020 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other chips. The processor 10020 is generally used to control the overall operation of the computer device 10000, such as performing control and processing related to data interaction or communication with the computer device 10000. In this embodiment, the processor 10020 is used to run the program code stored in the memory 10010 or process data.

[0091] The network interface 10030 may include a wireless network interface or a wired network interface, and the network interface 10030 is generally used to establish a communication link between the computer device 10000 and other computer devices. For example, the network interface 10030 is used to connect the computer device 10000 to an external terminal through a network, and to establish a data transmission channel and a communication link between the computer device 10000 and the external terminal. The network may be a wireless or wired network such as an intranet, the Internet, the Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, etc.

[0092] It should be pointed out that Fig.12 Only a computer device having components 10010 - 10030 is shown, but it should be understood that implementation of all of the components shown is not a requirement, and more or fewer components may alternatively be implemented.

[0093] In this embodiment, the target object positioning method stored in the memory 10010 can also be divided into one or more program modules and executed by one or more processors (such as processor 10020) to complete the embodiment of the present application.

[0094] Embodiment 4 An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the target object positioning method in the embodiment are implemented.

[0095] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the computer-readable storage medium can be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium can also be an external storage device of a computer device, such as a plug-in hard disk equipped on the computer device, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the computer-readable storage medium can also include both the internal storage unit of the computer device and its external storage device. In this embodiment, the computer-readable storage medium is generally used to store an operating system and various application software installed on the computer device, such as the program code of the target object positioning method in the embodiment. In addition, the computer-readable storage medium can also be used to temporarily store various types of data that have been output or are to be output.

[0096] Embodiment 5 An embodiment of the present application also provides a computer program product, including a computer program, which implements the method in the above embodiment when executed by a processor.

[0097] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present application can be implemented by general-purpose computer devices, they can be concentrated on a single computer device, or distributed on a network composed of multiple computer devices, optionally, they can be implemented by executable program codes of computer devices, so that they can be stored in a storage device and executed by the computer device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0098] It should be noted that the above are only preferred embodiments of the present application, and the patent protection scope of the present application is not limited thereto. Any equivalent structure or equivalent process transformation made using the contents of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for locating a target object, characterized in that: The method comprises: Acquire a positioning rule, wherein the positioning rule includes one or more target tags matching the target object, and a target tag value corresponding to each target tag; generating a first language type sentence according to the positioning rule; Acquire tag metadata, where the tag metadata is determined according to user data in a preset database, where the user data includes tag values ​​of each user under one or more tags, where the one or more tags include the one or more target tags; generating a second language type sentence for database query according to the first language type sentence and the tag metadata; The target object is determined according to the second language type sentence and the preset database.

2. The method according to claim 1, characterized in that: The tag metadata includes: the name and version information of the tag; Correspondingly, generating a target SQL statement according to the first language type statement and the tag metadata includes: Determining the one or more target tags and a target tag value of each target tag according to the first language type sentence; Determining, according to the tag metadata, names and version information corresponding to each of the one or more target tags; The second language type statement is generated based on preset rules according to the name, version information, and target tag value corresponding to each of the one or more target tags.

3. The method according to claim 1, characterized in that The preset database stores the user data in the form of a bitmap: Acquire the user data, and determine a tag value set for each tag, where the tag value set includes one or more tag values; Establishing a corresponding bitmap for each label value in the label value set of each label, the bitmap being used to represent users having the corresponding label value; The bitmap corresponding to each tag value is stored in the preset database.

4. The method according to claim 3, characterized in that: A corresponding bitmap is established for each label value in the label value set of each label, including: Acquire a user code set, the user code set including a user code of each user in the user data; Based on a preset rule, the user code set is divided into a plurality of user code slices, each user code slice including one or more user codes; For each user code slice: a corresponding bitmap is established for each label value in the label value set of each label, and the bitmap is used to represent the user code of the user with the corresponding label value in the user code slice.

5. The method according to claim 4, characterized in that Based on a preset rule, the user code set is divided into a plurality of user code segments, including: Obtaining a value of a predetermined area in each user code in the user code set; A modulo operation is performed on the value of the predetermined area, and the user code set is divided into the plurality of user code fragments according to the result of the modulo operation.

6. The method according to any one of claims 1 to 5, characterized in that: The second language type statement is used to execute the positioning rule; Correspondingly, determining the target object according to the second language type sentence and the preset database includes: By running the second language type statement, a bit map corresponding to the target label value in the positioning rule is obtained, and a bit operation is performed on the obtained bit map to obtain a target bit map; The target bitmap is used to represent the target object.

7. The method according to claim 6, characterized in that Also includes: Converting the target bitmap into user data corresponding to the target object in a preset manner; The user data corresponding to the target object is synchronized to a preset data warehouse.

8. A target object positioning device, characterized in that: The device comprises: A first acquisition module is used to acquire a positioning rule, wherein the positioning rule includes one or more target tags matching the target object and a target tag value corresponding to each target tag; A first generating module, used for generating a first language type sentence according to the positioning rule; A second acquisition module is used to acquire tag metadata, where the tag metadata is determined according to user data in a preset database, where the user data includes tag values ​​of each user under one or more tags, where the one or more tags include the one or more target tags; A second generating module, configured to generate a second language type statement for database query according to the first language type statement and the tag metadata; A determination module is used to determine the target object according to the second language type sentence and the preset database.

9. A computer device, characterized in that: include: at least one processor; and a memory communicatively connected to the at least one processor; wherein: The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to claims 1 to 7 are implemented.

Citation Information

Cited By

  • Object query method and device, computer equipment and readable storage medium

    CN121350066A