A label system and processing method for multiple entities

Abstract

The present invention discloses a multi-entity labeling system and processing method. The system includes: a label multi-entity expansion module, a label rule setting module, and a universal label content storage module. The label entity expansion module is used to configure label entities and input the configured label entities into the label rule setting module. The label rule setting module is used to configure label processing rules of different levels and process label entities according to the configured label processing rules to obtain labels. The universal label content storage module is used to store, update, and maintain the labels obtained by the label rule setting module. The present invention realizes the expansion of label multi-entities, improves the system expansion capability, and simultaneously realizes the flexible setting of label rules and the storage of universal label content.

Description

Technical Field

[0001] The present invention relates to the technical field of labeling systems, and more particularly, to a multi-entity oriented labeling system and processing method. Background Art

[0002] As we transition from the internet era to the big data era, businesses and consumers alike are experiencing a series of significant changes and reshaping. The most significant change is the visibility of all consumer behavior data for businesses. However, the reality is that not all the massive amounts of data provided to businesses is usable information. The most effective way to help businesses quickly and accurately understand consumer spending intentions from this vast amount of data is to build clear user profiles. Tags are the fundamental elements of user profiles. They are statistically calculated based on user behavior and consumption data, encompassing all user dimensions. Their values are accurate, providing the data foundation for user profile creation. This is driven by the capabilities provided by the entire tagging system.

[0003] The most commonly used tagging system is a capability system based on a user's feature set. Among existing capability service providers, both in PaaS and SaaS, tagging systems generally suffer from insufficient scalability.

[0004] How can the system be expanded to include non-user (person) objects and other objects that can be labeled?

[0005] How to generate labeling rules flexibly and dynamically according to agreed rule syntax?

[0006] How to store the output of the tag content and support multiple tags (10,000 dimensions)?

[0007] Tag data, how to update, how to maintain, how to quickly and conveniently retrieve and construct tag content analysis.

[0008] When a tagging system faces the aforementioned issues in a specific business scenario, it indicates that the current system is insufficient and inadequate for building tag content in diverse scenarios. These weaknesses include insufficient scalability for different tagging entities, inflexible rules, insufficient storage capacity for tag content, and insufficient analysis capabilities. As data volumes increase and business becomes increasingly complex, the entire system may need to be redesigned (restructured) and undergoes other design considerations.

[0009] The prior art discloses a method, user data center, and system for creating entity tags. The method includes: receiving an XCAP operation message sent by an application server; decomposing the XCAP operation in the XCAP operation message and sending the decomposed XCAP operation; receiving an operation response returned by the XCAP server, the operation response carrying the entity tag of an XML document; creating an entity tag for the XML document that conforms to the application view, and associating the entity tag with the entity tag created by the XCAP server for the XML document; and returning an operation response message to the application server, the operation response carrying the entity tag created for the XML document that conforms to the application view. This solution does not implement extensions to tag entities, making it impossible to set flexible rules. Summary of the Invention

[0010] In order to overcome the above-mentioned defects of the existing labeling system, such as insufficient expansion capability, inflexible rules, and insufficient storage capacity of label content, the present invention provides a multi-entity oriented labeling system and processing method.

[0011] The primary purpose of the present invention is to solve the above technical problems, and the technical solutions of the present invention are as follows:

[0012] The first aspect of the present invention provides a multi-entity labeling system, comprising: a label multi-entity extension module, a label rule setting module, and a universal label content storage module. The label entity extension module is used to extend the label entity and input the extended label entity into the label rule setting module. The label rule setting module is used to set label processing rules of different levels and process the label entity according to the set label processing rules to obtain a label. The universal label content storage module is used to store and update and maintain the labels processed by the label rule setting module.

[0013] Furthermore, the label multi-entity extension module extends the label entity by creating a label tree type. The attributes of the label tree type include: type name, material table, labeling object, and description, where the material table is the input data source. The material table contains several related tables, and one or more material tables can be selected to form a material table group.

[0014] Furthermore, after the label tree type is created, it is stored in the form of a table, recorded as a label tree type table. The label tree type table includes the following fields: tree type ID, tree type name, primary key field of the entity table, output label Hive table name, data source list included in the tree type, user ID, user name, whether frozen, creation time, and update time. Among them, the primary key field of the entity table is the corresponding entity tag attribute, and the output label Hive table name is the corresponding material table.

[0015] Furthermore, the label entity expansion process is as follows:

[0016] Get material information in the material table;

[0017] Construct a tag tree type based on the material information to obtain several tag trees;

[0018] Import the corpus and generate tags using each tagging rule, and generate tagging tasks for the tags at set intervals;

[0019] The label task enters the task queue and performs asynchronous offline operations according to the queue strategy to generate different labels under the entity, thus completing the expansion of the label entity.

[0020] Furthermore, the specific process of obtaining the material information in the material table is as follows:

[0021] Call the preset data engine capability interface to obtain the metadata of the data source; based on the metadata, obtain the data information of the corresponding data source and complete the acquisition of material information in the material table;

[0022] The tag tree type is constructed based on the material information to obtain several tag trees. The specific process is as follows:

[0023] Select the generated material table as input, select the storage address of the output tag table, determine the entity primary key of the tag, and complete the construction of the tag tree;

[0024] The imported corpus generates labels using each labeling rule, and generates labeling tasks from the labels at set intervals. The specific process is: establishing screening rules, establishing label value rules, submitting tasks according to the screening rules and label value rules, and forming labeling tasks;

[0025] The label task enters the task queue and performs asynchronous offline operations according to the queue strategy to generate different labels under the entity, that is, to complete the expansion of the label entity. The specific process is: setting the label's operating cycle and operating parameters, and executing the label task according to the operating cycle and operating parameters.

[0026] Furthermore, the tag rule setting module includes: a rule protocol unit and a parser. The rule protocol unit is used to reversely drive the interactive construction of the front-end page, the definition of logical rules, the interface parameter transmission form and the design of the background rule parser; the parser converts the protocol into SQL syntax for different storage engines.

[0027] Furthermore, the rule protocol unit includes: a label rule rule part and a metadata meta part, wherein the label rule rule part includes: entityField, labelTable, valueType, valueField, ruleLeo, ruleCondition, aggLeo, aggCondition, ruleSql, the entityField represents the ID field of the label entity, and for user labels, it is the user ID field of the user table; the labelTable represents the table name of the output label;

[0028] The valueType represents the value of the final tag;

[0029] The valueField is used to determine the corpus type according to the value of the final label;

[0030] The ruleLeo represents the basic screening condition;

[0031] The ruleCondition represents the parsing result of the basic condition Leo expression;

[0032] The aggLeo represents the aggregation screening condition;

[0033] The aggCondition represents the parsing result of the aggregation condition Leo expression;

[0034] The ruleSql represents the SQL statement parsed by Leo after basic drag and drop, or the SQL statement directly input by advanced users;

[0035] The metadata meta part includes: fieldKeyMap, joinSql, where fieldKeyMap represents the definition of the corpus key, and the corpus key includes: type, groupField, and groupFunc. The type represents the field type, the groupField represents the group field, and the groupFunc represents the group function supported by the current field; the joinSql represents that the join clauses between multiple tables used in the current query condition are retrieved from a preset list.

[0036] Furthermore, the rule parsing process of the tag rule setting module is as follows:

[0037] Get the level of the rule and determine whether the rule level is basic or advanced;

[0038] If it is a basic rule, the front-end drags and drops the rule expression field name to be the key of the corpus, and the back-end parses it into the condition expression field name to be the key of the corpus. The computing platform is selected for computing based on the parsing target.

[0039] If it is an advanced rule, directly enter the SQL statement as the label task SQL statement.

[0040] Furthermore, a computing platform is selected for computing based on the analytical objectives, specifically:

[0041] If the parsing task is configured as an engine task, the preset distributed processing engine is selected for parsing. The steps are as follows:

[0042] Based on the field key used in the condition, the corresponding table and the fields to be used for join are searched from the corpus. The input data source operator is generated based on the found table. The conversion operator for the join table is generated based on the found table and the fields to be used for join. The common code table operator is constructed based on the condition expression, and the Hive output operator is constructed. The engine task configuration is completed based on the obtained operator.

[0043] If the parsing task is a tag task SQL statement, select SparkSQL or Hive for parsing. The steps are as follows:

[0044] According to the field key used in the condition, the corresponding table and the fields to be joined are searched from the corpus. A join clause is constructed based on the found table and the fields to be joined. A select clause is constructed based on the tag ID. According to the condition itself, the select is parsed into a where clause. The select clause, join clause, and where clause are concatenated to obtain an SQL statement.

[0045] Furthermore, the process of the Wanwei tag content storage module processing the tag job task is as follows:

[0046] Obtain entity tags and store all tag results in a temporary table in Hive, a big data technology stack, partitioned by date and tag ID.

[0047] Utilize Hbase's fast search function to integrate Hive and Hbase storage in a 1:1 data ratio.

[0048] Data is synchronized and updated to the Elasticsearch full-text search engine, providing the ability to quickly search for tagged data;

[0049] Combined with the backend script for regularly updating business data and the script for clearing Elasticsearch tasks, business data updates and expired tag clearing can be achieved.

[0050] A second aspect of the present invention provides a multi-entity oriented label processing method, which is applied to the multi-entity oriented labeling system, and includes the following steps:

[0051] S1: Use the tag entity expansion module to expand the tag entity and input the expanded tag entity into the tag rule setting module;

[0052] S2: The label rule setting module processes the input label entity according to the set label processing rules to obtain a label;

[0053] S3: Use the Wanwei tag content storage module to store, update and maintain the tags processed by the tag rule setting module.

[0054] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0055] The present invention constructs a multi-entity oriented label system, realizes the expansion of label multi-entities by using a label multi-entity expansion module, realizes the flexible setting of label rules by using a label rule setting module, and realizes the storage of universal label content by using a universal label content storage module. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a block diagram of a multi-entity labeling system according to an embodiment of the present invention.

[0057] Figure 2 A schematic diagram of a tag tree type is created for an embodiment of the present invention.

[0058] Figure 3 This is a schematic diagram of label rule settings according to an embodiment of the present invention.

[0059] Figure 4 This is a flowchart of the parsing process of the tag rule setting module in an embodiment of the present invention.

[0060] Figure 5 This is a flow chart of a multi-entity label processing method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0061] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein can be combined with each other.

[0062] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0063] Glossary

[0064] Material table: The data source for inputting labeling materials. Multiple material tables can be input in the labeling system.

[0065] Entity: The object that contains the tag content. There can only be one object in a tag tree type.

[0066] Tag tree type: The concept of an object that includes material tables and entity fields.

[0067] Tag tree: A tag node is a collection of tags organized in a tree-structured data structure with child nodes.

[0068] DSL: Domain Specific Language, domain specific language.

[0069] ETL: It is the abbreviation of Extract-Transform-Load, which is used to describe the process of extracting (extract), transforming (transform), and loading (load) data from the source end to the destination end.

[0070] Hive: A data warehouse tool based on Hadoop that can map structured data files into a database table and provide complete SQL query capabilities. It can convert SQL statements into MapReduce tasks for execution.

[0071] HBase: A distributed, column-oriented open source database.

[0072] ES: Short for Elasticsearch, an open source distributed search engine based on a RESTful web interface and built on Apache Lucene.

[0073] HiveSQL: SQL tool provided by the distributed big data engine Hive.

[0074] SparkSQL: SQL tool provided by the distributed big data engine Spark.

[0075] Example 1

[0076] like Figure 1As shown, a multi-entity labeling system includes: a label multi-entity expansion module, a label rule setting module, and a universal label content storage module. The label entity expansion module is used to expand the label entity and input the expanded label entity into the label rule setting module. The label rule setting module is used to set label processing rules of different levels and process the label entity according to the set label processing rules to obtain a label. The universal label content storage module is used to store and update and maintain the labels obtained by the label rule setting module.

[0077] It should be noted that the embodiment of the present invention supports users to expand label entities through the UI interface through the label multi-entity extension module and input the expanded label entities into the label rule setting module. The label rule setting module can customize the dynamic setting of label rules and process the label entities according to the set label processing rules. The processing results of the label rule setting module are stored and updated and maintained through the Wanwei label content storage module, thereby realizing the storage of labels and stable, low-latency label content updates and maintenance.

[0078] Furthermore, the label multi-entity extension module extends the label entity by creating a label tree type. The attributes of the label tree type include: type name, material table, labeling object, and description. The material table is the input data source, which contains several related tables. One or more material tables can be selected to form a material table group.

[0079] It should be noted that, in a specific embodiment, the extension is achieved by creating a tag tree type by setting a UI interface at the front end, such as Figure 2 As shown, the attributes of the tag tree type include: type name, material table, tag object, and description. Among them, the material table is the input data source, which contains several related tables. One or more material tables can be selected to form a material table group.

[0080] Furthermore, after the label tree type is created, it is stored in the form of a table, recorded as a label tree type table. The label tree type table includes the following fields: tree type ID, tree type name, primary key field of the entity table, output label Hive table name, data source list included in the tree type, user ID, user name, whether frozen, creation time, and update time. Among them, the primary key field of the entity table is the corresponding entity tag attribute, and the output label Hive table name is the corresponding material table.

[0081] It should be noted that the label tree type is associated with a label entity, such as user / product. After the label tree type is created, it is stored in the form of a table structure, recorded as a label tree type table. The label tree type integrates the concepts of material group information and table relationships. Table 1 shows the label tree type table.

[0082] Table 1 Tag tree type table

[0083]

[0084]

[0085] Among them, the primary key field of the entity table is the corresponding entity tag attribute, and the output tag Hive table name is the corresponding material table. For example, selecting the UserId field of a table means labeling the entity person, and selecting the productId field means labeling the entity product.

[0086] Furthermore, the process of label entity expansion is as follows:

[0087] Get material information in the material table;

[0088] More specifically, the capability interface of the preset data engine is called to obtain the metadata of the data source; based on the metadata, the data information of the corresponding data source is obtained to complete the acquisition of the material information of the material table;

[0089] Construct a tag tree type based on the material information to obtain several tag trees;

[0090] More specifically, select the generated material table as input, select the storage address of the output tag table, determine the entity primary key of the tag, and complete the construction of the tag tree;

[0091] Import the corpus and generate tags using each tagging rule, and generate tagging tasks for the tags at set intervals;

[0092] More specifically, establish filtering rules and label value rules, submit tasks based on the filtering rules and label value rules to form label tasks;

[0093] The label task enters the task queue and performs asynchronous offline operations according to the queue strategy, generating different labels under the entity, that is, completing the expansion of the label entity. More specifically, the label operation cycle and operation parameters are set, and the label task is executed according to the operation cycle and operation parameters.

[0094] Furthermore, the tag tree has multiple tags at multiple levels, and each tag tree includes fields such as: tree root tag ID and top-level periodic configuration; the tag includes a specific tagging rule, and the tag includes fields such as: screening rule, tag value rule, periodic task configuration, execution configuration, and tag tree ID; the tag task is a task generated periodically by the tag, and the tag task includes fields such as: tag ID, task type, runtime parameters, and generated HiveSQL; the corpus is imported when creating the tree type, and the tag screening conditions are selected from the corpus of the tree type, and the corpus includes fields such as: library, table, field, type, optional aggregation function, and join relationship between tables.

[0095] It should be noted that the label tree type entity in the present invention runs through the context of the label processing process, including: initialization construction - label job processing - label processing - label query. This link is based on the entity object, achieving low coupling and high scalability of configuration and code processing logic.

[0096] Example 2

[0097] This embodiment describes in detail the tag rule setting module and the rule parsing process.

[0098] The tag rule setting module includes: a rule protocol unit and a parser. The rule protocol unit is used to reversely drive the interactive construction of the front-end page, the definition of logical rules, the interface parameter transmission form and the design of the back-end rule parser; the parser converts the protocol into SQL syntax for different storage engines.

[0099] In the present invention, the rule protocol unit includes: a label rule rule part and a metadata meta part, wherein the label rule rule part includes: entityField, labelTable, valueType, valueField, ruleLeo, ruleCondition, aggLeo, aggCondition, ruleSql, and the entityField represents the ID field of the label entity. For user labels, it is the user ID field of the user table.

[0100] The labelTable represents the table name of the output label;

[0101] The valueType represents the final label value, which can be labelName (label name), fieldKey (field value), or groupFunc (group function calculation);

[0102] The valueField is used to determine the corpus type according to the value of the final tag; if valueType == 'fieldKey' fill in the corpus key, valueType == 'groupFunc' fill in <function> (<expected key>);

[0103] The ruleLeo represents the basic filtering conditions, and the front-end drag-and-drop structure, {"paramJson":{},"boolExpresssion":""};

[0104] The ruleCondition represents the result of parsing the basic condition Leo expression, {"must":[],"mustNot":[],"boolExpression":""};

[0105] The aggLeo represents the aggregation screening conditions, and the front-end drag-and-drop structure is {"paramJson":{},"boolExpresssion":""};

[0106] The aggCondition represents the parsing result of the aggregation condition Leo expression, {"must":[],"mustNot":

[0107] [],"boolExpression":""};

[0108] The ruleSql represents the SQL statement parsed by Leo after basic drag and drop or the SQL statement directly input by advanced users (must have fields such as id, labelID, labelName, etc.), "select user_id,now()+86400from****";

[0109] The metadata meta part includes: fieldKeyMap, joinSql, where fieldKeyMap represents the definition of the corpus key. The corpus key includes: type, groupField, and groupFunc. The type represents the field type, the groupField represents the group field, and the groupFunc represents the group function supported by the current field. The joinSql represents "from***join***on***" / / The join clauses between multiple tables used in the current query condition are found from the t_trident_corpus_join series tables.

[0110] It should be noted that, in a specific embodiment, the label rule setting module front end sets the rule items through the UI interface, such as Figure 3As shown, the rules can include basic rules and advanced rules. Several conditions can be added to both basic and advanced rules, and logical relationships can be set between different conditions. Based on the different tag entities obtained by the tag multi-entity extension module, the tag rule setting module uses basic rules and advanced rules to process data operations with different tags and generate different tag content.

[0111] Furthermore, the working process of the tag rule setting module is as follows: the end user chooses to create a tag entity information;

[0112] 1. Execute the labeling task and select the processing engine to execute the task from the labeling job execution workshop;

[0113] 2. Generate an execution plan, which includes:

[0114] a) Parse according to tag rules and generate task configuration;

[0115] b) Run specific job logic based on tag entity information;

[0116] 3. Generate an execution plan for a specific processing engine based on the selected processing engine and the parsed execution plan.

[0117] like Figure 4 As shown in the figure, the process of generating task configuration according to tag rule parsing is as follows (that is, the rule parsing working process of the tag rule setting module):

[0118] Get the level of the rule and determine whether the rule level is basic or advanced;

[0119] If it is a basic rule, the front-end drags and drops the rule expression field name to be the key of the corpus, and the back-end parses it into the condition expression field name to be the key of the corpus. The computing platform is selected for computing based on the parsing target.

[0120] If it is an advanced rule, directly enter the SQL statement as the label task SQL statement.

[0121] Furthermore, a computing platform is selected for computing based on the analytical objectives, specifically:

[0122] If the parsing task is configured as an engine task, the preset distributed processing engine is selected for parsing. The steps are as follows:

[0123] Based on the field key used in the condition, the corresponding table and the fields to be used for join are searched from the corpus. Based on the found table, an input data source operator is generated. Based on the found table and the fields to be used for join, a conversion operator for the join table is generated. Based on the condition expression, a common code table operator is constructed, and a Hive output operator is constructed. Based on the obtained operator, the factory task configuration is completed.

[0124] If the parsing task is a tag task SQL statement, select SparkSQL or Hive for parsing. The steps are as follows:

[0125] According to the field key used in the condition, the corresponding table and the fields to be joined are searched from the corpus. A join clause is constructed based on the found table and the fields to be joined. A select clause is constructed based on the tag ID. According to the condition itself, the select is parsed into a where clause. The select clause, join clause, and where clause are concatenated to obtain an SQL statement.

[0126] It should be noted that in the present invention, through the setting of the front-end page, the analysis and conversion in the background, and the final output to the front-end user experience, it greatly improves the efficiency of labeling for some non-technical operators and analysts.

[0127] Example 3

[0128] like Figure 5 As shown, the present invention also provides a multi-entity oriented label processing method, which is applied to the multi-entity oriented labeling system and includes the following steps:

[0129] S1: Use the tag entity extension module to expand the tag entity and input the expanded tag entity into the tag rule setting module;

[0130] S2: The label rule setting module processes the input label entity according to the set label processing rules to obtain a label;

[0131] S3: Use the Wanwei tag content storage module to store, update and maintain the tags processed by the tag rule setting module.

[0132] This embodiment describes in detail the processing process of the universal tag content storage module. The process of the universal tag content storage module processing the tag job task is as follows:

[0133] Obtain entity tags and store all tag results in a temporary table in Hive, a big data technology stack, partitioned by date and tag ID.

[0134] Utilize Hbase's fast search function to integrate Hive and Hbase storage in a 1:1 data ratio.

[0135] Data is synchronized and updated to the Elasticsearch full-text search engine, providing the ability to quickly search for tagged data;

[0136] Combined with the backend script for regularly updating business data and the script for clearing Elasticsearch tasks, business data updates and expired tag clearing can be achieved.

[0137] It should be noted that in a specific embodiment, the scanning cycle is a high-frequency scan at the minute level. The merging conditions that need to be considered when merging label jobs include: a. Merge timing: triggered by the periodic task time of the label tree, excluding tasks with individual custom times; b. Merge basis: join table and group by field; c. Merge threshold: number of tasks, task submission time, and expected task execution time / priority. When merging jobs, tasks are merged by label tree type (merging HBase tables and label tree types in a 1:1 ratio); when merging ES label refresh jobs, first a. merge tasks by label tree type (merging ES label tables and label tree types in a 1:1 ratio); second, b. through ES multi-index storage, by mapping the field dimension number, the target of the field associated with the universal label of an entity is maintained; when modifying the label job merge execution logic, all label tasks under the same label tree type will be put into a task for execution, and the tasks will be divided according to the tree ID. Each tree will be further divided into tasks according to the threshold of 20 labels, and so on.

[0138] Furthermore, the periodic processing of labeling tasks also includes: executing each labeling task, integrating the Hive vertical table into the HBase label table, and regularly updating the ES secondary index. The regular updating of the ES secondary index includes: regularly deleting expired ES labels and regularly updating ES parent document information.

[0139] It should be noted that the scheduling and distributed computing of the above-mentioned labeling jobs achieve the following technical effects: First, through ES's multi-index storage and the relational mapping of field dimensions, the goal of associating a field with a universal label for an entity is achieved; incremental label updates are performed regularly to ensure the timeliness of label data; all application-layer data is uniformly stored in ES for read-only operations, and only Hive temporary tables are used for write operations, ensuring the stability and reliability of label data applications throughout the entire system.

[0140] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A multi-entity labeling system, characterized in that: include: A tag multi-entity expansion module, a tag rule setting module, and a universal tag content storage module. The tag multi-entity expansion module is used to expand tag entities and input the expanded tag entities into the tag rule setting module. The tag rule setting module is used to set tag processing rules of different levels and process tag entities according to the set tag processing rules to obtain tags. The universal tag content storage module is used to store, update, and maintain tags processed by the tag rule setting module. The expansion process of the tag entity is as follows: Get material information in the material table; Construct a tag tree type based on the material information to obtain several tag trees; Importing a corpus and using each labeling rule to generate labels, and generating labeling tasks from the labels at set intervals; Importing a corpus and using each labeling rule to generate labels, and generating labeling tasks from the labels at set intervals, the specific process is: establishing screening rules, establishing label value rules, submitting tasks according to the screening rules and label value rules, and forming labeling tasks; The label task enters the task queue and performs asynchronous offline operations according to the queue strategy, generating different labels under the entity, that is, completing the expansion of the label entity; the label task enters the task queue and performs asynchronous offline operations according to the queue strategy, generating different labels under the entity, that is, completing the expansion of the label entity. The specific process is: setting the label's operating cycle and operating parameters, and executing the label task according to the operating cycle and operating parameters.

2. A multi-entity labeling system according to claim 1, characterized in that: The label multi-entity extension module extends the label entity by creating a label tree type. The attributes of the label tree type include: type name, material table, labeling object, and description. The material table is the input data source. The material table contains several related tables. One or more material tables are selected to form a material table group.

3. A multi-entity labeling system according to claim 2, characterized in that: After the tag tree type is created, it is stored in the form of a table, which is recorded as a tag tree type table. The tag tree type table includes the following fields: Tree type ID, tree type name, primary key field of the entity table, output label Hive table name, list of data sources included in the tree type, user ID, user name, whether frozen, creation time, and update time. The primary key field of the entity table is the corresponding entity tag attribute, and the output label Hive table name is the corresponding material table.

4. The multi-entity oriented labeling system according to claim 3, characterized in that: The specific process of obtaining the material information in the material table is as follows: Call the preset data engine capability interface to obtain the metadata of the data source; based on the metadata, obtain the data information of the corresponding data source and complete the acquisition of material information in the material table; The tag tree type is constructed based on the material information to obtain several tag trees. The specific process is as follows: Select the generated material table as input, select the storage address of the output tag table, determine the entity primary key of the tag, and complete the construction of the tag tree.

5. The multi-entity labeling system according to claim 1, characterized in that: The tag rule setting module includes: a rule protocol unit and a parser. The rule protocol unit is used to reversely drive the interactive construction of the front-end page, the definition of logical rules, the interface parameter transmission form and the design of the back-end rule parser; the parser converts the protocol into SQL syntax of different storage engines; The rule protocol unit includes: a label rule part and a metadata part. The label rule part includes: entityField, labelTable, valueType, valueField, ruleLeo, ruleCondition, aggLeo, aggCondition, and ruleSql. The entityField represents the ID field of the label entity. For user labels, it is the user ID field of the user table. The labelTable represents the table name of the output label. The valueType represents the value of the final tag; The valueField is used to determine the corpus type according to the value of the final label; The ruleLeo represents the basic screening condition; The ruleCondition represents the parsing result of the basic condition Leo expression; The aggLeo represents the aggregation screening condition; The aggCondition represents the parsing result of the aggregation condition Leo expression; The ruleSql represents the SQL statement parsed by Leo after basic drag and drop or the SQL statement directly input by advanced users; The metadata meta part includes: fieldKeyMap, joinSql, where fieldKeyMap represents the definition of the corpus key, and the corpus key includes: type, groupField, and groupFunc. The type represents the field type, the groupField represents the group field, and the groupFunc represents the group function supported by the current field; the joinSql represents that the join clauses between multiple tables used in the current query condition are retrieved from a preset list.

6. The multi-entity oriented labeling system according to claim 1, characterized in that: The rule parsing process of the tag rule setting module is as follows: Get the level of the rule and determine whether the rule level is basic or advanced; If it is a basic rule, the front-end drags and drops the rule expression field name to be the key of the corpus, and the back-end parses it into the condition expression field name to be the key of the corpus. The computing platform is selected for computing based on the parsing target. If it is an advanced rule, directly enter the SQL statement as the label task SQL statement.

7. The multi-entity oriented labeling system according to claim 6, characterized in that: Select a computing platform for processing based on the analytical objectives, specifically: If the parsing task is configured as an engine task, the preset distributed processing engine is selected for parsing. The steps are as follows: Based on the field key used in the condition, the corresponding table and the fields to be used for join are searched from the corpus. The input data source operator is generated based on the found table. The conversion operator for the join table is generated based on the found table and the fields to be used for join. The common code table operator is constructed based on the condition expression, and the Hive output operator is constructed. The engine task configuration is completed based on the obtained operator. If the parsing task is a tag task SQL statement, select SparkSQL or Hive for parsing. The steps are as follows: According to the field key used in the condition, the corresponding table and the fields to be joined are searched from the corpus. A join clause is constructed based on the found table and the fields to be joined. A select clause is constructed based on the tag ID. According to the condition itself, the select is parsed into a where clause. The select clause, join clause, and where clause are concatenated to obtain an SQL statement.

8. The multi-entity oriented labeling system according to claim 1, characterized in that: The process of the Wanwei tag content storage module processing the tag job task is as follows: Obtain entity tags and store all tag results in a temporary table in Hive, a big data technology stack, partitioned by date and tag ID. Utilize Hbase's fast search function to integrate Hive and Hbase storage in a 1:1 data ratio. Data is synchronized and updated to the Elasticsearch full-text search engine, providing the ability to quickly search for tagged data; Combined with the backend script for regularly updating business data and the script for clearing Elasticsearch tasks, business data updates and expired tag clearing can be achieved.

9. A multi-entity oriented label processing method, the method being applied to the multi-entity oriented labeling system according to any one of claims 1 to 8, characterized in that: The following steps are involved: S1: Use the label multi-entity extension module to expand the label entity and input the expanded label entity into the label rule setting module; S2: The label rule setting module processes the input label entity according to the set label processing rules to obtain a label; S3: Use the Wanwei tag content storage module to store, update and maintain the tags processed by the tag rule setting module.

Images