User profile data processing method and apparatus

Abstract

The application relates to the technical field of data processing, and provides a user portrait data processing method and device. The method comprises the following steps: determining label information corresponding to real-time data of a user according to a data identification calculation rule; determining a to-be-filled label, and acquiring the to-be-filled label from a set database according to the type of the to-be-filled label; filling object data according to the to-be-filled label, the label information and a field identifier; and determining a user portrait according to the object data and a calculation rule of the label. The embodiment of the application reduces a data landing link and full-memory data calculation, reduces I / O consumption, improves data calculation efficiency through a real-time portrait filling mode.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to a user profile data processing method and apparatus. Background Technology

[0002] Real-time data output is usually specialized data of a certain type, such as user signaling call detail records, user internet access log call detail records, and user billing call detail records. When using real-time data, it is necessary to combine it with user characteristic data and user mining data to present a comprehensive real-time user profile model.

[0003] Currently, the common approach is to first perform real-time computation, then store the data in a relational database, and finally perform join queries on the data. Because this approach requires both data storage and full-memory computation, it increases I / O consumption and therefore cannot meet the needs of real-time storage and computation of big data, thus reducing the efficiency of data computation. Summary of the Invention

[0004] This application provides a user profile data processing method and apparatus to solve the problem of low data computation efficiency.

[0005] In a first aspect, embodiments of this application provide a user profile data processing method, including:

[0006] The tag information corresponding to the user's real-time data is determined according to the data identification calculation rules;

[0007] Determine the label to be filled, and retrieve the label to be filled from a designated database according to the type of the label to be filled;

[0008] The object data is filled based on the label to be filled, the label information, and the field identifier;

[0009] User profiles are determined based on the object data and the calculation rules for the tags.

[0010] In one embodiment, retrieving the label to be filled from a designated database according to the type of the label to be filled includes:

[0011] If the type of the tag to be filled is a hot topic tag, then the hot topic tag is obtained from the first set database;

[0012] If the type of the tag to be filled is a non-hotspot tag, then the non-hotspot tag is obtained from the second set database;

[0013] The tags to be filled include the hot tags and the non-hot tags, and the setting database includes the first setting database and the second setting database.

[0014] In one embodiment, before determining the label to be filled and retrieving the label to be filled from a designated database according to the type of the label to be filled, the method further includes:

[0015] Determine the usage frequency of historical tags within a set time period, and designate historical tags whose usage frequency is greater than or equal to a set threshold as hot tags;

[0016] If the hotspot tag does not exist in the first set database, then the hotspot tag is stored in the first set database.

[0017] In one embodiment, determining the usage frequency of historical tags within a set time period, and then designating historical tags with usage frequencies greater than or equal to a set threshold as hotspot tags, includes:

[0018] The historical tags whose usage frequency is less than the set threshold are designated as non-hotspot tags;

[0019] Delete the non-hotspot tags from the first set database.

[0020] In one embodiment, before determining the tag information corresponding to the user's real-time data according to the data identifier calculation rule, the method further includes:

[0021] Identify all tags based on the user's phone number information;

[0022] The data of the full set of tags is stored according to the type of the full set of tags, and the full set of tags includes the hotspot tags and the non-hotspot tags.

[0023] In one embodiment, before determining the tag information corresponding to the user's real-time data according to the data identifier calculation rule, the method further includes:

[0024] Determine the data characteristics of the real-time data;

[0025] The data identifier calculation rule is determined based on the data characteristics.

[0026] In one embodiment, determining the user profile based on the object data and the tag calculation rules includes:

[0027] The label to be calculated is determined according to the label calculation rules.

[0028] The user profile is determined based on the object data and the tags to be calculated.

[0029] Secondly, embodiments of this application provide a user profile data processing apparatus, comprising:

[0030] The first determining module is used to determine the tag information corresponding to the user's real-time data according to the data identification calculation rules;

[0031] The second determining module is used to determine the label to be filled, and to retrieve the label to be filled from a set database according to the type of the label to be filled;

[0032] The fill module is used to fill object data according to the label to be filled, the label information, and the field identifier;

[0033] The third determining module is used to determine the user profile based on the object data and the calculation rules of the tags.

[0034] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory storing a computer program, wherein the processor executes the program to implement the steps of the user profile data processing method described in the first aspect.

[0035] Fourthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the steps of the user profile data processing method described in the first aspect.

[0036] The user profile data processing method and apparatus provided in this application embodiment determine the tag information corresponding to the user's real-time data according to data identifier calculation rules; determine the tags to be filled, and retrieve the tags to be filled from a set database according to the type of the tags to be filled; fill object data according to the tags to be filled, tag information, and field identifiers; and determine the user profile according to the object data and tag calculation rules. This application embodiment reduces the data persistence step and all-in-memory data calculation, thereby reducing I / O consumption and improving data calculation efficiency through real-time profile filling. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is one of the flowcharts illustrating the user profile data processing method provided in the embodiments of this application;

[0039] Figure 2 This is a second schematic flowchart of the user profile data processing method provided in the embodiments of this application;

[0040] Figure 3This is the third flowchart illustrating the user profile data processing method provided in the embodiments of this application;

[0041] Figure 4 This is the fourth flowchart illustrating the user profile data processing method provided in the embodiments of this application;

[0042] Figure 5 This is a schematic diagram of the user profile data processing method provided in the embodiments of this application;

[0043] Figure 6 This is a schematic diagram of the message-to-object processing flow provided in an embodiment of this application;

[0044] Figure 7 This is a schematic diagram of the real-time big data filling process provided in the embodiments of this application;

[0045] Figure 8 This is a flowchart illustrating the rule calculation method provided in the embodiments of this application;

[0046] Figure 9 This is a flowchart illustrating the multi-dimensional image calculation process provided in the embodiments of this application;

[0047] Figure 10 This is a schematic diagram of the structure of the number hash storage tag provided in an embodiment of this application;

[0048] Figure 11 This is a schematic diagram of the tag-based bucket storage structure provided in an embodiment of this application;

[0049] Figure 12 This is a schematic diagram of the structure of a random 4-bit character corresponding to a calculation slot provided in an embodiment of this application;

[0050] Figure 13 This is a schematic diagram of the storage structure of the full HBase tags provided in the embodiments of this application;

[0051] Figure 14 This is a schematic diagram of the user profile data processing device provided in the embodiments of this application;

[0052] Figure 15 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0054] Figure 1 This is one of the flowcharts illustrating the user profile data processing method provided in this application embodiment. (Refer to...) Figure 1 This application provides a user profile data processing method, which may include:

[0055] Step S10: Determine the tag information corresponding to the user's real-time data according to the data identifier calculation rules;

[0056] It should be noted that this embodiment uses Redis as an in-memory database and HBase as a real-time computing intermediate cache layer. HBase assists Redis in realizing full tag data queries, and linear scaling of HBase nodes satisfies highly parallel real-time computing. Redis is an open-source in-memory data structure server that can be used as a database, cache, and message queue broker; HBase is a distributed, column-oriented open-source database.

[0057] Specifically, the tag information corresponding to the user's real-time data is determined according to the data identification calculation rules. For example, the real-time data is labeled using the data identification calculation rules to obtain the corresponding tag information in the real-time data, such as gender, hobbies, age, etc.

[0058] It should be noted that the data identification calculation rules and label calculation rules are pre-set. For example, when the Flink program starts, the data identification calculation rules and hotspot label calculation rules are loaded and the data calculation rules are loaded into the Flink application cache. Flink can be understood as a distributed processing framework.

[0059] The calculation rules are as follows:

[0060] Data identifier calculation rules:

[0061] Original field name, new field name, calculation rule, field source (original field, original field processing, source profile);

[0062] Tag calculation rules:

[0063] Set multiple conditions, such as: (a!=xxx&&b!=xxx)||c=xxx, where a, b, and c are object field names, and xxx is the value set by the condition.

[0064] The tag field stores location information:

[0065] For example: Tag 1: Redis; Tag 2: HBase; Tag 3: HBase;

[0066] The data is loaded from the database into the Flink application's memory using Guava cache, which is set to update every 10 minutes. Guava cache is an open-source local caching tool library from Google that uses fine-grained locking with multiple segments to ensure thread safety while supporting high-concurrency scenarios. It also supports various types of cache cleanup strategies, including capacity-based cleanup, time-based cleanup, and reference-based cleanup.

[0067] In one embodiment, user data objects are pre-built to facilitate subsequent computation and processing. Specifically, Flink is used to parse data read from Kafka (an open-source stream processing platform). Based on defined message parsing rules, the message data is mapped to object fields using JSON to construct the user data object, such as location information, internet access information, and subscription information. For example, refer to... Figure 6 , Figure 6 This is a flowchart illustrating the message-to-object processing provided in this application embodiment. It receives a JSON string (i.e., a JSON string), verifies the JSON string, and if the JSON string is correct, constructs an object based on the JSON fields; if the JSON string is incorrect, it filters out non-JSON strings.

[0068] Step S20: Determine the label to be filled, and retrieve the label to be filled from the set database according to the type of the label to be filled;

[0069] It should be noted that after parsing the real-time data to determine the tag information, the tag information may not be sufficient to meet the needs of user profiling. Therefore, it is necessary to obtain the tags from the set database, which includes the Redis database (i.e., the first set database) and the HBase database (i.e., the second set database).

[0070] The process involves identifying the tags to be populated and retrieving them from a designated database based on their type. These tags can be understood as those not present in real-time data but required for user profiling. Specifically, if the tag type is a hot topic tag, it is retrieved from a first designated database; if the tag type is a non-hot topic tag, it is retrieved from a second designated database. The tags to be populated include both hot and non-hot topic tags, and the designated databases include both the first and second databases. In other words, hot topic tags are retrieved from a Redis database, and non-hot topic tags are retrieved from an HBase database. Retrieving tags from different databases based on their type improves tag retrieval efficiency.

[0071] Step S30: Fill in the object data according to the label to be filled, the label information, and the field identifier;

[0072] After determining the label information to be filled, the object data is filled based on the label to be filled, the label information, and the field identifier. The field identifier is used to identify the area to be filled for the label data. For example, refer to... Figure 7 During real-time big data population, data from source profiles is populated according to data identifier calculation rules. Based on the data source identifier of the field, the system automatically routes to Redis or HBase to retrieve tag data, which is then used to populate the streaming information for subsequent calculations. Specifically, Flink reads data from Kafka, parses the messages according to the set field rules, extracts fields from the original messages and assigns them to new object fields, and loads the rule engine to convert formulas requiring expression calculations into executable instances and cache them in memory to accelerate the streaming data calculation of similar formulas. Then, fields that need to be populated in Redis or HBase are identified, and messages for fields requiring HBase population are pushed back to Kafka, awaiting HBase query for streaming application calculation. This reduces the impact of slow HBase query speeds on overall real-time calculation efficiency. Data that does not require Redis or HBase population, or only requires Redis population, continues to the next step, such as using Redis to query tag population data.

[0073] Step S40: Determine the user profile based on the object data and the tag calculation rules.

[0074] It should be noted that user profiles refer to a tagged user model abstracted from information such as user attributes, user preferences, lifestyle habits, and user behavior.

[0075] After populating the object data, the user profile is determined based on the object data and the calculation rules for the tags. Specifically, the tags to be calculated are determined according to the tag calculation rules, and then the user profile is determined based on the object data and the tags to be calculated. For example, when performing multi-dimensional real-time calculation of the profile, based on a pre-set data identifier (which is a combination of multiple tags), the data of multiple tags is populated into the streaming data in real time. Corresponding calculation formulas are set, such as: Gender == Male && Terminal == iPhone. The TRUE or FLASE values ​​derived from the formula are used to determine in real time whether the record meets the conditions. Simultaneously, the data that meets the conditions is output, i.e., the user profile data, achieving the effect of real-time and accurate data acquisition while improving data calculation efficiency.

[0076] Based on the multidimensional label data queried and populated from Redis or HBase, and according to pre-defined calculation rules, the calculation formula is extracted. Figure 8The provided method converts the calculation formula into a class field string in the Groovy scripting language. Groovy is used to dynamically convert the script class into an executable instance. Then, the parameter values ​​corresponding to the calculation formula are substituted into the calculation formula, and the application code executes its calculation formula to obtain a TRUE or FLASE result. Data with a TRUE result is output, realizing real-time precision marketing based on individuals and real-time big data services related to individuals.

[0077] For example, refer to Figure 9 The program retrieves the data of the objects to be filled. Assuming the labels are "male" and "young man", it extracts the calculation formula, calculates the labels according to the formula, converts the formula into an instance, caches the formula, executes the calculation formula through the streaming application code, and outputs the result.

[0078] The user profile data processing method provided in this application involves determining the tag information corresponding to the user's real-time data according to data identifier calculation rules; determining the tags to be filled and retrieving the tags from a designated database according to their type; filling object data based on the tags to be filled, tag information, and field identifiers; and determining the user profile based on the object data and tag calculation rules. This application reduces the data persistence step and all-in-memory data computation, thereby reducing I / O consumption and improving data computation efficiency through real-time profile filling.

[0079] refer to Figure 2 , Figure 2 This is a second flowchart illustrating the user profile data processing method provided in this application embodiment. In this application embodiment, before determining the tag to be filled and retrieving the tag to be filled from a designated database according to the type of the tag, the method further includes:

[0080] Step S21: Determine the usage frequency of historical tags within a set time period, and designate historical tags whose usage frequency is greater than or equal to a set threshold as hotspot tags;

[0081] Step S22: If the hotspot tag does not exist in the first set database, then store the hotspot tag in the first set database.

[0082] It should be noted that trending tags refer to tags that are used frequently. Trending tags for different time periods can be predicted in advance by analyzing the usage of historical tags.

[0083] In this embodiment, the usage frequency of historical tags within a set time period (e.g., weekly / monthly) is determined. Historical tags with a usage frequency greater than or equal to a set threshold are designated as hot tags. If a hot tag does not exist in a first set database, it is stored in the first set database. For example, by using the usage frequency of historical tags, time-series data prediction is performed on the tag usage for each month of each year to predict in advance the tags with high usage for that month. These high-usage tags are then placed in a Redis database in advance. The usage of the current tag is calculated at a regular low frequency (e.g., every 10 minutes), and the current hot tags are switched to the Redis database in real time. Specifically, based on time-series data, hot tags are predicted. Each month, tags that are likely to have high usage in that month are calculated and loaded into the Redis database. By analyzing the annual usage patterns of tags, their cycle is defined as a month. Using the total monthly tag usage over the past five years as the coordinate points of the curve, the usage trend of tags is predicted based on the tag usage patterns over the past one or two years. At the same time, seasonal time-series data prediction is added. Combining the monthly and quarterly tag usage, the monthly tag usage is predicted. The tag data in the in-memory database is increased or decreased to meet the high-speed query of hot tags, and the source identifiers corresponding to the data tags are updated accordingly.

[0084] Instantly detect hot tags and use a timer to periodically scan tag usage. For currently detected hot tags, check if they exist in the in-memory database using the tag list stored in Redis. If not, extract the tag data from the tag database to the Redis database for high-speed tag querying, and update the source identifier corresponding to the data tag accordingly.

[0085] In one embodiment, historical tags with a usage frequency less than a set threshold are designated as non-hot tags, and then these non-hot tags are deleted from the first set database. It should be noted that, to ensure maximum efficiency of the Redis database and address the issues of low storage and high consumption, as well as the cluster congestion caused by automatic expiration of Redis database data, which impacts overall cluster computing performance, this embodiment employs a pre-computation method. It filters frequently used hot tags from historical tag usage data. Unlike the automatic expiration mechanism of the Redis database, newly added hot tags are imported into the Redis database every hour, while non-hot tag data is cleared daily. Based on this, the maximum efficiency of the Redis database is improved.

[0086] In one embodiment, to achieve fast retrieval of number tags and tag location in a Redis database, this application embodiment uses a hash model and a sortable list model to store the tags corresponding to the numbers, as well as a list of tags. For example, refer to... Figure 10 , Figure 10This is a schematic diagram of the structure of the number hash storage tag provided in an embodiment of this application.

[0087] To fully leverage the performance of Redis cluster nodes and accelerate tag location efficiency, tags are distributed across 16384 slots in the Redis database using a sortable list bucketing method to maximize tag matching performance. For example, refer to... Figure 11 , Figure 11 This is a schematic diagram of the tag binning storage structure provided in an embodiment of this application. Based on this, tag binning storage improves tag positioning efficiency.

[0088] To ensure slot 16384 is allocated, a random 4-character string is used. The CRC16 algorithm is used to calculate the value, and the remainder when divided by 16384 is taken. The character combinations corresponding to 0-16383 are then selected as the key values ​​for the sortable list. For example, see reference... Figure 12 , Figure 12 This is a schematic diagram illustrating the structure for calculating the random 4-digit character corresponding to a slot, as provided in an embodiment of this application. Based on this, the accuracy of slot allocation is improved by calculating the random 4-digit character corresponding to the slot.

[0089] refer to Figure 3 , Figure 3 This is the third flowchart illustrating the user profile data processing method provided in this application embodiment. In this application embodiment, before determining the tag information corresponding to the user's real-time data according to the data identifier calculation rules, the method further includes:

[0090] Step S11: Identify all tags based on the user's number information;

[0091] Step S12: Store the data of the full-volume tags according to the type of the full-volume tags, wherein the full-volume tags include the hotspot tags and the non-hotspot tags.

[0092] To alleviate storage pressure on the Redis database, all tags can be stored in a high-performance query-friendly HBase database. Specifically, all tags are identified based on the user's phone number information, and then the data is stored according to the type of each tag. It should be noted that while HBase's query performance is significantly lower than Redis's, it can utilize inexpensive disk space to store large amounts of data. Therefore, the last three digits of the phone number plus the number itself are used as the rowkey to store the tag information corresponding to the phone number. The phone number is divided into 1000 shards to accelerate query performance. Furthermore, the tags corresponding to the phone number are stored in multiple columns, categorized and stored in different column families. For example, see [reference]. Figure 13 , Figure 13 This is a schematic diagram of the storage structure of the full HBase tags provided in the embodiments of this application.

[0093] It should be noted that the full set of tags includes both hot and non-hot tags. The full set of tags is stored in an HBase database. When cached data is lost, the full set of tags can be read directly from the HBase database, thus achieving dual data protection.

[0094] It should be noted that in HBase databases, data sharding based on number ranges distributes data across different HBase storage nodes, thereby improving read speed and enhancing HBase's read performance through multiple hard drives.

[0095] This application embodiment identifies all tags based on the user's number information, and then stores the data of all tags according to the type of the all tags. Based on this, the storage pressure on the Redis database is reduced, and dual data protection is achieved.

[0096] refer to Figure 4 , Figure 4 This is the fourth flowchart illustrating the user profile data processing method provided in this application embodiment. In this application embodiment, before determining the tag information corresponding to the user's real-time data according to the data identifier calculation rules, the method further includes:

[0097] Step S13: Determine the data characteristics of the real-time data;

[0098] Step S14: Determine the data identifier calculation rule based on the data characteristics.

[0099] Specifically, the data characteristics of real-time data are determined, such as data call detail record type and data category. For example, in the telecommunications industry, this includes data characteristics such as power on / off, ordering, making / receiving calls, and sending / receiving SMS messages. Then, data identifier calculation rules are determined based on the data characteristics. The data identifier calculation rules are as follows:

[0100] 1. Obtain the original message field values ​​corresponding to the call detail record (CDR) and name the corresponding fields. For example, define the number as usrNbr to indicate that it is extracted from the original message.

[0101] 2. After processing the original message, add segment values ​​and set calculation formulas. For example, extract the first 5 digits from the card number identifier and set its expression as: substring(card number, 0, 6). At the same time, define more expressions, such as date calculation, string concatenation calculation, etc. Use string dynamic definition of operation expressions to obtain new field information.

[0102] 3. External data sources: Based on the profile identifiers defined in the rules, data source information is obtained from the Guava cache through the identifier field data. Data routing is performed according to the source of the tag information, and the data retrieval method is directed to the HBase database or Redis database.

[0103] 4. Build a rules engine using the Groovy scripting language, embedded in the Flink application. Rule expressions are pre-generated into the script, and a hashMap is used as the data input parameter. A common calling function is designed, and instances of the script transformations generated by the expressions are cached to accelerate subsequent data computation. For example, refer to... Figure 8 , Figure 8 This is a flowchart illustrating the rule calculation method provided in this application embodiment. Specifically, the calculation rules are loaded for the first time, converted into Groovy scripting language, then converted into Java instance, the Java instance is cached, the instance is executed according to the passed parameters, and finally the execution result is output.

[0104] Based on the characteristics of Groovy scripts, lexical analysis is used to break down expressions into segments, such as spaces, operators, and parentheses, to separate variables and functions. Based on predefined variable types, accurate class variables are defined. In the execution of calculation functions, variable values ​​are obtained through variable names and converted to the corresponding variable types, while rule calculations are performed.

[0105] Rule calculations are converted into class representations. For example, the calculation of a+b requires predefining the types of a and b. If a is defined as an integer and b is also defined as an integer, then the class representation is `int a, b`. The execution function is defined as `execute`, and the function body is implemented as `a = (int)map.get("a")` and `b = (int)map.get("b")`. The function return value is `return a+b`. For function calls, a common utility class is predefined and injected into the Groovy script using static functions. Functions can be directly defined in the class field strings generated by the Groovy script. When converted into an executable class, the defined static functions are directly called within the class, thus achieving the goal of converting strings into formulas. In Groovy, the `parseClass` method is used to convert dynamic strings into class instances, and the Guava cache is used to cache executable rule instances. Simultaneously, the rules are dynamically updated to achieve the effect of changing calculations as rules change, meeting the efficiency requirements of real-time big data computation.

[0106] This application embodiment improves the data calculation efficiency by determining the data characteristics of real-time data and then determining the data identifier calculation rules based on the data characteristics.

[0107] refer to Figure 5 , Figure 5 This is a schematic diagram of the user profile data processing method provided in the embodiments of this application.

[0108] In this embodiment, Redis is used as an in-memory database and HBase as a real-time computing intermediate cache layer. HBase assists Redis in querying the full set of tag data, and HBase nodes are linearly scaled to meet the requirements of highly parallel real-time computing. The specific implementation steps are as follows:

[0109] 1. When starting the Flink program, load the data identifier calculation rules and hotspot tag calculation rules, and load the data calculation rules into the Flink application cache;

[0110] 2. Flink is used to parse the data read from Kafka. Based on the set message parsing rules, the message data is mapped to the fields of the object in JSON format to construct the user's data object.

[0111] 3. Obtain data identifier calculation rules based on the data characteristics of real-time data;

[0112] 4. Tag storage, for example, storing hot tags in a Redis database and storing non-hot tags (i.e., cold tags) in an HBase database;

[0113] 5. Hotspot tag calculation, for example, preset hotspot tags for different time periods based on historical tag usage information;

[0114] 6. Real-time big data population;

[0115] 7. Real-time multi-dimensional image calculation.

[0116] This application uses an HBase database to assist a Redis database in querying all tagged data. By linearly expanding HBase nodes, it meets the requirements of highly parallel real-time computing. Based on this, the real-time performance of the data is improved, the data persistence process is reduced, and the full-memory data computing is reduced, thus reducing I / O consumption. This application can meet the real-time storage and computing needs of big data. By using real-time profile filling, it improves data computing efficiency to prepare for subsequent real-time acquisition of valuable data.

[0117] The user profile data processing apparatus provided in the embodiments of this application is described below. The user profile data processing apparatus described below can be referred to in correspondence with the user profile data processing method described above.

[0118] refer to Figure 14 , Figure 14 This is a schematic diagram of the user profile data processing device provided in the embodiments of this application. The user profile data processing device provided in the embodiments of this application includes a first determining module 1401, a second determining module 1402, a filling module 1403 and a third determining module 1404.

[0119] The first determining module 1401 is used to determine the tag information corresponding to the user's real-time data according to the data identification calculation rules;

[0120] The second determining module 1402 is used to determine the label to be filled and retrieve the label to be filled from a set database according to the type of the label to be filled.

[0121] The filling module 1403 is used to fill object data according to the label to be filled, the label information, and the field identifier;

[0122] The third determining module 1404 is used to determine the user profile based on the object data and the calculation rules of the tags.

[0123] The user profile data processing apparatus provided in this application embodiment determines the tag information corresponding to the user's real-time data according to data identifier calculation rules; determines the tags to be filled, and retrieves the tags to be filled from a set database according to the type of the tags to be filled; fills the object data according to the tags to be filled, tag information, and field identifiers; and determines the user profile according to the object data and tag calculation rules. This application embodiment reduces the data persistence step and the full-memory data calculation, thereby reducing I / O consumption and improving data calculation efficiency through real-time profile filling.

[0124] In one embodiment, the second determining module 1402 is specifically used for:

[0125] If the type of the tag to be filled is a hot topic tag, then the hot topic tag is obtained from the first set database;

[0126] If the type of the tag to be filled is a non-hotspot tag, then the non-hotspot tag is obtained from the second set database;

[0127] The tags to be filled include the hot tags and the non-hot tags, and the setting database includes the first setting database and the second setting database.

[0128] In one embodiment, the second determining module 1402 is specifically used for:

[0129] Determine the usage frequency of historical tags within a set time period, and designate historical tags whose usage frequency is greater than or equal to a set threshold as hot tags;

[0130] If the hotspot tag does not exist in the first set database, then the hotspot tag is stored in the first set database.

[0131] In one embodiment, the second determining module 1402 is specifically used for:

[0132] The historical tags whose usage frequency is less than the set threshold are designated as non-hotspot tags;

[0133] Delete the non-hotspot tags from the first set database.

[0134] In one embodiment, the first determining module 1401 is specifically used for:

[0135] Identify all tags based on the user's phone number information;

[0136] The data of the full set of tags is stored according to the type of the full set of tags, and the full set of tags includes the hotspot tags and the non-hotspot tags.

[0137] In one embodiment, the first determining module 1401 is specifically used for:

[0138] Determine the data characteristics of the real-time data;

[0139] The data identifier calculation rule is determined based on the data characteristics.

[0140] In one embodiment, the third determining module 1404 is specifically used for:

[0141] The label to be calculated is determined according to the label calculation rules.

[0142] The user profile is determined based on the object data and the tags to be calculated.

[0143] Figure 15 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 15 As shown, the electronic device may include: a processor 1510, a communication interface 1520, a memory 1530, and a communication bus 1540, wherein the processor 1510, the communication interface 1520, and the memory 1530 communicate with each other via the communication bus 1540. The processor 1510 can call a computer program in the memory 1530 to execute the steps of a user profile data processing method, such as including:

[0144] The tag information corresponding to the user's real-time data is determined according to the data identification calculation rules;

[0145] Determine the label to be filled, and retrieve the label to be filled from a designated database according to the type of the label to be filled;

[0146] The object data is filled based on the label to be filled, the label information, and the field identifier;

[0147] User profiles are determined based on the object data and the calculation rules for the tags.

[0148] Furthermore, the logical instructions in the aforementioned memory 1530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0149] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the steps of the user profile data processing method provided in the above embodiments, such as including:

[0150] The tag information corresponding to the user's real-time data is determined according to the data identification calculation rules;

[0151] Determine the label to be filled, and retrieve the label to be filled from a designated database according to the type of the label to be filled;

[0152] The object data is filled based on the label to be filled, the label information, and the field identifier;

[0153] User profiles are determined based on the object data and the calculation rules for the tags.

[0154] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0155] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0156] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for processing user profile data, characterized in that, include: The tag information corresponding to the user's real-time data is determined according to the data identification calculation rules; The data identification calculation rule includes a field source identifier, which includes a source profile. Based on the data identification calculation rules, the field whose source identifier is the source profile is determined, and the tag to be filled is retrieved from the set database according to the type of the tag to be filled. Based on the label to be filled, the label information, and the field identifier, the label to be filled and the label information are filled into the corresponding fields of the data object constructed for the user to obtain the filled object data; User profiles are determined based on the object data and the calculation rules for the tags; The step of retrieving the label to be filled from a designated database according to the type of the label to be filled includes: If the type of the tag to be filled is a hot topic tag, then the hot topic tag is obtained from the first set database; If the type of the tag to be filled is a non-hotspot tag, then the non-hotspot tag is obtained from the second set database; The tags to be filled include the hot tags and the non-hot tags, and the setting database includes the first setting database and the second setting database; the first setting database is an in-memory database used to store hot tag data; the second setting database is a real-time computing intermediate cache layer database used to store all tag data.

2. The user profile data processing method according to claim 1, characterized in that, Before determining the tag to be filled based on the field whose source identifier is the source profile in the data identifier calculation rule, and retrieving the tag to be filled from the set database according to the type of the tag to be filled, the method further includes: Determine the usage frequency of historical tags within a set time period, and designate historical tags whose usage frequency is greater than or equal to a set threshold as hot tags; If the hotspot tag does not exist in the first set database, then the hotspot tag is stored in the first set database.

3. The user profile data processing method according to claim 2, characterized in that, The step of determining the usage frequency of historical tags within a set time period, and then designating historical tags with usage frequencies greater than or equal to a set threshold as hotspot tags, includes: The historical tags whose usage frequency is less than the set threshold are designated as non-hotspot tags; Delete the non-hotspot tags from the first set database.

4. The user profile data processing method according to claim 1, characterized in that, Before determining the tag information corresponding to the user's real-time data according to the data identifier calculation rules, the method further includes: Identify all tags based on the user's phone number information; The data of the full set of tags is stored according to the type of the full set of tags, and the full set of tags includes the hotspot tags and the non-hotspot tags.

5. The user profile data processing method according to claim 1, characterized in that, Before determining the tag information corresponding to the user's real-time data according to the data identifier calculation rules, the method further includes: Determine the data characteristics of the real-time data; The data identifier calculation rule is determined based on the data characteristics.

6. The user profile data processing method according to claim 1, characterized in that, The step of determining the user profile based on the object data and the calculation rules of the tags includes: The label to be calculated is determined according to the label calculation rules; The user profile is determined based on the object data and the tags to be calculated.

7. A user profile data processing device, characterized in that, include: The first determining module is used to determine the tag information corresponding to the user's real-time data according to the data identification calculation rules; The data identification calculation rule includes a field source identifier, which includes a source profile. The second determining module is used to determine the label to be filled based on the field whose source identifier is the source profile in the data identifier calculation rule, and to obtain the label to be filled from the set database according to the type of the label to be filled; The filling module is used to fill the label to be filled and the label information into the corresponding fields of the data object constructed for the user according to the label to be filled, the label information and the field identifier, so as to obtain the filled object data; The third determining module is used to determine the user profile based on the object data and the calculation rules of the tags; The second determining module is further configured to: if the type of the tag to be filled is a hotspot tag, then obtain the hotspot tag from the first setting database; if the type of the tag to be filled is a non-hotspot tag, then obtain the non-hotspot tag from the second setting database; wherein the tag to be filled includes the hotspot tag and the non-hotspot tag, and the setting database includes the first setting database and the second setting database; the first setting database is an in-memory database used to store hotspot tag data; and the second setting database is a real-time computing intermediate cache layer database used to store all tag data.

8. An electronic device comprising a processor and a memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the user profile data processing method according to any one of claims 1 to 6.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the user profile data processing method according to any one of claims 1 to 6.

Images