Big data-based user portrait intelligent generation method and system

Abstract

The application provides a user portrait intelligent generation method and system based on big data, relates to the technical field of big data processing, and comprises the following steps: acquiring user multi-source heterogeneous data and performing space-time labeling; extracting feature dimensions to establish a cross-source mapping relationship, and screening core feature primitives through a feature verification graph; constructing a correlation graph based on the core feature primitives and calculating a propagation weight to expand feature topology; dividing concentric levels based on the topology distance, and generating a user portrait after redirecting the correlation path by using a level dependency matrix. The method improves the accuracy and comprehensiveness of the user portrait.

Description

Technical Field

[0001] This invention relates to the field of big data processing technology, and in particular to a method and system for intelligent generation of user profiles based on big data. Background Technology

[0002] With the deepening application of big data, user profiling technology has become a crucial support for precision marketing and personalized recommendations. Current conventional practices typically revolve around the collection, cleaning, and feature extraction of multi-source heterogeneous data. Specifically, this involves extracting tagged attributes from data sources such as user behavior logs, transaction records, and social networks, and then using statistical induction or machine learning models to segment users or predict their attributes. These methods often rely on predefined tag systems or feature templates, employing weighted fusion, cluster analysis, or deep learning embedding to integrate scattered data into structured profiles.

[0003] At the feature engineering level, the conventional approach focuses on extracting features independently from each data source, and then completing cross-source data fusion through simple concatenation or relevance filtering. The correlation between features is usually based on static correlation calculations or empirical weight allocation, with little consideration for dynamic changes over time and the causal logic between data sources.

[0004] This approach still has shortcomings. The lack of a unified verification mechanism at the feature level for multi-source heterogeneous data leads to the mechanical absorption of contradictory features from different data sources due to noise, format differences, or time shifts, thus reducing the consistency and accuracy of the user profile. Especially when there are implicit conflicts between data sources (such as inconsistent preferences for the same user in different scenarios), simple fusion can cause confusion in user attributes, potentially resulting in self-contradictory labels in the generated profile. Furthermore, it ignores the dynamic evolution and hierarchical structure between features. User behavior changes over time, and the relationships between features also change, but conventional approaches typically use fixed-period full updates, failing to capture the collaborative changes between features based on time shifts. Simultaneously, the high feature dimensionality and flat structure lack characterization of the propagation paths and influence weights between core and peripheral features, resulting in a lagging response to changes in user behavior and limited generalization ability. These shortcomings make it difficult for the generated user profiles to meet the needs of complex application scenarios in terms of real-time performance, robustness, and interpretability. Summary of the Invention

[0005] This invention provides a method and system for intelligent generation of user profiles based on big data, which can solve the problems in the prior art.

[0006] A first aspect of this invention provides a method for intelligently generating user profiles based on big data, comprising:

[0007] Acquire multi-source heterogeneous data from users and perform spatiotemporal annotation to obtain a labeled dataset;

[0008] Extract a set of feature dimensions from the labeled dataset, establish cross-source mapping relationships between data fields of different data sources for each feature dimension, construct a feature verification graph through the cross-source mapping relationships, identify a subset of feature dimensions that form a closed-loop verification path in the feature verification graph, and select the core feature primitives from the feature dimension subsets;

[0009] Using core feature primitives as topology center nodes, candidate features are extracted based on the degree of collaborative change over time offset and an association graph is constructed. Connectivity paths from core feature primitives to candidate features are calculated in the association graph, and propagation weights are determined based on path length and association strength. Candidate features with propagation weights exceeding a preset growth threshold are included to form an extended feature topology.

[0010] In the extended feature topology, the extended feature topology is divided into multiple concentric levels according to the shortest topological distance from each feature node to the core feature primitive. Inter-level constraint edges are identified and a hierarchical dependency matrix is ​​constructed based on the inter-level transmission coefficient. The cross-level feature node association path is redirected through the hierarchical dependency matrix, and the redirected extended feature topology is used as the user profile.

[0011] Output the user profile.

[0012] In one optional embodiment, a set of feature dimensions is extracted from the labeled dataset. For each feature dimension, a cross-source mapping relationship is established between data fields from different data sources. A feature verification graph is constructed using this cross-source mapping relationship, and a subset of feature dimensions forming a closed-loop verification path is identified within the feature verification graph. Core feature primitives are selected from this subset of feature dimensions.

[0013] Extract a set of feature dimensions from the labeled dataset according to a predefined dimensional classification.

[0014] For each feature dimension, identify the data fields describing that feature dimension in different data sources, and extract the semantic identifier and data type identifier of the data fields. Based on the semantic identifier and data type identifier, establish a bidirectional mapping relationship between the data fields in different data sources to determine the cross-source mapping relationship.

[0015] The feature verification graph is constructed by using each feature dimension in the feature dimension set as a graph node and the mapping relationship connecting the corresponding data fields of the same feature dimension in different data sources in the cross-source mapping relationship as graph edges.

[0016] Search the graph nodes in the feature verification graph where the starting point and ending point are the same feature dimension and pass through at least three different data sources to determine the closed-loop verification path. The feature dimensions that form the closed-loop verification path constitute a subset of feature dimensions.

[0017] Extract the timestamp sequences associated with each feature dimension in the feature dimension subset in the labeled dataset, construct the temporal dependency chain between the timestamp sequences, and identify the feature dimensions that form a directed acyclic propagation path in the temporal dependency chain as the core feature primitives.

[0018] In one optional embodiment, the method of searching the feature verification graph for graph nodes whose start and end points are the same feature dimension and which pass through at least three different data sources to determine the closed-loop verification path includes:

[0019] Select any feature dimension from the feature verification graph as the search starting point, perform a depth-first traversal and record the data source sequence and graph edge sequence traversed by the traversal path. When the traversal returns to the search starting point, count the number of different data sources in the data source sequence.

[0020] For traversal paths with at least three edges, the mapping intensity of each edge in the cross-source mapping relationship is extracted from the graph edge sequence, and the path intensity of the traversal path is obtained by accumulating the mapping intensity of each edge.

[0021] Extract the number of data records of each data source in the data source sequence from the standardized dataset, calculate the proportion of the number of data records in the total number of records in the standardized dataset to determine the data source weight, and sum the weights of each data source in the data source sequence to obtain the coverage of the traversal path.

[0022] The comprehensive score of the traversal path is calculated by weighting the path strength and coverage. The traversal path with a comprehensive score exceeding the preset score threshold is identified as the closed-loop verification path.

[0023] In one optional embodiment, using core feature primitives as topological center nodes, extracting candidate features and constructing an association graph based on the degree of collaborative change over time offsets includes:

[0024] Extract the value sequences of core feature primitives and other features from the labeled dataset, and calculate the degree of co-change between other features and core feature primitives at different time offsets;

[0025] Identify the time offset corresponding to the peak of the degree of collaborative change, mark other features whose absolute value of the time offset is less than a preset offset threshold as candidate features, and extract the peak value of the degree of collaborative change of the candidate feature as the association strength.

[0026] A graph structure is constructed with core feature primitives as the topological center node and candidate features as the peripheral nodes. Connection edges that carry the correlation strength are established between the core feature primitives and each candidate feature, and time offsets are assigned to each candidate feature node.

[0027] The frequency of co-occurrence among candidate features in the data records is statistically analyzed. For candidate feature pairs whose co-occurrence frequency exceeds a preset co-occurrence threshold, connection edges are established between the candidate features to form an association graph.

[0028] In one optional embodiment, the process of calculating connected paths from core feature primitives to candidate features in the association graph, determining propagation weights based on path length and association strength, and incorporating candidate features with propagation weights exceeding a preset growth threshold into the expanded feature topology includes:

[0029] In the association graph, starting from the core feature primitive, a breadth-first traversal is performed on each candidate feature along the connecting edges. The sequence of connecting edges from the core feature primitive to each candidate feature is recorded as a connected path. The path length is obtained by counting the number of connecting edges in each connected path.

[0030] For each candidate feature, the association strength carried by each connecting edge on the connected path corresponding to the candidate feature is extracted. The association strength is constructed into an intensity sequence according to the propagation order in the connected path. The association strength of each connecting edge is attenuated and adjusted according to the position index of each connecting edge in the intensity sequence. The attenuated and adjusted association strengths are accumulated and summed to obtain the path attenuation propagation value.

[0031] The propagation weight of the candidate feature is obtained by comparing the path decay propagation value with the path length of the connected path corresponding to the candidate feature.

[0032] Candidate features whose propagation weight exceeds the preset growth threshold are included in the extended feature set. The core feature primitive is used as the initial topology node. Candidate features in the extended feature set are added one by one as extended nodes. All connecting edges and passing nodes on the connecting path between the core feature primitive and each extended node are extracted from the association graph. The connecting edges and passing nodes are attached to the initial topology node according to the topological connection relationship to form a multi-level topology structure, which constitutes the extended feature topology.

[0033] In an optional embodiment, in the extended feature topology, based on the shortest topological distance from each feature node to the core feature primitive, the extended feature topology is divided into multiple concentric levels. Inter-level constraint edges are identified, and a hierarchical dependency matrix is ​​constructed based on the inter-level transmission coefficients. The hierarchical dependency matrix is ​​used to redirect the cross-level feature node association paths. The redirected extended feature topology is used as the user profile, including:

[0034] Calculate the shortest topological distance from each feature node to the core feature primitive, and group feature nodes with the same shortest topological distance into the same level to form multiple concentric hierarchical structures.

[0035] Traverse the connecting edges in the extended feature topology, identify the connecting edges connecting feature nodes at different levels as inter-layer constraint edges, extract the corresponding association strength for each inter-layer constraint edge, obtain the level number of the starting feature node and the target feature node connected by each inter-layer constraint edge, calculate the difference of the level number as the cross-layer span, and calculate the inter-layer transmission coefficient by performing a ratio operation between the association strength and the cross-layer span. Construct a level dependency matrix with the level number as the row and column index and the inter-layer transmission coefficient corresponding to each inter-layer constraint edge as the matrix element.

[0036] For feature node association paths spanning multiple levels, the inter-layer transmission coefficients corresponding to each adjacent level between the starting and ending levels are extracted from the hierarchical dependency matrix, and the path transmission attenuation value is obtained by multiplying the inter-layer transmission coefficients together.

[0037] Based on the path conduction attenuation value, related paths are filtered, and the paths to be redirected are adjusted to pass through the feature nodes with the largest inter-layer conduction coefficients in sequence along the hierarchical order, forming an extended feature topology after redirection as a user profile.

[0038] In one optional embodiment, associated paths are filtered based on path conduction attenuation values, and the paths to be redirected are adjusted to redirected paths that sequentially pass through the feature nodes with the largest inter-layer conduction coefficients along the hierarchical order, forming an extended feature topology after redirection as a user profile, including:

[0039] Traverse the associated paths of feature nodes spanning multiple levels in the extended feature topology, mark associated paths with a path propagation attenuation value less than a preset propagation threshold as paths to be redirected, and mark associated paths with a path propagation attenuation value greater than or equal to the propagation threshold as paths to be retained.

[0040] For each redirection path, the corresponding starting feature node's level is extracted as the starting level, and the ending feature node's level is extracted as the ending level. The starting and ending levels are traversed layer by layer in order of level number. For the current level, the inter-level transmission coefficient from the current level to the next level is queried from the level dependency matrix. The feature node of the current level with the maximum value of the inter-level transmission coefficient is selected as the path node. The path nodes are connected in order of level to form the redirection path.

[0041] Replace the paths to be redirected in the extended feature topology with the redirected paths, and retain the reserved paths to form the extended feature topology after redirection and determine the user profile.

[0042] A second aspect of this invention provides a user profile intelligent generation system based on big data, comprising:

[0043] The data annotation unit is used to acquire multi-source heterogeneous data from users and perform spatiotemporal annotation to obtain an annotated dataset.

[0044] The core feature unit is used to extract a set of feature dimensions from the labeled dataset. For each feature dimension, a cross-source mapping relationship is established between data fields of different data sources. A feature verification graph is constructed through the cross-source mapping relationship, and a subset of feature dimensions that form a closed-loop verification path is identified in the feature verification graph. The feature dimension subset is selected as the core feature primitive.

[0045] The feature expansion unit is used to extract candidate features and construct an association graph based on the degree of cooperative change of the core feature primitive as the topology center node. In the association graph, the connected path from the core feature primitive to the candidate feature is calculated, and the propagation weight is determined based on the path length and association strength. Candidate features with propagation weights exceeding the preset growth threshold are included to form an expanded feature topology.

[0046] The hierarchical repositioning unit is used to divide the extended feature topology into multiple concentric levels based on the shortest topological distance from each feature node to the core feature primitive in the extended feature topology, identify inter-level constraint edges and construct a hierarchical dependency matrix based on the inter-level transmission coefficient, redirect the cross-level feature node association path through the hierarchical dependency matrix, and use the redirected extended feature topology as a user profile.

[0047] The user profile output unit is used to output the user profile.

[0048] A third aspect of the present invention provides an electronic device, comprising:

[0049] processor;

[0050] Memory used to store processor-executable instructions;

[0051] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0052] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0053] In this embodiment of the invention, after spatiotemporal annotation processing of multi-source heterogeneous data, the timeliness and spatial correlation of user profiles are significantly enhanced, accurately reflecting user behavior patterns at different times and locations. The identification of closed-loop verification paths in the feature verification graph effectively filters data redundancy and false associations. Core feature primitives, as stable and reliable topological center nodes, significantly improve the noise resistance and basic accuracy of user profiles. The association graph constructed around the core feature primitives fully considers the degree of collaborative change under time offsets, making the extraction of candidate features dynamically adaptive and capable of capturing the evolutionary patterns of user behavior trends. The setting of propagation weights and growth thresholds ensures the association strength and logical rationality of feature expansion. Expanding the feature topology retains key associations while suppressing weak correlation interference, thereby enhancing the robustness and sensitivity of user profiles. The division of concentric levels, combined with the hierarchical dependency matrix of inter-level transmission coefficients, realizes intelligent redirection of cross-level feature node association paths. This mechanism avoids deviations and breaks in information transmission between layers, enabling the final output user profile to have a clear hierarchical structure and causal path, facilitating the accurate implementation of subsequent personalized recommendations and behavior predictions, and significantly improving the interpretability and practicality of the profile. Attached Figure Description

[0054] Figure 1 A flowchart illustrating the intelligent generation method for user profiles based on big data;

[0055] Figure 2 This is a flowchart of feature topology hierarchy dependency and redirection. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0058] Figure 1 This is a flowchart illustrating the intelligent generation method for user profiles based on big data, as described in an embodiment of the present invention.

[0059] The intelligent generation methods for user profiles based on big data include:

[0060] Acquire multi-source heterogeneous data from users and perform spatiotemporal annotation to obtain a labeled dataset;

[0061] Extract a set of feature dimensions from the labeled dataset, establish cross-source mapping relationships between data fields of different data sources for each feature dimension, construct a feature verification graph through the cross-source mapping relationships, identify a subset of feature dimensions that form a closed-loop verification path in the feature verification graph, and select the core feature primitives from the feature dimension subsets;

[0062] Using core feature primitives as topology center nodes, candidate features are extracted based on the degree of collaborative change over time offset and an association graph is constructed. Connectivity paths from core feature primitives to candidate features are calculated in the association graph, and propagation weights are determined based on path length and association strength. Candidate features with propagation weights exceeding a preset growth threshold are included to form an extended feature topology.

[0063] In the extended feature topology, the extended feature topology is divided into multiple concentric levels according to the shortest topological distance from each feature node to the core feature primitive. Inter-level constraint edges are identified and a hierarchical dependency matrix is ​​constructed based on the inter-level transmission coefficient. The cross-level feature node association path is redirected through the hierarchical dependency matrix, and the redirected extended feature topology is used as the user profile.

[0064] Output the user profile.

[0065] In one optional embodiment, a set of feature dimensions is extracted from the labeled dataset. For each feature dimension, a cross-source mapping relationship is established between data fields from different data sources. A feature verification graph is constructed using this cross-source mapping relationship, and a subset of feature dimensions forming a closed-loop verification path is identified within the feature verification graph. Core feature primitives are selected from this subset of feature dimensions.

[0066] Extract a set of feature dimensions from the labeled dataset according to a predefined dimensional classification.

[0067] For each feature dimension, identify the data fields describing that feature dimension in different data sources, and extract the semantic identifier and data type identifier of the data fields. Based on the semantic identifier and data type identifier, establish a bidirectional mapping relationship between the data fields in different data sources to determine the cross-source mapping relationship.

[0068] The feature verification graph is constructed by using each feature dimension in the feature dimension set as a graph node and the mapping relationship connecting the corresponding data fields of the same feature dimension in different data sources in the cross-source mapping relationship as graph edges.

[0069] Search the graph nodes in the feature verification graph where the starting point and ending point are the same feature dimension and pass through at least three different data sources to determine the closed-loop verification path. The feature dimensions that form the closed-loop verification path constitute a subset of feature dimensions.

[0070] Extract the timestamp sequences associated with each feature dimension in the feature dimension subset in the labeled dataset, construct the temporal dependency chain between the timestamp sequences, and identify the feature dimensions that form a directed acyclic propagation path in the temporal dependency chain as the core feature primitives.

[0071] In one specific implementation, after obtaining the labeled dataset, it is necessary to systematically extract dimensional information that reflects user characteristics. The extraction of feature dimensions follows a pre-defined dimensional classification system, which typically covers multiple categories such as basic user attributes, behavioral characteristics, preferences, social relationships, and spending power. For e-commerce scenarios, this might include dimensions such as purchase frequency, browsing duration, product category preference, price sensitivity, and active time periods; for social platforms, it might cover dimensions such as interaction frequency, content preference, relationship network density, and posting time. The extraction process involves traversing all data fields in the labeled dataset and classifying them into corresponding feature dimension categories based on field names, statistical characteristics of field values, and semantic meaning of fields in business logic, ultimately forming a set containing several feature dimensions.

[0072] For each feature dimension in the feature dimension set, it is necessary to locate the specific data field that can describe that dimension in multiple data sources. Due to differences in data collection, storage, and naming conventions among different data sources, the same feature dimension may be presented with different field names and data formats in different data sources. For example, user age information may be recorded as the "user_age" field and stored as an integer in an e-commerce system, recorded as the "birth_year" field in a social platform requiring age calculation, and stored as an age range in a third-party data source. To establish the relationships between these fields, the semantic identifier of each data field is first extracted. The semantic identifier is determined by analyzing the semantic features of the field name, field comments, field values, and the field's role in the business process. Natural language processing techniques can be used to perform lexical analysis and semantic parsing on the field names to extract key semantic components. Simultaneously, the data type identifier of the data fields is extracted, including basic data types such as integer, floating-point, string, and date, as well as derived data features such as numerical range, enumerated value set, and string format pattern.

[0073] Based on the extracted semantic and data type identifiers, a bidirectional mapping relationship is established between data fields from different data sources. Establishing this bidirectional mapping relationship requires comprehensive consideration of semantic similarity and data type compatibility. Semantic similarity is measured by calculating the degree of similarity between the semantic identifiers of two fields. This can be achieved using a word vector model to calculate the cosine similarity between semantic vectors, or by using a knowledge graph-based semantic distance metric. Data type compatibility determines whether the data types of two fields can be mutually converted using deterministic conversion rules. For example, integer age and birth year can be converted using subtraction, and age range and specific age can be converted using range inclusion relationships. When the semantic similarity between two fields exceeds a preset similarity threshold and the data types meet the compatibility requirements, a bidirectional mapping relationship is established between these two fields. This mapping relationship records information such as the source field, target field, conversion rules, and mapping confidence. By traversing all feature dimensions across all data fields in all data sources, a complete cross-source mapping relationship network is established.

[0074] A feature verification graph is constructed using the established cross-source mapping relationships. The feature verification graph is a directed graph structure where nodes represent individual feature dimensions in the feature dimension set, and edges represent cross-source mapping relationships. Specifically, each feature dimension in the feature dimension set is treated as a node in the graph. For cross-source mapping relationships connecting corresponding data fields of the same feature dimension in different data sources, these are transformed into directed edges in the graph. For example, if the feature dimension "user age" corresponds to the field "age_A" in data source A and the field "age_B" in data source B, and there is a bidirectional mapping relationship between them, then in the feature verification graph, starting from the node "user age," an edge labeled "data source A" points to an intermediate node, then an edge labeled "mapping relationship" points to an intermediate node labeled "data source B," and finally back to the node "user age." This graph structure clearly shows the mapping path and verification chain of feature dimensions across multiple data sources.

[0075] The search process involves searching for closed-loop verification paths within the constructed feature verification graph. A closed-loop verification path is defined as a path whose starting and ending points are the same feature dimension node, and which passes through at least three different data source nodes. The existence of a closed-loop verification path means that the data for that feature dimension can form a mutually verifying closed loop across multiple independent data sources, thereby improving the credibility and consistency of the data for that feature dimension. The search process employs either depth-first search or breadth-first search algorithms. Starting from each feature dimension node, the search traverses along the graph edges, recording the data source identifiers traversed by the traversal path. When the traversal returns to the starting node and the number of different data sources traversed by the path reaches or exceeds three, a closed-loop verification path is confirmed to have been found. Feature dimensions with existing closed-loop verification paths are included in a subset of feature dimensions. Feature dimensions in this subset have higher data quality assurance because their data can be used for multi-source cross-validation to identify and correct data anomalies.

[0076] Core feature primitives are further filtered from the feature dimension subset. Core feature primitives refer to feature dimensions that occupy the source or key transit node position in the temporal dependency relationship. The timestamp sequences associated with each feature dimension in the feature dimension subset are extracted from the labeled dataset. These timestamp sequences record the changes in the corresponding data of that feature dimension at different points in time. By analyzing the temporal correlation between the timestamp sequences of different feature dimensions, a temporal dependency chain is constructed. The temporal dependency chain reflects the causal or sequential relationship between feature dimensions in the temporal dimension. For example, a user's browsing behavior usually occurs before a purchase behavior; therefore, the "browsing product categories" feature dimension is temporally dependent on the "purchasing product categories" feature dimension. The identification of temporal dependencies can be achieved by calculating the temporal offset correlation between two timestamp sequences. When the change time point of feature dimension A is systematically earlier than the change time point of feature dimension B, and there is a statistically significant correlation between the two, a temporal dependency relationship from A to B is established.

[0077] In the constructed temporal dependency chain, feature dimensions forming directed acyclic propagation paths are identified. A directed acyclic propagation path refers to a path in the temporal dependency chain that starts from a certain feature dimension and propagates along the direction of the temporal dependency, without forming a circular dependency. Feature dimensions forming directed acyclic propagation paths are usually located upstream in the dependency chain; their changes trigger a chain reaction of changes in downstream feature dimensions, thus possessing stronger explanatory power and predictive value. The identification process is implemented using a topological sorting algorithm. First, it detects whether there are cycles in the temporal dependency chain. If cycles exist, they need to be eliminated by adjusting the dependency strength threshold or introducing time window constraints. In the acyclic temporal dependency chain, the in-degree and out-degree of each feature dimension are calculated. Feature dimensions with zero in-degree or a small in-degree and a large out-degree are usually located at the source of the dependency chain, and these feature dimensions are identified as core feature primitives. Core feature primitives constitute the basic framework for user profile construction; subsequent feature expansion and correlation analysis will all start from these core feature primitives.

[0078] In one optional embodiment, the method of searching the feature verification graph for graph nodes whose start and end points are the same feature dimension and which pass through at least three different data sources to determine the closed-loop verification path includes:

[0079] Select any feature dimension from the feature verification graph as the search starting point, perform a depth-first traversal and record the data source sequence and graph edge sequence traversed by the traversal path. When the traversal returns to the search starting point, count the number of different data sources in the data source sequence.

[0080] For traversal paths with at least three edges, the mapping intensity of each edge in the cross-source mapping relationship is extracted from the graph edge sequence, and the path intensity of the traversal path is obtained by accumulating the mapping intensity of each edge.

[0081] Extract the number of data records of each data source in the data source sequence from the standardized dataset, calculate the proportion of the number of data records in the total number of records in the standardized dataset to determine the data source weight, and sum the weights of each data source in the data source sequence to obtain the coverage of the traversal path.

[0082] The comprehensive score of the traversal path is calculated by weighting the path strength and coverage. The traversal path with a comprehensive score exceeding the preset score threshold is identified as the closed-loop verification path.

[0083] In one specific implementation, after the feature verification graph is constructed, it is necessary to identify a subset of feature dimensions with closed-loop verification capabilities from the graph structure. The core value of the closed-loop verification path lies in ensuring the reliability and consistency of feature dimensions through cross-validation across multiple data sources. Specifically, when a feature dimension can form a closed-loop path across different data sources, it means that the feature is reflected in multiple independent data sources, and there is a traceable mapping relationship between these data sources, thereby mutually verifying the accuracy of the feature values.

[0084] In practice, any feature dimension node is arbitrarily selected from the feature verification graph as the starting point for the search. Nodes in the feature verification graph consist of combinations of feature dimensions and data sources, such as "user age - e-commerce platform data source" or "user age - social media data source." After selecting the starting point, a depth-first search strategy is used to search the graph. The advantage of depth-first search is that it can continuously explore along a path until it cannot proceed further or returns to the starting point; this method is particularly suitable for discovering closed-loop structures.

[0085] During the traversal, two key types of information need to be recorded in real time: the data source sequence and the graph edge sequence. The data source sequence records all data source identifiers traversed by the traversal path, such as passing through data sources like "e-commerce platform," "social media," and "mobile application" in sequence. The graph edge sequence records the connecting edges between adjacent nodes in the path, with each edge representing a cross-source mapping relationship. When the traversal process returns to the initially selected search starting point, it indicates that a closed loop has been formed. At this point, the data source sequence is deduplicated, and the number of different data sources it contains is calculated.

[0086] For traversal paths with at least three data sources, their validation strength is further evaluated. The mapping strength value corresponding to each edge is extracted from the graph edge sequence. The mapping strength reflects the reliability of the feature mapping between two data sources and is usually calculated when establishing the cross-source mapping relationship; its value can be a continuous value between 0 and 1. The overall path strength is obtained by summing the mapping strengths of all graph edges in the traversal path. The formula for calculating the path strength is: ;

[0087] Where M represents the total number of graph edges in the traversal path. This represents the mapping strength of the k-th graph edge. The higher the path strength, the more reliable the mapping relationship between the data sources in the closed-loop path, and the stronger the cross-validation capability of the feature dimensions.

[0088] In addition to path strength, the data coverage of the traversal path also needs to be evaluated. The number of data records from each data source in the statistical data source sequence is calculated from the labeled dataset. For example, an e-commerce platform data source contains 50,000 user records, and a social media data source contains 80,000 user records. The proportion of data records from each data source to the total number of records in the labeled dataset is calculated to obtain the weight value of that data source. The formula for calculating the data source weight is: ;

[0089] in, This indicates the number of data records in data source s. This represents the total number of records in the labeled dataset. The coverage of the path is obtained by summing the weights of all data sources along the traversal path. ;

[0090] in, This represents the set of data sources traversed by the traversal path. Coverage reflects the representativeness of the data sources involved in the closed-loop path within the overall dataset. Higher coverage indicates that the data sources involved in the path contain more user data records, and the wider the applicability of its validation results.

[0091] After obtaining the path strength and coverage, a comprehensive score for the traversal path is calculated through a weighted combination. The purpose of the weighted combination is to balance the two dimensions of mapping reliability and data coverage. The formula for calculating the comprehensive score is: ;

[0092] in, and These are the weighting coefficients for path strength and coverage, respectively, which are typically adjusted based on the actual application scenario and satisfy the following conditions: In user profile generation scenarios, it is possible to... Set to 0.6, Set it to 0.4 to place greater emphasis on the reliability of the mapping relationship.

[0093] The calculated overall score is compared with a preset scoring threshold. Setting the scoring threshold requires considering both the rigor of feature verification and the number of feature dimensions to retain. If the threshold is set too high, a large number of feature dimensions will be filtered out, affecting the richness of the user profile; if the threshold is set too low, some feature dimensions with insufficient verification capabilities will be retained, reducing the reliability of the profile. In practice, a scoring threshold of 0.65 can be set, meaning that traversal paths with an overall score exceeding 0.65 are determined to be valid closed-loop verification paths.

[0094] For each traversal path identified as a closed-loop validation path, its starting feature dimension is included in a subset of feature dimensions. Since multiple different closed-loop validation paths may exist for the same feature dimension, deduplication of feature dimensions is required during the selection process to ensure that each feature dimension appears only once in the subset. The final subset of feature dimensions contains all feature dimensions that can pass cross-validation across multiple data sources. These feature dimensions possess high reliability and stability, making them suitable as a candidate set for subsequent core feature primitives.

[0095] In practical applications, assume that the closed-loop verification path for a certain "user consumption preference" feature dimension sequentially passes through four nodes: e-commerce platform, payment system, logistics data, and back to e-commerce platform, forming a closed loop. This path includes three different data sources. The mapping strengths of the three graph edges in the path are 0.85, 0.78, and 0.82, respectively, and the path strength is 2.45. The number of data records in the three data sources are 50,000, 120,000, and 80,000, respectively, and the total number of records in the annotated dataset is 300,000. The corresponding data source weights are 0.167, 0.400, and 0.267, respectively, with a coverage of 0.834. Weight coefficients are used... , Calculate the overall score as follows The overall score far exceeded the threshold of 0.65, therefore this traversal path was determined to be a closed-loop verification path, and the "user consumption preference" feature dimension was included in the feature dimension subset.

[0096] The above method enables a systematic identification of feature dimensions with multi-source cross-validation capabilities from the feature validation graph, laying a solid foundation for the subsequent selection of core feature primitives. The identification of closed-loop validation paths not only considers the diversity of data sources but also comprehensively evaluates the reliability of mapping relationships and the breadth of data coverage, ensuring that the selected feature dimensions possess both high reliability and broad applicability.

[0097] In one optional embodiment, using core feature primitives as topological center nodes, extracting candidate features and constructing an association graph based on the degree of collaborative change over time offsets includes:

[0098] Extract the value sequences of core feature primitives and other features from the labeled dataset, and calculate the degree of co-change between other features and core feature primitives at different time offsets;

[0099] Identify the time offset corresponding to the peak of the degree of collaborative change, mark other features whose absolute value of the time offset is less than a preset offset threshold as candidate features, and extract the peak value of the degree of collaborative change of the candidate feature as the association strength.

[0100] A graph structure is constructed with core feature primitives as the topological center node and candidate features as the peripheral nodes. Connection edges that carry the correlation strength are established between the core feature primitives and each candidate feature, and time offsets are assigned to each candidate feature node.

[0101] The frequency of co-occurrence among candidate features in the data records is statistically analyzed. For candidate feature pairs whose co-occurrence frequency exceeds a preset co-occurrence threshold, connection edges are established between the candidate features to form an association graph.

[0102] In one specific implementation, after obtaining the core feature primitives, it is necessary to discover other features that are temporally related to these core feature primitives, thereby constructing a correlation graph that reflects the dynamic relationships between features. The core of this process lies in identifying those features that exhibit a cooperative change pattern with the core feature primitives in the time dimension, and quantifying the strength of this cooperative change.

[0103] Extract the value sequences of core feature primitives and all other features from the labeled dataset. Since the labeled dataset has been spatiotemporally labeled, each data record carries timestamp information, so the data records can be sorted in chronological order to form time series data. For a given core feature primitive, extract its values ​​in all data records and arrange them in chronological order to form a value sequence. For numerical features, the value sequence is directly composed of numerical values; for categorical features, the category can be mapped to a numerical code or represented using one-hot encoding. Similarly, extract the value sequences of other features in the labeled dataset besides the core feature primitives.

[0104] When calculating the degree of co-change between other features and the core feature primitive, the impact of time offset needs to be considered. Time offset refers to the possibility that a change in a feature may lag behind or precede a change in the core feature primitive. For example, a change in a user's income level may lead to a change in consumption behavior after a period of time; in this case, the consumption behavior feature has a positive time offset relative to the income feature. Conversely, a user's search behavior may predict subsequent purchase behavior; the search behavior feature has a negative time offset relative to the purchase behavior feature. To capture this temporal correlation, for each other feature, the degree of co-change with the core feature primitive is calculated within a certain time offset range.

[0105] Specifically, a time offset range is set, for example, shifting from a negative offset to a positive offset by a certain number of time units. For a given time offset value, the value sequences of other features are shifted according to this offset value, and then the correlation between the shifted sequences and the value sequences of the core feature primitives is calculated. The correlation can be calculated using metrics such as Pearson correlation coefficient, Spearman rank correlation coefficient, or mutual information. For numerical features, the Pearson correlation coefficient is effective in measuring linear correlation; the calculation formula is as follows: ,in This represents the time offset, and L represents the sequence length. This indicates that the core feature primitive is at time 1000. The value of , Indicates other features at time... The value of , and These represent the means of the corresponding sequences. For categorical features or nonlinear associations, mutual information can be used as a measure of the degree of co-variation.

[0106] Iterate through all offset values ​​within the time offset range, calculate the degree of co-variation corresponding to each offset value, and form a co-variation degree curve. Identify the peak point on this curve, that is, the position where the degree of co-variation reaches its maximum value. The time offset corresponding to the peak point reflects the most significant temporal correlation between this feature and the core feature primitive, while the magnitude of the peak itself reflects the strength of this correlation.

[0107] Determining whether an additional feature should be labeled a candidate feature requires meeting two conditions simultaneously. The first condition is that the co-variance curve of the feature exhibits a significant peak exceeding a certain threshold, indicating a substantial co-variance relationship between the feature and the core feature primitive. The second condition is that the absolute value of the time offset corresponding to the peak is less than a preset offset threshold. The preset offset threshold is set based on the actual needs of the business scenario. If the time offset is too large, it indicates that the temporal correlation between the feature and the core feature primitive is too distant, potentially lacking a real causal relationship or meaningful association. For example, in user behavior analysis, if the optimal time offset between a feature and the core feature primitive reaches several months or even years, this association may be accidental or influenced by other confounding factors and should not be included as a candidate feature. For other features that meet the conditions, they are labeled as candidate features, and the peak value of their co-variance is extracted as the correlation strength between the candidate feature and the core feature primitive.

[0108] After identifying all candidate features and their correlation strengths, a graph structure for the correlation graph is constructed. The core feature primitive is used as the topological center node of the graph, and all candidate features are used as peripheral nodes. A connection edge is established between the core feature primitive and each candidate feature. This connection edge carries two pieces of information: first, the correlation strength, i.e., the peak value of the previously extracted degree of collaborative change; and second, the time offset, i.e., the time offset value corresponding to the peak value. The correlation strength can be used as the edge weight for subsequent calculation of propagation weights and path analysis; the time offset is used as an attribute of the edge to characterize the temporal relationship between features. For each candidate feature node, its corresponding time offset is assigned as a node attribute, thus preserving both the correlation strength information and the temporal relationship information within the graph structure.

[0109] In addition to the connections between core feature primitives and candidate features, the relationships between candidate features also need to be considered. Candidate features may exhibit common patterns, meaning they may both acquire specific values ​​or change simultaneously within the same data record. This co-occurrence pattern reflects a potential correlation between candidate features, possibly indicating that they are influenced by common factors or have an interaction relationship. The frequency of co-occurrence between candidate features in data records is calculated by iterating through all data records in the labeled dataset. For each pair of candidate features, it checks whether they both acquire valid values ​​or both change significantly within the same record. The total number of co-occurrences for each pair of candidate features is then accumulated to obtain a co-occurrence frequency matrix.

[0110] To avoid misclassifying accidental co-occurrences as meaningful associations, a preset co-occurrence threshold is set. Only when the co-occurrence frequency of a pair of candidate features exceeds this threshold is a significant association considered to exist, and a connection edge is established between these candidate features in the graph structure. The weights of the connection edges between candidate features can be set as normalized values ​​of co-occurrence frequency, or the association strength between candidate features can be calculated using metrics such as point mutual information. By introducing connection edges between candidate features, the association graph not only includes a star-shaped connection structure between core feature primitives and candidate features, but also a mesh-like connection structure between candidate features, forming a richer and more complete feature association network.

[0111] The constructed association graph is a weighted directed or undirected graph. Nodes include core feature primitives and candidate features, and edges include connections between core feature primitives and candidate features, as well as connections between candidate features. Each edge carries association strength information, and some edges also carry time offset information. This association graph provides the foundation for subsequent feature topology expansion and user profile generation. By performing path analysis and propagation calculations within the graph, extended features that significantly contribute to the user profile can be identified, and the dependencies between features can be quantified.

[0112] In one optional embodiment, the process of calculating connected paths from core feature primitives to candidate features in the association graph, determining propagation weights based on path length and association strength, and incorporating candidate features with propagation weights exceeding a preset growth threshold into the expanded feature topology includes:

[0113] In the association graph, starting from the core feature primitive, a breadth-first traversal is performed on each candidate feature along the connecting edges. The sequence of connecting edges from the core feature primitive to each candidate feature is recorded as a connected path. The path length is obtained by counting the number of connecting edges in each connected path.

[0114] For each candidate feature, the association strength carried by each connecting edge on the connected path corresponding to the candidate feature is extracted. The association strength is constructed into an intensity sequence according to the propagation order in the connected path. The association strength of each connecting edge is attenuated and adjusted according to the position index of each connecting edge in the intensity sequence. The attenuated and adjusted association strengths are accumulated and summed to obtain the path attenuation propagation value.

[0115] The propagation weight of the candidate feature is obtained by comparing the path decay propagation value with the path length of the connected path corresponding to the candidate feature.

[0116] Candidate features whose propagation weight exceeds the preset growth threshold are included in the extended feature set. The core feature primitive is used as the initial topology node. Candidate features in the extended feature set are added one by one as extended nodes. All connecting edges and passing nodes on the connecting path between the core feature primitive and each extended node are extracted from the association graph. The connecting edges and passing nodes are attached to the initial topology node according to the topological connection relationship to form a multi-level topology structure, which constitutes the extended feature topology.

[0117] In one specific implementation, based on the constructed association graph, it is necessary to identify candidate features that are effectively associated with the core feature primitives, and determine whether these candidate features should be included in the final extended feature topology through path analysis. Nodes in the association graph represent various types of features, while connecting edges carry information about the strength of associations between features. To systematically explore the connection relationships between the core feature primitives and each candidate feature, a breadth-first traversal strategy is used for path search.

[0118] Starting with the core feature primitive node, mark it as visited and add it to the traversal queue. Remove the current node from the queue and check all its directly connected neighbors. For each unvisited neighbor, record the complete sequence of edges traversed from the core feature primitive to that node; this sequence constitutes the connected path. Mark the newly discovered node as visited and add it to the end of the queue, then continue processing the next node in the queue. This layered expansion ensures that candidate features closer to the core feature primitive are discovered first, while guaranteeing that each candidate feature can find a valid connected path originating from the core feature primitive.

[0119] During the traversal, the number of connecting edges in the corresponding connected path is recorded for each candidate feature; this number is the path length. The path length reflects the topological distance between the candidate feature and the core feature primitives; a shorter path length usually indicates a more direct correlation between the two. For candidate features with multiple connected paths, the shortest path is selected as the representative path for that candidate feature to reflect the closest correlation.

[0120] After obtaining the connected paths of each candidate feature, it is necessary to quantify the propagation effect of the association strength along the paths. Each connecting edge in the association graph carries a numerical value of the association strength between the corresponding feature pairs. These values ​​have already been calculated based on the degree of co-variance when constructing the association graph. For a candidate feature, the association strength of all connecting edges on its connected paths is extracted and arranged in the propagation order starting from the core feature primitive, forming a strength sequence. The first element in the strength sequence corresponds to the association strength between the core feature primitive and its directly adjacent features, and subsequent elements correspond to the association strength of each segment of the path.

[0121] In actual feature association propagation, information attenuation occurs as it travels along the path. The farther a connection edge is from the core feature primitive, the smaller its contribution to the final propagation effect. To simulate this attenuation effect, a position-dependent adjustment is made to each association strength value in the strength sequence. Let h be the position index of a certain association strength value in the strength sequence (counting from 1), and multiply this association strength value by an attenuation factor. ,in This is the attenuation coefficient, typically ranging from 0.7 to 0.9. A larger position index results in a higher power of the attenuation factor and a more pronounced attenuation effect. After attenuation adjustment, all adjusted associated intensity values ​​in the intensity sequence are summed to obtain the path attenuation propagation value. The calculation formula is: ;

[0122] Where H represents the length of the intensity sequence (i.e., the number of connecting edges in the connected path), This represents the original value of the association strength at the h-th position in the intensity sequence. The path decay propagation value comprehensively reflects the cumulative effect of the association information propagation from the core feature primitive to the candidate feature, taking into account both the strength of the association at each segment along the path and the decay effect caused by the propagation distance.

[0123] To establish comparable evaluation criteria among candidate features with different path lengths, the path decay propagation value is normalized to the path length. The path decay propagation value is then divided by the corresponding path length. The propagation weights of the candidate features are obtained. : ;

[0124] The propagation weight represents the average effective association strength per unit path length. A larger value indicates a stronger association between the candidate feature and the core feature primitive, and higher propagation efficiency. This normalization method avoids the undue influence of path length differences on feature selection results, allowing candidate features that are geographically distant but have high association strength to be included in the expanded feature topology.

[0125] Set preset growth threshold This threshold serves as the criterion for selecting candidate features. The threshold is set based on the specific application scenario and the required level of detail in the user profile, typically ranging from 0.3 to 0.6. The propagation weight of each candidate feature is compared to the preset growth threshold. If a candidate feature is found to have a sufficiently strong correlation with the core feature primitives, it is considered to be included in the extended feature set. This screening mechanism can filter out candidate features with weak correlations or those that have excessively long propagation paths leading to significant information decay, while retaining feature dimensions that substantially contribute to the construction of user profiles.

[0126] After candidate feature selection, the extended feature topology is constructed. The core feature primitives are used as the central nodes of the topology and marked as initial topology nodes. Candidate features in the extended feature set are processed one by one, and each candidate feature is added to the topology as an extended node. For each extended node, its corresponding connected path in the association graph is traced back, and all intermediate nodes and connecting edges along the path are extracted. These intermediate nodes may be other candidate features, or features that were passed through during the path search but were not directly selected as extended nodes.

[0127] The extracted intermediate nodes are sequentially appended to the initial topology node according to their positional relationship in the connected path. If an intermediate node already exists in the topology, it is not added again, but the existing node is reused. The connecting edges are connected according to the original topology connection relationship, ensuring that the path from the core feature primitive to each extended node is fully represented in the topology. As extended nodes are added one by one, the topology gradually grows from a single core node into a network structure containing multiple levels.

[0128] During the construction process, a multi-level topology naturally forms around the core feature primitives. Nodes with the same path length from the core feature primitives are at the same level; nodes with a path length of 1 form the first layer, nodes with a path length of 2 form the second layer, and so on. This hierarchical topology clearly demonstrates the hierarchy of feature propagation, providing a structured foundation for subsequent hierarchical dependency analysis and path redirection operations. The resulting extended feature topology not only includes the selected highly correlated candidate features but also preserves the propagation paths and hierarchical relationships between features, forming the core feature network framework of the user profile.

[0129] like Figure 2 The diagram shown illustrates the feature topology hierarchy dependency and redirection flowchart.

[0130] In an optional embodiment, in the extended feature topology, based on the shortest topological distance from each feature node to the core feature primitive, the extended feature topology is divided into multiple concentric levels. Inter-level constraint edges are identified, and a hierarchical dependency matrix is ​​constructed based on the inter-level transmission coefficients. The hierarchical dependency matrix is ​​used to redirect the cross-level feature node association paths. The redirected extended feature topology is used as the user profile, including:

[0131] Calculate the shortest topological distance from each feature node to the core feature primitive, and group feature nodes with the same shortest topological distance into the same level to form multiple concentric hierarchical structures.

[0132] Traverse the connecting edges in the extended feature topology, identify the connecting edges connecting feature nodes at different levels as inter-layer constraint edges, extract the corresponding association strength for each inter-layer constraint edge, obtain the level number of the starting feature node and the target feature node connected by each inter-layer constraint edge, calculate the difference of the level number as the cross-layer span, and calculate the inter-layer transmission coefficient by performing a ratio operation between the association strength and the cross-layer span. Construct a level dependency matrix with the level number as the row and column index and the inter-layer transmission coefficient corresponding to each inter-layer constraint edge as the matrix element.

[0133] For feature node association paths spanning multiple levels, the inter-layer transmission coefficients corresponding to each adjacent level between the starting and ending levels are extracted from the hierarchical dependency matrix, and the path transmission attenuation value is obtained by multiplying the inter-layer transmission coefficients together.

[0134] Based on the path conduction attenuation value, related paths are filtered, and the paths to be redirected are adjusted to pass through the feature nodes with the largest inter-layer conduction coefficients in sequence along the hierarchical order, forming an extended feature topology after redirection as a user profile.

[0135] In one specific implementation, after obtaining the extended feature topology, it needs to be hierarchically organized and path optimized to form a user profile representation with a clear structure and efficient transmission. Although the extended feature topology contains rich feature nodes and their relationships, the distance difference between feature nodes and core feature primitives means that directly using the original topology may lead to low information transmission efficiency, and cross-level connection paths may contain redundancy or inefficient connections. Therefore, dividing the hierarchy by the shortest topological distance and redirecting the connection paths based on inter-layer transmission characteristics can significantly improve the expression quality and computational efficiency of the user profile.

[0136] For each feature node in the extended feature topology, calculate its shortest topological distance to the core feature primitive. The shortest topological distance is the minimum number of edges required to reach the core feature primitive from the feature node along the edges in the topology graph. A breadth-first search algorithm can be used to expand outwards layer by layer from the core feature primitive, recording the distance value when each feature node is first visited. The shortest topological distance of the core feature primitive itself is defined as 0, the shortest topological distance of the feature nodes directly connected to the core feature primitive is 1, and so on. In this way, the shortest topological distance from the core feature primitive to all feature nodes in the extended feature topology can be labeled.

[0137] After obtaining the shortest topological distance for each feature node, feature nodes with the same shortest topological distance are grouped into the same level. Specifically, nodes with a shortest topological distance of 0 constitute level 0, which is the level containing the core feature primitive; all feature nodes with a shortest topological distance of 1 constitute level 1; all feature nodes with a shortest topological distance of 2 constitute level 2, and so on. This division forms multiple concentric hierarchical structures centered on the core feature primitive. Within each level, the feature nodes have the same topological distance to the core feature primitive, and the levels exhibit a radial relationship from the inside out. The concentric hierarchical structure not only intuitively reflects the close relationship between feature nodes and the core feature primitive but also provides a clear organizational framework for subsequent inter-layer transmission analysis.

[0138] After completing the hierarchical division, it is necessary to identify the inter-layer constraint edges in the extended feature topology. Inter-layer constraint edges are edges that connect feature nodes at different levels, representing cross-level feature relationships. All connecting edges in the extended feature topology are traversed, and for each edge, the level numbers of the two connected feature nodes are extracted. If the two level numbers are different, the edge is identified as an inter-layer constraint edge. Inter-layer constraint edges play a bridging role in the hierarchical structure of the user profile, serving as a crucial channel for information transmission between different levels.

[0139] For each inter-layer constraint edge, its corresponding association strength needs to be extracted. The association strength is calculated during the construction of the association graph and reflects the degree of collaborative change or correlation strength between the two feature nodes. Simultaneously, the layer numbers of the starting and target feature nodes connected by the inter-layer constraint edge are obtained, and the difference between the two layer numbers is calculated; this difference is called the cross-layer span. The cross-layer span indicates how many layers the edge traverses; a larger cross-layer span indicates a greater distance difference between the two feature nodes connected by the edge in the topology.

[0140] To quantify the transmission capacity of inter-layer constraint edges, the correlation strength is calculated as a ratio to the cross-layer span, yielding the inter-layer transmission coefficient. The formula for calculating the inter-layer transmission coefficient is as follows: ,in Indicates connection hierarchy with hierarchy The inter-layer conduction coefficient of the inter-layer constraint edge. This indicates the strength of the association between the edges. Indicates hierarchy with hierarchy The span between layers, i.e. The interlayer conduction coefficient takes into account both the correlation strength and the cross-layer span. The higher the correlation strength and the smaller the cross-layer span, the larger the interlayer conduction coefficient, indicating that the conduction ability of the interlayer constraint edge is stronger.

[0141] After calculating the inter-layer transmission coefficients of all inter-layer constraint edges, a layer dependency matrix is ​​constructed using the layer number as the row and column index and the inter-layer transmission coefficient corresponding to each inter-layer constraint edge as the matrix element. The layer dependency matrix is ​​a square matrix, where the rows and columns correspond to the respective layer numbers. The elements in the matrix... Indicates from hierarchy to level The interlayer conduction coefficient. If the layer with hierarchy There are multiple inter-layer constraint edges, and the maximum or average value of the inter-layer transmission coefficients of these edges can be taken as matrix elements. The hierarchical dependency matrix clearly describes the transmission relationship between different layers, providing a quantitative basis for subsequent path redirection.

[0142] In extended feature topology, there may be feature node association paths that span multiple levels. These paths start from a starting feature node, pass through several intermediate feature nodes, and finally reach the target feature node, with the starting and target nodes belonging to different levels. For such cross-level association paths, it is necessary to evaluate their propagation efficiency. The inter-level propagation coefficients corresponding to each adjacent level between the starting and ending levels are extracted from the hierarchical dependency matrix. It is assumed that the path sequentially traverses the following levels. Then extract the interlayer conduction coefficient. Multiplying these interlayer conduction coefficients together yields the path conduction attenuation value. The path conduction attenuation value reflects the cumulative attenuation of information as it travels along the path; the larger the value, the higher the conduction efficiency of the path.

[0143] Filter associated paths based on path conduction attenuation values. Set a path conduction attenuation threshold. Paths with a path propagation attenuation value below a certain threshold are marked as paths to be redirected. These paths, due to their low propagation efficiency, may cause information transmission distortion or delay, requiring optimization. For paths to be redirected, a strategy is adopted to redirect them sequentially through the feature nodes with the largest inter-layer propagation coefficients along the hierarchical order. Specifically, starting from the initial level, the feature node with the largest inter-layer propagation coefficient connected to the current node is selected as the next hop node in the next level, and so on, until the final level is reached. The redirection path constructed in this way selects the connection with the strongest propagation capacity at each step, thereby maximizing the overall path propagation efficiency.

[0144] Redirection not only optimizes the transmission performance of individual paths but also improves the structural quality of the extended feature topology at the global level. By converting inefficient paths into efficient ones, redundant connections are reduced, enhancing the efficiency of information flow between layers. The redirected extended feature topology retains the original hierarchical structure and the central position of core feature primitives, while further optimizing path connections, resulting in a user profile representation that is highly efficient in transmission and has a clear structure.

[0145] In practical applications, the construction of the hierarchical dependency matrix and path redirection can be performed using an iterative optimization approach. Initially, the hierarchical dependency matrix is ​​calculated based on the original extended feature topology, the paths to be redirected are identified, and the first round of redirection is performed. After redirection, the topology changes, and the hierarchical dependency matrix can be recalculated for a second round of redirection. The iterative process continues until the improvement in path propagation attenuation values ​​falls below a preset convergence threshold, or the maximum number of iterations is reached. Iterative optimization can gradually approach the optimal topology, further improving the quality of user profiles.

[0146] The redirected extended feature topology, serving as the final user profile, not only contains rich feature nodes and relationships but also possesses a hierarchical organizational structure and efficient transmission paths. This user profile clearly reflects users' attributes and behavioral patterns across different feature dimensions, supporting subsequent applications such as user analysis, recommendation systems, and precision marketing. Furthermore, the introduction of hierarchical structure and transmission coefficients makes the user profile interpretable and controllable, facilitating understanding and application by business personnel.

[0147] In one optional embodiment, associated paths are filtered based on path conduction attenuation values, and the paths to be redirected are adjusted to redirected paths that sequentially pass through the feature nodes with the largest inter-layer conduction coefficients along the hierarchical order, forming an extended feature topology after redirection as a user profile, including:

[0148] Traverse the associated paths of feature nodes spanning multiple levels in the extended feature topology, mark associated paths with a path propagation attenuation value less than a preset propagation threshold as paths to be redirected, and mark associated paths with a path propagation attenuation value greater than or equal to the propagation threshold as paths to be retained.

[0149] For each redirection path, the corresponding starting feature node's level is extracted as the starting level, and the ending feature node's level is extracted as the ending level. The starting and ending levels are traversed layer by layer in order of level number. For the current level, the inter-level transmission coefficient from the current level to the next level is queried from the level dependency matrix. The feature node of the current level with the maximum value of the inter-level transmission coefficient is selected as the path node. The path nodes are connected in order of level to form the redirection path.

[0150] Replace the paths to be redirected in the extended feature topology with the redirected paths, and retain the reserved paths to form the extended feature topology after redirection and determine the user profile.

[0151] In one specific implementation, after completing the hierarchical division of the extended feature topology and the construction of the hierarchical dependency matrix, it is necessary to conduct quality assessment and optimization of the cross-hierarchical association paths existing in the topology. Since some cross-hierarchical paths may have low feature transmission efficiency due to excessive hierarchical span or weak intermediate transmission links, it is necessary to identify these inefficient paths and perform redirection processing to ensure that the feature association relationships in the user profile have sufficient reliability and interpretability.

[0152] For all feature node association paths in the extended feature topology, a global traversal is first performed. During the traversal, for each association path spanning multiple concentric levels, the sequence of level numbers traversed by the path is extracted. Based on the constructed hierarchical dependency matrix, the path propagation attenuation value of the path is calculated. The calculated path conduction attenuation value is compared with the preset conduction threshold. Comparison. When When this occurs, it indicates that the path experiences significant information attenuation during cross-level propagation, and the reliability of the correlation between features is insufficient; such paths are marked as paths to be redirected. Conversely, when... If the path exhibits good propagation performance and its feature association strength is sufficient to support the needs of profile construction, then such paths are marked as retained paths. This filtering mechanism allows for the division of associated paths in the topology into two categories, laying the foundation for subsequent differentiated processing.

[0153] For all associated paths marked as paths to be redirected, path reconstruction needs to be performed one by one. For a given path to be redirected, first identify the starting and ending feature nodes. Extract the concentric level number to which the starting feature node belongs, and denote it as the starting level; extract the concentric level number to which the ending feature node belongs, and denote it as the ending level. Since the extended feature topology adopts a concentric hierarchical structure, the level number increases outward from the central layer where the core feature primitive is located, so there is a clear hierarchical order relationship between the starting and ending levels.

[0154] After determining the starting and ending levels, the hierarchy is traversed layer by layer in ascending order of level numbers. Starting from the starting level, the currently processed level is set as the starting level. Within the current level, all inter-level transmission coefficients from the current level to the next level need to be retrieved from the hierarchy dependency matrix. The hierarchy dependency matrix contains the following matrix elements. Recorded from the hierarchy to level The transmission capacity, where the values ​​of the matrix elements are determined by the inter-layer transmission coefficients of the inter-layer constraint edges connecting the two layers. The decision is as follows: In the current level, there are multiple feature nodes, each of which may be connected to different feature nodes in the next level through different inter-level constraint edges. For all feature nodes in the current level, the inter-level transmission coefficients from each node to each feature node in the next level are queried, and the node with the maximum value is selected. The feature node in the current level corresponding to this maximum value is the node with the strongest transmission capability in the current level, and it is determined as the path node.

[0155] After selecting the path nodes for the current level, update the current level to the next level and continue performing the same query and selection operations. Repeat the above process until the current level reaches the level before the terminating level. During the entire layer-by-layer traversal, a path node with optimal propagation capability is selected for each level visited. The starting feature node, the path nodes of each level, and the terminating feature node are connected sequentially according to the level number to form a new association path, which is the redirection path. This redirection path strictly follows the level order, and each hop in the inter-level propagation selects the optimal propagation channel from the current level to the next level, thus maximizing the overall propagation efficiency of the path and ensuring the reliability of feature association.

[0156] After reconstructing all paths to be redirected, the extended feature topology needs to be updated. Existing paths to be redirected are removed from the graph structure and replaced with newly generated redirected paths. For associated paths marked as retained, their original connections within the topology remain unchanged. Through this replacement operation, cross-level associated paths in the extended feature topology are optimized; inefficient propagation paths are replaced by efficient redirected paths, while paths with good propagation performance are retained. After path redirection, the extended feature topology exhibits more rational relationships between feature nodes, significantly improved information transfer efficiency between levels, and clearer and more interpretable dependencies between features.

[0157] The redirected extended feature topology is used as the final user profile output. This user profile is centered on core feature primitives, forming a concentric topological structure through multi-level feature nodes. The association paths between feature nodes undergo propagation attenuation evaluation and redirection optimization to ensure the efficiency and reliability of feature associations in the profile. Each feature node in the profile corresponds to a user's attribute or behavioral feature in a certain feature dimension, and the association paths between nodes reflect the dependencies and propagation mechanisms between features. Through hierarchical structure, the distance relationship between core features and derived features can be intuitively identified, providing a structured feature representation foundation for subsequent applications such as user behavior prediction and personalized recommendation.

[0158] In practical applications, conduction threshold The setting of the propagation threshold directly impacts the path selection results. A higher threshold results in more paths being identified as requiring redirection, leading to greater topology optimization, but potentially causing some otherwise acceptable paths to be unnecessarily reconstructed. Conversely, a lower threshold only redirects paths with extremely poor propagation performance, resulting in a higher proportion of retained paths and relatively less optimization. Therefore, in practical deployments, the appropriate range of propagation threshold values ​​can be determined through experimentation, based on the specific business scenario's requirements for user profile quality, to achieve a balance between path optimization intensity and computational overhead.

[0159] Furthermore, when selecting path nodes layer by layer, if multiple feature nodes in a certain level have the same and maximum inter-layer transmission coefficients to the next level, an auxiliary judgment mechanism can be used for selection. For example, the degrees of these candidate path nodes in the current level can be compared, prioritizing nodes with higher degrees, as nodes with higher degrees typically have stronger connectivity and information aggregation capabilities in the topology. Alternatively, the shortest topological distances from candidate nodes to core feature primitives can be compared, prioritizing nodes with closer distances to maintain the compactness of the redirection path. Through these auxiliary strategies, the selection of path nodes can be further optimized when the inter-layer transmission coefficients are the same, improving the overall quality of the redirection path.

[0160] A second aspect of this invention provides a user profile intelligent generation system based on big data, comprising:

[0161] The data annotation unit is used to acquire multi-source heterogeneous data from users and perform spatiotemporal annotation to obtain an annotated dataset.

[0162] The core feature unit is used to extract a set of feature dimensions from the labeled dataset. For each feature dimension, a cross-source mapping relationship is established between data fields of different data sources. A feature verification graph is constructed through the cross-source mapping relationship, and a subset of feature dimensions that form a closed-loop verification path is identified in the feature verification graph. The feature dimension subset is selected as the core feature primitive.

[0163] The feature expansion unit is used to extract candidate features and construct an association graph based on the degree of cooperative change of the core feature primitive as the topology center node. In the association graph, the connected path from the core feature primitive to the candidate feature is calculated, and the propagation weight is determined based on the path length and association strength. Candidate features with propagation weights exceeding the preset growth threshold are included to form an expanded feature topology.

[0164] The hierarchical repositioning unit is used to divide the extended feature topology into multiple concentric levels based on the shortest topological distance from each feature node to the core feature primitive in the extended feature topology, identify inter-level constraint edges and construct a hierarchical dependency matrix based on the inter-level transmission coefficient, redirect the cross-level feature node association path through the hierarchical dependency matrix, and use the redirected extended feature topology as a user profile.

[0165] The user profile output unit is used to output the user profile.

[0166] A third aspect of the present invention provides an electronic device, comprising:

[0167] processor;

[0168] Memory used to store processor-executable instructions;

[0169] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0170] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0171] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0172] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A user profile intelligent generation method based on big data, characterized in that, include: Acquire multi-source heterogeneous data from users and perform spatiotemporal annotation to obtain a labeled dataset; Extract a set of feature dimensions from the labeled dataset, establish cross-source mapping relationships between data fields of different data sources for each feature dimension, construct a feature verification graph through the cross-source mapping relationships, identify a subset of feature dimensions that form a closed-loop verification path in the feature verification graph, and select the core feature primitives from the feature dimension subsets; Using core feature primitives as topology center nodes, candidate features are extracted based on the degree of collaborative change over time offset and an association graph is constructed. Connectivity paths from core feature primitives to candidate features are calculated in the association graph, and propagation weights are determined based on path length and association strength. Candidate features with propagation weights exceeding a preset growth threshold are included to form an extended feature topology. In the extended feature topology, the extended feature topology is divided into multiple concentric levels according to the shortest topological distance from each feature node to the core feature primitive. Inter-level constraint edges are identified and a hierarchical dependency matrix is ​​constructed based on the inter-level transmission coefficient. The cross-level feature node association path is redirected through the hierarchical dependency matrix, and the redirected extended feature topology is used as the user profile. Output the user profile.

2. The method according to claim 1, characterized in that, A set of feature dimensions is extracted from the labeled dataset. For each feature dimension, a cross-source mapping relationship is established between data fields from different data sources. A feature verification graph is constructed through the cross-source mapping relationship, and a subset of feature dimensions that form a closed-loop verification path is identified in the feature verification graph. The core feature primitives are selected from the feature dimension subsets, including: Extract a set of feature dimensions from the labeled dataset according to a predefined dimensional classification. For each feature dimension, identify the data fields describing that feature dimension in different data sources, and extract the semantic identifier and data type identifier of the data fields. Based on the semantic identifier and data type identifier, establish a bidirectional mapping relationship between the data fields in different data sources to determine the cross-source mapping relationship. The feature verification graph is constructed by using each feature dimension in the feature dimension set as a graph node and the mapping relationship connecting the corresponding data fields of the same feature dimension in different data sources in the cross-source mapping relationship as graph edges. Search the graph nodes in the feature verification graph where the starting point and ending point are the same feature dimension and pass through at least three different data sources to determine the closed-loop verification path. The feature dimensions that form the closed-loop verification path constitute a subset of feature dimensions. Extract the timestamp sequences associated with each feature dimension in the feature dimension subset in the labeled dataset, construct the temporal dependency chain between the timestamp sequences, and identify the feature dimensions that form a directed acyclic propagation path in the temporal dependency chain as the core feature primitives.

3. The method according to claim 2, characterized in that, In the feature verification graph, search for graph nodes whose start and end points are in the same feature dimension and pass through at least three different data sources to determine the closed-loop verification path, including: Select any feature dimension from the feature verification graph as the search starting point, perform a depth-first traversal and record the data source sequence and graph edge sequence traversed by the traversal path. When the traversal returns to the search starting point, count the number of different data sources in the data source sequence. For traversal paths with at least three edges, the mapping intensity of each edge in the cross-source mapping relationship is extracted from the graph edge sequence, and the path intensity of the traversal path is obtained by accumulating the mapping intensity of each edge. Extract the number of data records of each data source in the data source sequence from the standardized dataset, calculate the proportion of the number of data records in the total number of records in the standardized dataset to determine the data source weight, and sum the weights of each data source in the data source sequence to obtain the coverage of the traversal path. The comprehensive score of the traversal path is calculated by weighting the path strength and coverage. The traversal path with a comprehensive score exceeding the preset score threshold is identified as the closed-loop verification path.

4. The method according to claim 1, characterized in that, Using core feature primitives as topological center nodes, candidate features are extracted and association graphs are constructed based on the degree of collaborative change over time offsets, including: Extract the value sequences of core feature primitives and other features from the labeled dataset, and calculate the degree of co-change between other features and core feature primitives at different time offsets; Identify the time offset corresponding to the peak of the degree of collaborative change, mark other features whose absolute value of the time offset is less than a preset offset threshold as candidate features, and extract the peak value of the degree of collaborative change of the candidate feature as the association strength. A graph structure is constructed with core feature primitives as the topological center node and candidate features as the peripheral nodes. Connection edges that carry the correlation strength are established between the core feature primitives and each candidate feature, and time offsets are assigned to each candidate feature node. The frequency of co-occurrence among candidate features in the data records is statistically analyzed. For candidate feature pairs whose co-occurrence frequency exceeds a preset co-occurrence threshold, connection edges are established between the candidate features to form an association graph.

5. The method according to claim 4, characterized in that, In the association graph, the connected paths from core feature primitives to candidate features are calculated, and the propagation weights are determined based on the path length and association strength. Candidate features whose propagation weights exceed a preset growth threshold are included in the formation of an expanded feature topology, including: In the association graph, starting from the core feature primitive, a breadth-first traversal is performed on each candidate feature along the connecting edges. The sequence of connecting edges from the core feature primitive to each candidate feature is recorded as a connected path. The path length is obtained by counting the number of connecting edges in each connected path. For each candidate feature, the association strength carried by each connecting edge on the connected path corresponding to the candidate feature is extracted. The association strength is constructed into an intensity sequence according to the propagation order in the connected path. The association strength of each connecting edge is attenuated and adjusted according to the position index of each connecting edge in the intensity sequence. The attenuated and adjusted association strengths are accumulated and summed to obtain the path attenuation propagation value. The propagation weight of the candidate feature is obtained by comparing the path decay propagation value with the path length of the connected path corresponding to the candidate feature. Candidate features whose propagation weight exceeds the preset growth threshold are included in the extended feature set. The core feature primitive is used as the initial topology node. Candidate features in the extended feature set are added one by one as extended nodes. All connecting edges and passing nodes on the connecting path between the core feature primitive and each extended node are extracted from the association graph. The connecting edges and passing nodes are attached to the initial topology node according to the topological connection relationship to form a multi-level topology structure, which constitutes the extended feature topology.

6. The method according to claim 1, characterized in that, In the extended feature topology, based on the shortest topological distance from each feature node to the core feature primitive, the extended feature topology is divided into multiple concentric levels. Inter-level constraint edges are identified, and a hierarchical dependency matrix is ​​constructed based on the inter-level transmission coefficient. The hierarchical dependency matrix is ​​used to redirect the cross-level feature node association paths. The redirected extended feature topology is used as the user profile, including: Calculate the shortest topological distance from each feature node to the core feature primitive, and group feature nodes with the same shortest topological distance into the same level to form multiple concentric hierarchical structures. Traverse the connecting edges in the extended feature topology, identify the connecting edges connecting feature nodes at different levels as inter-layer constraint edges, extract the corresponding association strength for each inter-layer constraint edge, obtain the level number of the starting feature node and the target feature node connected by each inter-layer constraint edge, calculate the difference of the level number as the cross-layer span, and calculate the inter-layer transmission coefficient by performing a ratio operation between the association strength and the cross-layer span. Construct a level dependency matrix with the level number as the row and column index and the inter-layer transmission coefficient corresponding to each inter-layer constraint edge as the matrix element. For feature node association paths spanning multiple levels, the inter-layer transmission coefficients corresponding to each adjacent level between the starting and ending levels are extracted from the hierarchical dependency matrix, and the path transmission attenuation value is obtained by multiplying the inter-layer transmission coefficients together. Based on the path conduction attenuation value, related paths are filtered, and the paths to be redirected are adjusted to pass through the feature nodes with the largest inter-layer conduction coefficients in sequence along the hierarchical order, forming an extended feature topology after redirection as a user profile.

7. The method according to claim 6, characterized in that, Based on the path conduction attenuation value, related paths are filtered, and the paths to be redirected are adjusted to pass sequentially through the feature nodes with the largest inter-layer conduction coefficients along the hierarchical order. This forms the extended feature topology after redirection, which serves as the user profile, including: Traverse the associated paths of feature nodes spanning multiple levels in the extended feature topology, mark associated paths with a path propagation attenuation value less than a preset propagation threshold as paths to be redirected, and mark associated paths with a path propagation attenuation value greater than or equal to the propagation threshold as paths to be retained. For each redirection path, the corresponding starting feature node's level is extracted as the starting level, and the ending feature node's level is extracted as the ending level. The starting and ending levels are traversed layer by layer in order of level number. For the current level, the inter-level transmission coefficient from the current level to the next level is queried from the level dependency matrix. The feature node of the current level with the maximum value of the inter-level transmission coefficient is selected as the path node. The path nodes are connected in order of level to form the redirection path. Replace the paths to be redirected in the extended feature topology with the redirected paths, and retain the reserved paths to form the extended feature topology after redirection and determine the user profile.

8. A user profile intelligent generation system based on big data, used to implement the method as described in any one of claims 1-7, characterized in that, include: The data annotation unit is used to acquire multi-source heterogeneous data from users and perform spatiotemporal annotation to obtain an annotated dataset. The core feature unit is used to extract a set of feature dimensions from the labeled dataset. For each feature dimension, a cross-source mapping relationship is established between data fields of different data sources. A feature verification graph is constructed through the cross-source mapping relationship, and a subset of feature dimensions that form a closed-loop verification path is identified in the feature verification graph. The feature dimension subset is selected as the core feature primitive. The feature expansion unit is used to extract candidate features and construct an association graph based on the degree of cooperative change of the core feature primitive as the topology center node. In the association graph, the connected path from the core feature primitive to the candidate feature is calculated, and the propagation weight is determined based on the path length and association strength. Candidate features with propagation weights exceeding the preset growth threshold are included to form an expanded feature topology. The hierarchical repositioning unit is used to divide the extended feature topology into multiple concentric levels based on the shortest topological distance from each feature node to the core feature primitive in the extended feature topology, identify inter-level constraint edges and construct a hierarchical dependency matrix based on the inter-level transmission coefficient, redirect the cross-level feature node association path through the hierarchical dependency matrix, and use the redirected extended feature topology as a user profile. The user profile output unit is used to output the user profile.

9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 7.

Images