A communication user complex behavior pattern mining method and system based on a heterogeneous graph neural network

Abstract

This application discloses a method and system for mining complex behavior patterns of communication users based on heterogeneous graph neural networks. The method includes: acquiring real-time user data; acquiring a trained heterogeneous graph neural network model and a trained downstream behavior pattern recognition model; updating a preset heterogeneous graph using real-time user data to obtain a subgraph of the heterogeneous graph with updated attributes; inputting the updated subgraph into the trained heterogeneous graph neural network model to obtain an updated user embedding vector; and inputting the updated user embedding vector into the trained downstream behavior pattern recognition model to obtain the recognition result. This application's solution can effectively capture multi-hop high-order associations on large-scale communication heterogeneous graph data, and compared to traditional independent label methods, it significantly improves the accuracy, generalization ability, and scalability of complex behavior pattern recognition.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to a method and system for mining complex behavior patterns of communication users based on heterogeneous graph neural networks. Background Technology

[0002] With the widespread adoption of mobile communication networks and smart terminals, operators have accumulated a large amount of structured data over the long term, including basic user attributes, recharge and call bills, call details, product subscription records, and application usage behavior. How to extract complex user behavior patterns from this structured data to support precision marketing, risk control, and service recommendations is a crucial issue for operators' refined operations.

[0003] In existing technologies, mainstream user profiling methods typically employ rule-based or simple statistical approaches to extract features from various types of data and transform them into a series of independent labels, such as "average monthly voice call duration exceeds a certain threshold," "subscribed to a certain type of data plan," or "monthly usage time on a certain application exceeds a certain threshold." While these methods are simple to implement and easy to interpret, they suffer from the following shortcomings:

[0004] Ignoring multi-hop, high-order relationships. In real-world business scenarios, complex networks of connections exist between users, between users and products, and between users and applications. For example, two users may frequently communicate and simultaneously subscribe to the same product, or multiple users may use the same type of application through different paths; these relationships often span multiple hops. Independent labeling methods struggle to characterize these high-order connections and fail to identify potential social propagation relationships, group preference patterns, and implicit churn risks.

[0005] It is difficult to unify the modeling of multiple types of entities and edges. Operator data contains multiple types of entities, such as user nodes, product nodes, and application nodes, as well as various types of relationships, such as calls, orders, clicks, searches, and usage. Traditional models usually piece together this information by manually designing features, which not only fails to fully preserve structural information but also easily introduces feature redundancy, thus limiting model performance and computational efficiency.

[0006] In large-scale scenarios, it is difficult to balance accuracy and efficiency. In large-scale network environments containing hundreds of millions of nodes and tens of billions of edges, directly using homogeneous graph neural networks for full-graph training can easily lead to excessive memory consumption, excessively long training time, and insufficient ability to distinguish between multiple types of entities and multiple types of relationships.

[0007] Therefore, how to model communication data containing multiple types of nodes and edges within a unified heterogeneous graph framework, effectively mine multi-hop high-order relationships through multi-layer message passing, and simultaneously ensure the accuracy and scalability of the model, has become a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0008] The purpose of this invention is to provide a method for mining complex behavior patterns of communication users based on heterogeneous graph neural networks to at least solve one of the above-mentioned technical problems.

[0009] One aspect of the present invention provides a method for mining complex behavior patterns of communication users based on heterogeneous graph neural networks, the method comprising:

[0010] Obtain real-time user data;

[0011] Obtain a trained heterogeneous graph neural network model and a trained downstream behavior pattern recognition model;

[0012] The preset heterogeneous graph is updated using real-time user data to obtain a local heterogeneous graph subgraph with updated attributes.

[0013] The updated local subgraph is input into a trained heterogeneous graph neural network model to obtain the updated user embedding vector.

[0014] The updated user embedding vector is input into the trained downstream behavior pattern recognition model to obtain the recognition result.

[0015] Optionally, the heterogeneous graph neural network model and the downstream behavior pattern recognition model are trained in the following manner:

[0016] Obtain user attribute datasets, call log datasets, product order datasets, and application usage datasets;

[0017] A heterogeneous graph is generated based on user attribute datasets, call record datasets, product order datasets, and application usage datasets; the heterogeneous graph includes user nodes, product nodes, and application nodes, and the edges include call edges, order edges, click edges, search edges, and usage edges;

[0018] Acquire heterogeneous graph neural networks and downstream multi-task learning networks;

[0019] Iterative training is performed on the heterogeneous graph neural network and the downstream multi-task learning network: user embedding vectors that integrate multi-hop higher-order relation information are generated based on the acquired heterogeneous graph; the heterogeneous graph neural network and the downstream multi-task learning network are jointly trained by the user embedding vectors that integrate multi-hop higher-order relation information, thereby obtaining the trained heterogeneous graph neural network model and the trained downstream behavior pattern recognition model.

[0020] Optionally, the user attribute dataset includes multiple user attribute data, each user attribute data including a unique user identifier, basic user information, and basic product information;

[0021] The call record dataset includes multiple call record data, and each call record data includes a unique call record identifier and basic call information;

[0022] The product order dataset includes multiple product order data, and each product order data includes a unique product order identifier, a unique product identifier, and basic order information.

[0023] The application usage dataset includes multiple application usage data, and each application usage data includes an application unique identifier, an application unique identifier, and application usage data.

[0024] Optionally, generating a heterogeneous graph based on the user attribute dataset, call record dataset, product order dataset, and application usage dataset includes:

[0025] Extract the unique user identifier from the user attribute data and assign a globally unique user node ID to each unique identifier; associate the basic user information and basic product information in the user attribute data with the corresponding user node;

[0026] Extract the unique product order identifier from the product order data, assign a globally unique product node ID to each product order identifier, and associate the basic order information with the corresponding product node.

[0027] Extract the unique application usage identifier from the application usage data, assign a globally unique application node ID to each application usage identifier, and associate the application usage data with the corresponding application node.

[0028] Map the unique identifier of the caller in the call record to the corresponding user node ID, and map the unique identifier of the called party in the call record to the corresponding user node ID. Use these two user nodes as the two endpoints of the call edge, and assign the edge attributes of the call edge through the basic call information.

[0029] Map the user's unique identifier to the user node ID, and the product's unique identifier to the product node ID. Use these two nodes as the two endpoints of the ordering edge, and assign the edge attributes of the ordering edge based on the basic ordering information.

[0030] Map the user's unique identifier to the user node ID, and the application's unique identifier to the application node ID. Use these two nodes as the two endpoints of the click edge, and assign the edge attributes of the click edge through the application's usage data.

[0031] Map the user's unique identifier to the user node ID, and the application uses the unique identifier to map to the application node ID. Use these two nodes as the two endpoints of the search edge, and assign the edge attributes of the search edge through the application's data.

[0032] The user's unique identifier is mapped to the user node ID, and the application's unique identifier is mapped to the application node ID. These two nodes are used as the two endpoints of the edge, and the edge attributes of the edge are assigned through the application's usage data.

[0033] Optionally, generating a user embedding vector that fuses multi-hop higher-order relation information based on the acquired heterogeneous graph includes:

[0034] Feature extraction is performed on the heterogeneous graph to obtain initial feature vectors of nodes with uniform dimensions;

[0035] User embedding vectors that integrate multi-hop higher-order relationship information are generated based on the initial feature vectors of nodes with a unified dimension and heterogeneous graphs.

[0036] Optionally, the step of generating a user embedding vector that fuses multi-hop higher-order relationship information based on the initial feature vector of nodes of a unified dimension and the heterogeneous graph includes:

[0037] Hierarchical neighbor sampling is performed on the heterogeneous graph to obtain each training subgraph;

[0038] Encode edge temporal information for each training subgraph to obtain a batch of training subgraphs with temporal features;

[0039] Layer-by-layer message passing and neighborhood feature aggregation are performed on batch training subgraphs containing time features to obtain the aggregation vector of each target user node;

[0040] Generate a user embedding vector that incorporates multi-hop higher-order relationship information based on the aggregation vector of each target user node.

[0041] Optionally, the step of performing hierarchical neighbor sampling on the heterogeneous graph to obtain each training subgraph includes:

[0042] Traverse each user node, and based on the edge association relationship of the heterogeneous graph, traverse its neighbor nodes from 1 hop to k hops respectively, divide its neighbor set according to edge type, and thus obtain the three-dimensional mapping table of hop number-edge type-neighbor node for each user node.

[0043] Generate a four-dimensional mapping table of hop count, edge type, neighbor node, and sampling probability for each user node based on the three-dimensional mapping table of hop count, edge type, neighbor node, and sampling probability for each user node.

[0044] Hierarchical sampling is performed on each four-dimensional mapping table of hop count-edge type-neighbor node-sampling probability to obtain the set of sampled neighbors from 1 hop to k hops for each user node;

[0045] Each user node generates an independent training subgraph based on its 1-hop to k-hop sampled neighbor set. Each independent training subgraph includes a node table and an edge table. The edge table fields include source node ID, target node ID, edge type, edge attribute dictionary, and timestamp field.

[0046] Optionally, the step of encoding edge temporal information for each training subgraph to obtain a batch of training subgraphs containing temporal features includes:

[0047] Each training subgraph is processed as follows:

[0048] Each edge table is timestamped to obtain a standardized edge table;

[0049] Generate an edge-time feature vector mapping table for each standardized edge table;

[0050] The edge-time feature vector mapping table, the node table, and the initial feature vectors of nodes of the same dimension are fused together to obtain training subgraphs with time features. The training subgraphs with time features are then combined to form a batch training subgraph with time features.

[0051] Optionally, the step of performing layer-by-layer message passing and neighborhood feature aggregation on the batch training subgraphs containing time features to obtain the aggregation vector for each target user node includes:

[0052] Generate input features based on batch training subgraphs containing time features;

[0053] Perform the following iterative operation until the iteration condition is met:

[0054] Based on the edge table of the batch training subgraph with time features, the neighbor subsets of the current layer of the user node are divided according to edge type, the input features are associated, and the features to be aggregated in each subset are determined, thus obtaining the correspondence table of edge type-neighbor subset-features to be aggregated.

[0055] Based on the edge type-neighbor subset-feature to be aggregated correspondence table, the linear projection matrix corresponding to different node types is called to map the features to be aggregated of each subset to a unified dimension feature space, thereby obtaining the node type normalized feature vector of each neighbor node.

[0056] Based on the node type normalized feature vectors of each neighbor node, the relationship transformation matrix and attention parameters corresponding to different edge types are called to calculate the attention weight of each neighbor node. After softmax normalization, the normalized attention coefficient of each neighbor node is obtained.

[0057] Based on the normalized attention coefficients of each neighbor node, the node type normalized feature vectors of the corresponding neighbor nodes are weighted and aggregated. After cross-edge type fusion, the vectors are connected to the residuals through a nonlinear activation function to obtain the aggregated vector of the target user node at the current layer.

[0058] Based on the l-th layer aggregate representation of the target user node, update the layer-node-input feature mapping table to obtain the input features of the next layer;

[0059] The iteration condition is:

[0060] The iteration stops when the current layer reaches the upper limit of the number of hops k or the cosine similarity between the aggregate representations of two adjacent layers is ≥0.99. The aggregate representations of layers 1 to k for each user node are output, thus obtaining the set of aggregate vectors of the target user node.

[0061] This application also provides a system for mining complex behavior patterns of communication users based on heterogeneous graph neural networks, the system comprising:

[0062] User real-time data acquisition module, the user real-time data acquisition module is used to acquire user real-time data;

[0063] The model acquisition module is used to acquire a trained heterogeneous graph neural network model and a trained downstream behavior pattern recognition model.

[0064] A local heterogeneous graph subgraph update module is used to update a preset heterogeneous graph using real-time user data, thereby obtaining a local heterogeneous graph subgraph with updated attributes.

[0065] The updated user embedding vector generation module is used to input the local subgraph with updated attributes into a trained heterogeneous graph neural network model to obtain the updated user embedding vector.

[0066] The identification result acquisition module is used to embed the updated user into the communication operation data proposed by the present invention, which is to uniformly model the communication operation data as a heterogeneous graph containing user nodes, product nodes and application nodes, and perform multi-layer message passing based on the improved heterogeneous graph neural network, and fully aggregate different types of neighbors and their edge attribute information.

[0067] This application cleanses and standardizes structured data collected from telecommunications operation networks, constructing a heterogeneous graph containing user nodes, product nodes, application nodes, call edges, order edges, click edges, search edges, and usage edges. Initial feature vectors are generated for different node types and input into the heterogeneous graph neural network. Utilizing a multi-head relationship attention mechanism based on node and edge types, message passing and aggregation are performed in multi-layer neighborhoods to obtain user embedding vectors that fuse multi-hop higher-order relationships. These user embedding vectors are then input into a downstream behavioral pattern recognition model, outputting prediction results and scores for complex behavioral patterns such as potential user churn and product preferences, and generating corresponding labels and strategy recommendation information. This application's solution can effectively capture multi-hop higher-order relationships on large-scale heterogeneous communication graph data, significantly improving accuracy, generalization ability, and scalability in complex behavioral pattern recognition compared to traditional independent label methods. Attached Figure Description

[0068] Figure 1 This is a flowchart illustrating a method for mining complex behavior patterns of communication users based on heterogeneous graph neural networks according to an embodiment of this application.

[0069] Figure 2 This is a schematic diagram of heterogeneous graph construction in one embodiment of this application;

[0070] Figure 3 This is a schematic diagram of a multi-layer message passing mechanism in one embodiment of this application;

[0071] Figure 4 This is a schematic diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0072] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be described in more detail below with reference to the accompanying drawings. In the drawings, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The described embodiments are some, but not all, embodiments of this application. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0073] like Figure 1 The method for mining complex behavioral patterns of communication users based on heterogeneous graph neural networks, as shown, includes:

[0074] Obtain real-time user data;

[0075] Obtain a trained heterogeneous graph neural network model and a trained downstream behavior pattern recognition model;

[0076] The preset heterogeneous graph is updated using real-time user data to obtain a local heterogeneous graph subgraph with updated attributes.

[0077] The updated local subgraph is input into a trained heterogeneous graph neural network model to obtain the updated user embedding vector.

[0078] The updated user embedding vector is input into the trained downstream behavior pattern recognition model to obtain the recognition result.

[0079] In this embodiment, the heterogeneous graph neural network model and the downstream behavior pattern recognition model are trained in the following manner:

[0080] Obtain user attribute datasets, call log datasets, product order datasets, and application usage datasets;

[0081] A heterogeneous graph is generated based on user attribute datasets, call record datasets, product order datasets, and application usage datasets; the heterogeneous graph includes user nodes, product nodes, and application nodes, and the edges include call edges, order edges, click edges, search edges, and usage edges;

[0082] Acquire heterogeneous graph neural networks and downstream multi-task learning networks;

[0083] Iterative training is performed on the heterogeneous graph neural network and the downstream multi-task learning network: user embedding vectors that integrate multi-hop higher-order relation information are generated based on the acquired heterogeneous graph; the heterogeneous graph neural network and the downstream multi-task learning network are jointly trained by the user embedding vectors that integrate multi-hop higher-order relation information, thereby obtaining the trained heterogeneous graph neural network model and the trained downstream behavior pattern recognition model.

[0084] In this embodiment, the user attribute dataset, call log dataset, product order dataset, and application usage dataset are obtained in the following ways:

[0085] Structured data such as user basic attributes, call details, product order records, and application usage logs are collected from the operation support system, billing system, and business support system. Missing values ​​are filled or removed, outliers are identified and corrected, time fields are formatted uniformly, and category fields are coded.

[0086] In this embodiment, the user attribute dataset includes multiple user attribute data, each of which includes a unique user identifier, basic user information, and basic product information; for example, user attribute data includes age, gender, city, network duration, current package type, average call charges, average data usage, etc.

[0087] The call record dataset includes multiple call record data, each of which includes a unique call record identifier and basic call information; for example, the call record data includes the caller's number, the called number, the call start time, the call duration, and the caller and called party identifiers, etc.

[0088] The product order dataset includes multiple product order data sets, each containing a unique product order identifier, a unique product identifier, and basic order information. For example, product order data may include a user identifier, product identifier, order time, cancellation time, and order status. The application usage dataset includes multiple application usage data sets, each containing a unique application usage identifier, a unique application identifier, and application usage data. For example, application usage data may include a user identifier, application identifier, usage start time, usage duration, and number of uses.

[0089] In this embodiment, the process of generating a heterogeneous graph based on user attribute datasets, call log datasets, product order datasets, and application usage datasets is described. include:

[0090] Extract the unique user identifier from the user attribute data and assign a globally unique user node ID to each unique identifier; associate the user's basic information and product basic information from the user attribute data with the corresponding user node; specifically, use fields such as age, gender, and city from the user attribute data as the basic attributes of the user node and associate them with the corresponding user node to ensure that one user identifier corresponds to one user node and there are no duplicate nodes.

[0091] Extract the unique product order identifier from the product order data and assign a globally unique product node ID to each product order identifier; associate the basic order information with the corresponding product node; supplement the basic product attributes (obtained from the operator's product database, including product category: 5G package / data package / value-added service, etc., tariff level, whether it is a promotional product, and applicable user group), and associate them with the corresponding product node to ensure that "one product identifier corresponds to one product node";

[0092] Extract the unique application usage identifier from the application usage data, and assign a globally unique application node ID to each application usage identifier; associate the application usage data with the corresponding application node; supplement the application's basic attributes (obtained from the application management library, including application category: social / video / tool / game, etc., developer, user group penetration rate, etc.), and associate them with the corresponding application node to ensure that "one application identifier corresponds to one application node";

[0093] Map the caller's unique identifier in the call log to the corresponding user node ID, and map the called party's unique identifier in the call log to the corresponding user node ID. Using these two user nodes as the two endpoints of the call edge, assign values ​​to the edge attributes based on the basic call information. Specifically, extract and calculate the following attributes from the call log and bind them to the call edge:

[0094] Call count: The cumulative number of calls made by the same calling-called combination within a preset time window (such as the last 30 days);

[0095] Total call duration: The cumulative call duration (in seconds) for the same group;

[0096] Caller ID: The user node ID of the initiator in the call record corresponding to the current side (marking the caller's identity);

[0097] Call timestamp: The timestamp of the most recent call;

[0098] Call connection rate: Number of connections for the same combination / Total number of calls;

[0099] Uniqueness rule for edges: Only one call edge is constructed for the same calling-called combination, and the attribute values ​​are accumulated as subsequent data is updated.

[0100] The user's unique identifier is mapped to the user node ID, and the product's unique identifier is mapped to the product node ID. These two nodes serve as the two endpoints of the order edge, and the edge attributes of the order edge are assigned values ​​based on the basic order information. Specifically, the following attributes are extracted from the product order data and bound to the order edge:

[0101] Product order time: Order timestamp (accurate to the second);

[0102] Order period: Monthly / Quarterly / Annual (consistent with product order data);

[0103] Fee Information: Monthly Fee (Unit: Yuan), Cumulative Consumption Amount (Subscription Duration × Monthly Fee);

[0104] Subscription status: In use (unsubscription timestamp is null) / Unsubscribed (unsubscription timestamp recorded) / Expired (subscription period exceeded and not renewed);

[0105] Renewal Intention Indicator: Whether there has been any proactive renewal activity in the past 6 months (1=Yes / 0=No);

[0106] Edge uniqueness rule: Only one order edge is constructed for the same user-product combination, and attributes such as order status and cumulative consumption amount are dynamically adjusted as the data is updated.

[0107] Map the user's unique identifier to the user node ID, and the application's unique identifier to the application node ID. Use these two nodes as the two endpoints of the click edge, and assign values ​​to the edge attributes of the click edge using application usage data. Specifically, extract and calculate the following attributes from the click record and bind them to the click edge:

[0108] Click timestamp: The timestamp of the latest click;

[0109] Click frequency: The cumulative number of clicks in the past 30 days;

[0110] Click time distribution: Percentage of clicks during morning (6:00-12:00), noon (12:00-18:00), evening (18:00-24:00), and early morning (0:00-6:00);

[0111] Edge uniqueness rule: Only one click edge is constructed for the same user-application combination, and the attribute values ​​are updated cumulatively with subsequent click behaviors.

[0112] The user's unique identifier is mapped to the user node ID, and the application's unique identifier is mapped to the application node ID. These two nodes serve as the endpoints of the search edge, and the application uses data to assign values ​​to the edge attributes of the search edge. Specifically, the following attributes are extracted and calculated from the search records and bound to the search edge:

[0113] Search timestamp: The timestamp of the latest search;

[0114] Search frequency: The cumulative number of searches in the past 30 days;

[0115] Search keyword relevance: Extracting the matching degree between user search keywords and application core functions (0-1 points, 1 point is a perfect match).

[0116] Edge uniqueness rule: Only one search edge is constructed for the same user-application combination, and the attribute values ​​are updated cumulatively with subsequent search behaviors.

[0117] The user's unique identifier is mapped to the user node ID, and the application's unique identifier is mapped to the application node ID. These two nodes serve as the endpoints of the usage edge, and the edge attributes are assigned using application usage data. Specifically, the following attributes are extracted and calculated from the usage record and bound to the usage edge:

[0118] Use timestamp: The timestamp of the most recent use;

[0119] Usage frequency: The cumulative number of times used in the past 30 days;

[0120] Total usage time: Cumulative usage time (in seconds) over the past 30 days;

[0121] Average daily usage time: Total usage time in the past 30 days / Number of days used (0 if there are no days used);

[0122] Edge uniqueness rule: Only one usage edge is constructed for the same user-application combination, and the attribute values ​​are updated cumulatively with subsequent usage behavior.

[0123] In this embodiment, the heterogeneous diagram In, node set Includes a set of user nodes Product Node Set and application node set edge set It includes a set of call edges, a set of order edges, and sets of interaction edges such as clicks, searches, and usage.

[0124] In this embodiment, when constructing a call edge, the user nodes corresponding to the calling and called numbers in the call record are used as endpoints to establish a call edge between them, and attributes such as the number of calls, total call duration, and call time distribution are added to the call edge; when constructing an order edge, the user node and product node are used as endpoints to establish an order edge between them, and attributes such as product order time, order cycle, and status are added to the order edge; when constructing click edges, search edges, and usage edges, the user node and application node are used as endpoints, and attributes such as the number of clicks, search counts, and usage duration of the user within a certain time window are added.

[0125] In this embodiment, the heterogeneous graph specifically includes:

[0126] Node sets: User node set (U_XXX): contains a unique ID and basic attributes (age, gender, etc.) for each node; Product node set (P_XXX): contains a unique ID and basic attributes (product category, pricing, etc.) for each node; Application node set (A_XXX): contains a unique ID and basic attributes (application category, penetration rate, etc.) for each node.

[0127] Edge sets: Call edge set: Each edge contains a unique edge ID, user node IDs at both ends, and attributes (number of calls, total duration, etc.); Order edge set: Each edge contains a unique edge ID, user node ID, product node ID, and attributes (order time, cost, etc.); Click edge set: Each edge contains a unique edge ID, user node ID, application node ID, and attributes (click frequency, time period distribution, etc.); Search edge set: Each edge contains a unique edge ID, user node ID, application node ID, and attributes (search frequency, keyword relevance, etc.); Usage edge set: Each edge contains a unique edge ID, user node ID, application node ID, and attributes (usage duration, average daily usage duration, etc.).

[0128] It supports adjacency list files (including node attribute tables and edge attribute tables) stored in graph databases (such as Neo4j and NebulaGraph) or directly readable by the model, ensuring that the relationship between nodes and edges is clear and that attribute fields can be called by the subsequent model.

[0129] In this embodiment, iterative training is performed on the heterogeneous graph neural network and the downstream multi-task learning network: a user embedding vector incorporating multi-hop higher-order relation information is generated based on the acquired heterogeneous graph; the heterogeneous graph neural network and the downstream multi-task learning network are jointly trained using the user embedding vector incorporating multi-hop higher-order relation information, thereby obtaining a trained heterogeneous graph neural network model and a trained downstream behavior pattern recognition model, including:

[0130] Generate user embedding vectors that fuse multi-hop higher-order relationship information based on the obtained heterogeneous graph; wherein, generating user embedding vectors that fuse multi-hop higher-order relationship information based on the obtained heterogeneous graph includes:

[0131] Feature extraction is performed on the heterogeneous graph to obtain initial feature vectors of nodes with uniform dimensions;

[0132] User embedding vectors that integrate multi-hop higher-order relationship information are generated based on the initial feature vectors of nodes with a unified dimension and heterogeneous graphs.

[0133] See Figure 2 In this embodiment, feature extraction of the heterogeneous graph to obtain initial feature vectors of nodes with uniform dimensions includes:

[0134] The original feature vectors of nodes are extracted using preset feature extraction rules;

[0135] Generate initial feature vectors of nodes with uniform dimensions based on the original feature vectors of the nodes.

[0136] In this embodiment, extracting the original feature vector of a node using a preset feature extraction rule includes:

[0137] General basic rules:

[0138] Feature validity screening rules: Remove feature fields with a missing rate of more than 30%, and retain fields in the heterogeneous graph node attributes and associated edge attributes that are strongly related to business objectives (churn prediction, product preference);

[0139] Feature redundancy removal rule: Redundant features with an absolute correlation coefficient ≥ 0.8 are removed by screening based on Pearson correlation coefficient, and representative features are retained;

[0140] Outlier handling rules: Use the 3σ criterion to identify outliers of continuous features and replace them with the 95th percentile of the feature (to avoid the influence of extreme values).

[0141] 2. Node type-specific rules

[0142] (1) User node feature extraction rules

[0143] Feature field filtering rules: From the user node attributes of the heterogeneous graph, filter the following attributes: population attributes (age, gender, city, network duration), consumption capacity attributes (average call charges in the past 3 months, current package type, average monthly data usage), and behavioral statistics attributes (total call duration in the past 3 months, call frequency, total application interaction time, number of products ordered).

[0144] Category feature coding rules: Gender (male / female), city (city code within the province), and package type (basic / advanced / premium) are encoded using one-hot coding to generate binary feature vectors;

[0145] Continuous feature normalization rules: Continuous features such as age, network duration, and average call charges are normalized using Min-Max and mapped to the [0,1] interval.

[0146] (2) Product node feature extraction rules

[0147] Feature field filtering rules: Filter the core product attributes (product category, price level, subscription period, promotion status, service type) from the product node attributes of the heterogeneous graph.

[0148] Category feature coding rules: Product category (voice package / data package / integrated package) and promotion status (yes / no) use unique hot coding, and service type uses tag coding (levels 1-5 correspond to different service levels).

[0149] Continuous feature normalization rules: Tariff level (monthly fee) and subscription period (number of months) are normalized using Z-Score.

[0150] (3) Apply node feature extraction rules

[0151] Feature field filtering rules: Filter application basic attributes (application category, developer, online duration) and usage statistics attributes (target user group penetration rate, monthly average usage frequency, average usage time per user) from the application node attributes of the heterogeneous graph.

[0152] Category feature coding rules: Application category (social / entertainment / tools / office) and developer (top / middle / tail) use one-hot coding;

[0153] Continuous feature normalization rules: penetration rate, usage frequency, and usage duration are normalized to the [0,1] interval using Min-Max.

[0154] Feature extraction is performed using the above rules to obtain the original, unchanging feature vectors of the nodes (user). ,product ,application ),user ,product ,application The original feature vectors of the constituent nodes;

[0155] Repeat the following steps until the joint loss function converges:

[0156] Generate initial feature vectors of nodes with uniform dimensions based on the original feature vectors of the nodes.

[0157] Call the projection matrix of the current iteration to perform dimension mapping on the original feature vectors of the nodes:

[0158] User node: ;

[0159] Product nodes: ;

[0160] Application Node: ;

[0161] The final output is the feature vector of the same dimension for the current iteration;

[0162] In this embodiment, different node types are considered. Set the corresponding linear projection matrix , the initial feature vector Projecting onto a feature space of uniform dimension yields... .

[0163] In this embodiment, the step of generating a user embedding vector that fuses multi-hop higher-order relationship information based on the initial feature vector of nodes of a unified dimension and the heterogeneous graph includes:

[0164] Hierarchical neighbor sampling is performed on the heterogeneous graph to obtain each training subgraph;

[0165] In this embodiment, hierarchical neighbor sampling is performed on the heterogeneous graph to obtain each training subgraph, including:

[0166] Traverse each user node, and based on the edge association relationship of the heterogeneous graph, traverse its neighbor nodes from 1 hop to k hops respectively, divide its neighbor set according to edge type, and thus obtain the three-dimensional mapping table of hop number-edge type-neighbor node for each user node.

[0167] In this embodiment, starting from any user node, nodes are directly connected by an edge. Jump to neighbor (directly associated node); A hop neighbor's directly associated node (not the user node itself) is a 2-hop neighbor; and so on, the hop neighbor... Jump neighbor refers to the interval between the target user node and the target user node. The nodes of the edge, and all their neighboring nodes come from the heterogeneous graph.

[0168] In this embodiment, the neighbor set refers to the set of nodes that are associated with the target user node, categorized by edge type. The set of nodes associated with jumps, denoted as ( For user nodes, Number of jumps Edge type: =1→Call side, =2→Order edge, =3→Click on the edge, =4→Search edge, =5→Use edges).

[0169] In this embodiment, for a user node, based on the edge association relationship of the heterogeneous graph G, it recursively traverses its 1-hop to k-hop neighbor node. The traversal terminates when k hops are reached or no new neighbor node is found.

[0170] For each jump (1≤ ≤k), by edge type (1-5) Divide the neighbor nodes and generate Jump to the corresponding 5 neighbor subsets , , ;

[0171] Record the association path between each neighbor node and the user node, and finally obtain a three-dimensional mapping table of hop count-edge type-neighbor node for each user node u, denoted as Map_u.

[0172] Generate a four-dimensional mapping table of hop count, edge type, neighbor node, and sampling probability for each user node based on the three-dimensional mapping table of hop count, edge type, neighbor node, and sampling probability for each user node.

[0173] In this embodiment, generating a four-dimensional mapping table for each user node based on the hop count-edge type-neighbor node-sampling probability three-dimensional mapping table includes:

[0174] For each neighbor subset in Map_u Extract edge type from node v in the data. Given the original weight values ​​of all edges, calculate the minimum weight (min(value_r)) and the maximum weight (max(value_r)) of all edges of this type under this edge type;

[0175] For user nodes and The associated edge e between (or through -1 Jump node to In the path, the last edge Extract its original weight value value_e, and calculate the edge weight coefficient ω_e according to the Min-Max normalization formula;

[0176] Calculate the sampling probability of node v ( edge type Business weight, (sampling rate);

[0177] For each neighbor subset Normalize the sampling probabilities of all nodes to ensure the probability distribution is effective; finally, obtain the sampling probabilities of each user node. The corresponding four-dimensional mapping table of hop count, edge type, neighbor node, and sampling probability is denoted as ProbMap_u.

[0178] Hierarchical sampling is performed on each hop count-edge type-neighbor node-sampling probability four-dimensional mapping table to obtain the corresponding data for each user node. Jump to Skip sampling neighbor set;

[0179] In this embodiment, hierarchical sampling is performed on each four-dimensional mapping table of hop count-edge type-neighbor node-sampling probability to obtain the corresponding data for each user node. Jump to The skip sampling neighbor set includes:

[0180] For each hop, calculate the total number of nodes in all neighbor subsets of that hop. ;

[0181] like (Upper limit of this hop sampling): Directly select all nodes of all neighbor subsets of this hop to form the l-hop sampling neighbor set. ;

[0182] like For each neighbor subset Weighted random sampling is performed according to the sampling probability P_{u,l,r,v}, and the number of samples is allocated according to the proportion of the number of nodes in the subset × S_l” (e.g., If the proportion is 30%, then the number of samples is 0.3 × S_l (rounded up). After sampling, the sampling nodes of all subsets are merged to form an l-hop sampling neighbor set. ;

[0183] During the sampling process, the associated edge attributes (including normalized ω_e) and sampling probability P_{u,l,r,v} of each sampling node v are retained; finally, the set of sampling neighbors from 1 hop to k hop corresponding to each user node u is obtained, denoted as SAMPLE_u, and each set contains a quadruple of node ID, edge type, associated edge attributes, and sampling probability.

[0184] Each user node generates an independent training subgraph based on its 1-hop to k-hop sampled neighbor set. Each independent training subgraph includes a node table and an edge table. The edge table fields include source node ID, target node ID, edge type, edge attribute dictionary, and timestamp field.

[0185] In this embodiment, generating an independent training subgraph for each user node based on the 1-hop to k-hop sampled neighbor set includes:

[0186] Merge u itself with all nodes in SAMPLE_u to obtain the subgraph node set. ;

[0187] Summarize edges: Extract from heterogeneous graph G The set of edges in the subgraph is obtained by considering all the associated edges (including edge attributes) between all nodes. (Ensure that both ends of the edge are in) (in the middle); finally, obtain the independent training subgraph corresponding to each user node u. =( , ).

[0188] Encode edge temporal information for each training subgraph to obtain a batch of training subgraphs with temporal features;

[0189] In this embodiment, edge temporal information encoding is performed on each training subgraph to obtain a batch of training subgraphs containing temporal features, including:

[0190] Each training subgraph is processed as follows:

[0191] Normalize the timestamps of each edge table to obtain a normalized edge table;

[0192] In this embodiment, normalizing the timestamps of each edge table to obtain a normalized edge table includes:

[0193] Traverse the edge table Edge_Table of each subgraph, and extract the timestamp field of each edge (e.g., extract the "call timestamp" for E1 call edges and the "subscription timestamp" for E2 subscription edges);

[0194] Format conversion: If it is an ISO 8601 time string, convert it to UTC time zone through a time zone conversion tool and then parse it into a Unix timestamp in seconds (e.g., "2024-05-20T18:30:00+08:00" → 1716207000 seconds); if it is a Unix timestamp in milliseconds, divide it by 1000 to convert it to seconds;

[0195] Missing value filling: For edges with missing timestamps, calculate the mean of all valid timestamps of the same subgraph and the same edge type as the filling value (e.g., there are 50 E5 usage edges in the subgraph, and the mean of 49 valid timestamps is 1715000000 seconds, and use this value to fill the missing edge);

[0196] Outlier removal: If the timestamp exceeds the reasonable business time range (e.g., earlier than January 1, 2010 or later than the current system time), directly remove this edge (because abnormal times have no practical meaning), and finally obtain a normalized edge table.

[0197] Generate an edge-time feature vector mapping table according to each normalized edge table respectively;

[0198] In this embodiment, generating an edge-time feature vector mapping table according to each normalized edge table respectively includes:

[0199] Load a preset sine-cosine encoding function, and the formula is as follows:

[0200] For the timestamp t (in seconds), the i-th dimension (0 ≤ i < D) of the time feature vector time_feat is calculated as:

[0201] When i is even: [[ID=3)]] ;

[0202] When i is odd: ;

[0203] Where, = 10 4 (default base period), D is the time encoding dimension (consistent with the initial feature vector dimension);

[0204] Edge-by-edge calculation: For each edge in Edge_Table_Norm, generate a D-dimensional time feature vector according to the above formula. The vector data type is float32, and the value range is [-1,1].

[0205] Finally, obtain the edge-time feature vector mapping table (Edge_TimeFeat_Map) for each subgraph, with the structure {edge ID: time feature vector (D-dimensional float32 array)}.

[0206] The edge-time feature vector mapping table, the node table, and the initial feature vectors of nodes of the same dimension are fused together to obtain training subgraphs with time features. The training subgraphs with time features are then combined to form a batch training subgraph with time features.

[0207] In this embodiment, in each training iteration, a target user node and its multi-layer neighbor nodes are selected to form a training subgraph according to a preset hierarchical sampling strategy; the size of the training subgraph is controlled not to exceed a preset resource constraint, and a computing resource allocation strategy is configured accordingly to improve the system training efficiency.

[0208] In this embodiment, graph representation learning can be deployed on a server with GPU acceleration capabilities. The size of the training subgraph can be controlled through techniques such as hierarchical neighbor sampling, thereby supporting model training on large-scale heterogeneous communication graphs. Behavioral pattern recognition can provide interfaces to the outside world in a microservice manner, providing online inference services to downstream business systems.

[0209] In this embodiment, the edge-time feature vector mapping table, the node table, and the initial node feature vectors of a unified dimension are fused to obtain a training subgraph containing time features. The various training subgraphs containing time features constitute a batch of training subgraphs containing time features, including:

[0210] For each subgraph's node table Node_Table, the initial feature vectors (dimension D) of all neighboring nodes of that subgraph are extracted from the node initial feature vector Init_Feature_Matrix of the same dimension through the node ID-initial feature vector row index mapping table.

[0211] Edge-node association: For each neighboring node v, find the associated edge e between it and the target user node u (matched from the subgraph Edge_Table_Norm), and extract the time feature vector (dimension D) corresponding to edge e from Edge_TimeFeat_Map.

[0212] Dimensional concatenation: Perform dimensional concatenation in the order of initial feature vector (first D dimensions) + time feature vector (second D dimensions) to obtain a 2D enhanced feature vector, as shown in the following formula:

[0213] (“;” indicates dimension concatenation), finally obtaining the Enhance_Feature_Table of the training subgraph containing time features (including neighbor node IDs and corresponding 2D enhanced feature vectors);

[0214] Layer-by-layer message passing and neighborhood feature aggregation are performed on batch training subgraphs containing time features to obtain the aggregation vector of each target user node;

[0215] In this embodiment, layer-by-layer message passing and neighborhood feature aggregation are performed on the batch training subgraph containing time features to obtain the aggregated vector of each target user node, including:

[0216] Generate input features based on batch training subgraphs containing time features;

[0217] Specifically, initialize the layer counter l=1, construct the layer-node-input feature mapping table, store the initial feature vectors of nodes of the same dimension into the mapping table, and obtain the input features of the first layer;

[0218] Perform the following iterative operation until the iteration condition is met:

[0219] Based on the edge table of the batch training subgraph with time features, the neighbor subsets of the current layer of the user node are divided according to the edge type (call edge, order edge, click edge, search edge, use edge). The input features in the layer-node-input feature mapping table are associated to determine the features to be aggregated for each subset, thus obtaining the correspondence table of edge type-neighbor subset-features to be aggregated.

[0220] Based on the edge type-neighbor subset-feature to be aggregated correspondence table, the linear projection matrix corresponding to different node types is called to map the features to be aggregated of each subset to a unified dimension feature space, thereby obtaining the node type normalized feature vector of each neighbor node.

[0221] Based on the node type normalized feature vectors of each neighbor node, the relationship transformation matrix and attention parameters corresponding to different edge types are called to calculate the attention weight of each neighbor node. After softmax normalization, the normalized attention coefficient of each neighbor node is obtained.

[0222] Based on the normalized attention coefficients of each neighbor node, the node type normalized feature vectors of the corresponding neighbor nodes are weighted and aggregated. After cross-edge type fusion, the vectors are connected to the residuals through a nonlinear activation function to obtain the aggregated vector of the target user node at the current layer.

[0223] Based on the l-th layer aggregate representation of the target user node, update the layer-node-input feature mapping table to obtain the input features of the next layer;

[0224] The iteration condition is:

[0225] The iteration stops when l reaches the upper limit of the number of hops k or the cosine similarity between two adjacent aggregate representations is ≥0.99. The aggregate representations of each user node from level 1 to level k are output, thus obtaining the set of aggregate vectors of the target user node.

[0226] In this embodiment, constructing the layer-node-input feature mapping table includes:

[0227] Create a three-dimensional mapping table of layer number, node ID, and input feature vector. The fields include: layer number (initially only l=1 is supported); node ID (a unique identifier for all target user nodes and neighboring nodes in the batch training subgraph); and input feature vector (stores the unified dimensional feature after fusing temporal features).

[0228] After concatenating the initial feature vector of the node with the time feature vector of the corresponding edge according to the same dimension, the node is associated with the node ID and stored in the l=1 entry of the mapping table to ensure that the first layer input features of each node include the node's own features and the edge time features.

[0229] In this embodiment, based on the edge table of the batch training subgraph containing time features, the neighbor subsets of the current layer of the user node are divided according to edge type, and the input features are associated to determine the features to be aggregated for each subset, thereby obtaining the edge type-neighbor subset-feature to be aggregated correspondence table, including:

[0230] Edge type filtering: Extract all associated edges of the target user node from the edge table and divide them into 5 categories according to edge type (call edge, order edge, click edge, search edge, usage edge), with each category of edge corresponding to an independent subset of neighbors;

[0231] Neighbor node matching: For each type of edge, filter out all neighbor nodes that are associated with the target user node through that type of edge, forming an edge type-neighbor node set mapping relationship (e.g., call edge neighbor subset = {user A, user B}, order edge neighbor subset = {product C, product D}).

[0232] Input feature association: Match the l-th layer input feature vector of each neighboring node in the mapping table by node ID, and use it as the feature to be aggregated for that neighboring node;

[0233] Construct a correspondence table: Create a three-dimensional correspondence table of edge type, neighbor subset, and features to be aggregated. Fields include: edge type (identifying each of the 5 edge types); neighbor subset ID (a unique identifier for the neighbor set corresponding to each edge type); and list of features to be aggregated (the l-level input feature vectors of all neighbor nodes within this subset). The final output is the edge type-neighbor subset-feature to be aggregated correspondence table (clearly defining the neighbor nodes and aggregable features for each edge type).

[0234] In this embodiment, based on the edge type-neighbor subset-feature to be aggregated correspondence table, the linear projection matrix corresponding to different node types is called to map the features to be aggregated in each subset to a unified dimension feature space, resulting in the node type normalized feature vector of each neighbor node, including:

[0235] Node type identification: Traverse the nodes of each neighbor subset in the corresponding relationship table and distinguish the node type (user / product / application).

[0236] Projection matrix matching: Calling the corresponding linear projection matrix for different types of nodes (e.g., calling the linear projection matrix for user nodes). Product node call );

[0237] Dimension mapping operation: Perform matrix multiplication mapping on the feature vector to be aggregated for each neighbor node, as shown in the following formula:

[0238] (in For nodes type For nodes The features to be aggregated in layer l, (This is the normalized feature after mapping); finally, the node type normalized feature vector of each neighbor node (grouped and stored according to edge type - neighbor subset, with a unified dimension of D).

[0239] In this embodiment, based on the node type normalized feature vectors of each neighbor node, the relation transformation matrix and attention parameters corresponding to different edge types are called to calculate the attention weights of each neighbor node. After softmax normalization, the normalized attention coefficients of each neighbor node are obtained, including:

[0240] For all neighboring nodes under the same edge type r, the normalized features are processed using the relation transformation matrix of that edge type, as shown in the following formula:

[0241] ;

[0242] Attention weight calculation: transforming neighbor features Layer l input features of the target user node Concatenation (2D×1 dimension), and edge type attention vector The attention weights are obtained through calculation:

[0243] ;

[0244] Softmax normalization: for the same type The attention weights of all neighboring nodes are subjected to a softmax operation to obtain the normalized attention coefficients, as shown in the formula:

[0245] (in Indicates the edge type Next node (the set of neighbors).

[0246] In this embodiment, generating a user embedding vector that fuses multi-hop higher-order relationship information based on the aggregation vector of each target user node includes:

[0247] For the same type The aggregated features of this subset are obtained by summing the neighboring nodes according to their normalized attention coefficients:

[0248] ;

[0249] Cross-edge type fusion: The aggregated features of the subsets corresponding to the 5 edge types are fused by weighted summation (the weights are learnable parameters) to obtain preliminary aggregated features:

[0250] (in edge type The fusion weight, );

[0251] See Figure 3 Activation and Residual Connection: The initial aggregated features are input into the ReLU activation function, and then added to the l-layer input features of the target user node after mapping with the residual matrix. The formula is as follows:

[0252] ;

[0253] Aggregate representation optimization: L2 normalization is applied to the results (ensuring the vector norm is 1) to avoid numerical inflation. Finally, the l-th layer aggregate vector of the user node is obtained.

[0254] Based on the l-th layer aggregate representation of the target user node, the layer-node-input feature mapping table is updated to obtain the input features of the next layer, including:

[0255] Add a new mapping entry for the next level: Add a field entry for level l+1 in the mapping table;

[0256] Target user node feature update: The l-th layer aggregate representation of the target user node is used as the input feature of its l+1 layer and stored in the corresponding node ID entry of the l+1 layer of the mapping table;

[0257] Neighbor node feature synchronization update: For all neighbor nodes participating in aggregation in layer l, repeat sub-step ad to generate their layer l aggregate representation, and store it in the mapping table as the input feature of layer l+1;

[0258] Increment the layer counter: Set l=l+1 to prepare for the next iteration and obtain the updated "layer-node-input feature mapping table" (containing the input features of all nodes in layer l+1).

[0259] After the iteration is complete, the different layers will be aggregated. The final user embedding vector is obtained by merging the data through concatenation or weighted summation. :

[0260]

[0261] in, These are the learnable layer weight coefficients.

[0262] By overlaying several layers of message passing, this embodiment can progressively aggregate the features of one-hop, two-hop, and even higher-hop neighbors into the target user node representation, thereby capturing multi-hop high-order relationship information. Finally, the representations of each layer are fused in a weighted summation manner to obtain the user embedding vector.

[0263] In this embodiment, the heterogeneous graph neural network and the downstream multi-task learning network are jointly trained by fusing user embedding vectors containing multi-hop higher-order relation information, thereby obtaining a trained heterogeneous graph neural network model and a trained downstream behavior pattern recognition model, including:

[0264] nth iteration - loss calculation and parameter update:

[0265] Joint loss calculation: based on Calculation error; where, The cross-entropy loss is the prediction of churn. For ranking or regression loss of recommendation tasks, For regularization loss, , , These are the weighting coefficients;

[0266] Gradient calculation: Through backpropagation, solve for the gradient of the loss with respect to all parameters (including the projection matrix, relation transformation matrix, and downstream model weights).

[0267] Parameter update: Fine-tune all parameters using the Adam optimizer (learning rate 0.001-0.01) to obtain the parameters for the (n+1)th iteration; d. Convergence judgment: If the loss fluctuation is <1e-5 for 10 consecutive iterations, stop the iteration.

[0268] In this embodiment, the downstream behavior pattern recognition model is a multi-task learning network, which includes a classification branch for predicting potential user churn risk and a recommendation branch for predicting user product preferences.

[0269] The classification branch uses a multilayer perceptron with a softmax output layer to output the probability that a user belongs to each churn risk level. The recommendation branch uses a structure based on dot product or matching network to calculate the matching degree between the user vector and the candidate product vector and output the product preference score. The above output results are converted into churn labels, preference labels and corresponding scores, and sorted by score to obtain the recommendation list.

[0270] The churn risk tags output from the churn prediction branch and the recommendation list output from the product recommendation branch are written into the user profile database in the form of tags and ratings. Operators can access these tags and recommendation results through marketing or customer service systems for targeted marketing, differentiated services, and risk warnings.

[0271] The trained model is deployed to an online inference environment. When newly generated call records, product order data, or application usage data are received, the edge attributes of the corresponding nodes and their neighbors are updated. Based on the updated local subgraph, local message passing and embedding vector updates are performed on the affected target user nodes, thereby achieving near real-time updates of user behavior patterns without completely retraining the model.

[0272] This application also provides a system for mining complex behavior patterns of communication users based on heterogeneous graph neural networks, the system comprising:

[0273] User real-time data acquisition module, the user real-time data acquisition module is used to acquire user real-time data;

[0274] The model acquisition module is used to acquire a trained heterogeneous graph neural network model and a trained downstream behavior pattern recognition model.

[0275] A local heterogeneous graph subgraph update module is used to update a preset heterogeneous graph using real-time user data, thereby obtaining a local heterogeneous graph subgraph with updated attributes.

[0276] The updated user embedding vector generation module is used to input the local subgraph with updated attributes into a trained heterogeneous graph neural network model to obtain the updated user embedding vector.

[0277] The recognition result acquisition module is used to input the updated user embedding vector into the trained downstream behavior pattern recognition model to obtain the recognition result.

[0278] In this embodiment, user tag query and strategy recommendation services are provided to marketing systems, customer service systems and other business systems in the form of an interface; batch query and export of prediction results of complex behavior patterns are supported by user identifier, city dimension or product dimension.

[0279] The offline training subsystem accesses historical data, is responsible for performing model training and periodically updating model parameters; the online inference subsystem accesses real-time data streams, calls the latest model parameters, performs complex behavior pattern predictions for specified users, and returns the prediction results to the business system.

[0280] The behavior pattern recognition module includes a churn prediction submodule, a product recommendation submodule, and a multi-task training submodule. The churn prediction submodule is used to predict the risk of user churn within a preset time window based on user embedding vectors. The product recommendation submodule is used to calculate the matching degree between users and candidate products based on user embedding vectors and output a product preference list. The multi-task training submodule is used to jointly train the graph representation learning module, the churn prediction submodule, and the product recommendation submodule under the constraint of the joint loss function.

[0281] The graph representation learning module includes a type projection unit, a relation attention unit, and a multi-layer message aggregation unit. The type projection unit is used to linearly project the initial feature vector for different node types, so that it maps to a feature space of a unified dimension. The relation attention unit is used to learn the corresponding relation transformation matrix and attention parameters for different edge types, so as to calculate the information transfer weight from the source node to the target node. The multi-layer message aggregation unit is used to repeatedly perform the neighborhood feature aggregation operation according to a preset number of layers, thereby obtaining node representations at different levels and combining them into the final user embedding vector.

[0282] This application has the following advantages:

[0283] Fully explore multi-hop higher-order relationships

[0284] This invention utilizes heterogeneous graph modeling and a multi-layer message passing mechanism (see...). Figure 3 It can leverage information from multiple paths such as "user-user-product" and "user-application-product" within a unified framework to fully explore the complex behavioral patterns of communication users and enhance the model's ability to identify implicit preferences and potential risks.

[0285] Unified modeling of multiple node types and multiple edge types

[0286] This invention employs a heterogeneous graph neural network to uniformly model different types of nodes and edges. Through type projection and relational attention mechanisms, different parameters are assigned to each type of entity and relation, enabling the model to learn more granular structural information for different business relationships.

[0287] Balancing accuracy and scalability

[0288] This invention introduces mechanisms such as hierarchical neighbor sampling in the multi-layer message passing process. While ensuring the effective use of high-order relations, it limits the size of the subgraph involved in each training, thereby reducing computation and storage overhead. It is suitable for large-scale communication scenarios with hundreds of millions of nodes and tens of billions of edges.

[0289] Support for multi-task behavior pattern recognition

[0290] This invention applies user embedding vectors to multiple downstream tasks such as churn prediction and product recommendation, and performs joint training through a joint loss function to achieve feature sharing and task collaboration, which helps to improve overall prediction accuracy and model generalization ability.

[0291] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the computer program, it implements the above-described method for mining complex behavioral patterns of communication users based on heterogeneous graph neural networks.

[0292] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the above-mentioned method for mining complex behavioral patterns of communication users based on heterogeneous graph neural networks.

[0293] Figure 4 This is an exemplary structural diagram of an electronic device capable of implementing the method for mining complex behavioral patterns of communication users based on heterogeneous graph neural networks according to an embodiment of this application.

[0294] like Figure 2As shown, the electronic device includes an input device 501, an input interface 502, a central processing unit 503, a memory 504, an output interface 505, and an output device 506. The input interface 502, central processing unit 503, memory 504, and output interface 505 are interconnected via a bus 507. The input device 501 and output device 506 are connected to the bus 507 via the input interface 502 and output interface 505, respectively, and thus connected to other components of the electronic device. Specifically, the input device 504 receives input information from the outside and transmits it to the central processing unit 503 via the input interface 502. The central processing unit 503 processes the input information based on computer-executable instructions stored in the memory 504 to generate output information, temporarily or permanently storing the output information in the memory 504, and then transmitting the output information to the output device 506 via the output interface 505. The output device 506 outputs the output information to the outside of the electronic device for user use.

[0295] In other words, Figure 2 The illustrated electronic device may also be implemented as including: a memory storing computer-executable instructions; and one or more processors, which can be coupled when executing the computer-executable instructions. Figure 1 This describes a method for intelligent recommendation based on geographic location information.

[0296] In one embodiment, Figure 2 The illustrated electronic device can be implemented to include: a memory 504 configured to store executable program code; and one or more processors 503 configured to run the executable program code stored in the memory 504 to perform the method for intelligent recommendation based on geographic location information in the above embodiments.

[0297] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0298] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0299] Computer-readable media include both permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, DVD or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0300] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0301] Furthermore, it is clear that the word "comprising" does not exclude other units or steps. Multiple units, modules, or devices recited in the apparatus claims may also be implemented by a single unit or overall apparatus via software or hardware.

[0302] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which includes one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutively marked blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or the overall flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0303] In this embodiment, the processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0304] Memory can be used to store computer programs and / or modules. The processor implements various functions of the device / terminal equipment by running or executing the computer programs and / or modules stored in the memory, and by accessing data stored in the memory. Memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc.; the data storage area can store data created based on the use of the mobile phone (such as audio data, phonebook, etc.). In addition, memory can include high-speed random access memory, and can also include non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart media cards (SMC), secure digital cards (SD cards), flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0305] In this embodiment, if the modules / units integrated into the device / terminal equipment are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. Although this application discloses preferred embodiments as described above, it is not intended to limit this application. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of this application. Therefore, the scope of protection of this application should be determined by the scope defined in the claims of this application.

[0306] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0307] Furthermore, it is clear that the word "comprising" does not exclude other units or steps. Multiple units, modules, or devices recited in the apparatus claims may also be implemented by a single unit or overall apparatus via software or hardware.

[0308] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. A method for mining complex behavior patterns of communication users based on a heterogeneous graph neural network, characterized in that, The method for mining complex behavior patterns of communication users based on heterogeneous graph neural networks includes: Obtain real-time user data; Obtain a trained heterogeneous graph neural network model and a trained downstream behavior pattern recognition model; The preset heterogeneous graph is updated using real-time user data to obtain a local heterogeneous graph subgraph with updated attributes. The updated local subgraph is input into a trained heterogeneous graph neural network model to obtain the updated user embedding vector. The updated user embedding vector is input into the trained downstream behavior pattern recognition model to obtain the recognition result. 2.The communication user complex behavior pattern mining method based on the heterogeneous graph neural network of claim 1, wherein, The heterogeneous graph neural network model and the downstream behavior pattern recognition model are trained in the following manner: Obtain user attribute datasets, call log datasets, product order datasets, and application usage datasets; A heterogeneous graph is generated based on user attribute datasets, call record datasets, product order datasets, and application usage datasets; the heterogeneous graph includes user nodes, product nodes, and application nodes, and the edges include call edges, order edges, click edges, search edges, and usage edges; Acquire heterogeneous graph neural networks and downstream multi-task learning networks; Iterative training is performed on the heterogeneous graph neural network and the downstream multi-task learning network: user embedding vectors that fuse multi-hop high-order relation information are generated based on the acquired heterogeneous graph; By fusing user embedding vectors with multi-hop higher-order relation information, the heterogeneous graph neural network and the downstream multi-task learning network are jointly trained to obtain a trained heterogeneous graph neural network model and a trained downstream behavior pattern recognition model. 3.The communication user complex behavior pattern mining method based on the heterogeneous graph neural network of claim 2, wherein, The user attribute dataset includes multiple user attribute data, and each user attribute data includes a unique user identifier, basic user information, and basic product information. The call record dataset includes multiple call record data, and each call record data includes a unique call record identifier and basic call information; The product order dataset includes multiple product order data, and each product order data includes a unique product order identifier, a unique product identifier, and basic order information. The application usage dataset includes multiple application usage data, and each application usage data includes an application unique identifier, an application unique identifier, and application usage data.

4. The method for mining complex behavior patterns of communication users based on heterogeneous graph neural networks as described in claim 3, characterized in that, The process of generating a heterogeneous graph based on user attribute datasets, call log datasets, product order datasets, and application usage datasets includes: Extract the unique user identifier from the user attribute data and assign a globally unique user node ID to each unique identifier; associate the basic user information and basic product information in the user attribute data with the corresponding user node; Extract the unique product order identifier from the product order data, assign a globally unique product node ID to each product order identifier, and associate the basic order information with the corresponding product node. Extract the unique application usage identifier from the application usage data, assign a globally unique application node ID to each application usage identifier, and associate the application usage data with the corresponding application node. Map the unique identifier of the caller in the call record to the corresponding user node ID, and map the unique identifier of the called party in the call record to the corresponding user node ID. Use these two user nodes as the two endpoints of the call edge, and assign the edge attributes of the call edge through the basic call information. Map the user's unique identifier to the user node ID, and the product's unique identifier to the product node ID. Use these two nodes as the two endpoints of the ordering edge, and assign the edge attributes of the ordering edge based on the basic ordering information. Map the user's unique identifier to the user node ID, and the application's unique identifier to the application node ID. Use these two nodes as the two endpoints of the click edge, and assign the edge attributes of the click edge through the application's usage data. Map the user's unique identifier to the user node ID, and the application uses the unique identifier to map to the application node ID. Use these two nodes as the two endpoints of the search edge, and assign the edge attributes of the search edge through the application's data. The user's unique identifier is mapped to the user node ID, and the application's unique identifier is mapped to the application node ID. These two nodes are used as the two endpoints of the edge, and the edge attributes of the edge are assigned through the application's usage data.

5. The method for mining complex behavior patterns of communication users based on heterogeneous graph neural networks as described in claim 4, characterized in that, The step of generating a user embedding vector that fuses multi-hop higher-order relationship information based on the acquired heterogeneous graph includes: Feature extraction is performed on the heterogeneous graph to obtain initial feature vectors of nodes with uniform dimensions; User embedding vectors that integrate multi-hop higher-order relationship information are generated based on the initial feature vectors of nodes with a unified dimension and heterogeneous graphs.

6. The method for mining complex behavior patterns of communication users based on heterogeneous graph neural networks as described in claim 5, characterized in that, The process of generating a user embedding vector that fuses multi-hop higher-order relationship information based on the initial feature vector of nodes of a unified dimension and the heterogeneous graph includes: Hierarchical neighbor sampling is performed on the heterogeneous graph to obtain each training subgraph; Encode edge temporal information for each training subgraph to obtain a batch of training subgraphs with temporal features; Layer-by-layer message passing and neighborhood feature aggregation are performed on batch training subgraphs containing time features to obtain the aggregation vector of each target user node; Generate a user embedding vector that incorporates multi-hop higher-order relationship information based on the aggregation vector of each target user node.

7. The method for mining complex behavior patterns of communication users based on heterogeneous graph neural networks as described in claim 6, characterized in that, The step of performing hierarchical neighbor sampling on the heterogeneous graph to obtain each training subgraph includes: Traverse each user node, and based on the edge association relationship of the heterogeneous graph, traverse its neighbor nodes from 1 hop to k hops respectively, divide its neighbor set according to edge type, and thus obtain the three-dimensional mapping table of hop number-edge type-neighbor node for each user node. Generate a four-dimensional mapping table of hop count, edge type, neighbor node, and sampling probability for each user node based on the three-dimensional mapping table of hop count, edge type, neighbor node, and sampling probability for each user node. Hierarchical sampling is performed on each four-dimensional mapping table of hop count-edge type-neighbor node-sampling probability to obtain the set of sampled neighbors from 1 hop to k hops for each user node; Each user node generates an independent training subgraph based on its 1-hop to k-hop sampled neighbor set. Each independent training subgraph includes a node table and an edge table. The edge table fields include source node ID, target node ID, edge type, edge attribute dictionary, and timestamp field.

8. The method for mining complex behavior patterns of communication users based on heterogeneous graph neural networks as described in claim 7, characterized in that, The step of encoding edge temporal information for each training subgraph to obtain a batch of training subgraphs containing temporal features includes: Each training subgraph is processed as follows: Each edge table is timestamped to obtain a standardized edge table; Generate an edge-time feature vector mapping table for each standardized edge table; The edge-time feature vector mapping table, the node table, and the initial feature vectors of nodes of the same dimension are fused together to obtain training subgraphs with time features. The training subgraphs with time features are then combined to form a batch training subgraph with time features.

9. The method for mining complex behavior patterns of communication users based on heterogeneous graph neural networks as described in claim 8, characterized in that, The step of performing layer-by-layer message passing and neighborhood feature aggregation on the batch training subgraphs containing time features to obtain the aggregation vector for each target user node includes: Generate input features based on batch training subgraphs containing time features; Perform the following iterative operation until the iteration condition is met: Based on the edge table of the batch training subgraph with time features, the neighbor subsets of the current layer of the user node are divided according to edge type, the input features are associated, and the features to be aggregated in each subset are determined, thus obtaining the correspondence table of edge type-neighbor subset-features to be aggregated. Based on the edge type-neighbor subset-feature to be aggregated correspondence table, the linear projection matrix corresponding to different node types is called to map the features to be aggregated of each subset to a unified dimension feature space, thereby obtaining the node type normalized feature vector of each neighbor node. Based on the node type normalized feature vectors of each neighbor node, the relationship transformation matrix and attention parameters corresponding to different edge types are called to calculate the attention weight of each neighbor node. After softmax normalization, the normalized attention coefficient of each neighbor node is obtained. Based on the normalized attention coefficients of each neighbor node, the node type normalized feature vectors of the corresponding neighbor nodes are weighted and aggregated. After cross-edge type fusion, the aggregated vector of the target user node is obtained by connecting it with the residual through a nonlinear activation function. Based on the l-th layer aggregate representation of the target user node, update the layer-node-input feature mapping table to obtain the input features of the next layer; The iteration condition is: The iteration stops when l reaches the upper limit of the number of hops k or the cosine similarity between two adjacent aggregate representations is ≥0.

99. The aggregate representations of each user node from level 1 to level k are output, thus obtaining the set of aggregate vectors of the target user node.

10. A system for mining complex behavioral patterns of communication users based on heterogeneous graph neural networks, characterized in that, The communication user complex behavior pattern mining system based on heterogeneous graph neural networks includes: User real-time data acquisition module, the user real-time data acquisition module is used to acquire user real-time data; The model acquisition module is used to acquire a trained heterogeneous graph neural network model and a trained downstream behavior pattern recognition model. A local heterogeneous graph subgraph update module is used to update a preset heterogeneous graph using real-time user data, thereby obtaining a local heterogeneous graph subgraph with updated attributes. The updated user embedding vector generation module is used to input the local subgraph with updated attributes into a trained heterogeneous graph neural network model to obtain the updated user embedding vector. The recognition result acquisition module is used to input the updated user embedding vector into the trained downstream behavior pattern recognition model to obtain the recognition result.

Images