Theme sensing node identification method based on deep reinforcement learning

Abstract

The invention belongs to the technical field of artificial intelligence, and relates to a topic perception node identification method based on deep reinforcement learning, which comprises the following steps: constructing a directed social network graph, calculating topic perception similarity among nodes by adopting a multi-head attention mechanism, and constructing a reward prediction network; generating community groups, and executing a reverse reachable set in a community to preliminarily estimate the influence of nodes; performing theme grouping according to the theme preference matrix, and scoring the theme groups; constructing a deep Q network based on grouping relative strategy optimization, and carrying out model training by adopting an experience playback mechanism and a delay updating strategy; network parameters are updated through back propagation, and an Adam optimizer is adopted for parameter optimization; performing dynamic seed scoring; and calculating the propagation weight of each theme group and dynamically allocating seed budget, and selecting an optimal seed node set by adopting a greedy strategy. According to the method, influence overlapping is reduced, the calculation efficiency is improved, and the seed node set with the most influence under the specific semantic condition can be recognized more accurately.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a topic-aware node identification method based on deep reinforcement learning. BACKGROUND

[0002] In the analysis and processing of graph structure data, in order to maximize the propagation effect of information in the network graph, researchers have proposed various algorithms and techniques to identify the most influential seed node set. These methods usually rely on knowledge in the fields of graph theory, machine learning and optimization theory, aiming to filter out a seed node set that can activate the maximum number of nodes through efficient computing strategies.

[0003] According to a specific diffusion propagation rule, a seed node can activate its neighbor nodes, and at the next moment these newly activated nodes will continue this process as new seed nodes until the diffusion ends. This ability to activate other nodes is called influence. Early research mainly focused on sampling or simulation-based methods, such as using the independent cascade model to simulate the information diffusion process, which has been widely recognized and applied. Sampling-based methods such as the IMM algorithm are considered one of the best algorithms currently available, capable of achieving seed set filtering in a single topic in linear time. However, although this method can capture the basic characteristics of the graph, when faced with multiple topic conditions for information propagation, it can only achieve seed set filtering through repeated sampling simulation, resulting in a linear increase in time cost as the number of topics increases.

[0004] With the development of deep learning technology, some research has begun to explore how to apply deep learning to the information propagation problem of graph data structure. Methods based on inductive representation learning achieve efficient node embedding on large-scale graphs, effectively capturing the non-linear relationships between nodes and improving the prediction accuracy of the model. However, existing methods still face challenges when dealing with topic-aware information propagation, especially in terms of adaptability in considering influence overlap and dynamic environments. In addition, excessive reliance on topological structure information ignores the diversity of node text features, making it impossible to accurately reflect the sensitivity of nodes to specific content based on node interest.

[0005] Subsequent research combined deep reinforcement learning with graph data analysis and proposed a topic-aware influence maximization method based on deep reinforcement learning, which dynamically evaluates the long-term gains of node combinations by constructing a learning framework for agent and environment interaction, thereby improving the accuracy of seed node selection. However, most of these methods focus on specific application scenarios or specific types of graph structures, lacking universality and flexibility. Especially when there is rich semantic information in the network, existing node filtering strategies often fail to fully utilize the characteristics of graph structure for efficient retrieval and generation tasks specific to specific topic information.

[0006] At present, the existing still has the following shortcomings:

[0007] 1) The traditional strategy based on sampling or simulation can capture the basic characteristics of the graph and ensure the accuracy of node screening, but when facing multi-topic information propagation, it faces significant time efficiency problems.

[0008] 2) The method based on graph neural network performs well in processing complex graph structure, but relies too much on topological structure information and ignores the diversity of node text features, which cannot accurately reflect the sensitivity of specific content according to node interest.

[0009] 3) The method based on deep reinforcement learning has limitations in dealing with influence overlap and generalization under multiple topics, making it difficult to achieve rapid response and efficient propagation under the condition of topic change.

[0010] Therefore, for a given topic, how to select a set of propagation sources that can maximize the number of activated nodes under the condition of limited budget, i.e., the seed set, has become a problem to be solved in data analysis and processing under graph data structure. SUMMARY

[0011] According to the above technical problems, a topic-aware node identification method based on deep reinforcement learning is provided. The present application optimizes node selection by capturing the coupling between network structure and topic, reduces influence overlap and improves computational efficiency, so as to more accurately identify the most influential seed node set under specific semantic conditions (topic). This strategy not only applies to knowledge graph assisted large language model to perform question and answer operations, but also can enhance the node selection process in information retrieval and generation tasks in the retrieval augmented generation system RAG, so that the system can more effectively utilize the knowledge of related topics for accurate link prediction or content generation.

[0012] The technical means adopted by the present application are as follows:

[0013] A topic-aware node identification method based on deep reinforcement learning, comprising:

[0014] A directed social network graph is constructed, network topological features and user content features are fused through a double-channel embedding network, a multi-head attention mechanism is used to calculate the topic-aware similarity between nodes, and a reward prediction network is constructed;

[0015] Community detection is performed based on a community partitioning algorithm to generate community groups, and the influence of nodes is estimated by performing reverse reachable set estimation within the community;

[0016] Theme grouping is performed according to the theme preference matrix, the theme groups are scored, and nodes with high propagation potential are retained;

[0017] A deep Q network based on packet relative strategy optimization is constructed, a composite reward function is designed to balance intra-community cooperation and inter-community competition, an experience replay mechanism and a delayed update strategy are adopted for model training.

[0018] Network parameters are updated through back propagation, Adam optimizer is used for parameter optimization, ε-greedy strategy is set to balance exploration and utilization, and the model converges until the model converges.

[0019] According to the target theme distribution T, the propagation weight of each theme group is calculated and the node is dynamically scored as a seed;

[0020] The propagation weight of each theme group is calculated and the seed budget is dynamically allocated, the node theme relevance, community importance and global influence are comprehensively evaluated, and the greedy strategy is used to select the optimal seed node set.

[0021] Further, the directed social network graph G=(V,E) includes a node set V={v1,v2,...,v N} and an edge set The edge weight p u,v ∈[0,1] represents the propagation probability;

[0022] The double-channel embedding network includes a structure channel and a content channel, the structure channel generates a structure feature matrix M∈R N×d , and the content channel uses a Sentence Transformer model to generate a content feature matrix X∈R N×d ;

[0023] Calculate the adaptive adjacency matrix:

[0024]

[0025] Where A is the adjacency matrix and D is the degree matrix.

[0026] The multi-head attention mechanism learns different feature subspaces through independent attention heads:

[0027] α ij =softmax((W q M i ) T (W k M j ) / d 1 / 2

[0028] Where α ij represents the attention weight from the jth position to the ith position in the sequence, W q , W k ∈R d×dis the training parameter; d represents the vector dimension; the reward prediction network is constructed

[0029]

[0030] The composite loss function is represented as the influence difference L in , the proximity loss L prox and the overfitting loss L reg :

[0031]

[0032] Wherein, R pred represents the result predicted by the reward prediction network, R true represents the ground-truth value obtained based on Monte Carlo simulation, E represents expectation, and the composite loss function is:

[0033]

[0034] Further, the community division algorithm adopts the Leiden algorithm to divide the community to obtain the community where each node is located;

[0035] The community influence ratio r(u) of node u is calculated:

[0036]

[0037] Wherein, d * is the execution maximum depth of the reverse reachable set, R(u) represents the frequency of node u appearing after executing the local reverse reachable set in the community, v represents any node in the community, φ(u) represents the community where node u is located, and φ(v) represents the community where node v is located;

[0038] The community size is calculated and normalized to generate the weight:

[0039] w φ(u) ←|V φ(u) | / |V|, V c ={v|φ(v)=c}

[0040] Wherein, represents the local influence weight of node u, represents the number of all nodes in the community where u is located, V represents the number of nodes in the whole network, and V C represents the number of nodes in the community c, and c represents the index number of the community where node v is located;

[0041] The community score is generated:

[0042]

[0043] wherein, denotes the community score of node u, composed of an integer part and a decimal part , for preventing the decimal part from being zero.

[0044] Further, the topic grouping score is calculated, specifically comprising:

[0045] A topic distribution matrix T ∈ R N×m is established using a Gaussian distribution model, wherein T i,j denotes the preference degree of node v i to topic τ, and m is the total number of topics;

[0046] For each node u, top-θ topics are selected:

[0047]

[0048] wherein k denotes an index, k ∈ {1,...,|τ|}, and nodes with a degree of 0 and nodes with an out-degree less than δ are filtered out.

[0049] Further, the deep Q network based on grouping relative strategy optimization is constructed, specifically comprising:

[0050] The state S t is defined as a multi-hot encoding vector with |V| dimensions, and the initial value of S t is all zero, and when node u is selected as a seed, the S t of node u is set to 1 through action a t , and the corresponding reward function is represented as:

[0051] r t =σ(S t ∪{a t})-σ(S t )

[0052] wherein σ denotes the influence range of the seed set;

[0053] The reward prediction network and the community score of node u are combined with global influence to obtain a reward preset function:

[0054]

[0055] wherein β denotes a hyperparameter;

[0056] The composite reward function is represented as:

[0057]

[0058] wherein J denotes an experience replay pool;

[0059] updating the Q value in the deep Q network:

[0060]

[0061] wherein Q is an action value function, representing the value of performing action a in state S t t the action value obtained, a represents a discount factor, and γ represents a hyperparameter.

[0062] Further, the parameter optimization using the Adam optimizer specifically includes:

[0063] using the agent optimizer to perform E p training, following the use and exploration process of the seed node screening based on the ε-greedy strategy, using the Adam optimizer to perform parameter updating θ;

[0064] updating the ε-greedy strategy, gradually converging the model by attenuating each round, and selecting nodes with larger Q values.

[0065] Further, the dynamic seed score specifically includes:

[0066] According to the target theme distribution T, the propagation weight of each theme group is calculated, T τ represents the importance score of theme τ, and N out (v) represents the out-degree of node v;

[0067] The propagation weights of each theme group are normalized:

[0068]

[0069] wherein G τ represents the theme group corresponding to theme τ;

[0070] According to the normalized weight, the initial budget is allocated, and the remaining budget is allocated to the theme group with the highest weight:

[0071]

[0072] wherein k τ represents the weight allocated to theme τ, and K represents the budget of the seed node set.

[0073] The comprehensive score of each node is calculated:

[0074]

[0075] wherein γ1 is a theme relevance coefficient, γ2 is a normalized out-degree coefficient, γ3 is a standardized Q value coefficient, and γ4 is a community diversity coefficient.

[0076] ​Further, the optimal seed set is selected by using a greedy strategy, and specifically includes the following steps.

[0077] Initialize the seed set Community counter count = {0} m For each topic group G τ , rank the nodes in descending order according to the score f {score} ; select the nodes in turn to join S, and update count, when , skip the community node, and terminate when the distribution budget k is reached; if |S| < K, select the remaining nodes from the highest weight topic group; return the optimal seed node set. τ

[0078] Compared with the prior art, the present application has the following advantages:

[0079] The method for identifying topic-aware nodes based on deep reinforcement learning provided by the present application designs a topic-aware node identification method (GR-TIM) based on deep reinforcement learning. By fusing topic relevance and network structure features, the node screening efficiency under multi-topic conditions is enhanced, and the influence range is increased by 5% under the condition of constant budget. At the same time, the grouping relative strategy optimization mechanism can automatically adapt to different topic distributions and network structure changes. Compared with the traditional simulation-based method, the calculation efficiency is improved by 2-3 orders of magnitude, and it is suitable for real-time processing of large-scale social networks.

[0080] The method of the present application not only applies to knowledge graph assisted large language model execution question and answer operations, but also can enhance the node selection process in information retrieval and generation tasks in the retrieval augmented generation (RAG) system of the graph structure, so that the system can more effectively utilize the knowledge of related topics for accurate link prediction or content generation. In addition, the present application not only overcomes the limitations of traditional methods in node selection under graph data structure, but also greatly improves the efficiency of information dissemination and the diversity of application scenarios.

[0081] For the above reasons, the present application can be widely promoted in the field of artificial intelligence. BRIEF DESCRIPTION OF DRAWINGS

[0082] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0083] Figure 1 The flow chart of the method for identifying topic-aware nodes based on deep reinforcement learning in the present application.

[0084] ​Figure 2 The architecture diagram of the subject-aware node identification method (GR-TIM) in the embodiment of the present application.

[0085] Figure 3 The architecture diagram of the reward prediction network (RPN) in the embodiment of the present application.

[0086] Figure 4 The comparison diagram of the GR-TIM model in the embodiment of the present application and the prior art in the subject-aware node screening effect.

[0087] Figure 5 The comparison diagram of the GR-TIM model in the embodiment of the present application and the prior art in the subject-aware node screening running time effect. DETAILED DESCRIPTION

[0088] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0089] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings of the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. The description of the at least one exemplary embodiment is actually only illustrative, but not as any limitation on the present application and its application or use. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0090] It should be noted that the terms used herein are only for describing specific embodiments, and are not intended to limit the exemplary embodiments according to the present application. As used herein, the singular form is intended to include the plural form unless the context clearly indicates otherwise, and it should also be understood that when the terms "comprise" and / or "include" are used in the specification, there is a feature, step, operation, device, component and / or combination thereof.

[0091] The foregoing is considered as illustrative only of the principles of the application. Other variations and modifications will occur to those skilled in the art upon consideration of the foregoing description. Accordingly, it is intended that the application be construed as including all such variations and modifications as falling within the scope of the application. Unless otherwise indicated, the relative arrangement of components and steps, the numerical expressions and numerical values set forth in these embodiments are not limiting the scope of the application. It should be understood that all the technical and scientific terms used herein are not limited to the meaning in particular examples, but have meanings that are consistent with the skill in the art. Moreover, it should be understood that the size, shape and geometric relationships of the various parts shown in the drawings are not necessarily to scale. Techniques, methods, and apparatus known to those of ordinary skill in the art can not be discussed in detail, but should be considered as part of the specification, where appropriate. In all the examples shown and discussed herein, any specific value should be interpreted as merely exemplary, and not as a limitation. Thus, other examples of the exemplary embodiments can have different values. It should be noted that like reference numerals and letters refer to like items in the following drawings, and thus, once an item is defined in one drawing, it need not be discussed further in subsequent drawings.

[0092] As shown in Figure 1 , the application provides a topic-aware node identification method based on deep reinforcement learning, the architecture of the method is as shown in Figure 2 , comprising:

[0093] A directed social network graph is constructed, network topology features and user content features are fused through a double-channel embedding network, a multi-head attention mechanism is used to calculate topic-aware similarity between nodes, and a reward prediction network is constructed;

[0094] In specific implementation, as a preferred embodiment of the application, the directed social network graph G=(V,E) includes a node set V={v1,v2,...,v N} and an edge set Edge weight p u,v ∈[0,1] is used to represent the propagation probability;

[0095] The double-channel embedding network includes a structure channel and a content channel, the structure channel generates a structure feature matrix M∈R N×d through a Node2Vec algorithm, and the content channel uses a Sentence Transformer model to generate a content feature matrix X∈R N×d ;

[0096] An adaptive adjacency matrix is calculated:

[0097]

[0098] Wherein, A is an adjacency matrix, and D is a degree matrix.

[0099] Due to personal preferences, the user similarity between different topics varies greatly. To solve this problem, a multi-head attention mechanism is adopted to aggregate multi-topic information, enhance key feature representation, and reduce redundancy and GNN over-smoothing. The multi-head attention mechanism learns different feature subspaces through independent attention heads:

[0100] α ij =softmax((W q M i ) T (W k M j ) / d 1 / 2

[0101] wherein, α ij represents the attention weight from the jth position to the ith position in the sequence, W q , W k ∈R d×d are training parameters; d represents the vector dimension;

[0102] A reward prediction network is constructed, and the model architecture diagram is shown in Figure 3 In implementation, the reward prediction network is a three-layer graph convolution network, which is used to predict the global influence of nodes

[0103]

[0104] The composite loss function is represented as the sum of the influence difference L in , the proximity loss L prox and the overfitting loss L reg :

[0105]

[0106] wherein, R pred represents the result predicted by the reward prediction network, R true represents the ground-truth value obtained based on Monte Carlo simulation, E represents expectation, and the composite loss function is:

[0107]

[0108] Based on the community division algorithm, community detection is performed to generate community grouping, and the reverse reachable set is executed within the community to estimate the influence of nodes;

[0109] In implementation, as a preferred embodiment of the present application, the community division algorithm adopts the Leiden algorithm to divide the community to obtain the community where each node is located;

[0110] The community internal influence ratio r(u) of the node u is calculated:

[0111]

[0112] wherein, d * is the execution maximum depth of reverse reachable set, in implementation, 2 is taken, R(u) represents the frequency of node u appearing after executing local reverse reachable set in the community, v represents any node in the community, φ(u) represents the community where node u is located, and φ(v) represents the community where node v is located;

[0113] Calculate the community size and normalize the generated weight:

[0114] w φ(u) ←|V φ(u) | / |V|, V c ={v|φ(v)=c}

[0115] wherein, represents the local influence weight of node u, represents the number of all nodes in the community where u is located, V represents the number of nodes in the whole network, V C represents the number of nodes in the community c, and c represents the index number of the community where node v is located;

[0116] Generate community score:

[0117]

[0118] wherein, represents the community score of node u, which is composed of an integer part and a decimal part , for preventing the decimal part from being zero, and the default is 10 -6 .

[0119] According to the theme preference matrix, theme grouping is performed, theme grouping is scored, and high propagation potential nodes are reserved;

[0120] In implementation, as a preferred embodiment of the present application, the scoring of the theme grouping specifically includes:

[0121] A theme distribution matrix T ∈ R N×m is established by using a Gaussian distribution model, wherein T i,j represents the preference degree of node vi to theme τ, and m is the total number of themes, and the default is 20;

[0122] For each node u, top-θ themes are selected:

[0123]

[0124] Wherein, k represents an index, k belongs to {1,...,|τ|}, filter out the nodes with degree 0 and the nodes with out-degree less than delta, and delta is 3 by default.

[0125] A deep Q network based on grouping relative strategy optimization is constructed, a composite reward function is designed to balance intra-community cooperation and inter-community competition, an experience replay mechanism and a delayed update strategy are adopted for model training.

[0126] In specific implementation, as a preferred embodiment of the present application, the deep Q network based on grouping relative strategy optimization is constructed, and specifically includes:

[0127] Define state S t , which is a multi-hot encoding vector with |V| dimensions, S t The initial value is all zero, when selecting node u as a seed, through action a t , S t (u) of node u is set to 1; the corresponding reward function is represented as:

[0128] r t =σ(S t ∪{a t})-σ(S t )

[0129] Wherein, sigma represents the influence range of the seed set.

[0130] Using the reward prediction network and the community score of node u, combined with global influence, the reward preset function is obtained:

[0131]

[0132] Wherein, beta represents a hyperparameter, and the default is 0.5.

[0133] The composite reward function is represented as:

[0134]

[0135] Wherein, J represents an experience replay pool.

[0136] Update the Q value in the deep Q network:

[0137]

[0138] Wherein, Q is an action value function, which represents the action value obtained by performing action a t in state S t , alpha represents a discount factor, when the discount factor is close to 0, more attention is paid to timely rewards, and the default alpha is 0.1, and gamma represents a hyperparameter, and the default is 0.1.

[0139] The network parameters are updated by back propagation, the Adam optimizer is used for parameter optimization, an epsilon-greedy strategy is set to balance exploration and utilization, and the model converges until the model converges.

[0140] In specific implementation, as a preferred embodiment of the present application, the parameter optimization using the Adam optimizer specifically includes:

[0141] The Agent optimizer is used to execute E p The utilization and exploration process of seed node screening is followed in the secondary training, the initial epsilon is 0.9, which means that the Agent has a 0.9 probability of executing exploration, that is, randomly selecting nodes, and the Agent is encouraged to execute exploration in the early stage of training. The Adam optimizer is used to execute parameter update θ, and the epsilon-greedy strategy is updated. In each round, 90% is attenuated to gradually converge the model, and the node with a larger Q value is selected.

[0142] According to the target theme distribution T, the propagation weight of each theme group is calculated and the dynamic seed score of the node is calculated; in specific implementation, as a preferred embodiment of the present application, the dynamic seed score specifically includes:

[0143] According to the target theme distribution T, the propagation weight of each theme group is calculated, T τ The importance score of theme τ, N out (v) represents the out-degree of node v;

[0144] The propagation weight of each theme group is normalized:

[0145]

[0146] Wherein, G τ The theme group corresponding to theme τ; the initial budget is allocated according to the normalized weight, and the remaining budget is allocated to the theme group with the highest weight:

[0147]

[0148] Wherein, k τ Indicates the weight allocated to theme τ, K represents the budget of the seed node set;

[0149] The comprehensive score of each node is calculated:

[0150]

[0151] Wherein, γ1 is the theme correlation coefficient, γ2 is the normalized out-degree coefficient, γ3 is the normalized Q value coefficient, and γ4 represents the community diversity coefficient, all of which are 0.25 by default.

[0152] The propagation weight of each topic group is calculated and the seed budget is dynamically allocated. The node topic relevance, community importance and global influence are comprehensively evaluated, and a greedy strategy is used to select the optimal seed node set.

[0153] In specific implementation, as a preferred embodiment of the present invention, the greedy strategy is used to select the optimal seed set, specifically including:

[0154] Initialize the seed set Community counter count = {0} m , for each subject group G τ , by rating f {score} Arrange the nodes in descending order; select nodes to join S in turn and update count at the same time. When skipping the community node, the allocation budget k is reached τ terminates when |S| <K,则从最高权重主题组补选剩余节点;返回最优种子节点集合。

[0155] Example

[0156] In this example, we explicitly model the interaction patterns between users on social networks by introducing user historical behavior features, and demonstrate the propagation effects of the node sets selected by GR-TIM and the baseline method on the network graph through experiments on six real datasets (Eu-core, Reddit, Yelp, YouTube, Twitter, and Aminer). The results are shown in Figure 2. Figure 4 As shown in Figure 3, GR-TIM's impact range is generally better than that of the baseline method.

[0157] This embodiment also captures the intrinsic relationship between structure and topic, and directly selects nodes under multiple topics using pre-trained agents and grouping results, thereby reducing the computational complexity of simulation and sampling methods. The experimental results on six real datasets are as follows: Figure 5 As shown in the figure, CELF++ represents the improved cost-effective lazy forward algorithm, IMM represents the influence maximization based on the martingale method, and TIM+ represents the threshold influence maximization with improved approximation ratio. Figure 5 As can be seen from the figure, GR-TIM achieves an efficiency improvement of 2-3 orders of magnitude compared to simulation and sampling based methods.

[0158] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions recorded in the above embodiments can be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A topic-aware node recognition method based on deep reinforcement learning, characterized in that: include: Construct a directed social network graph, fuse network topology features and user content features through a dual-channel embedding network, use a multi-head attention mechanism to calculate the topic-aware similarity between nodes, and build a reward prediction network; Perform community detection based on the community partitioning algorithm to generate community groups, and perform reverse reachable set analysis within the community to preliminarily estimate the influence of nodes; Group topics according to the topic preference matrix, score the topic groups, and retain nodes with high dissemination potential; Build a deep Q-network based on group-based relative strategy optimization, design a composite reward function to balance intra-community collaboration and inter-community competition, and use an experience replay mechanism and delayed update strategy for model training; Update network parameters through backpropagation, use Adam optimizer for parameter optimization, set ε-greedy strategy to balance exploration and exploitation until the model converges; According to the target topic distribution T, the propagation weight of each topic group is calculated and the nodes are dynamically seeded. The propagation weight of each topic group is calculated and the seed budget is dynamically allocated. The node topic relevance, community importance and global influence are comprehensively evaluated, and a greedy strategy is used to select the optimal seed node set.

2. The topic-aware node identification method based on deep reinforcement learning according to claim 1 is characterized in that: The directed social network graph G=(V,E) includes a node set V={v1,v2,...,v N } and edge sets Using edge weight p u,v ∈[0,1] represents the propagation probability; The dual-channel embedding network includes a structure channel and a content channel. The structure channel generates a structure feature matrix M∈R through the Node2Vec algorithm. N×d , the content channel uses the Sentence Transformer model to generate the content feature matrix X∈R N×d ; Compute the adaptive adjacency matrix: Among them, A is the adjacency matrix and D is the degree matrix; The multi-head attention mechanism learns different feature subspaces through independent attention heads: α ij =softmax((W q M i ) T (W k M j ) / d 1 / 2 Among them, α ij represents the attention weight from the jth position to the ith position in the sequence, W q 、W k ∈R d×d is the training parameter; d represents the vector dimension; Build a reward prediction network: The composite loss function is expressed as the influence difference L in , approaching loss L prox and overfitting loss L reg The sum of the influence difference L in The calculation formula is: Among them, R pred Represents the result predicted by the reward prediction network, R true represents the ground-truth value obtained based on Monte Carlo simulation, E represents the expectation, and the composite loss function is:

3. The topic-aware node identification method based on deep reinforcement learning according to claim 1, characterized in that: The community division algorithm adopts the Leiden algorithm to divide the community and obtain the community where each node is located; Calculate the influence ratio r(u) of node u within the community: Among them, d * is the maximum execution depth of the reverse reachable set, R(u) represents the frequency of node u appearing after executing the local reverse reachable set in the community, v represents any node in the community, φ(u) represents the community where node u is located, and φ(v) represents the community where node v is located; Calculate the community size and normalize the generated weights: w φ (u)←|V φ(u) | / |V|,V c ={v|φ(v)=c} in, represents the local influence weight of node u, represents the number of all nodes in the community where u is located, V represents the number of nodes in the entire network, V C represents the number of nodes in community c, where c represents the index number of the community where node v is located; Generate community ratings: in, Represents the community score of node u, which consists of a positive integer part and the decimal part composition, Used to prevent decimal places from being zero.

4. The topic-aware node identification method based on deep reinforcement learning according to claim 1, characterized in that The scoring of the subject groups specifically includes: Use Gaussian distribution model to establish topic distribution matrix T∈R N×m , where T i,j Represents node v i The preference for topic τ, m is the total number of topics; For each node u, select the top-θ topics: Here, k represents the index, k∈{1,...,|τ|}, filtering out nodes with degree 0 and nodes with out-degree less than δ.

5. The topic-aware node identification method based on deep reinforcement learning according to claim 1, characterized in that The construction of a deep Q network based on group relative strategy optimization specifically includes: Define state S t is a |V|-dimensional multi-hot encoding vector, S t The initial value is all zero. When node u is selected as the seed, through action a t Set S of node u t (u) is set to 1; the corresponding reward function is expressed as: r t =σ(S t ∪{a t })-σ(S t ) Among them, σ represents the influence range of the seed set; Using the reward prediction network and the community score of node u, combined with the global influence, we get the reward preset function: Among them, β represents the hyperparameter; The composite reward function is expressed as: Among them, J represents the experience replay pool; Update the Q value in the deep Q network: Among them, Q is the action value function, which means that in state S t Next, perform action a t The obtained action value, α represents the discount factor, and γ represents the hyperparameter.

6. The topic-aware node identification method based on deep reinforcement learning according to claim 1, characterized in that: The Adam optimizer is used to perform parameter optimization, specifically including: Utilize the Agent optimizer to execute E p Second training, following the ε-greedy strategy to perform the utilization and exploration process of seed node screening, using the Adam optimizer to perform parameter update θ; Update the ε-greedy strategy, gradually converge the model with each round of attenuation, and select nodes with larger Q values.

7. The topic-aware node identification method based on deep reinforcement learning according to claim 1, characterized in that: The dynamic seed scoring specifically includes: According to the target topic distribution T, calculate the propagation weight of each topic group, T τ represents the importance score of topic τ, N out (v) represents the out-degree of node v; Normalize the propagation weight of each topic group: Among them, G τ represents the topic group corresponding to the topic τ; Allocate the initial budget according to the normalized weights and allocate the remaining budget to the most heavily weighted topic groups: Among them, k τ Represents the weight assigned to the topic τ, K represents the budget of the seed node set; a comprehensive score is given for each node: Among them, γ1 is the topic correlation coefficient, γ2 is the normalized out-degree coefficient, γ3 is the standardized Q value coefficient, and γ4 represents the community diversity coefficient.

8. The topic-aware node identification method based on deep reinforcement learning according to claim 1, characterized in that: The greedy strategy is used to select the optimal seed set, specifically including: Initialize the seed set Community counter count = {0} m , for each topic group G τ , sort the nodes in descending order according to the score f {score} ; sequentially select nodes to join S, and at the same time update count. When , skip the community nodes and terminate when the allocation budget k is reached τ ; if |S| < K, supplement the remaining nodes from the topic group with the highest weight; return the optimal seed node set.

Images