An influence maximization method combining large language model and deep reinforcement learning

Abstract

The application discloses a method for influence maximization combining a large language model and deep reinforcement learning in the technical fields of social network influence maximization, deep reinforcement learning, large language model assisted training, experience pool management and interpretable decision making, and contains the following steps: S1, influence maximization problem modeling; S2, deep reinforcement learning environment design; S3, LLM candidate action space construction; S4, LLM guided epsilon-greedy exploration; S5, LLM assisted high-quality experience enhancement; S6, LLM assisted experience pool management; S7, parameter update based on Double DQN; S8, training stability protection and final explanation generation. By introducing the LLM, the reinforcement learning training process can be verified while keeping the action execution and reward calculation, and the structured prior, training sample quality control and interpretable output capability can be obtained.

Description

Technical Field

[0001] This invention relates to a method for maximizing influence by combining large language models and deep reinforcement learning in the fields of maximizing influence in social networks, deep reinforcement learning, large language model-assisted training, experience pool governance, and interpretable decision-making. Background Technology

[0002] Influence Maximization (IM) is a fundamental problem in complex network analysis. Its goal is to maximize the spread of influence across the network, given a fixed network size and a limited number of seed nodes. Traditional IM methods typically rely on greedy search, heuristic centrality metrics, or extensive Monte Carlo simulations. These methods perform well in small-scale networks, but as the network size increases, the number of candidate nodes, the time required for Monte Carlo simulations, and the combinatorial search space all increase rapidly, leading to a significant rise in computational costs.

[0003] The "offline training, online decision-making" characteristics of Deep Reinforcement Learning (DRL) provide a feasible direction for alleviating the efficiency bottleneck of traditional influence maximization algorithms. The existing standard framework for DRL-based IM methods can be described as follows: First, a Deep Q-Network (DQN) is used to generate the Q-values ​​of each node in the current state, and the node with the highest Q-value is selected from the non-seed nodes. Next, the marginal gain reward brought by this node is estimated based on Monte Carlo simulation. Then, the current state, action, reward, next-time state, and termination flag are combined into a quintuple of empirical data for iterative optimization of node representation and Q-value estimation. Finally, a complete seed set is obtained through k consecutive rounds of seed selection.

[0004] While DRL-based IM methods have garnered significant attention, existing approaches still exhibit considerable shortcomings in practical applications. First, using the entire graph network as a candidate set necessitates evaluating the Q-values ​​of all nodes at each step, leading to slow training and hindering the full exploration of the propagation potential of different node combinations. Second, the epsilon-greedy action selection strategy relies on random exploration in the early stages of training, randomly selecting seed nodes with epsilon probability, resulting in decreased training sample quality and reduced exploration efficiency. Furthermore, the priority experience replay mechanism manages the experience pool, primarily adjusting the sampling probability of training samples based on TD error. However, TD error is not equivalent to the actual propagation value of experience samples and cannot remove low-quality or duplicate experiences, nor can it expand high-quality experiences, thus limiting the quality of training data. Finally, DRL models lack user-explanatory text; this black-box output limits the auditability and credibility of the method in real-world deployments.

[0005] Therefore, how to introduce the task understanding, policy induction, and natural language interpretation capabilities of Large Language Models (LLMs) into the deep reinforcement learning framework, and systematically enhance the candidate action space, exploration process, experience pool quality, and final decision interpretation, remains a key problem that urgently needs to be solved. Summary of the Invention

[0006] The purpose of this invention is to provide an influence maximization method that combines large language models and deep reinforcement learning, which solves the problems of excessive action space, lack of purpose in random exploration, lack of quality governance in priority experience replay, and lack of interpretability in the final decision in existing DRL-based IM methods.

[0007] To achieve the above objectives, this invention provides a method for maximizing influence by combining large language models with deep reinforcement learning, comprising the following steps:

[0008] S1. Modeling the influence maximization problem;

[0009] S2, Deep Reinforcement Learning Environment Design;

[0010] S3 and LLM candidate action space construction;

[0011] S4, LLM-guided epsilon-greedy exploration;

[0012] S5 and LLM-assisted high-quality experience enhancement;

[0013] S6 and LLM-assisted experience pool management;

[0014] S7. Parameter update based on Double DQN;

[0015] S8, Training Stability Protection and Final Interpretation Generation.

[0016] As a further improvement to the present invention, S1 is specifically described below.

[0017] Given a network ,in For a set of nodes, Let be the set of edges, and the size of the seed set be... In the node set Select seed set And satisfy The goal is to maximize the expected number of activated nodes under the propagation model; its optimization objective can be expressed as follows:

[0018] ;

[0019] in, Represents the seed set The expected scope of influence;

[0020] The propagation environment adopts a weighted independent concatenation model, i.e., nodes Activate unactivated neighbors The probability can be expressed as ,in For nodes The degree; because each neighbor activates a node independently. ,so exist The probability of being activated at any given time is ,in For nodes exist The number of active neighbors at any given time.

[0021] As a further improvement to the present invention, S2 is specifically described below.

[0022] Let the first The set of seeds selected at the given time is Environmental conditions The set of legal actions consists of node structural features and seed tags. Policy Network Used to estimate in state Select Node As the long-term reward of the next seed node; the action selection in the non-random selection branch is:

[0023] ;

[0024] When node After being added to the seed set, the additional marginal reward is estimated based on Monte Carlo simulation. The reward function can be expressed as:

[0025] ;

[0026] This yields the empirical quintuple. ,in Indicates whether the current seed set has reached its size. .

[0027] As a further improvement to the present invention, S3 is specifically described below.

[0028] To avoid random exploration and direct sampling of the entire network action space, it is necessary to construct a network context. Its structural information includes the number of nodes, the number of edges, the average degree, the number of cores, the number of communities, the community size, the number of nodes with height, the nodes connecting across communities, and the representative nodes of each community;

[0029] For each candidate node Construct candidate summaries This includes structural centrality, community ID, cross-community connectivity ratio, community coverage effect, and local structure score;

[0030] LLM receiving Then, output the candidate action space. , means as follows:

[0031] ;

[0032] in, For early exploration pool, For bridging node pool, For a diverse node pool; through the candidate action space The validity check only retains nodes that belong to the current graph node set and are not duplicated.

[0033] As a further improvement to the present invention, S4 is specifically described below.

[0034] Let the total number of training rounds be... The current training round number is The initial guidance probability is The ratio of guided activity is The LLM candidate sampling probability can then be expressed as:

[0035] ;

[0036] in, , When selecting actions, the epsilon-greedy framework is still retained, that is, based on... The Q-value selection in the probability execution formula (2) is used; The probability of entering the exploration branch;

[0037] In the exploration branch, then... The probability from Select an action from the set of legal actions; otherwise, select from the set of legal actions. Randomly select actions;

[0038] In addition, when Prioritize Nodes with high structural centrality, when Prioritize and There are nodes that spread across communities; when Prioritize coverage of uncovered communities. .

[0039] As a further improvement to the present invention, the specific content of S5 is as follows.

[0040] Let the current seed quantity stage be... The experience reward statistics for this stage are as follows: Rewards obtained from random exploration actions Exceeding the preset high reward threshold for this stage This triggers experience enhancement, and the enhancement request is constructed as follows:

[0041] ;

[0042] in, Estimate the current Q-value of the candidate node. This is a set of candidate node structure summaries. Large language models output recommended action sets. And for any Calculate its marginal reward Only when Extended experience will only be granted when a node is a legitimate unselected node and receives a valid reward after recalculation based on the propagation environment. Write it into the experience pool.

[0043] As a further improvement to the present invention, the specific content of S6 is as follows.

[0044] Construct candidate experience summaries based on the experience pool This includes experience index, actions, rewards, seed set before executing an action, termination flag, current priority, stage, and quality score;

[0045] LLM is based on candidate empirical summaries Stage reward statistics Based on the best historical exploration results, output governance plans , can be represented as follows:

[0046] ;

[0047] in, For the set of sampling suggestions of the priority experience index, This is a set of suggestions for adjusting the priority ratio. A set of deletion suggestions for low-quality experience indexes. For governance reasons;

[0048] For priority adjustments, the multiplier is limited to a preset range and a smooth update is used; for deletion suggestions, hard deletion is only performed when safety conditions are met, otherwise it is used as a reference for soft deletion.

[0049] As a further improvement to the present invention, the specific contents of S7 are as follows.

[0050] when During training, a portion of the empirical data samples come from the priority sampling set recommended by LLM, while the remaining samples are still drawn according to the priority empirical replay probability; the objective value of Double DQN can be expressed as:

[0051] ;

[0052] in, For policy network parameters, For the target network parameters, The discount factor is used; the model training loss employs Huber loss with importance weights.

[0053] ;

[0054] in, For training batches, Sampling weights are assigned to prioritize experience replays based on their importance. For training samples The target value of Double DQN; after training, the system is based on the TD error. Prioritize updating experience.

[0055] As a further improvement to the present invention, the specific content of S8 is as follows.

[0056] During training, the target network is periodically synchronized. If multiple consecutive synchronized LLM-guided actions cause the loss to increase, the LLM candidate action space is disabled. If the current policy evaluation result is significantly lower than the historical best policy, the policy is rolled back to the historical best policy and the learning rate is reduced.

[0057] After training, the optimal policy network is used to continuously select according to formula (2). From seed nodes, we obtain the final seed set. ;

[0058] Next, construct the final evidence. This includes seed node order, structural features, community coverage, marginal reward, neighborhood overlap, experience pool stage statistics, and training policy evolution records, enabling LLM to generate explanatory text. This is used to explain why each seed node is selected, the coverage logic of the overall seed set, and the impact of experience pool governance on the training results.

[0059] Compared with existing technologies, the advantages of this invention lie in that, by introducing LLM, the reinforcement learning training process achieves structured priors, training sample quality control, and interpretable output capabilities while maintaining verifiable action execution and reward calculation. Specifically, this invention constructs a candidate action space using LLM. This invention enables the model to prioritize candidate nodes with structural significance during the exploration phase, thereby reducing the proportion of invalid actions and improving the targeting of seed combination search. When random exploration actions yield relatively high rewards at a given stage, LLM recommends candidate actions with complementary propagation value. Furthermore, this invention uses LLM-assisted experience pool governance to ensure that priority experience replay no longer relies solely on TD error, and improves the effectiveness of training sample selection. Finally, this invention generates explanatory text while outputting the final set of seed nodes to explain the structural role, community coverage value, marginal reward contribution, and training strategy evolution process of the seed nodes.

[0060] Therefore, this invention establishes a collaborative optimization mechanism in key aspects such as candidate action space construction, random exploration guidance, high-quality experience enhancement, experience pool management, and final result interpretation. This mechanism allows LLM and DRL to complement each other, ensuring both the verifiability and stability of the training process while fully leveraging the knowledge reasoning and structural analysis capabilities of LLM. Experimental verification on multiple typical network datasets demonstrates that the method of this invention outperforms existing methods in terms of influence propagation effect, training efficiency, convergence performance, and result interpretability. It provides a solution for maximizing influence in large-scale social networks that combines accuracy, interpretability, and engineering practicality. Attached Figure Description

[0061] Figure 1 This is a flowchart illustrating the framework of the present invention.

[0062] Figure 2 This is a schematic diagram illustrating the construction of network context summaries in this invention.

[0063] Figure 3 This is a schematic diagram of the LLM-guided candidate action space construction in this invention.

[0064] Figure 4 This is a schematic diagram of LLM-assisted high-quality experience enhancement in this invention.

[0065] Figure 5 This is a schematic diagram of LLM-assisted experience pool management in this invention.

[0066] Figure 6 This is a schematic diagram illustrating the generation of the final seed set using LLM in this invention.

[0067] Figure 7 This is a performance comparison chart between the method of the present invention and the IM method based on DRL (DRL-IM). Detailed Implementation

[0068] The present invention will be further described below with reference to the accompanying drawings:

[0069] like Figure 1-7The method shown combines large language models with deep reinforcement learning to maximize influence, and includes the following steps:

[0070] S1. Modeling the problem of maximizing influence.

[0071] Given a network ,in For a set of nodes, Let be the set of edges, and the size of the seed set be... In the node set Select seed set And satisfy The goal is to maximize the expected number of activated nodes under the propagation model; its optimization objective can be expressed as follows:

[0072] ;

[0073] in, Represents the seed set The expected scope of influence;

[0074] The propagation environment adopts a weighted independent concatenation model, i.e., nodes Activate unactivated neighbors The probability can be expressed as ,in For nodes The degree; because each neighbor activates a node independently. ,so exist The probability of being activated at any given time is ,in For nodes exist The number of active neighbors at any given time.

[0075] S2, Deep Reinforcement Learning Environment Design.

[0076] Let the first The set of seeds selected at each time point is Environmental conditions The set of legal actions consists of node structural features and seed tags. Policy Network Used to estimate in state Select Node As the long-term reward of the next seed node; the action selection in the non-random selection branch is:

[0077] ;

[0078] When node After being added to the seed set, the additional marginal reward is estimated based on Monte Carlo simulation. The reward function can be expressed as:

[0079] ;

[0080] This yields the empirical quintuple. ,in Indicates whether the current seed set has reached its size. .

[0081] S3 and LLM candidate action space construction.

[0082] To avoid random exploration and direct sampling of the entire network action space, it is necessary to construct a network context. Its structural information includes the number of nodes, the number of edges, the average degree, the number of cores, the number of communities, the community size, the number of nodes with height, the nodes connecting across communities, and the representative nodes of each community;

[0083] For each candidate node Construct candidate summaries This includes structural centrality, community ID, cross-community connectivity ratio, community coverage effect, and local structure score;

[0084] LLM receiving Then, output the candidate action space. , means as follows:

[0085] ;

[0086] in, For early exploration pool, For bridging node pool, For a diverse node pool; through the candidate action space The validity check only retains nodes that belong to the current graph node set and are not duplicated.

[0087] S4, LLM-guided epsilon-greedy exploration.

[0088] Let the total number of training rounds be... The current training round number is The initial guidance probability is The ratio of guided activity is The LLM candidate sampling probability can then be expressed as:

[0089] ;

[0090] in, , When selecting actions, the epsilon-greedy framework is still retained, that is, based on... The Q-value selection in the probability execution formula (2) is used; The probability of entering the exploration branch;

[0091] In the exploration branch, then... The probability from Select an action from the set of legal actions; otherwise, select from the set of legal actions. Randomly select actions;

[0092] In addition, when Prioritize Nodes with high structural centrality, when Prioritize and There are nodes that spread across communities; when Prioritize coverage of uncovered communities. .

[0093] S5 and LLM-assisted high-quality experience enhancement.

[0094] Let the current seed quantity stage be... The experience reward statistics for this stage are as follows: Rewards obtained from random exploration actions Exceeding the preset high reward threshold for this stage This triggers experience enhancement, and the enhancement request is constructed as follows:

[0095] ;

[0096] in, Estimate the current Q-value of the candidate node. This is a set of candidate node structure summaries. Large language models output recommended action sets. And for any Calculate its marginal reward Only when Extended experience will only be granted when a node is a legitimate unselected node and receives a valid reward after recalculation in the propagation environment. Write it into the experience pool.

[0097] S6 and LLM-assisted experience pool management.

[0098] Construct candidate experience summaries based on the experience pool This includes experience index, actions, rewards, seed set before executing an action, termination flag, current priority, stage, and quality score;

[0099] LLM is based on candidate empirical summaries Stage reward statistics Based on the best historical exploration results, export governance plans. , can be represented as follows:

[0100] ;

[0101] in, For the set of sampling suggestions of the priority experience index, This is a set of suggestions for adjusting the priority ratio. A set of deletion suggestions for low-quality experience indexes. For governance reasons;

[0102] For priority adjustments, the multiplier is limited to a preset range and a smooth update is used; for deletion suggestions, hard deletion is only performed when safety conditions are met, otherwise it is used as a reference for soft deletion.

[0103] S7. Parameter update based on Double DQN.

[0104] when During training, a portion of the empirical data samples come from the priority sampling set recommended by LLM, while the remaining samples are still drawn according to the priority empirical replay probability; the objective value of Double DQN can be expressed as:

[0105] ;

[0106] in, For policy network parameters, For the target network parameters, The discount factor is used; the model training loss employs Huber loss with importance weights.

[0107] ;

[0108] in, For training batches, Prioritize the sampling weights for the importance of the experience replay. For training samples The target value of Double DQN; after training, the system is based on the TD error. Prioritize updating experience.

[0109] S8, Training Stability Protection and Final Interpretation Generation.

[0110] During training, the target network is periodically synchronized. If multiple consecutive synchronized LLM-guided actions cause the loss to increase, the LLM candidate action space is disabled. If the current policy evaluation result is significantly lower than the historical best policy, the policy is rolled back to the historical best policy and the learning rate is reduced.

[0111] After training, the optimal policy network is used to continuously select according to formula (2). From seed nodes, we obtain the final seed set. ;

[0112] Next, construct the final evidence. This includes seed node order, structural features, community coverage, marginal reward, neighborhood overlap, experience pool stage statistics, and training policy evolution records, enabling LLM to generate explanatory text. This is used to explain why each seed node is selected, the coverage logic of the overall seed set, and the impact of experience pool governance on the training results.

[0113] The purpose of this invention is to address the problems of excessively large action space, lack of purposefulness in random exploration, lack of quality governance in priority experience replay, and lack of interpretability in the final decision in existing DRL-based IM methods. This invention proposes an influence maximization method that combines a large language model with deep reinforcement learning. Specifically, the technical problems addressed by this invention include: first, solving the problem of low exploration efficiency and difficulty in fully evaluating candidate node combinations during DRL training due to the excessively large candidate action space across the entire graph; second, solving the problem of lack of task guidance and easy generation of low-value training experience in random exploration actions in the epsilon-greedy strategy; third, solving the problem that traditional priority experience replay mechanisms cannot actively expand high-quality experience, cannot govern low-quality experience, and over-rely on TD error in training sample selection; and fourth, solving the problem that the seed selection results lack explanatory textual descriptions, making it difficult to review, verify, and deploy the method during application.

[0114] The core of this invention lies in proposing an influence maximization method that combines large language models and deep reinforcement learning. This method uses a DRL model as the basic decision-maker for seed node selection and an LLM model as the policy advisor and result interpreter for model training, providing specialized auxiliary information for four key stages: candidate set generation, random exploration action selection, experience pool management, and selection result interpretation. Therefore, this invention can improve the purposefulness of the exploration process, the quality of training experience, and the interpretability of the final result while maintaining the closed loop of reinforcement learning training.

[0115] First, in the candidate set generation stage, this invention constructs a network context summarization mechanism. By compressing network structural information, LLM can summarize the network structural context and generate an early exploration pool, a bridging node pool, and a diversity node pool, thereby forming a structurally meaningful candidate action space at the beginning of training.

[0116] Then, in the random exploration action selection phase, this invention proposes an LLM-guided epsilon-greedy exploration strategy. The guiding weight decreases with each training round, so that structural guidance is obtained in the early stage of training, and the strategy selection gradually reverts to Q-value-dominated in the later stage of training.

[0117] Secondly, in the experience pool management phase, this invention introduces an LLM-assisted experience enhancement and governance mechanism. When a high-reward action is obtained during random exploration, the LLM will recommend several candidate actions with complementary propagation value. Furthermore, after training reaches a preset interval, the LLM outputs a priority sampling set, priority adjustment suggestions, and low-quality experience processing suggestions, followed by legality checks, safety filtering, and priority smoothing updates to avoid training instability that may result from complete adoption.

[0118] Finally, in the result interpretation stage, this invention designs an evidence construction and interpretation generation mechanism. By constructing an evidence summary of the final seed node set, LLM can generate information such as the selection rationale and influence coverage strategy for each seed node, thereby enhancing the interpretability of the results.

[0119] Therefore, this invention forms an overall technical roadmap of "reinforcement learning training environment - LLM candidate action space - LLM guided exploration - LLM experience enhancement and governance - reinforcement learning policy update - LLM result interpretation". Compared with existing influence maximization methods that only rely on DQN and priority experience replay, this roadmap does not change the verifiability of action execution and reward calculation, but rather improves the verifiability of action execution and reward calculation through the interconnection of LLM-assisted action space compression, random exploration improvement, experience pool quality enhancement, and result interpretation generation.

[0120] To verify the effectiveness of this invention, comparative experiments were conducted on 12 typical network datasets. The number of seed nodes was set to 5 in the experiments. Figure 2 This demonstrates an example of building a network context summary for the polbooks dataset. Figure 3 This demonstrates an example of LLM guided candidate action space construction for the polbooks dataset. Figure 4 This demonstrates an example of LLM-assisted high-quality experience enhancement on the polbooks dataset. Figure 5 This demonstrates an example of LLM-assisted experience pool governance for the polbooks dataset. Figure 6 This demonstrates an example of how the LLM on the polbooks dataset generates explanations for the final seed set. Figure 7 The training phase reward curves of the two methods are shown. The DRL-IM curve represents the DRL-based IM method without LLM, and the LLM-DRL curve represents the method proposed in this invention.

[0121] Depend on Figure 3It is evident that after receiving the network context and a summary of the candidate node structure, LLM can output an early exploration pool, a bridging node pool, and a diversity node pool. This result means that the random exploration phase no longer directly faces all nodes, but instead prioritizes selecting actions from a set of nodes with high height, high core count, cross-community connectivity, or community supplementary value.

[0122] Depend on Figure 4 It can be seen that when the epsilon-greedy random exploration generates a high-reward action in a certain stage, the LLM can recommend multiple candidate nodes based on the current seed prefix and candidate action information, and give a brief reason. Then, the recommended action is verified in the propagation environment, and only the verified experience is written into the priority experience replay pool, thereby ensuring the reliability of experience enhancement.

[0123] Depend on Figure 5 As can be seen, this invention can extend experience pool management from simply relying on TD error to comprehensive quality governance. After training reaches a preset number of rounds, LLM can return a priority sampling index, suggestions for handling low-quality samples, suggestions for priority adjustment, and a strategy summary. Subsequently, a safety filter is performed on the result to protect high-reward experiences, stage-related high-value experiences, and experiences aligned with historical best exploration results.

[0124] Depend on Figure 6 As can be seen, LLM can output the reasons for the selection of each seed node, the final seed set coverage strategy, the evolution of the training strategy, the stage reward analysis, and the relationship with the optimal exploration seed set. Therefore, this invention can transform the originally black-box node selection results of reinforcement learning models into an auditable natural language evidence chain, thereby improving the understandability of the method in practical applications.

[0125] from Figure 7In terms of the final propagation range, the LLM-DRL method shows a significant advantage over the DRL-IM method on 11 datasets, with an average improvement of approximately 119.0%, demonstrating the strong applicability of this invention to large-scale networks or networks with obvious community structures. Furthermore, analysis of the training reward curves shows that the LLM-DRL method not only improves the quality of the final seed set but also enhances the efficiency of reward acquisition during training. In the first 100 rounds of training, the LLM-DRL method achieves a significant advantage on 10 datasets; in the last 100 rounds, the LLM-DRL method maintains its advantage on 11 datasets. It is worth noting that the improvement of this invention is limited on some small-scale or structurally simple datasets. For example, on the Lesmis dataset, the final propagation range of the DRL-IM method is 38.2, while that of the LLM-DRL method is 33.7. Therefore, the main advantages of this invention are concentrated in scenarios with large action spaces, complex network structures, and high costs of random exploration. Furthermore, LLM-guided exploration is particularly helpful in the early stages of training because when the model has not yet formed a stable Q-value estimate, the LLM candidate action space can provide a structured exploration direction, reducing low-value actions caused by completely random exploration.

[0126] The experimental results above demonstrate that the proposed method significantly improves the scope of influence propagation and substantially enhances reward acquisition efficiency in the early stages of training. Four types of LLM interactions—candidate action space, experience enhancement, experience pool governance, and final interpretation—demonstrate the effectiveness and advancement of the proposed method from four aspects: exploration direction, experience expansion, experience selection, and result interpretability. Compared to traditional DRL-IM methods that rely solely on DQN and priority experience replay, this invention introduces a reviewable large language model auxiliary mechanism while maintaining verifiable reward calculation and policy updates, making it more suitable for influence maximization tasks in complex network scenarios.

[0127] This invention is not limited to the above embodiments. Based on the technical solutions disclosed herein, those skilled in the art can make some substitutions and modifications to some of the technical features without creative effort, and all such substitutions and modifications are within the protection scope of this invention.

Claims

1. A method for maximizing influence by combining large language models and deep reinforcement learning, characterized in that: Includes the following steps, S1. Modeling the influence maximization problem; S2, Deep Reinforcement Learning Environment Design; S3 and LLM candidate action space construction; S4, LLM-guided epsilon-greedy exploration; S5 and LLM-assisted high-quality experience enhancement; S6 and LLM-assisted experience pool management; S7. Parameter update based on Double DQN; S8, Training Stability Protection and Final Interpretation Generation.

2. The influence maximization method combining large language models and deep reinforcement learning according to claim 1, characterized in that: The specific content of S1 is as follows: Given a network ,in For a set of nodes, Let be the set of edges, and the size of the seed set be... In the node set Select seed set And satisfy The goal is to maximize the expected number of activated nodes under the propagation model; its optimization objective can be expressed as follows: ； in, Represents the seed set The expected scope of influence; The propagation environment adopts a weighted independent concatenation model, i.e., nodes Activate unactivated neighbors The probability can be expressed as ,in For nodes The degree; because each neighbor activates a node independently. ,so exist The probability of being activated at any given time is ,in For nodes exist The number of active neighbors at any given time.

3. The method for maximizing influence by combining a large language model with deep reinforcement learning according to claim 2, characterized in that: The specific details of S2 are as follows. Let the first The set of seeds selected at each time point is Environmental conditions The set of legal actions consists of node structural features and seed tags. Policy Network Used to estimate in state Select Node As the long-term reward of the next seed node; the action selection in the non-random selection branch is: ； When node After being added to the seed set, the additional marginal reward is estimated based on Monte Carlo simulation. The reward function can be expressed as: ； This yields the empirical quintuple. ,in Indicates whether the current seed set has reached its size. .

4. The method for maximizing influence by combining a large language model with deep reinforcement learning as described in claim 3, characterized in that: The specific details of S3 are as follows. To avoid random exploration and direct sampling of the entire network action space, it is necessary to construct a network context. Its structural information includes the number of nodes, the number of edges, the average degree, the number of cores, the number of communities, the community size, the number of nodes with height, the nodes connecting across communities, and the representative nodes of each community; For each candidate node Construct candidate summaries This includes structural centrality, community ID, cross-community connectivity ratio, community coverage effect, and local structure score; LLM receiving Then, output the candidate action space. , means as follows: ； in, For early exploration pool, For bridging node pools, For a diverse node pool; through the candidate action space The validity check only retains nodes that belong to the current graph node set and are not duplicated.

5. The influence maximization method combining large language models and deep reinforcement learning according to claim 4, characterized in that: The specific details of S4 are as follows. Let the total number of training rounds be... The current training round number is The initial guidance probability is The ratio of guided activity is The LLM candidate sampling probability can then be expressed as: ； in, , When selecting actions, the epsilon-greedy framework is still retained, that is, based on... The Q-value selection in the probability execution formula (2) is used; The probability of entering the exploration branch; In the exploration branch, then... The probability from Select an action from the set of legal actions; otherwise, select from the set of legal actions. Randomly select actions; In addition, when Prioritize A node with high structural centrality, when Prioritize and There are nodes that spread across communities; when Prioritize coverage of uncovered communities. .

6. The influence maximization method combining large language models and deep reinforcement learning according to claim 5, characterized in that: The specific details of S5 are as follows. Let the current seed quantity stage be... The experience reward statistics for this stage are as follows: Rewards obtained from random exploration actions Exceeding the preset high reward threshold for this stage This triggers experience enhancement, and the enhancement request is constructed as follows: ； in, Estimate the current Q-value of the candidate node. This is a set of candidate node structure summaries. Large language models output recommended action sets. And for any Calculate its marginal reward Only when Extended experience will only be granted when a node is a legitimate unselected node and receives a valid reward after recalculation based on the propagation environment. Write it into the experience pool.

7. The influence maximization method combining large language models and deep reinforcement learning according to claim 6, characterized in that: The specific details of S6 are as follows. Construct candidate experience summaries based on the experience pool This includes experience index, actions, rewards, seed set before executing an action, termination flag, current priority, stage, and quality score; LLM is based on candidate empirical summaries Stage reward statistics Based on the best historical exploration results, export governance plans. , can be represented as follows: ； in, For the set of sampling suggestions for the priority experience index, This is a set of suggestions for adjusting the priority ratio. A set of deletion suggestions for low-quality experience indexes. For governance reasons; For priority adjustments, the multiplier is limited to a preset range and a smooth update is used; for deletion suggestions, hard deletion is only performed when safety conditions are met, otherwise it is used as a reference for soft deletion.

8. The method for maximizing influence by combining a large language model with deep reinforcement learning according to claim 7, characterized in that: The specific content of S7 is as follows. when During training, a portion of the empirical data samples come from the priority sampling set recommended by LLM, while the remaining samples are still drawn according to the priority empirical replay probability; the objective value of Double DQN can be expressed as: ； in, For policy network parameters, For the target network parameters, The discount factor is used; the model training loss employs Huber loss with importance weights. ； in, For training batches, Sampling weights are assigned to prioritize experience replays based on their importance. For training samples The target value of Double DQN; after training, the system is based on the TD error. Prioritize updating experience.

9. The method for maximizing influence by combining a large language model with deep reinforcement learning as described in claim 8, characterized in that: The specific details of S8 are as follows. During training, the target network is periodically synchronized. If multiple consecutive synchronized LLM-guided actions cause the loss to increase, the LLM candidate action space is disabled. If the current policy evaluation result is significantly lower than the historical best policy, the policy is rolled back to the historical best policy and the learning rate is reduced. After training, the optimal policy network is used to continuously select according to formula (2). From seed nodes, we obtain the final seed set. ; Next, construct the final evidence. This includes seed node order, structural features, community coverage, marginal reward, neighborhood overlap, experience pool stage statistics, and training policy evolution records, enabling LLM to generate explanatory text. This is used to explain why each seed node is selected, the coverage logic of the overall seed set, and the impact of experience pool governance on the training results.

Images