Session content key point extraction method based on reinforcement learning

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a reinforcement learning-based session content key point extraction method, which comprises the following steps of: preprocessing an input text sentence, constructing a composite feature vector, inputting a preprocessed session composite feature vector sequence into a Transform multi-head attention layer, and obtaining a key point of a session content; after exponential decay is applied to attention weights of historical sentences, emotional polarity and entity density are introduced as regulatory factors, attention weight calculation is optimized, and key statements are screened according to a threshold value and stored in a cache region; constructing a focus state vector and graph evolution, then establishing semantic edges and time sequence edges according to rules to form a graph, enhancing nodes through a graph convolutional network, and finally pooling active nodes to generate a vector as input of reinforcement learning; according to the method, through time decay and multi-head attention, time sequence association and semantic key points in dialogues are dynamically captured, and the timeliness of key information identification is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence, and particularly relates to a method for extracting key points from conversation content based on reinforcement learning. Background Art

[0002] Currently, the technical solutions for intelligent dialogue research based on online mediation corpus still have deficiencies in topic guiding ability and adaptability.

[0003] The technical solutions for intelligent dialogue research based on online mediation corpus still have deficiencies in topic guiding ability and adaptability. First, they cannot conduct targeted topic guidance according to the mediation goal. Second, they cannot provide targeted and adaptive answers and dialogues according to the feedback content of the parties. Summary of the Invention

[0004] In view of the technical problems existing in the prior art, the present invention provides a method for extracting key points from conversation content based on reinforcement learning.

[0005] The technical solution of the present invention to solve the above technical problems is as follows: A method for extracting key points from conversation content based on reinforcement learning, comprising the following steps: S101. Preprocess the input text sentence, identify the action intention of the verb phrase, and splice the semantic vector, entity label, sentiment score, and action phrase into a composite feature vector in multiple dimensions.

[0006] S102. Input the preprocessed dialogue composite feature vector sequence into the Transformer multi-head attention layer. After applying exponential decay to the attention weights of historical sentences, introduce sentiment polarity and entity density as adjustment factors to optimize the calculation of attention weights, and filter key statements according to a threshold and store them in the buffer area; S103. Construct a focus state vector and graph evolution. Set the named entities and action phrases of the key statements as graph nodes and endow them with attributes, and then establish semantic edges and temporal edges according to rules to form a graph. Enhance the nodes through a graph convolutional network, and finally pool the active nodes to generate a vector as the input of reinforcement learning; S104. When using reinforcement learning to generate guiding topics, first splice the focus state vector and sentiment statistic to construct a state space, design an action space containing guiding topics and corresponding templates, then set reward and punishment rules, train a policy network through a reinforcement learning algorithm, and select actions according to probabilities to call templates; S105. Based on the current state vector, adopt a greedy strategy to select the action with the largest Q value from the action space, perform repeated verification and logical verification on the candidate topics, call predefined templates according to the action type, and optimize the sentence fluency with a pre-trained Seq2Seq model to output a guiding topic that conforms to the mediation scenario; In a preferred embodiment, in S101, the input text T is first segmented into independent statements according to sentence boundaries by using regular expressions in combination with a syntactic analysis tool , where S represents the set of sentences after segmentation, denotes the function of using regular expressions to segment the text. After denoising, the specific calculation formula for denoising is as follows: ; where, represents the i-th sentence, denotes the text cleaning function for removing blank lines and noise, and a pre-trained BERT model is used to generate semantic vectors for each statement , where, represents the word segmentation result of the i-th sentence, denotes the word segmentation function to capture context semantic associations, and at the same time, key entities such as person names, times, locations, and amounts are annotated through named entity technology , where, represents the set of entity types, denotes the named entity function, and the sentiment polarity score of the statement is calculated by means of training a sentiment classification model. The specific calculation formula is as follows: ; where, denotes the sentiment classification model function, represents the sentiment polarity score of the i-th sentence, where, ( ) Then, verb phrases are extracted through part-of-speech tagging to identify action intentions. Finally, the semantic vectors, entity labels, sentiment scores, and action phrases are embedded into multi-dimensional features and concatenated into a composite feature vector. After completing the standardization process, it is stored as structured data including statement text, feature vectors, entity sentiment, and action phrases.

[0007] In a preferred embodiment, in S102, the sequence of composite feature vectors obtained by preprocessing the dialogue text is input into the Transformer multi-head attention layer, and a query matrix , a key matrix and a value matrix are obtained through linear transformation, where, represents, in the i-th attention head, the weight matrix used to map the composite feature vector to the query matrix, represents, in the i-th attention head, the weight matrix used to map the composite feature vector to the key matrix, represents, in the i-th attention head, the weight matrix used to map the composite feature vector The weight matrix mapped to the value matrix calculates the original attention score matrix of the current sentence and all historical sentences. The specific calculation formula for the single-head attention score is as follows: ; Among them, represents the original attention score between the current round t and the historical round i in the h-th attention head. t represents the current dialogue round, that is, the index of the current sentence when calculating attention. i represents the historical dialogue round, that is, the index of the historical sentence when calculating attention. h represents the attention head number, corresponding to the h-th subspace in the multi-head attention mechanism. represents the feature dimension of each attention head. represents the query vector of the current round t in the h-th attention head. represents the key vector of the historical round i in the h-th attention head. represents matrix transpose, which transposes the key vector into a column vector for matrix multiplication with the query vector, and applies exponential decay to the attention weights of historical sentences. The specific calculation formula is as follows: ; Among them, represents the exponential decay coefficient. represents the time decay rate. The specific calculation formula for the decayed attention score is as follows: ; Among them, represents the decayed attention score. represents the fused attention score between the current round t and the historical round i, highlighting the influence of recent conversations and introducing sentiment polarity and entity density . Among them, represents the number of entities. represents the maximum number of entities, which is used as a regulatory factor to optimize the calculation of attention weights. The specific calculation formula is as follows: ; Among them, represents the adjusted normalized attention weight, which is used to measure the importance of the historical round i to the current round t. represents the exponential function. , represents a hyperparameter that adjusts the influence weights of sentiment polarity and entity density. Key statements are screened according to the set attention threshold and stored in the dynamic buffer.

[0008] In a preferred embodiment, in S103, the named entities and action phrases in the key statements are used as graph nodes, and the nodes are assigned attributes such as frequency of occurrence, sentiment tendency, and first appearance timestamp. Subsequently, semantic edges and temporal edges are established according to edge connection rules. If two nodes appear in the same statement or in adjacent rounds and the semantic similarity is higher than a threshold, an undirected semantic edge with a weight of semantic similarity is established. At the same time, a directed temporal edge reflecting the logical sequence relationship is established according to the order of the first appearance of the nodes to form a focus evolution graph. The node neighborhood information is then aggregated through a graph convolutional network to enhance the node representation. Finally, the enhanced representation of all currently active nodes is mean pooled to generate a focus state vector that integrates the semantics, sentiment, and temporal relationship of the focus of controversy as one of the input features of reinforcement learning.

[0009] In a preferred embodiment, in S104, when using the reinforcement learning strategy to generate the guiding topic, the focus state vector is first concatenated with the emotion statistics of the current conversation to construct a state space that fully reflects the semantic focus and emotional state of the conversation, providing basic input for subsequent strategic decisions. Next, the action space is designed to divide the guiding topics into three categories: fact clarification, responsibility confirmation, and solution negotiation. For each type of topic, a preset natural language template is prepared. These templates can generate specific guiding content by filling in the relevant information of the current focus node. In the reward function design stage, an effective guiding topic is generated for the incentive strategy network, and positive and negative reward rules are set: if the generated topic is successfully guided to a new focus or promotes consensus, a + reward is given. The specific calculation formula for the reward is as follows: ; in, Represents the new focus guidance reward value, represents the reward coefficient, An indicator function indicating the emergence of a new focus, Indicates the indicator function of consensus. If the topic increases the emotional polarity of the conversation and reduces the negative emotion, the reward is + , the specific calculation formula is as follows: ; in, represents the reward value of sentiment reward optimization, represents the reward function, Indicates the change in sentiment polarity between the current round and the previous round (s ), Indicates the decrease in the proportion of negative emotions l( ), when the topic is repeated and fails to elicit effective responses from users, causing them to repeat existing statements, give - The negative reward is calculated as follows: ; Among them, represents the penalty value, represents the penalty coefficient, represents the indicator function for topic repetition, represents the indicator function for the user's ineffective response, 、 、 are core parameters that can be dynamically adjusted, used to balance the weights and influence degrees of different reward conditions. The policy network is trained using the deep reinforcement learning algorithm. During the training process, the policy network takes the state vector as input and outputs the probability distribution of each action. By continuously interacting with the environment and optimizing the parameters according to the reward feedback, in the inference stage, based on the action probability distribution output by the policy network for the current state, the action with the highest probability is selected, and then the corresponding natural language template is called to fill in the focus node parameters, and finally a guiding topic in the form of natural language is generated.

[0010] In a preferred embodiment, in S105, in the inference stage, first, based on the current state vector, the action with the largest Q value is selected from the action space through a greedy strategy. After selecting the action, duplicate detection and logical consistency verification are performed on the candidate topics. By calculating the semantic similarity between the candidate topic and the current focus state vector and setting a threshold, topics that are irrelevant or repetitive to the current focus are excluded. After completing the verification, enter the natural language generation link. According to the topic type to which the action belongs, a predefined template is called, and the entity, keyword and other parameters in the focus state are filled into the corresponding positions of the template to generate a preliminary guiding statement. Finally, the pre-trained Seq2Seq model is used to optimize the fluency of the statement generated by the template. By adjusting the statement structure through the encoding-decoding process, ensure that the language is natural and conforms to the professional norms of the mediation scenario, and finally output the guiding topic.

[0011] The beneficial effects of the present invention are as follows: The present invention combines multi-modal information of semantics, entities, emotions, and action phrases to improve the richness and accuracy of content representation. Through time decay and multi-head attention, it dynamically captures the temporal correlation and semantic focus in the conversation, enhances the timeliness of key information recognition, uses GCN to construct a focus evolution graph, intuitively presents the logical evolution and interaction relationship of the controversial points, supports the interpretability analysis of the mediation process, designs a reward function driven by the mediation goal, realizes the adaptive optimization of the topic generation strategy, and improves the mediation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is the method flow chart of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0013] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present application.

[0014] As Figure 1 , this embodiment provides: A method for extracting key points from conversation content based on reinforcement learning, including the following steps: S101. Preprocess the input text sentence, identify the action intention of the verb phrase, and splice the semantic vector, entity label, sentiment score, and action phrase into a composite feature vector in multi-dimensional features.

[0015] Further, first use a regular expression combined with a syntactic analysis tool to split the input text T into independent statements according to sentence boundaries , where S represents the set of split sentences, represents the function of splitting the text using a regular expression according to . After denoising, the specific calculation formula for denoising is as follows: ; Among them, represents the i-th sentence, represents the text cleaning function for removing blank lines and noise, and use the pre-trained BERT model to generate semantic vectors for each statement , where represents the word segmentation result of the i-th sentence, represents the word segmentation function to capture context semantic associations, and at the same time label key entities such as names, times, locations, and amounts through named entity technology , where represents the set of entity types, represents the named entity function, and calculate the sentiment polarity score of the statement by means of training a sentiment classification model. The specific calculation formula is as follows: ; Among them, represents the sentiment classification model function, represents the sentiment polarity score of the i-th sentence, where, ( ) Then extract the verb phrase through part-of-speech tagging to identify the action intention, and finally splice the semantic vector, entity label, sentiment score, and action phrase into a composite feature vector in multi-dimensional features. After completing the standardization process, store it as structured data including statement text, feature vector, entity sentiment, and action phrase.

[0016] S102. Input the preprocessed sequence of dialogue composite feature vectors into the Transformer multi-head attention layer. After applying exponential decay to the attention weights of historical sentences, introduce sentiment polarity and entity density as adjustment factors to optimize the calculation of attention weights, and filter out key statements according to a threshold and store them in the buffer area; Further, input the sequence of composite feature vectors obtained by preprocessing the dialogue text into the Transformer multi-head attention layer, and obtain the query matrix through linear transformation , key matrix and value matrix , where represents the weight matrix in the i-th attention head for mapping the composite feature vector to the query matrix, represents the weight matrix in the i-th attention head for mapping the composite feature vector to the key matrix, represents the weight matrix in the i-th attention head for mapping the composite feature vector to the value matrix. Calculate the original attention score matrix of the current sentence and all historical sentences. The specific calculation formula for the single-head attention score is as follows: ; where represents the original attention score between the current round t and the historical round i in the h-th attention head. t represents the current dialogue round, that is, the index of the current sentence when calculating attention. i represents the historical dialogue round, that is, the index of the historical sentence when calculating attention. h represents the attention head number, corresponding to the h-th subspace in the multi-head attention mechanism, represents the feature dimension of each attention head, represents the query vector of the current round t in the h-th attention head, represents the key vector of the historical round i in the h-th attention head, represents matrix transpose, transposing the key vector into a column vector for matrix multiplication with the query vector. Apply exponential decay to the attention weights of historical sentences. The specific calculation formula is as follows: ; where represents the exponential decay coefficient, represents the time decay rate. The specific calculation formula for the decayed attention score is as follows: ; where represents the decayed attention score, represents the fused attention score between the current round t and the historical round i, highlighting the influence of recent conversations. Introduce sentiment polarity and entity density , where represents the number of entities, represents the maximum number of entities, which is used as a regulation factor to optimize the calculation of attention weights. The specific calculation formula is as follows: ; where represents the adjusted normalized attention weight, which is used to measure the importance of the historical round i to the current round t, represents the exponential function, , represents a hyperparameter that adjusts the influence weights of sentiment polarity and entity density, and filters key statements according to the set attention threshold and stores them in the dynamic buffer.

[0017] S103. Construct the focus state vector and graph evolution. Set the named entities and action phrases of the key statements as graph nodes and endow them with attributes, then establish semantic edges and temporal edges according to the rules to form a graph. Enhance the nodes through the graph convolutional network, and finally pool the active nodes to generate a vector as the input of reinforcement learning; Furthermore, take the named entities and action phrases in the key statements as graph nodes, and endow the nodes with attributes such as appearance frequency, sentiment tendency, and first appearance timestamp. Subsequently, establish semantic edges and temporal edges according to the edge connection rules. If two nodes appear in the same statement or in adjacent rounds and the semantic similarity is higher than the threshold (such as cosine similarity > 0.7), establish an undirected semantic edge with the weight of the semantic similarity. At the same time, establish a directed temporal edge reflecting the logical sequence according to the order of the first appearance of the nodes to form a focus evolution graph. Then, aggregate the neighborhood information of the nodes through the graph convolutional network to enhance the node representation. Finally, perform a mean pooling operation on the enhanced representations of all current active nodes to generate a focus state vector that fuses the semantics, sentiment, and temporal relationships of the controversial focus, which is used as one of the input features of reinforcement learning.

[0018] S104. When using reinforcement learning to generate guiding topics, first splice the focus state vector and the sentiment statistic to construct the state space, design the action space containing the guiding topics and the corresponding templates, then set the reward and punishment rules, train the policy network through the reinforcement learning algorithm, and select actions according to the probability to call the templates; Furthermore, when using reinforcement learning strategies to generate guiding topics, the focus state vector is first concatenated with the emotional statistics of the current conversation (including average emotional polarity, proportion of negative emotions, etc.) to construct a state space that fully reflects the semantic focus and emotional state of the conversation, providing basic input for subsequent strategic decisions. Next, the action space is designed to divide the guiding topics into three categories: fact clarification, responsibility confirmation, and solution negotiation. For each type of topic, preset natural language templates are prepared. These templates can generate specific guiding content by filling in relevant information of the current focus node (such as entity name, dispute type). In the reward function design stage, effective guiding topics are generated for the incentive strategy network, and positive and negative reward rules are set: if the generated topic successfully guides to a new focus or promotes consensus, a + The specific calculation formula for the reward is as follows: ; in, Represents the new focus guidance reward value, represents the reward coefficient, An indicator function indicating the emergence of a new focus, Indicates the indicator function of consensus. If the topic increases the emotional polarity of the conversation and reduces the negative emotion, the reward is + , the specific calculation formula is as follows: ; in, represents the reward value of sentiment reward optimization, represents the reward function, Indicates the change in sentiment polarity between the current round and the previous round (s ), Indicates the decrease in the proportion of negative emotions l( ), when the topic is repeated and fails to elicit effective responses from users, causing them to repeat existing statements, give - The negative reward is calculated as follows: ; in, represents the penalty value, represents the penalty coefficient, An indicator function indicating topic repetition, An indicator function indicating that the user did not respond effectively. , , It is a core parameter that can be dynamically adjusted and is used to balance the weights and influence degrees of different reward conditions. The deep reinforcement learning algorithm is adopted to train the policy network. During the training process, the policy network takes the state vector as the input and outputs the probability distribution of each action. By continuously interacting with the environment and optimizing the parameters according to the reward feedback, in the inference stage, based on the action probability distribution output by the policy network for the current state, the action with the highest probability is selected, and then the corresponding natural language template is called to fill in the focus node parameters, and finally a guiding topic in the form of natural language is generated.

[0019] S105. Based on the current state vector, adopt a greedy strategy to select the action with the largest Q value from the action space, perform repeated verification and logical verification on the candidate topics, and call the predefined template according to the action type. The pre-trained Seq2Seq model optimizes the sentence fluency and outputs a guiding topic that conforms to the mediation scenario. Furthermore, in the inference stage, first, based on the current state vector, select the action with the largest Q value from the action space through a greedy strategy. After selecting the action, perform repeated detection and logical consistency verification on the candidate topics. By calculating the semantic similarity between the candidate topic and the current focus state vector and setting a threshold, exclude topics that are irrelevant or repeated with the current focus. After completing the verification, enter the natural language generation link. According to the topic type of the action (such as fact clarification, liability determination, compensation negotiation), call the predefined template, fill in the parameters such as entities and keywords in the focus state into the corresponding positions of the template to generate a preliminary guiding statement. Finally, use the pre-trained Seq2Seq model to optimize the fluency of the statement generated by the template, adjust the sentence structure through the encoding-decoding process, ensure that the language is natural and conforms to the professional norms of the mediation scenario, and finally output the guiding topic.

[0020] It should be noted that the method for extracting key points in the present invention, after splitting the dialogue text into sentences, generates semantic vectors through BERT, synchronously extracts features such as entities, emotions, and action phrases, splices them into a composite vector, constructs a multi-dimensional data basis, calculates the semantic weights between sentences based on the multi-head attention of Transformer, superimposes a time decay factor to highlight recent content, combines the sentiment polarity and entity density to optimize the weights, screens out high-weight sentences as key statements, abstracts the entities and action phrases in the key statements into graph nodes, constructs a graph through semantic edges (similarity) and temporal edges (logical order), uses GCN to aggregate neighborhood information to enhance the node representation, pools the active nodes to generate a focus state vector, fuses the focus state and sentiment statistics to define the state space, designs three types of guiding topic action sets, takes focus advancement and sentiment improvement as positive rewards, and repeated invalid actions as negative punishments, trains the policy network through the PPO algorithm, selects actions according to probability, and calls the template to generate a natural language guiding topic.

[0021] It should be noted that in the above embodiments, the descriptions of the respective embodiments have their own emphases. For parts not described in detail in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0022] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

Claims

1. A method for extracting key points from conversation content based on reinforcement learning, characterized in that, It includes the following steps: S101. Preprocess the input text sentence, identify the action intention of the verb phrase, and splice the semantic vector, entity label, sentiment score, and action phrase into a composite feature vector in multi-dimensional features; S102. Input the preprocessed sequence of dialogue composite feature vectors into the Transformer multi-head attention layer. After applying exponential decay to the attention weights of historical sentences, introduce sentiment polarity and entity density as adjustment factors to optimize the calculation of attention weights, and filter key statements according to the threshold and store them in the buffer; S103. Construct the focus state vector and graph evolution. Set the named entities and action phrases of the key statements as graph nodes and endow them with attributes, then establish semantic edges and temporal edges according to the rules to form a graph. Enhance the nodes through the graph convolutional network, and finally pool the active nodes to generate a vector as the input of reinforcement learning; S104. When using reinforcement learning to generate guiding topics, first splice the focus state vector and sentiment statistics to construct the state space, design the action space including guiding topics and corresponding templates, then set the reward and punishment rules, train the policy network through the reinforcement learning algorithm, and select actions according to the probability to call the templates; S105. Based on the current state vector, adopt a greedy strategy to select the action with the largest Q value from the action space, perform repeated verification and logical verification on the candidate topics, call the predefined template according to the action type, and optimize the sentence fluency with the pre-trained Seq2Seq model to output a guiding topic that conforms to the mediation scenario.

2. The method for extracting key points from conversation content based on reinforcement learning according to claim 1, wherein In S101, first use regular expressions combined with syntactic analysis tools to split the input text into independent sentences at the sentence boundary. After removing blank lines and noises, use the pre-trained BERT model to generate semantic vectors for each sentence to capture the context semantic association. At the same time, annotate the key entities of person names, time, location, and amount through the named entity technology, calculate the sentiment polarity score of the sentence by means of training a sentiment classification model, extract the verb phrase through part-of-speech tagging to identify the action intention, and finally splice the semantic vector, entity label, sentiment score, and action phrase into a composite feature vector in multi-dimensional features. After completing the standardization process, store it as structured data including sentence text, feature vector, entity sentiment, and action phrase.

3. The method for extracting key points from conversation content based on reinforcement learning according to claim 1, wherein In S102, the sequence of composite feature vectors obtained by preprocessing the dialogue text is input into the Transformer multi-head attention layer, and a query matrix is obtained through linear transformation , key matrix and value matrix , where represents the weight matrix in the i-th attention head for mapping the composite feature vector to the query matrix, represents the weight matrix in the i-th attention head for mapping the composite feature vector to the key matrix, represents the weight matrix in the i-th attention head for mapping the composite feature vector to the value matrix. Calculate the original attention score matrix of the current sentence and all historical sentences. The specific calculation formula for the single-head attention score is as follows: ; Among them, represents the original attention score of the current round t and the historical round i in the h-th attention head. t represents the current conversation round, that is, the current sentence index when calculating attention. i represents the historical conversation round, that is, the historical sentence index when calculating attention. h represents the attention head number, corresponding to the h-th subspace in the multi-head attention mechanism. represents the feature dimension of each attention head. represents the query vector of the current round t in the h-th attention head. represents the key vector of the historical round i in the h-th attention head. represents matrix transpose, which transposes the key vector into a column vector and applies exponential decay to the attention weights of historical sentences. The specific calculation formula is as follows: ; Among them, represents the exponential decay coefficient, represents the time decay rate, and the specific calculation formula for the decayed attention score is as follows: ; Among them, represents the attention score after attenuation, represents the attention score after the fusion of the current round t and the historical round i, highlighting the influence of recent conversations and introducing sentiment polarity and entity density , where represents the number of entities, represents the maximum number of entities, which is used as a regulatory factor to optimize the calculation of attention weights. The specific calculation formula is as follows: ; Among them, represents the adjusted normalized attention weight, which is used to measure the importance of the historical round i to the current round t. represents the exponential function. , represents a hyperparameter that adjusts the influence weights of sentiment polarity and entity density, and screens key statements according to the set attention threshold and stores them in the dynamic buffer.

4. The method for extracting key points from conversation content based on reinforcement learning according to claim 1, characterized in that, In S103, the named entities and action phrases in the key statements are used as graph nodes, and attributes such as occurrence frequency, sentiment tendency, and first appearance timestamp are endowed to the nodes. Subsequently, semantic edges and temporal edges are established according to the edge connection rules. If two nodes appear in adjacent rounds and the semantic similarity is higher than the threshold, an undirected semantic edge with the weight of semantic similarity is established. At the same time, a directed temporal edge reflecting the logical sequence relationship is established according to the order of the first appearance of the nodes to form a focus evolution graph. Then, aggregate the neighborhood information of the nodes through the graph convolutional network to enhance the node representation. Finally, perform an average pooling operation on the enhanced representations of all current active nodes to generate a focus state vector that fuses the semantics, sentiment, and temporal relationship of the dispute focus as one of the input features of reinforcement learning.

5. The method for extracting key points from conversation content based on reinforcement learning according to claim 1, characterized in that In S104, when generating guiding topics using the reinforcement learning strategy, first, the focus state vector is concatenated with the sentiment statistics of the current conversation to construct a state space that comprehensively reflects the semantic focus and emotional state of the conversation, providing the basic input for subsequent policy decisions. Next, the action space is designed, and the guiding topics are divided into three categories: fact clarification, responsibility confirmation, and solution negotiation. Preset natural language templates are prepared for each category of topics. In the reward function design section, positive and negative reward rules are set to encourage the policy network to generate effective guiding topics.

6. The method for extracting key points from conversation content based on reinforcement learning according to claim 5, wherein Positive and negative reward rules: If the generated topic successfully leads to a new focus, a reward of + will be given. The specific calculation formula is as follows: ; Among them, represents the new focus guidance reward value, represents the reward coefficient, represents the indication function of the appearance of the new focus, represents the indication function of reaching a consensus. If the topic improves the emotional polarity of the conversation and reduces negative emotions, the reward is + , and the specific calculation formula is as follows: ; Among them, represents the emotional reward optimization reward value, represents the reward function, represents the change in emotional polarity between the current round and the previous round (s ), represents the decrease in the proportion of negative emotions l( ). When the topic appears repeatedly and fails to trigger an effective response from the user, resulting in the user repeating existing statements, a negative reward of - is given. The specific calculation formula is as follows: ; Among them, represents the penalty value, represents the penalty coefficient, represents the indication function for topic repetition, represents the indication function for the user's ineffective response, , , are core parameters that can be dynamically adjusted, used to balance the weights and influence degrees of different reward conditions. The policy network is trained using the deep reinforcement learning algorithm. During the training process, the policy network takes the state vector as input and outputs the probability distribution of each action. By continuously interacting with the environment and optimizing the parameters according to the reward feedback, in the inference stage, based on the action probability distribution output by the policy network for the current state, the action with the highest probability is selected, and then the corresponding natural language template is called to fill in the focus node parameters, and finally a guiding topic in the form of natural language is generated.

7. A method for extracting key points from conversation content based on reinforcement learning according to claim 1, characterized in that, In S105, during the inference phase, first, based on the current state vector, the action with the largest Q value is selected from the action space through the greedy strategy. After selecting the action, duplicate detection and logical consistency verification are performed on the candidate topics. By calculating the semantic similarity between the candidate topics and the current focus state vector and setting a threshold, topics that are irrelevant or duplicate to the current focus are excluded. After completing the verification, it enters the natural language generation phase. According to the topic type to which the action belongs, the predefined template is called, and the entities and keyword parameters in the focus state are filled into the corresponding positions of the template to generate a preliminary guiding statement. Finally, the pre-trained Seq2Seq model is used to optimize the fluency of the statement generated by the template, and the statement structure is adjusted through the encoding-decoding process to finally output the guiding topic.

Images