A personalized reply generation method fusing role reasoning and adaptive attention

Abstract

The application belongs to the field of natural language processing, and particularly relates to a personalized reply generation method fusing role reasoning and adaptive attention; comprising: constructing inter-sentence concept graph and intra-sentence concept graph; encoding context dialogue data and concepts therein to obtain context semantic representation and concept representation; decoding the context semantic representation to obtain decoding output; expanding the concept representation according to the decoding output, the inter-sentence and intra-sentence concept graphs to obtain a role sentence set; processing the role sentence set and the context dialogue data to obtain role sentence self-attention representation, cross-role representation and cross-context representation; and calculating the personalized reply according to the role sentence self-attention representation, the cross-role representation and the cross-context representation; the application fully utilizes role information and context content to improve role consistency and dialogue coherence of the generated reply.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing, specifically relating to a personalized response generation method that integrates role reasoning and adaptive attention. Background Technology

[0002] In recent years, dialogue systems have brought convenience to people's lives in areas such as medical consultation and psychotherapy through intelligent question answering and casual conversation. The widespread adoption of the internet has led to a rapid increase in internet social data, with massive amounts of dialogue data constantly emerging, providing a solid data foundation for building data-driven dialogue systems. The wave of deep learning development has spurred major technology companies to invest in the dialogue field, resulting in a surge of commercial applications for dialogue systems. Many products have already entered the mass production stage, and a series of dialogue system applications, represented by Baidu Xiaodu, Apple Siri, Microsoft Xiaoice, and the currently popular ChatGPT, have permeated all aspects of life.

[0003] Personalized dialogue systems aim to provide context-appropriate, personalized responses based on given role information and dialogue history. Currently, one of the challenges facing personalized dialogue systems is the inconsistency between roles in responses. For example, even minor inconsistencies during speech can easily cause users to lose touch with the current dialogue context and lose interest in further conversation. Furthermore, the contextual coherence of the responses generated by the dialogue system is also an important indicator of response quality.

[0004] Existing role-based dialogue generation models often focus on correcting inconsistencies in the role-based response generation stage, while neglecting the fact that selecting appropriate role information based on context before response generation is crucial for generating personalized responses with role consistency. Furthermore, existing research considers both role information and dialogue context when generating dialogue responses, but rarely considers the impact of changes in the weight of these two elements during the response generation stage on the generation of personalized dialogue responses. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention proposes a personalized response generation method that integrates role reasoning and adaptive attention. The method includes: inputting the dialogue to be responded to into a trained personalized response generation model to obtain a personalized response that is consistent with the role.

[0006] The training process for the personalized response generation model includes:

[0007] S1: Obtain contextual dialogue data and construct inter-sentence concept maps and intra-sentence concept maps based on the contextual dialogue data;

[0008] S2: Perform Transformer encoding on the contextual dialogue data to obtain the contextual semantic representation; perform GRU encoding on the concepts in the contextual dialogue data to obtain the concept representation;

[0009] S3: Perform Transformer decoding on the context semantic representation to obtain the decoding output; expand the concept representation based on the decoding output, inter-sentence concept graph and intra-sentence concept graph, and obtain the role statement subset based on the probability expansion result;

[0010] S4: Encode the subset of character statements and contextual dialogue data to obtain the character encoded representation and the context encoded representation;

[0011] S5: Perform self-attention and weighted calculations on the subset of character statements and contextual dialogue data to obtain the self-attention representation and weighted representation of character statements;

[0012] S6: Perform cross-attention calculation on the role encoding representation and the context encoding representation based on the role statement self-attention representation and the role statement weighted representation to obtain the cross-role representation and the cross-context representation;

[0013] S7: Calculate personalized responses that maintain role consistency based on role statement self-attention representation, cross-role representation, and cross-context representation;

[0014] S8: Calculate the total model loss and adjust the model parameters based on the total model loss to obtain a trained personalized response generation model.

[0015] Preferably, in step S3, the process of expanding the concept representation based on the decoding output, the inter-sentence concept map, and the intra-sentence concept map includes:

[0016] Obtain the neighbor concept representations of the head concept representation in the last dialogue context from the inter-sentence concept graph of the last dialogue context; calculate the fusion probability based on the decoding output, the head concept representation, and the neighbor concept representations;

[0017] The concept transition probability of the head concept is calculated based on the fusion probability, decoding output, head concept representation, and neighbor concept representation.

[0018] Calculate the final probability distribution of concepts in the context of the last dialogue sentence based on the fusion probability and concept transformation probability; select role-related concepts from the concepts in the context of the last dialogue sentence based on the final probability distribution;

[0019] Obtain the intra-sentence concept map of the candidate related character statements, calculate the concept expansion probability based on the concept representation of the character-related concepts, the concept representation in the intra-sentence concept map, and the decoding output; calculate the final probability distribution of the concepts in the intra-sentence concept map based on the concept expansion probability and the concept transformation probability; select the expanded concepts from the concepts in the intra-sentence concept map based on the final probability distribution.

[0020] Furthermore, the formula for calculating the fusion probability is:

[0021]

[0022]

[0023] in, This represents the fusion probability of the k-th head concept and the m-th neighbor concept; This represents the information representation of all neighboring concepts of the k-th head concept. and Indicates the first and second training parameters, o j Indicates the decoded output, p N Represents the original character statement. This represents the k-th head concept in the inter-sentence concept graph. This represents the m-th neighboring concept in the inter-sentence concept graph.

[0024] Furthermore, the formula for calculating the probability of concept transition is:

[0025]

[0026]

[0027]

[0028] in, This represents the probability of concept switching, i.e., when the head concept is represented as At that time, the probability of choosing a concept related to the role; g k N represents the probability of selecting the k-th head concept. k This represents the number of neighbors as represented by the concept of neighbors. This represents the concept transition probability of the k-th head concept. With W g Indicates the third and fourth training parameters, o j Indicates the decoded output, p N Represents the original character statement. This represents the m-th neighboring concept in the inter-sentence concept graph.

[0029] Furthermore, the formula for calculating the concept expansion probability is:

[0030]

[0031]

[0032] in, This represents the concept expansion probability of the k-th head concept. Information representing the concept of all neighbors, Indicates the first training parameter. This represents the fifth training parameter, o. j Indicates the decoded output. This represents the k-th head concept in the sentence-level concept map. This represents the m-th neighboring concept in the sentence's concept map.

[0033] Furthermore, in step S6, the formula for calculating cross-attention between the role encoding representation and the context encoding representation is as follows:

[0034]

[0035]

[0036] Among them, o P Indicates cross-role representation, o U Q represents cross-context representation. r K represents the linear transformation of the weighted representation of character statements. p V represents the first linear transformation of the role encoding representation. p K represents the second linear transformation of the role encoding representation. u V represents the first linear transformation of the context-encoded representation. u d represents the second linear transformation of the context encoding representation, and d′ represents the dimension of the attention head.

[0037] Preferably, the process of calculating a personalized response with role consistency based on the role's statement self-attention representation, cross-role representation, and cross-context representation includes:

[0038] Generate initial character weights based on the character statement self-attention representation and cross-character representation;

[0039] The weighted cross-role representation and weighted cross-context representation are calculated based on the initial role weights, cross-role representations, and cross-context representations.

[0040] Perform mask transformation on the initial weights of the character to obtain the character mask and the context mask;

[0041] Multiply the role mask and the weighted cross role representation element by element, and multiply the context mask and the weighted cross context representation element by element to obtain two mask results; sum the two mask results to obtain a personalized response with consistent roles.

[0042] Furthermore, the formula for generating the initial weights of the characters is:

[0043] m p =FC([h R ;o P ])

[0044] w persona =Sigmoid(m p )

[0045] Among them, w persona Indicates the initial weight of the role, m p h represents the mapping result. R This represents the self-attention representation of character statements, o P The 'c' indicates a cross-role representation, FC indicates a linear layer, and Sigmoid(·) indicates an activation function.

[0046] Furthermore, the formula for masking the initial weights of the characters is as follows:

[0047] m persona =M(w persona >σ)

[0048] m context =M(1-w persona >σ)

[0049] Where, m persona Indicates the role mask, m context This represents the context mask, M represents the binary generator function, and w persona σ represents the initial weight of the role, and σ represents the mask control parameter.

[0050] Preferably, the total loss of the model is the negative log-likelihood loss.

[0051] The beneficial effects of this invention are as follows: First, this invention utilizes role reasoning for role selection. By employing graph attention fusion, it filters out the subset of role statements most relevant to the current context, effectively enhancing the contextual relevance of role statements and providing a strong foundation for generating high-quality personalized dialogue responses. Then, by introducing an adaptive attention mechanism, it dynamically adjusts the weight relationship between role information and context, thereby deriving personalized responses that conform to the context. Simultaneously, during the training phase, a dynamic masking mechanism is introduced, which not only removes redundant information from the context and role statements but also serves as a regularization mechanism to avoid overfitting. Compared with existing technologies, this invention can fully utilize role information and contextual content on a limited dataset, thereby maximizing the role consistency and dialogue coherence of the responses generated by the dialogue model. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the personalized response generation method that integrates role reasoning and adaptive attention in this invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] This invention proposes a personalized response generation method that integrates role-based reasoning and adaptive attention, such as... Figure 1 As shown, the method includes the following:

[0055] Role statements are statements that describe a person, such as "I'm a student," which describes "my" profession. Dialogue usually revolves around certain "concepts," which are prominent keywords in the context. Different role statements typically represent different concepts (e.g., profession, hobbies, age). For example, given the context "What do you do for living? I do consulting at a management company," the extracted concepts are "living," "management," and "company." Therefore, selecting role information that aligns with the current dialogue context is equivalent to predicting role-related concepts in the context. Personalized responses based on role consistency must adhere to the principle that the generated dialogue response is consistent with the role information and the context, based on the given role information.

[0056] The objective of this invention is to provide a given role statement P = {p1, ..., p...} e} and the dialogue context U = {u h,1 ,u m,1 ,…,u h,n Under the premise of}, generate personalized responses r=u that are consistent with the role. m,n Among them, the character statement P consists of e character description statements, u h,1 The first question representing humanity, u m,1 The first reply represents the machine, and so on...

[0057] Input the dialogue to be replied to into the trained personalized reply generation model to obtain a personalized reply that is consistent with the user's role. The training process of the personalized reply generation model includes:

[0058] S1: Obtain contextual dialogue data and construct inter-sentence concept maps and intra-sentence concept maps based on the contextual dialogue data.

[0059] Constructing an inter-sentence concept graph G τ With intra-sentence concept map G ε First, concepts are identified from the context and used as vertices in the concept graph, forming the inter-sentence concept graph G. τ The vertices in the graph represent concepts between different sentences, and the intra-sentence concept graph G is... ε The vertices in the graph represent concepts within the same sentence; then, the boundary values ​​of the graph are calculated using inter-point mutual information (PMI) to characterize the associations between concept pairs in the context. The specific process is as follows:

[0060]

[0061] Where P(A,B) represents the probability that word A and word B appear together, P(A) represents the frequency of word A, and P(B) represents the probability of word B appearing.

[0062] S2: Perform Transformer encoding on the contextual dialogue data to obtain the contextual semantic representation; perform GRU encoding on the concepts in the contextual dialogue data to obtain the concept representation.

[0063] Given the dialogue context U = {u h,1 ,u m,1 ,…,u h,n} is used as input to the Transformer encoder, ultimately yielding the contextual semantic representation H = {h} h,1 ,h m,1 ,…,h h,n The specific process is as follows:

[0064] H = Encoder(U)

[0065] in, d h It is the hidden dimension of the Transformer encoder.

[0066] Given a set of concepts F = {f1, f2, ..., f} in a given context N}, where f i The concept set corresponding to the i-th sentence context is then used to obtain the concept representation in the dialogue context using a GRU unit with an attention mechanism. The specific process is as follows:

[0067] m i =GRU(m i-1 ,f i )

[0068] S3: Perform Transformer decoding on the context semantic representation to obtain the decoding output; perform concept expansion on the concept representation based on the decoding output, inter-sentence concept graph and intra-sentence concept graph, and obtain the role statement subset based on the probability expansion result.

[0069] Obtain contextual semantic representation With concept representation Then, the special token sequence The context semantic representation, along with the input, is fed into the Transformer decoder to obtain the decoded output, represented as...

[0070]

[0071] Where, x j It is a special token <pad>It can be considered as the initial input token in non-autoregressive generation, N j It is a hyperparameter that represents the maximum number of target concepts.

[0072] Conceptual expansion is performed on the concept representation based on the decoded output, inter-sentence concept maps, and intra-sentence concept maps. The process includes:

[0073] Based on the context of the last sentence of the dialogue N Inter-sentence concept map G τ Retrieve the neighboring concept representations of the head concept representation in the context of the last dialogue sentence. Specifically: u N Each concept in the list can be represented as a head concept, denoted as... Only retain the concept representation of its neighbors. N k This represents the number of neighbor concept representations, which are included in the original set of roles. Then, based on the decoded output, the head concept representation, and the neighbor concept representations, the fusion probability is calculated, i.e., dynamic graph attention is performed to fuse the information of all concept neighbors.

[0074]

[0075]

[0076] in, This represents the fusion probability of the k-th head concept and the m-th neighbor concept; This represents the information representation of all neighboring concepts of the k-th head concept. and Indicates the first and second training parameters, o j Indicates the decoded output, p N Represents the original character statement. This represents the k-th head concept in the inter-sentence concept graph. This represents the m-th neighboring concept in the inter-sentence concept graph.

[0077] The concept transition probability of the head concept is calculated based on the fusion probability, decoded output, head concept representation, and neighbor concept representation. That is, when the head concept is represented as... When choosing a concept related to the role:

[0078]

[0079]

[0080]

[0081] in, This represents the probability of concept switching, i.e., when the head concept is represented as At that time, the probability of choosing a concept related to the role; g k N represents the probability of selecting the k-th head concept. k This represents the number of neighbors as represented by the concept of neighbors. This represents the concept transition probability of the k-th head concept. With W g Indicates the third and fourth training parameters, o j Indicates the decoded output, p N This indicates the original character statement.

[0082] The final probability distribution of concepts in the last dialogue context is calculated based on the fusion probability and concept transformation probability. That is, the final probability distribution of the target role-related concepts based on concept transformation is as follows:

[0083]

[0084] Finally, based on the final probability distribution, kernel sampling is used to select character-related concepts from the context of the last dialogue sentence.

[0085] Obtain the intra-sentence concept map of the relevant character statements, and calculate the concept expansion probability based on the concept representation of the character-related concepts, the concept representation in the intra-sentence concept map, and the decoding output; specifically:

[0086] Unlike the concept transformation process, the concept expansion process is based on the intra-sentence concept map G. ε Sharing the same decoder, the concept expansion probability is calculated as follows:

[0087]

[0088]

[0089] in, This represents the concept expansion probability of the k-th head concept. Information representing the concept of all neighbors, This represents the fifth training parameter, o. j Indicates the decoded output. This represents the k-th head concept in the sentence-level concept map. This represents the m-th neighboring concept in the sentence's concept map.

[0090] The final probability distribution of concepts in the intra-sentence concept map is calculated based on the concept expansion probability and concept transformation probability. That is, the final probability distribution of role-related concepts is as follows:

[0091]

[0092] Based on the probability expansion results, a subset of character statements is obtained; specifically: based on the final probability distribution, extended concepts are selected from the concepts in the sentence-specific concept map using kernel sampling, and these are the character-related concepts (extended concepts) predicted above. A beam search method is used to select character statements and output them as a subset.

[0093] S4: Encode the subset of character statements and context dialogue data to obtain the character encoding representation and the context encoding representation.

[0094] The character statement subset P and the context dialogue data U are processed by two separate encoders, allowing... Let P be a concatenation representation of all sentences in P, where Let be the i-th marker in character P, and at the same time, let This represents the token sequence used to connect the context content. Then, a bidirectional Transformer encoder is used to encode the role P and the context U:

[0095] h P =Encoder P (I P )

[0096] h U =Encoder U (I U )

[0097] Among them, h P The role code is represented by h. U Encoder represents the context encoding. P With Encoder U This represents a bidirectional Transformer encoder used for roles and context. and It is the hidden state from the last pooling layer of the encoder, where d is the output dimension of the encoder.

[0098] S5: Perform self-attention and weighted calculations on the subset of character statements and contextual dialogue data to obtain the self-attention representation and weighted representation of character statements.

[0099] To obtain the (r+1)th token in the response. For response input Self-attention computation is performed, where [BOS] represents the sentence start token. This is the i-th decoded response token. The specific self-attention and weighted operation process is as follows:

[0100] h R =Self-Attention(I R )+M R

[0101]

[0102] in, h R This represents the self-attention representation of character statements. Weighted representation of role statements; M R It is the mask of the decoder, which enables unidirectional computation of self-attention.

[0103] S6: Perform cross-attention calculation on the role encoding representation and context encoding representation based on the role statement self-attention representation and the role statement weighted representation to obtain cross-role representation and cross-context representation.

[0104] In obtaining the encoder's hidden state h P with h U and the decoder's self-attention output h R Then, based on (h) P ,h R ) and (h U ,h R Cross-attention is calculated using K and V, where K and V are provided by the encoder and Q is provided by the decoder. The specific calculation process is as follows:

[0105]

[0106]

[0107] Among them, o P Indicates cross-role representation, o U Indicates cross-context representation; Q r Weighted representation of role statements linear transformation; K p , K p The role code is represented by h. P The first linear transformation, V p The role code is represented by h. P The second linear transformation; K u The context encoding represents h U The first linear transformation, V u The context encoding represents h U The second linear transformation, d′ represents the dimension of the attention head.

[0108] S7: Calculate personalized responses that align with the role based on the role's self-attention representation, cross-role representation, and cross-context representation.

[0109] Based on the self-attention representation of the character's statements h R and cross-role representation o P Generate initial character weight w persona Specifically: using a linear layer FC and h R ,o P Mapped to Then Sigmoid(·) is used to apply m P This process is performed to obtain the initial weights of the characters, represented as follows:

[0110] m p =FC([h R ;o P ])

[0111] w persona =Sigmoid(m p )

[0112] Among them, w persona Indicates the initial weight of the role, [;] indicates cascading operations, m p h represents the mapping result. R This represents the self-attention representation of character statements, o P The 'c' indicates a cross-role representation, FC indicates a linear layer, and Sigmoid(·) indicates an activation function.

[0113] The weighted cross-role representation is calculated based on the initial role weight, cross-role representation, and cross-context representation. and weighted cross-context representation Specifically:

[0114]

[0115]

[0116] The initial weights of the roles are masked to obtain the role mask and the context mask; specifically, this is done to dynamically remove redundant information and regularize the two cross-attention results. and Here will be w persona Convert to m persona With m context , represented as:

[0117] m persona =M(w persona >σ)

[0118] m context =M(1-w persona >σ)

[0119] Where, m persona Indicates the role mask, m context This represents the context mask, where M represents the binary generator function; the mask m persona With m context Generated by the binary generator function M, which will output 1 and 0 based on specified conditions; w persona This represents the initial weight of the role; σ represents the mask control parameter, which is defined here as σ = |I U | / (|I U |+|I P |), where |I U | indicates the length of the context statement input, |I P | indicates the length of the character's statement input.

[0120] Multiply the role mask and the weighted cross-role representation element-wise, and multiply the context mask and the weighted cross-context representation element-wise to obtain two mask results; sum the two mask results to obtain a personalized response with consistent role representation; represented as:

[0121]

[0122]

[0123]

[0124] Here, ⊙ represents element-wise multiplication to obtain the mask result. and Then, summation is performed to obtain H. PAA As the final output.

[0125] S8: Calculate the total model loss and adjust the model parameters based on the total model loss to obtain a trained personalized response generation model.

[0126] To obtain the optimal output, this invention applies the negative log-likelihood loss function to the training of the model, and the specific implementation process is as follows:

[0127]

[0128] Among them, I R Indicates a response to input. Indicate I R The i-th token, p represents the first to i-1 response tokens. θ This represents the model of this method, and θ represents the model parameters.

[0129] Backpropagation is performed using negative log-likelihood loss to continuously adjust model parameters. When the model parameters are optimal, the model is saved, resulting in a trained personalized response generation model. The dialogue to be responded to is then input into the trained personalized response generation model to obtain a personalized response consistent with the user's role.

[0130] The above-described embodiments further illustrate the purpose, technical solution, and advantages of the present invention. It should be understood that the above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made to the present invention within the spirit and principles of the present invention should be included within the protection scope of the present invention.< / pad>

Claims

1. A personalized response generation method integrating role-based reasoning and adaptive attention, characterized in that, include: Input the dialogue to be replied to into the trained personalized reply generation model to obtain a personalized reply that is consistent with the role; The training process for the personalized response generation model includes: S1: Obtain contextual dialogue data and construct inter-sentence concept maps and intra-sentence concept maps based on the contextual dialogue data; S2: Perform Transformer encoding on the contextual dialogue data to obtain the contextual semantic representation; perform GRU encoding on the concepts in the contextual dialogue data to obtain the concept representation; S3: Perform Transformer decoding on the context semantic representation to obtain the decoded output; expand the concept representation based on the decoded output, inter-sentence concept graph, and intra-sentence concept graph, and obtain the role statement subset based on the probability expansion result; the process of expanding the concept representation based on the decoded output, inter-sentence concept graph, and intra-sentence concept graph includes: Obtain the neighbor concept representations of the head concept representation in the last sentence's dialogue context from the inter-sentence concept graph; calculate the fusion probability based on the decoded output, the head concept representation, and the neighbor concept representations; the formula for calculating the fusion probability is: ； ； in, Indicates the first The concept of the head and the first The fusion probability of each neighbor concept; Indicates the first Information representation of all neighboring concepts of a head concept. and Indicates the first and second training parameters. Indicates the decoded output. Represents the original character statement. The first in the inter-sentence concept map The concept of a head is represented. The first in the inter-sentence concept map The concept of a neighbor is represented; The concept transition probability of the head concept is calculated based on the fusion probability, decoded output, head concept representation, and neighbor concept representation; the formula for calculating the concept transition probability is: ； ； ； in, This represents the probability of concept switching, i.e., when the head concept is represented as When choosing a concept related to a character; Indicates selecting the first The probability of a leading concept. This represents the number of neighbors as represented by the concept of neighbors. Indicates the first The probability of concept transformation of a head concept and Indicates the third and fourth training parameters; Calculate the final probability distribution of concepts in the context of the last dialogue sentence based on the fusion probability and concept transformation probability; select role-related concepts from the concepts in the context of the last dialogue sentence based on the final probability distribution; Obtain the intra-sentence concept map of the candidate related character statements. Calculate the concept expansion probability based on the concept representations of the character-related concepts, the concept representations in the intra-sentence concept map, and the decoding output. Calculate the final probability distribution of the concepts in the intra-sentence concept map based on the concept expansion probability and the concept transformation probability. Select expanded concepts from the concepts in the intra-sentence concept map based on the final probability distribution. The formula for calculating the concept expansion probability is: ； ； in, Indicates the first The probability of concept expansion of a head concept, Information representing the concept of all neighbors, This represents the fifth training parameter. Indicates the first in the sentence concept map The concept of a head is represented. Indicates the first in the sentence concept map The concept of a neighbor is represented; S4: Encode the subset of character statements and contextual dialogue data to obtain the character encoded representation and the context encoded representation; S5: Perform self-attention and weighted calculations on the subset of character statements and contextual dialogue data to obtain the self-attention representation and weighted representation of character statements; S6: Perform cross-attention calculation on the role encoding representation and the context encoding representation based on the role statement self-attention representation and the role statement weighted representation to obtain the cross-role representation and the cross-context representation; S7: Calculate personalized responses that maintain role consistency based on role statement self-attention representation, cross-role representation, and cross-context representation; S8: Calculate the total model loss and adjust the model parameters based on the total model loss to obtain a trained personalized response generation model.

2. The personalized response generation method integrating role reasoning and adaptive attention as described in claim 1, characterized in that, In step S6, the formula for calculating cross-attention between the role encoding representation and the context encoding representation is as follows: ； ； in, Indicates cross-role representation, This indicates a cross-context representation. A linear transformation representing the weighted representation of character statements. This represents the first linear transformation of the role encoding representation. The second linear transformation represents the character encoding representation. This represents the first linear transformation of the context-encoded representation. The second linear transformation representing the context encoding representation, This represents the dimension of the attention head.

3. The personalized response generation method integrating role reasoning and adaptive attention as described in claim 1, characterized in that, The process of calculating a personalized response with role consistency based on role-based statement self-attention representation, cross-role representation, and cross-context representation includes: Generate initial character weights based on the character statement self-attention representation and cross-character representation; The weighted cross-role representation and weighted cross-context representation are calculated based on the initial role weights, cross-role representations, and cross-context representations. Perform mask transformation on the initial weights of the character to obtain the character mask and the context mask; Multiply the role mask and the weighted cross role representation element by element, and multiply the context mask and the weighted cross context representation element by element to obtain two mask results; sum the two mask results to obtain a personalized response with consistent roles.

4. The personalized response generation method integrating role reasoning and adaptive attention as described in claim 3, characterized in that, The formula for generating the initial weights of a character is: ； ； in, Indicates the initial weight of the role. This represents the mapping result. This represents the self-attention representation of character statements. Indicates cross-role representation, Indicates a linear layer. This represents the activation function.

5. The personalized response generation method integrating role reasoning and adaptive attention as described in claim 3, characterized in that, The formula for masking the initial weights of the characters is: ； ； in, Indicates the role mask. Indicates the context mask, Represents a binary generator function. Indicates the initial weight of the role. This represents the mask control parameters.

6. The personalized response generation method integrating role reasoning and adaptive attention according to claim 1, characterized in that, The total loss of the model is the negative log-likelihood loss.

Images