Text processing method and electronic equipment

Abstract

The invention provides a text processing method and electronic equipment, and is applied to the technical field of artificial intelligence. The text processing method comprises the steps that a first dialogue text and first prompt information are input into a large language model, first summary information is obtained, and the first prompt information is used for guiding generation of the first summary information; based on the first summary information, determining a first feature by using a first model, the first feature representing context information corresponding to each word in the first summary information; according to the first feature, a first heterogeneous graph is constructed, the heterogeneous graph comprises at least two nodes and an edge used for connecting the nodes of the two heterogeneous graphs, the nodes represent the first feature of the words in the first summary information, and the edge represents that the two connected nodes are related in semantics; based on the first heterogeneous graph, a first label corresponding to the first dialogue text is determined by utilizing a multi-layer graph attention network, and the first label comprises a first semantic label and a first entity label.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a text processing method and an electronic device. Background Technology

[0002] Currently, large language models generate insufficient text labels during the process of extracting labels from text content, resulting in poor performance when using these text labels for further task processing. Summary of the Invention

[0003] In view of this, the present disclosure provides a text processing method and an electronic device.

[0004] According to a first aspect of this disclosure, a text processing method is provided, comprising: inputting a first dialogue text and first prompt information into a large language model to obtain first summary information, wherein the first prompt information is used to guide the generation of the first summary information; based on the first summary information, using a first model, determining a first feature, wherein the first feature represents context information corresponding to each word in the first summary information; constructing a first heterogeneous graph based on the first feature, wherein the heterogeneous graph contains at least two nodes and edges for connecting nodes of two heterogeneous graphs, wherein the nodes represent the first features of words in the first summary information, and the edges represent that the two connected nodes are semantically related; and based on the first heterogeneous graph, using a multi-layer graph attention network, determining a first label corresponding to the first dialogue text, wherein the first label includes a first semantic label and a first entity label.

[0005] According to embodiments of this disclosure, the Nth layer graph attention network is used to process node information within N hops corresponding to each node. The hop count represents the number of edges included in the path between two nodes, where N is an integer greater than 1. Each layer of the graph attention network embeds a multi-hop gating mechanism. Based on the first heterogeneous graph, the multi-layer graph attention network is used to determine the first label corresponding to the first dialogue text. This includes, for any node in the first heterogeneous graph: taking the first feature of any node as the initial input, sequentially inputting the output of the (N-1)th layer graph attention network and the image information of the first heterogeneous graph into the Nth layer graph attention network to obtain the output of the Nth layer graph attention network, until the second feature is output. The output of the Nth layer of the target node is the fusion of the N-hop neighboring node information of the target node. The hop count represents the number of edges included in the path between two nodes, where N is an integer greater than 1. The multi-hop gating mechanism is used to assign different weights to node information with different hop counts relative to the target node. The second feature is the result of fusing the N-hop neighboring node information of each node. Based on the second feature, the first semantic label and the first entity label are determined.

[0006] According to embodiments of this disclosure, determining a first semantic label and a first entity label based on a second feature includes: inputting the second feature and semantic information from the first summary information into a first prediction layer to obtain a first semantic label; and inputting the second feature and entity information from the first summary information into a second prediction layer to obtain a first entity label.

[0007] According to embodiments of this disclosure, the first dialogue text and the first prompt information are input into a large language model to obtain first summary information, including: segmenting the first dialogue text according to the speaking subject turn to obtain at least one subsequence, where the speaking subject turn is the basic unit of multiple speaking subjects speaking alternately; determining the topic clusters corresponding to each subsequence, where different topic clusters represent different topics; determining the local summary corresponding to the subsequence and the topic summary corresponding to the topic cluster according to the first prompt information; and determining the first summary information according to the first prompt information, the local summary, and the topic summary.

[0008] According to embodiments of this disclosure, the first model is obtained through the following operations: obtaining multiple sample data, the sample data including a second dialogue text and a second label corresponding to the second dialogue text, the second label including a second semantic label and a second entity label; determining second summary information based on the second dialogue text and the second prompt information, the second prompt information being used to guide the generation of the second summary information; inputting the second summary information into the first model to obtain a third feature, the third feature representing the context information corresponding to each word in the second summary information; constructing a second heterogeneous graph based on the third feature, the second heterogeneous graph containing at least two nodes and edges for connecting the nodes of the two heterogeneous graphs, the nodes representing the third features of words in the second summary information, the edges representing that the two connected nodes are semantically related; based on the second heterogeneous graph, using a multi-layer graph attention network, determining a predicted label corresponding to the second dialogue text, the predicted label including a predicted semantic label and a predicted entity label; adjusting the parameters of the first model based on the difference information between the second label and the predicted label, to obtain the trained first model.

[0009] According to embodiments of this disclosure, the method further includes: determining a tag prototype vector based on a third feature, the tag prototype vector representing a vector representation of the tags contained in the second summary information.

[0010] According to embodiments of this disclosure, adjusting the parameters of a first model based on the difference information between a second label and a predicted label to obtain a trained first model includes: determining a first loss based on a second semantic label and a predicted semantic label; determining a second loss based on a second entity label and a predicted entity label; determining a target loss based on the first loss and the second loss; iteratively calculating the gradient of the target loss with respect to the parameters of the first model using a backpropagation algorithm, updating the parameters of the first model in the direction of the gradient, until the target loss converges, thereby obtaining a trained first model.

[0011] According to embodiments of this disclosure, determining a target loss based on a first loss and a second loss includes: determining a target bag-of-words distribution based on second summary information, wherein the target bag-of-words distribution represents the word frequency statistics in the second summary information; determining a predicted bag-of-words distribution based on predicted tags, wherein the predicted bag-of-words distribution represents the word frequency statistics in the predicted tags; determining a third loss based on the target bag-of-words distribution and the predicted bag-of-words distribution; and determining the target loss based on the first loss, the second loss, and the third loss.

[0012] According to embodiments of this disclosure, determining a target loss based on a first loss, a second loss, and a third loss includes: determining word vectors based on predicted labels; determining a fourth loss based on label prototype vectors and word vectors; and determining the target loss based on the first loss, the second loss, the third loss, and the fourth loss.

[0013] A second aspect of this disclosure provides an electronic device, comprising: a processor configured to input a first dialogue text and first prompt information into a large language model to obtain first summary information, wherein the first prompt information is used to guide the generation of the first summary information; based on the first summary information, using a first model, determining a first feature, wherein the first feature represents context information corresponding to each word in the first summary information; constructing a first heterogeneous graph according to the first feature, wherein the first heterogeneous graph contains at least two nodes and edges for connecting nodes of two heterogeneous graphs, wherein the edges represent that the two nodes are semantically related; and based on the first heterogeneous graph, using a multi-layer graph attention network, determining a first label corresponding to the first dialogue text, wherein the first label includes a first semantic label and a first entity label.

[0014] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0015] The above and other objects, features and advantages of this disclosure will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0016] Figure 1 This illustration schematically depicts an application scenario of a text processing method and an electronic device according to embodiments of the present disclosure.

[0017] Figure 2 A flowchart illustrating text processing according to an embodiment of the present disclosure is shown schematically;

[0018] Figure 3 The illustration shows a schematic diagram of a multilayer graph attention network method for processing a first heterogeneous graph according to an embodiment of the present disclosure;

[0019] Figure 4A schematic diagram illustrating a method for obtaining first digest information according to an embodiment of the present disclosure is shown.

[0020] Figure 5 A schematic diagram illustrating a text processing method according to an embodiment of the present disclosure is shown.

[0021] Figure 6 A block diagram schematically illustrates an electronic device suitable for implementing a text processing method according to an embodiment of the present disclosure. Detailed Implementation

[0022] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0023] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0024] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0025] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0026] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0027] This disclosure provides a text processing method and an electronic device. Before introducing the technical solutions provided by this disclosure, the relevant technologies involved in this disclosure will be described first.

[0028] Currently, large language models generate insufficient text labels during the process of extracting labels from text content, resulting in poor performance in subsequent downstream tasks that utilize these text labels.

[0029] For example, in multi-turn dialogue scenarios, accurately extracting high-level text tags that summarize the core content of the dialogue is a crucial task. Text tags play a key role in downstream tasks such as dialogue semantic understanding, customer intent recognition, accurate intelligent recommendation, business opportunity mining, and optimizing information retrieval efficiency. These include various business scenarios such as enterprise outbound call list acquisition and prioritization, customer profiling and refined operations, and intelligent sales assistants.

[0030] To obtain high-quality tags, the following solutions are typically used in related technologies:

[0031] I. A method based on a predefined tag system and LLM (Large Language Model) zero-sample or few-sample prompts is used for extraction. This involves using predefined business tags to design prompt words for text tag extraction. However, this approach is severely limited by the quality of the prompting process and the completeness of the predefined tag system. The main drawbacks are as follows: (1) Extraction performance is highly dependent on the design quality of the prompt words: its extraction performance is extremely sensitive to small changes in the prompt words, and there is significant instability and unreliability; (2) Long context processing is limited: traditional prompting methods are difficult to handle long contexts, and the inherent window length limitation of LLM leads to the loss or forgetting of key information; (3) The computation and reasoning costs are too high: when LLM directly processes long texts, the computational overhead is huge, and the reasoning latency and resource consumption are significantly increased, making it difficult to meet the real-time requirements of large-scale dialogue scenarios; (4) Lack of structured hierarchical modeling mechanism: the lack of structured multi-level information compression and reasoning chain makes it easy to miss implicit semantics across rounds or generate redundant labels, resulting in lengthy or abstract prediction results; (5) Difficulty in handling multiple intentions and complex semantic relationships: in business dialogues with multiple intentions and multiple topics, LLM direct extraction often makes it difficult to accurately distinguish and model the complex relationship between entities and semantics, which limits the ability to support downstream tasks (such as customer profiling and strategy recommendation).

[0032] II. Extracting Entities and Semantic Labels Based on an Integrated Generation-Extraction Keyphrase Prediction Framework. This scheme proposes an end-to-end keyphrase prediction framework (UniKeyphrase) that achieves integrated modeling of extraction and generation by jointly learning two sub-tasks: PresentKeyphrase Extraction (PKE) and AbsentKeyphrase Generation (AKG). This joint learning strategy, along with the design of Stacked Relational Layer (SRL) and Bag-of-Words Constraint (BWC), considers the interaction between tasks and global word frequency information, providing a reasonable foundation for keyphrase prediction. However, the original framework still has several limitations in practical applications: First, SRL relies solely on the co-attention mechanism to implicitly capture the relationships between tasks, making it difficult to fully model the potential hierarchical semantic structure in the text, such as the relationship between "product-procurement needs-business scenarios," which is particularly evident in complex documents or cross-domain corpora. Furthermore, multi-layer stacking easily leads to representation degradation, making it difficult to effectively transmit low-level information, while also increasing computational overhead and affecting the model's efficiency on large-scale data. Secondly, BWC only reflects the overall semantic profile of the label in the dataset. If only global statistics are relied upon, it is difficult to distinguish the subtle differences in the current context.

[0033] Third, extract the main content of the dialogue and corresponding tags using other text classification and generation methods.

[0034] However, this solution addresses a task analogous to combining intent recognition with slot filling and keyphrase extraction. The former primarily targets dialogues with a single dominant intent. In complex, multi-turn business dialogues, user expressions often contain multiple intertwined needs, making this method weak in multi-intent processing. Furthermore, the slot filling mechanism, reliant on intent recognition, easily misses key information or tags corresponding to non-dominant intents, leading to insufficient information mining of the overall dialogue content. The latter extracts tags typically as raw keywords or phrases, lacking necessary semantic abstraction and generalization, resulting in poor tag interpretability. It also easily generates a large number of difficult-to-distinguish or semantically redundant tags, failing to effectively form a clear, high-level, business-meaning tag system, making it difficult to directly serve downstream tasks requiring precise semantic mapping. Moreover, the uncertainty of multi-turn dialogues (omissions, tone, context jumps) may introduce errors into unsupervised extraction.

[0035] An ideal tag not only needs to be accurate and have a high level of abstraction, but also needs to reflect multi-dimensional characteristics and cover multiple key information points that are intertwined in the conversation.

[0036] Before providing a further detailed description of the embodiments of this disclosure, the nouns and terms involved in the embodiments of this disclosure will be explained, and the nouns and terms involved in the embodiments of this disclosure shall be interpreted as follows.

[0037] Chain of Thought (CoT) refers to a series of logically related thinking steps that form a complete thought process. Chain of Thought is often used in cue-based learning of large models, breaking down the reasoning process of a large model into individual steps and visually representing them, thereby improving the accuracy of the final result.

[0038] In one or more embodiments of this disclosure, a large model refers to a deep learning model with a large number of model parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even tens of trillions of model parameters. A large model can also be called a foundation model. It is pre-trained using large-scale unlabeled corpora to produce a pre-trained model with hundreds of millions of parameters. Such models can adapt to a wide range of downstream tasks and have good generalization ability. Examples include large-scale language models (LLMs) and multi-modal pre-training models.

[0039] Figure 1 The illustration shows an application scenario of a text processing method and an electronic device according to embodiments of the present disclosure.

[0040] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a client 110 and a server 120;

[0041] Client 110 is used to send the first dialogue text and the first prompt information to server 120;

[0042] Server 120 is used to input the first dialogue text and the first prompt information into a large language model to obtain the first summary information. The first prompt information is used to guide the generation of the first summary information. Based on the first summary information, a first feature is determined using a first model. The first feature represents the context information corresponding to each word in the first summary information. According to the first feature, a first heterogeneous graph is constructed. The heterogeneous graph contains at least two nodes and edges for connecting the nodes of two heterogeneous graphs. The nodes represent the first features of the words in the first summary information, and the edges represent that the two connected nodes are semantically related. Based on the first heterogeneous graph, a first label corresponding to the first dialogue text is determined using a multi-layer graph attention network. The first label includes a first semantic label and a first entity label. The first label corresponding to the first dialogue text is fed back to the client.

[0043] Client 110 is also used to receive the first tag sent by server 120.

[0044] The text processing system may include multiple clients 110 and a server 120. Clients 110 can be referred to as client devices, and servers 120 can be referred to as server devices. Multiple clients 110 can establish communication connections through the server 120. In a text tag generation scenario, the server 120 provides text processing and generation services between the multiple clients 110. Each client 110 can act as a sender or receiver, communicating through the server 120.

[0045] Users can interact with server 120 through client 110 to receive data sent by other clients 110, or send data to other clients 110, etc.

[0046] In this system, client 110 and server 120 establish a connection via a network. The network provides a communication link between client 110 and server 120. The network can include various connection types, such as wired or wireless communication links or fiber optic cables. Data transmitted by client 110 may need to undergo encoding, transcoding, compression, or other processing before being published to server 120.

[0047] Client 110, or client application, can be a browser, APP (Application), web application such as H5 (HyperText Markup Language 5), lightweight application (also known as a mini-program), or cloud application. Client 110 can be developed based on the software development kit (SDK) provided by server 120, such as a real-time communication (RTC) SDK. Client 110 can be deployed on electronic devices and depends on the device or certain APPs on the device to run. Electronic devices may have displays and support information browsing, such as personal mobile terminals like mobile phones, tablets, and personal computers. Various other types of applications can also be configured on electronic devices, such as human-computer interaction applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, and social media platform software.

[0048] Server 120 may include servers providing various services, such as servers providing communication services to multiple clients, servers supporting backend training of models used on clients, and servers processing data sent by clients. It should be noted that server 120 can be implemented as a distributed server cluster composed of multiple servers, or as a single server. The server can also be a server in a distributed system, or a server integrated with blockchain. The server can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0049] It is worth noting that the text processing method provided in the embodiments of this disclosure is generally executed by the server. However, in other embodiments of this disclosure, the client may also have similar functions to the server, thereby executing the text processing method provided in the embodiments of this disclosure. In other embodiments, the text processing method provided in the embodiments of this disclosure may also be executed jointly by the client and the server.

[0050] The following will be based on Figure 1 The described scene, through Figures 2-5 The text processing method of the present disclosure embodiments will be described in detail.

[0051] Figure 2 A flowchart illustrating a text processing method according to an embodiment of the present disclosure is shown schematically.

[0052] like Figure 2 As shown, the text processing method of this embodiment includes operations S210 to S240.

[0053] In operation S210, the first dialogue text and the first prompt information are input into the large language model to obtain the first summary information. The first prompt information is used to guide the generation of the first summary information.

[0054] In operation S220, based on the first summary information, the first model is used to determine the first feature, which represents the context information corresponding to each word in the first summary information.

[0055] In operation S230, a first heterogeneous graph is constructed based on the first feature. The heterogeneous graph contains at least two nodes and edges for connecting the nodes of the two heterogeneous graphs. The nodes represent the first features of words in the first summary information, and the edges represent that the two connected nodes are semantically related.

[0056] In operation S240, based on the first heterogeneous graph, a multi-layer graph attention network is used to determine the first label corresponding to the first dialogue text. The first label includes a first semantic label and a first entity label.

[0057] For example, the first dialogue text can be text information from an original dialogue record. The original dialogue record includes at least one of the following forms of information: voice information, text information, etc. For instance, the original dialogue record could be a voice communication record between user A and user B in social media, a telephone communication record between user A and customer service, or a text communication record between user A and customer service in shopping software. For example, a conversation between user 1 and customer service regarding cloud server pricing via telephone could include multiple rounds of dialogue information, such as user inquiry, agent quotes, and user negotiation.

[0058] The initial prompt can be an instruction or template text used to guide and control the generation behavior of the large language model. It employs a Chain-of-Thought (CoT) structure to guide the model in structured reasoning, ensuring that the style, content, and format of the generated results meet the requirements of downstream task processing. For example, the initial prompt could be: "Please summarize the following customer service dialogue into a concise summary, including the user's core needs and the agent's key responses. The summary must be in Chinese and no more than 100 characters."

[0059] The first summary information can be compressed text extracted and summarized from the first dialogue text by a large language model based on the first prompt information, retaining the core information of the dialogue and removing redundant content. For example, the first summary information could be: User 1 inquired about the price and configuration of Cloud Server Package B. Customer service informed him that the price was 800 yuan per month, with a configuration of 8 cores and 16GB of memory, and mentioned that there was a 10% discount at present. User 1 then inquired about its comparative advantages compared to competitor Company A.

[0060] The first model can be a pre-trained language model capable of extracting contextual features from summary information. For example, the first model can be a model using the UNILM (Unified Language Model) architecture.

[0061] The first feature can be a vector output by the first model corresponding to each word. This vector is not an isolated word meaning, but rather a word meaning that incorporates the entire summary context. For example, the first feature can be a sequence of hidden state vectors obtained by the first model encoding the first summary information, with each vector representing the contextual semantic information of the corresponding word. For instance, for the word "price" in the first summary, its corresponding first feature vector not only contains the general semantics of "price," but also encodes specific contextual information such as "the price of Package B," "800 yuan," and "the focus of user inquiries."

[0062] The first heterogeneous graph can be a graph structure constructed based on the first feature, used to explicitly model semantic dependencies between labels. The heterogeneous graph contains different types of nodes and edges, capable of simultaneously representing the relationship between entity labels and semantic labels. For example, a node is the basic unit in the heterogeneous graph, representing a word in the summary. The data stored in each node is the first feature corresponding to that word. Each word in the first summary information creates two nodes: a PLE node (for entity recognition) and an ALG node (for semantic label generation). An edge is a line segment connecting two nodes in the first heterogeneous graph, representing a semantic correlation between the words represented by these two nodes.

[0063] A multilayer graph attention network can be a deep learning model for graph-structured data. It consists of multiple stacked graph attention layers.

[0064] The first set of labels can be obtained by classifying and predicting nodes in the first heterogeneous graph using a multi-layer graph attention network. It includes two subsets: first semantic labels and first entity labels. The first semantic labels are the result of a high-level semantic abstraction of the dialogue content. For example, for the first summary information, semantic labels such as product inquiry, competitor comparison, and price objection might be predicted. The first entity labels are the identification and classification of specific entities in the dialogue, usually appearing directly in the first dialogue text. For example, "Package B" in the first summary information is labeled as the product name, "800 yuan" as the price, and "10% off" as the discount.

[0065] Understandably, by combining prompts, the model is guided to generate first summary information, and the first feature of the first summary information is extracted using the first model to construct a heterogeneous graph, forming dependencies between features. At the same time, a multi-layer graph attention network is combined to predict labels, achieving collaborative reasoning. The model predicts labels not only based on the words themselves, but also on their position and relationship in the entire dialogue semantic network, which greatly improves the accuracy, consistency, and understanding of complex contexts of label extraction. Finally, it outputs structured, multi-granular dialogue labels, providing accurate and rich input for downstream business scenarios such as customer profiling, business opportunity mining, and intelligent recommendation.

[0066] Figure 3 The illustration shows a schematic diagram of a multilayer graph attention network method for processing a first heterogeneous graph according to an embodiment of the present disclosure.

[0067] As mentioned above, the Nth layer graph attention network is used to process node information within N hops corresponding to each node. The hop count represents the number of edges included in the path between two nodes, where N is an integer greater than 1. Each layer of the graph attention network embeds a multi-hop gating mechanism.

[0068] In operation S240, based on the first heterogeneous graph, a multi-layer graph attention network is used to determine the first label corresponding to the first dialogue text. In one possible implementation, such as... Figure 3 As shown, the operation can further include performing the following operations for any node in the first heterogeneous graph: taking the first feature of any node as the initial input, sequentially inputting the output of the (N-1)th layer graph attention network and the image information of the first heterogeneous graph into the Nth layer graph attention network to obtain the output of the Nth layer graph attention network until the second feature is output. The output of the Nth layer of the target node is the fusion of the N-hop neighbor node information of the target node. The hop count represents the number of edges included in the path between two nodes, and N is an integer greater than 1. The multi-hop gating mechanism is used to assign different weights to the node information with different hop counts relative to the target node. The second feature is the result of fusing the N-hop neighbor node information of each node. Based on the second feature, the first semantic label and the first entity label are determined.

[0069] For example, the Nth layer graph attention network can be the Nth layer processing unit in a multi-layer graph attention network, used to aggregate the neighbor node information of the target node within the Nth hop range. For instance, when the multi-layer graph attention network contains 3 layers, the first layer graph attention network processes 1-hop neighborhood information, the second layer graph attention network processes 2-hop neighborhood information, and the third layer graph attention network processes 3-hop neighborhood information. For example, in a first heterogeneous graph constructed from the first summary information, if the "price" node is taken as the target node, its 1-hop neighbors may include the "Package B" node and the "800 yuan" node, its 2-hop neighbors may include the "cloud server" node and the "10% discount" node, and its 3-hop neighbors may include the "competitor company A" node.

[0070] N-hop neighborhood node information can be the set of feature information carried by all nodes whose shortest path to the target node in the first heterogeneous graph contains exactly N edges. For example, the N-hop neighborhood does not include nodes that have a shorter path to the target node. For example, for the target node "price", if the "B package" node is directly connected to the "price" node, then the "B package" node belongs to the 1-hop neighborhood; if the "cloud server" node needs to go through the "B package" node to reach the "price" node, and there is no shorter path, then the "cloud server" node belongs to the 2-hop neighborhood. In this case, when calculating the 2-hop neighborhood node information, only the features of 2-hop neighbors such as "cloud server" and "10% off" are aggregated, and the features of 1-hop neighbors such as "B package" and "800 yuan" are not calculated repeatedly.

[0071] Multi-hop gating can be an adaptive weight allocation mechanism used to control the contribution of neighborhood information with different hop counts to the feature update of the target node. This mechanism calculates a gating coefficient for each hop count, allowing the model to dynamically adjust the influence of nearby neighbors based on the specific context, thus preventing excessive interference from noise information from distant nodes in the target node's representation learning. For example, a multi-hop gating mechanism may include two sub-steps: gating score calculation and gating coefficient normalization. In the gating score calculation stage, for the aggregated information of each hop count received by the target node at layer N, combined with the target node's current representation, a score is calculated using trainable gating parameters. This score reflects the importance of the corresponding hop count information. In the gating coefficient normalization stage, the scores of all hop counts are normalized to obtain gating coefficients that sum to 1, which are used for subsequent weighted fusion. For example, when processing the target node "price" in the second-layer graph attention network, it is assumed that the 1-hop neighborhood information aggregation result emphasizes the direct association between "Package B" and "800 yuan", while the 2-hop neighborhood information aggregation result introduces the supplementary semantics of "cloud server" and "10% discount". The multi-hop gating mechanism can calculate that the gating coefficient of the 1-hop information is 0.7 and the gating coefficient of the 2-hop information is 0.3, indicating that in the current layer, the nearest neighbor information contributes more to the feature update of the "price" node. As the number of network layers increases to the third layer, if the "competitor company A" information introduced by the 3-hop neighborhood is weakly related to the core topic of the dialogue, the multi-hop gating mechanism can adaptively reduce its gating coefficient, for example, by assigning a weight of 0.1, while assigning weights of 0.6 and 0.3 to the 1-hop and 2-hop information respectively, thereby suppressing the influence of distant noise nodes.

[0072] The second feature can be the final vector representation of each node in the first heterogeneous graph after processing by a multi-layer graph attention network and a multi-hop gating mechanism. This representation integrates the contextual semantic information of the node within its multi-hop neighborhood and achieves adaptive weighting of different hop count information through the gating mechanism. For example, compared to the first feature, the second feature not only includes the node's own initial contextual semantics but also integrates information passed down through multiple levels of relationships, such as its neighboring nodes and their neighboring nodes, while reasonably balancing the contributions of near and far neighbors. For instance, for the target node "price," its first feature only encodes the contextual meaning of the word "price" in the first summary information. After passing through the first-layer graph attention network, the representation of this node incorporates the information of the 1-hop neighbor "Package B" and "800 yuan," expanding its semantics to "the price of Package B at 800 yuan." After passing through the second-layer graph attention network and applying the multi-hop gating mechanism, the node further integrates the information of the 2-hop neighbor "cloud server" and "10% discount," and through weighting with gating coefficients of 0.7 and 0.3, forms an intermediate representation that emphasizes direct associations while appropriately introducing indirect semantics. Finally, after the third layer of processing, the second feature of this node not only covers the core semantics of "the price of the B package cloud server is 800 yuan and a 10% discount", but also incorporates the background information of "comparison with competitor A company" through a lower gating weight, forming a rich and focused vector representation. At this point, all nodes in the first heterogeneous graph have completed a similar multi-hop information fusion process, obtaining their respective second features. These second features together constitute the feature set used for subsequent label prediction.

[0073] In one example, we will use a multilayer graph attention network (GAT) that introduces a hop-specific gating mechanism.

[0074] If only a single attention layer is used, the same attention weights are applied to all attributes of the neighboring nodes, which will weaken the model's learning ability. Therefore, a multi-layer GAT is constructed for label node updates.

[0075] A target node propagating at layer l Represented as ,node The aggregation result of the l-th layer is essentially a fusion of the l-hop neighborhood information of node i, denoted as t-hop neighborhood of node i. It includes only neighbors with a shortest path length of exactly t (excluding shorter paths), where the maximum propagation hop count is L. Calculate the message of its neighbors that have exactly t hops:

[0076]

[0077] in, Let represent the trainable transformation matrix of the l-th layer and the t-th hop. Indicates the relationship with neighboring nodes Attention weights (including one's own) Weight Measuring neighbors Information for the target node Importance. Weight Typically, this is calculated using a shared attention mechanism 'a':

[0078]

[0079] Here, || represents the vector concatenation operation.

[0080] To avoid over-smoothing due to indiscriminate aggregation of hop count information, a hop-specific gating mechanism is introduced. The idea is to adaptively learn the importance of different hop information for each node within each layer. First, the gating score for each hop is calculated:

[0081]

[0082] in, This represents the gated projection matrix, which projects the concatenated input into a low-dimensional space to capture the interaction features between the node's own representation and the corresponding hop message. This represents the hop-specific gating vector, used to map the projected hidden layer features to the score value of that hop, reflecting the learnable weights of different hops. Then, softmax normalization is applied to the scores of all hops to obtain the gating coefficients:

[0083]

[0084] The information from different jumps is then weighted and fused:

[0085]

[0086] Finally, the update representation of the l-th layer node is obtained through residual connection and normalization as follows:

[0087]

[0088] in, For trainable transformation matrices, It is a non-linear activation function.

[0089] At this time, each node Obtain the final representation containing rich contextual and structural information. The first semantic label sequence is determined based on the final representation (second feature) of each node. and the first entity label sequence .

[0090] Understandably, by propagating layer by layer through a multi-layer graph attention network, each node can capture its multi-hop neighborhood information in the first heterogeneous graph. Simultaneously, the multi-hop gating mechanism adaptively weights information with different hop counts, effectively avoiding the oversmoothing problem caused by excessive noise propagation from distant nodes in traditional graph neural networks, and preventing the semantic understanding limitations caused by relying solely on nearest neighbor information. This mechanism ensures that the second feature reflects both the close local relationships between nodes and appropriately integrates long-range dependencies in the global structure, thus providing semantically rich and structure-aware feature representations for accurately predicting the first semantic label and the first entity label. This significantly improves the accuracy and robustness of label extraction in complex multi-turn dialogue scenarios.

[0091] As described above, the operation involves determining the first semantic label and the first entity label based on the second feature. In one feasible approach, this is further referenced... Figure 3 The operation may further include the following operations: inputting the semantic information from the second feature and the first summary information into the first prediction layer to obtain the first semantic label; inputting the entity information from the second feature and the first summary information into the second prediction layer to obtain the first entity label.

[0092] For example, the semantic information in the first summary information can be a high-level abstract feature obtained after semantic parsing the first summary information, used to characterize the intent category, topic tendency, or semantic attribute of the dialogue content. This semantic information does not directly correspond to specific words in the summary, but reflects the semantic framework of the entire dialogue. For example, the semantic information can include abstract representations of dimensions such as the business scenario type to which the dialogue belongs, the core demand category expressed by the user, and the emotional tendency of the dialogue. For example, for the first summary information "User 1 inquired about the price and configuration of the B package cloud server. The customer service informed him that the price was 800 yuan per month, the configuration was 8 cores and 16G memory, and mentioned that there was a 10% discount at present. User 1 then inquired about the comparative advantages with competitor A company," its semantic information can include vector representations of high-level semantic categories such as product consultation intent, price sensitivity attributes, and competitor comparison needs. These representations are extracted by a specific semantic parsing module when the first model encodes the first summary information.

[0093] The entity information in the first summary can be a feature representation of a specific entity object obtained after entity recognition of the first summary information. It is used to identify explicitly referential noun phrases appearing in the dialogue. This entity information directly corresponds to the actual words or phrases in the summary, reflecting the specific things involved in the dialogue. For example, entity information can include vector representations of specific entities such as product names, price values, configuration parameters, company names, and time limits. For instance, for the first summary information mentioned above, its entity information can include the product name entity representation of "Package B", the price value entity representation of "800 yuan", the configuration parameter entity representation of "8 cores and 16GB of memory", the discount entity representation of "10% off", and the company name entity representation of "Competitor Company A". These representations are obtained by locating the entity boundaries in the summary and extracting the corresponding latent state vectors through the entity recognition module during the encoding process of the first model.

[0094] The first prediction layer can be a neural network layer that maps input features to semantic label classification results. This network layer receives second features that fuse graph structure information and semantic information that reflects the semantic framework of the dialogue. Through parameter transformation and activation function processing, it outputs the predicted probability distribution of each semantic label category.

[0095] The second prediction layer can be a neural network layer that maps input features to entity label classification results. This network layer receives second features that fuse graph structure information and entity information that reflects specific entity objects. Through parameter transformation and activation function processing, it outputs the predicted probability distribution of each entity label category.

[0096] The first semantic label can be a set of labels reflecting the high-level semantic categories of the dialogue, obtained by filtering the data after classifying and predicting the nodes in the first heterogeneous graph through the first prediction layer. The labels in this set represent the intent, theme, or semantic attributes of the dialogue and are abstract and generalized. For example, the first semantic label usually does not directly correspond to a specific word in the summary, but rather is a summary and deduplication of the prediction results for multiple related nodes. For instance, after applying the first prediction layer to all nodes in the first heterogeneous graph, the node "price" is predicted as a product inquiry category, the node "10% discount" is predicted as a price objection category, the node "Company A (Competitor)" is predicted as a competitor comparison category, and the node "configuration" is predicted as a configuration consultation category. After summarizing and deduplication, the final first semantic label can include four labels: product inquiry, price objection, competitor comparison, and configuration consultation. These labels collectively summarize the multi-dimensional semantic needs expressed by the user in the first dialogue text.

[0097] The first entity label can be a set of labels reflecting specific entity objects in the dialogue, obtained by filtering the labels after classifying and predicting each node in the first heterogeneous graph through the second prediction layer. The labels in this set directly correspond to the actual words or phrases in the summary, and are specific and referential. For example, the first entity label associates the predicted categories with the corresponding text fragments by locating the position of each node in the summary, forming a structured entity annotation result. For example, after applying the second prediction layer to all nodes in the first heterogeneous graph, the node "Package B" is predicted as the product name category, the node "800 yuan" is predicted as the price category, the node "8 cores 16G memory" is predicted as the configuration parameter category, the node "10% off" is predicted as the discount category, and the node "Company A (Competitor)" is predicted as the company name category. Aligning these prediction results with the summary text, the final first entity label can be represented as: Product Name - Package B, Price - 800 yuan, Configuration Parameters - 8 cores 16G memory, Discount - 10% off, Company Name - Company A (Competitor). These labels accurately identify the specific entity objects and their types involved in the dialogue.

[0098] In one example, we will continue to use the multilayer graph attention network (GAT) with the introduction of a hop-specific gating mechanism as an example.

[0099] At this time, each node Obtain the final representation containing rich contextual and structural information. The final representation of each node is then input into two parallel prediction layers. The entity label prediction layer predicts the entity label of the token:

[0100]

[0101] This represents the final node representation corresponding to the PKE task. and This represents the training parameter weights and bias vector. Similarly, the semantic label prediction layer predicts the semantic label at the location of the token:

[0102]

[0103] This represents the final node representation corresponding to task AKG. and This represents the training parameter weights and bias vector.

[0104] Finally, the predicted semantic label sequence (first semantic label) is output. And the predicted entity label sequence (first entity label) .

[0105] It is understandable that by combining the second feature with the semantic and entity information in the first summary information respectively, and inputting them into their respective independent prediction layers for processing, the task separation and parallel prediction of semantic labels and entity labels are achieved, avoiding feature confusion and prediction conflict caused by a single prediction layer processing two heterogeneous tasks at the same time.

[0106] Figure 4 A schematic diagram illustrating a method for obtaining first summary information according to an embodiment of the present disclosure is shown.

[0107] As described above, in operation S210, the first dialogue text and the first prompt information are input into the large language model to obtain the first summary information. In one possible implementation, such as... Figure 4 As shown, the operation may further include the following steps: segmenting the first dialogue text according to the turn of the speaker to obtain at least one subsequence, where the turn of the speaker is the basic unit of multiple speakers taking turns speaking; determining the topic clusters corresponding to each subsequence, where different topic clusters represent different topics; determining the local summary corresponding to the subsequence and the topic summary corresponding to the topic cluster according to the first prompt information; and determining the first summary information according to the first prompt information, the local summary, and the topic summary.

[0108] For example, a speaker turn can be a complete unit of expression formed when a speaker switches from one party to another during a dialogue. A speaker turn can contain one or more sentences spoken consecutively by that speaker until another speaker begins to speak. For instance, in a dialogue scenario between a user and customer service, multiple related questions asked consecutively by the user constitute a user speaker turn, and subsequent responses given consecutively by the customer service constitute a customer service speaker turn. For example, in the original dialogue record corresponding to the first dialogue text, User 1 first says, "Hello, I would like to learn about your cloud server products" and "I am mainly concerned about the price and configuration information." These two sentences are consecutive in time and are both spoken by User 1, thus constituting the first user speaker turn. Subsequently, the customer service responds with three sentences: "Hello, we currently mainly promote Package B cloud servers," "The price is 800 yuan per month," and "Configured with 8 cores and 16GB of memory." These three sentences are consecutively spoken by the customer service, constituting the first customer service speaker turn. User 1 then continued to ask, "Are there any current promotional activities?" and "What advantages does our product have compared to competitor A?", forming the second round of user responses. Customer service responded again, "We currently have a 10% discount" and "Our product has advantages in stability and after-sales service," forming the second round of customer service responses.

[0109] A subsequence can be a continuous dialogue segment obtained by dividing the first dialogue text according to the speaker's turn. Each subsequence contains several speaker turns, and these turns are semantically relatively complete or related. For example, the division of subsequences can be based on criteria such as natural pauses in the dialogue, topic transitions, or semantic completeness, so that each subsequence can independently express a relatively complete interaction process. For example, after dividing the above first dialogue text according to the speaker's turn, two subsequences can be obtained. The first subsequence contains the first two speaker turns, i.e., User 1 inquires about cloud server products and their concerns, and the customer service introduces the price and configuration of Package B. This subsequence focuses on consulting and informing about basic product information. The second subsequence contains the last two speaker turns, i.e., User 1 inquires about promotional activities and competitor comparisons, and the customer service responds with a 10% discount and product advantages. This subsequence focuses on exchanging information about promotions and competitor comparisons. The two subsequences form a progressive relationship in terms of topics, from basic consultation to in-depth comparison.

[0110] A topic cluster can be a set of topic categories obtained by semantic clustering analysis of subsequences. Subsequences or dialogue fragments within the same topic cluster revolve around the same or similar topics, while there are significant differences in topics between different topic clusters. For example, topic clusters can be determined using methods based on keyword extraction, semantic vector similarity calculation, or topic models, grouping semantically similar subsequences into the same topic cluster.

[0111] A local summary can be a concise summary text generated by a large language model based on the first cue information for a single subsequence. This text extracts the core information from the subsequence, removes redundant expressions, and retains key interactive content. For example, the local summary generation process can involve concatenating the subsequence text with the first cue information and inputting it into the large language model. The model then outputs a compressed representation of the subsequence based on the cue requirements. For instance, for the first subsequence, the text "User 1: Hello, I would like to learn about your cloud server product. User 1: I am mainly concerned about the price and configuration information. Customer Service: Hello, we are currently promoting the B package cloud server. Customer Service: The price is 800 yuan per month. Customer Service: The configuration is 8 cores and 16GB of memory" is concatenated with the first cue information and input into the large language model. The local summary generated by the model could be "User 1 inquired about the price and configuration of the cloud server product. Customer service introduced that the B package price is 800 yuan per month, and the configuration is 8 cores and 16GB of memory." For the second subsequence, the text "User 1: Are there any promotional activities now? User 1: What are the advantages compared to competitor A? Customer Service: There is a 10% discount currently. Customer Service: Our product has advantages in stability and after-sales service" can be processed in the same way to generate a partial summary that reads "User 1 inquires about promotional activities and comparisons with competitor A; customer service informs them of a 10% discount and emphasizes the advantages in stability and after-sales service."

[0112] A topic summary can be a topic-level summary text generated by a large language model, using the first cue information to guide all subsequences within the same topic cluster. This text abstracts and integrates the core semantics of the topic cluster at a higher level, highlighting the overall characteristics of the topic. For example, the topic summary generation process can merge multiple subsequences or partial summaries belonging to the same topic cluster, concatenate them with the first cue information, and input them into the large language model. The model outputs an abstract summary of the topic cluster. For instance, for a product consultation topic cluster, if the topic cluster contains only the first subsequence, then the partial summary or original text of that subsequence is concatenated with the first cue information and input into the model. The generated topic summary could be "The dialogue involves basic information consultation about Cloud Server Package B, including the price of 800 yuan and the configuration of 8 cores and 16GB of memory." For a promotion and comparison topic cluster, if the topic cluster contains a second subsequence and other possible subsequences about discounts or competitor comparisons, then the partial summaries of these subsequences are merged and input into the model. The generated topic summary could be "The dialogue involves inquiries about a 10% discount and a comparison of the advantages of competitor A in terms of stability and after-sales service." If the dialogue also includes service consultation topics, its topic summary could be: The dialogue involves consultation on after-sales service policies and safeguards.

[0113] The first summary information can be a comprehensive summary text generated by a large language model, combining all local summaries and topic summaries and using the first prompt information. This text, while retaining the core information of each subsequence, integrates the semantic content of different topic clusters to form a complete and concise representation of the entire first dialogue text. For example, the generation process of the first summary information can arrange all local summaries and topic summaries in chronological order or topic logic, concatenate them with the first prompt information, and input them into the large language model. The model performs information fusion and redundancy elimination to output the final summary text. For example, the local summary of the first subsequence above, "User 1 inquires about the price and configuration of cloud server products. Customer service introduces Package B, which costs 800 yuan per month and has an 8-core and 16GB memory configuration," the local summary of the second subsequence, "User 1 inquires about promotional activities and comparisons with competitor A. Customer service informs users of a 10% discount and emphasizes the advantages of stability and after-sales service," the product consultation topic summary, "The dialogue involves basic information about Package B cloud servers, including the price of 800 yuan and the configuration of 8 cores and 16GB memory," and the promotion and comparison topic summary, "The dialogue involves a 10% discount..." The phrase "user 1 inquired about the price and configuration of cloud server package B, and customer service informed them that the price was 800 yuan per month, with a configuration of 8 cores and 16GB of memory, and mentioned a current 10% discount. User 1 then inquired about the comparative advantages with competitor A, and customer service emphasized the advantages in stability and after-sales service." This summary not only covers the key information points of each subsequence but also reflects the overall semantic framework of the two topic clusters of product inquiry and promotional comparison, providing a clearly structured and complete input for subsequent feature extraction and label prediction.

[0114] In one example, input long text dialogue records from different scenarios. (First dialogue text).

[0115] Hierarchical input segmentation: The dialogue is divided into subsequences based on user and agent turn:

[0116]

[0117] in, This indicates the m-th turn, where M is the total number of turns. Indicates the first Statements, This indicates the starting statement number of this turn. .

[0118] Subsequently, in each speech wheel sequence Internally, using the topic assignment function Map statements to topic clusters k , here This is achieved through semantic embedding similarity calculation, thus forming a hierarchical structure of "turn-to-topic-global".

[0119] Hierarchical summary generation: The first prompt and subsequence are input into the large language model to obtain a local summary; the first prompt and topic clusters are input into the large language model to obtain a topic summary. Local summaries are generated within each talk round, and then topic summaries are generated based on each topic cluster. The local summaries and topic summaries are combined to generate the first summary information.

[0120] This method effectively compresses redundant information and ensures that the summary covers the core content globally. Here, a local summary and a topic summary are generated for each round of question and answer. Then, the summary results of multiple rounds of question and answer in a dialogue are integrated to obtain the final summary (first summary information).

[0121] In the LLM summary generation process, a thought chain structure is adopted, forcing the model to interpret before generating. This includes three stages: First, key facts are extracted, identifying key information and its rationale in successive rounds. Next, the facts are summarized into themes, and the meaning of the themes is explained. Finally, the themes are integrated into a summary, with an explicit reasoning chain designed to force the model to output intermediate reasoning steps, avoiding the direct generation of a one-sided summary.

[0122] The summary is input into a pre-trained Unified Language Model (UNILM). UNILM is a Tranformer-based pre-trained model widely used in end-to-end natural language processing tasks. This approach imposes specific constraints on UNILM: the internal ALG latent state generation can capture PLE information, but PLE does not capture ALG information. In the UNILM framework, each word is first projected into an initial word vector e through an embedding matrix. w These word vectors are then encoded using multiple Transformer layers to obtain context-enhanced hidden states.

[0123] Inputting the summary into the UNILM model yields a context-aware hidden state (first feature) representation. Where n is the number of tokens in the digest, and these hidden states It integrates the generated semantics and the extracted entity information.

[0124] Hidden state sequence Input the first model, where PLE 0 =ALG 0 ={ }, PLE 0and ALG 0 This represents the initial representation of the PLE and ALG tasks.

[0125] Node definition: Represent PLE and ALG tasks as hidden states. Defined as a graph node ,Right now .

[0126] Edge Definition: Define the connection relationships (edges E) between nodes. The following strategies can be adopted: Connect the internal edges of hidden nodes representing task PLE using a sliding window; if two nodes are close in the text, an edge is established, similar to the local dependency assumption of n-grams, ensuring that adjacent or nearest neighbor labels can pass information to each other; connect the internal edges of hidden nodes representing task ALG sequentially; and calculate node features by connecting the two across task edges. and Edges are established between node pairs whose similarity exceeds a threshold m0. Whether in outbound calling or large customer list scenarios, edges are prioritized between customer identity entity nodes and procurement demand tag nodes.

[0127] Based on the node set V and the edge set E, construct the graph structure G=(V,E) of the heterogeneous graph.

[0128] Understandably, segmenting the first dialogue text according to the turn of the speakers decomposes the long dialogue into multiple semantically complete subsequences, effectively reducing the text length and complexity of a single processing iteration and avoiding information omissions and semantic shifts that occur when large language models handle extremely long dialogues. By performing topic clustering on the subsequences, multiple different topics involved in the dialogue are identified, achieving a structured understanding of complex multi-topic dialogues, enabling the summary generation process to distinguish and retain the core semantics of each topic. By generating local summaries and topic summaries separately, both accurate extraction of detailed information from individual subsequences and high-level abstract integration of multiple subsequences under the same topic are ensured, forming a multi-level summary system from local to global and from concrete to abstract. By inputting the local summaries and topic summaries together with the first prompt information into the large language model for fusion, it is ensured that the first summary information covers all key information while eliminating redundancy and repetition between different levels of summaries. The final output summary text is both concise and maintains completeness and structure.

[0129] As described above, the first model in this embodiment can be obtained through the following operations: The first model is obtained by: acquiring multiple sample data, including a second dialogue text and a second label corresponding to the second dialogue text, the second label including a second semantic label and a second entity label; determining second summary information based on the second dialogue text and the second prompt information, the second prompt information being used to guide the generation of the second summary information; inputting the second summary information into the first model to obtain a third feature, the third feature representing the context information corresponding to each word in the second summary information; constructing a second heterogeneous graph based on the third feature, the second heterogeneous graph containing at least two nodes and edges connecting the nodes of the two heterogeneous graphs, the nodes representing the third features of words in the second summary information, and the edges representing that the two connected nodes are semantically related; based on the second heterogeneous graph, using a multi-layer graph attention network to determine the predicted label corresponding to the second dialogue text, the predicted label including a predicted semantic label and a predicted entity label; adjusting the parameters of the first model based on the difference information between the second label and the predicted label to obtain the trained first model.

[0130] In one example, during the training phase, the system can retrieve multiple labeled sample data from a historical customer service dialogue database. For instance, the second dialogue text of one of the sample data might be:

[0131] User 2: Hello, I would like to inquire about enterprise email services.

[0132] Customer Service: Hello, we offer the C package corporate email.

[0133] User 2: What's the price?

[0134] Customer service: Package C costs 1200 yuan per year.

[0135] User 2: How many accounts are included?

[0136] Customer service: Includes 50 email accounts, each with 5GB of storage space.

[0137] User 2: How does your product differ from Company B's?

[0138] Customer service: Our products offer greater assurance in terms of security and speed of technical support response.

[0139] The second label corresponding to the sample data may include: the second semantic label is product consultation, price inquiry, and function comparison; the second entity label is product name - Package C, price - 1200 yuan, configuration parameters - 50 email accounts, configuration parameters - 5G storage space, and company name - Company B.

[0140] The system inputs the second dialogue text and the second prompt message, "Please summarize the following customer service dialogue into a concise summary, retaining key elements such as product name, price, configuration parameters, and competitor information," into the large language model to obtain the second summary information: "User 2 inquired about the price and configuration of Package C enterprise email. Customer service informed them that the price is 1200 yuan per year, including 50 email accounts and 5G storage space. User 2 asked about the differences between the product and Company B's product. Customer service emphasized the advantages of security and technical support response speed."

[0141] The second summary information is input into the initial first model (e.g., a pre-trained BERT model). The model encodes each word in the summary to obtain the third feature. For example, the third feature of the word "C package" is a 768-dimensional vector that encodes the semantic information of "C package" in the summary context, including its association as a product name with words such as "corporate email," "price," and "1200 yuan." The third feature of the word "1200 yuan" encodes its contextual dependence as a price value with words such as "C package" and "annual."

[0142] Based on these third features, the system constructs a second heterogeneous graph. For example, the graph contains nodes such as "Package C," "Enterprise Email," "Price," "1200 Yuan," "50 Email Accounts," "5G Storage Space," "Company B," "Security," and "Technical Support," with each node carrying the third feature of the corresponding word. The system calculates the semantic relevance between nodes. For instance, "Package C" and "Enterprise Email" have a strong association due to their direct description of the product category; "Price" and "1200 Yuan" have a strong association due to their attribute-value relationship; "Package C" and "50 Email Accounts" have a moderately strong association due to their product-configuration relationship; and "Company B" and "Security" have an association due to the context of competitor comparison. The final constructed second heterogeneous graph contains 9 nodes and 15 edges.

[0143] The second heterogeneous graph is input into a multi-layer graph attention network for processing. In the first layer of the graph attention network, the node "Price" aggregates information from its one-hop neighbors "Package C" and "1200 yuan". The node "Package C" aggregates information from its one-hop neighbors "Corporate Email", "Price", and "50 Email Accounts". After applying a multi-hop gating mechanism, the system assigns a gating coefficient of 0.75 to the one-hop information. In the second layer of processing, the node "Price" further integrates information from its two-hop neighbors "Corporate Email", "50 Email Accounts", and "5G Storage Space". The gating mechanism assigns a coefficient of 0.6 to the one-hop information and a coefficient of 0.4 to the two-hop information. After the third layer of processing, the final representation of the node "Price" incorporates information from remote nodes within the three-hop range, including "Company B" and "Security", with gating coefficients of 0.5 (one hop), 0.35 (two hops), and 0.15 (three hops), respectively.

[0144] After processing by a multi-layer graph attention network, the system obtains the fourth features of all nodes in the second heterogeneous graph (corresponding to the second features in the inference stage). These fourth features, along with the semantic and entity information from the second summary information, are input into the first and second prediction layers, respectively. The first prediction layer outputs predicted semantic labels: for the node "price," it is predicted as a product consultation category (probability 0.7) or a price inquiry category (probability 0.2); for the node "Company B," it is predicted as a function comparison category (probability 0.8). The summaries yield the predicted semantic labels as product consultation, price inquiry, and function comparison. The second prediction layer outputs predicted entity labels: the node "Package C" is predicted as the product name category (probability 0.92), the node "1200 yuan" is predicted as the price category (probability 0.94), the node "50 email accounts" is predicted as the configuration parameter category (probability 0.88), the node "5G storage space" is predicted as the configuration parameter category (probability 0.85), and the node "Company B" is predicted as the company name category (probability 0.91). The final predicted entity labels are: Product Name - Package C, Price - 1200 yuan, Configuration Parameter - 50 email accounts, Configuration Parameter - 5G storage space, and Company Name - Company B.

[0145] The system compares the predicted label with the second label. In this sample, the predicted semantic label and the second semantic label are completely identical, and the predicted entity label and the second entity label are also completely identical. The calculated loss value is relatively small, for example, the cross-entropy loss is 0.08. Based on this loss value, the system calculates the gradient through the backpropagation algorithm and updates the parameters of the pre-trained encoder in the first model, the weight parameters of each layer of the graph attention network, the gating parameters of the multi-hop gating mechanism, and the classification parameters of the first and second prediction layers.

[0146] In another sample dataset, the second dialogue text involves user 3 inquiring about database services. The second tag includes semantic tags for product inquiry, performance inquiry, and deployment method inquiry, and entity tags for product name - Package D, configuration parameters - 4 cores 8GB, and deployment method - cloud deployment. After the same summary generation, feature extraction, graph construction, and tag prediction process, the system predicts the semantic tags as product inquiry and performance inquiry, but lacks the deployment method inquiry category. The predicted entity tags are product name - Package D and configuration parameters - 4 cores 8GB, but lack the deployment method - cloud deployment. The system calculates a relatively large loss value, for example, 0.65. This loss value generates a large gradient signal, which significantly adjusts the parameters of the first model. In particular, it enhances the feature extraction capability of words related to deployment methods, increases the attention weight of semantic nodes related to deployment methods and surrounding nodes, improves the sensitivity of the first prediction layer to the recognition of deployment method inquiry categories, and optimizes the classification accuracy of the second prediction layer for deployment method entities.

[0147] The system repeats the above process, processing thousands of samples in the training dataset. The difference between the second label and the predicted label for each sample is used to adjust the parameters of the first model. As the number of training iterations increases, the average loss between the predicted and second labels gradually decreases from the initial 1.2 to 0.15, indicating that the model parameters are gradually converging to the optimal state. When the loss on the validation set no longer decreases for 10 consecutive iterations, the system determines that training is complete and obtains the final first model.

[0148] As described above, the text processing method of this embodiment may further include the operation of: determining a tag prototype vector based on a third feature, wherein the tag prototype vector represents the vector representation of the tags contained in the second summary information.

[0149] For example, the label prototype vector can be a set of vector representations obtained based on the third feature through specific vector extraction and clustering operations. Each vector corresponds to a potential label category, used to characterize the typical feature patterns of that category in the vector space. For instance, the process of determining the label prototype vector can employ a prototype learning method, which aggregates semantically similar feature vectors in the third feature space into prototype representations of the same category.

[0150] In one example, during model training, the hidden state representation (third feature) corresponding to the second dialogue text is obtained. Subsequently, the label prototype vector is learned through linear transformation. :

[0151] n

[0152] in, The training matrix is ​​used to make the label prototype vector closer to the positive sample span vector corresponding to the Summary output after step one through contrastive loss, and to align it with the bag-of-words distribution vector. The positive sample span vector is obtained by mapping the pseudo-annotated fragments generated by LLM based on the summary to the context semantic representation after being encoded by the UNILM encoder. This is used to replace manually annotated data and expand the diversity and coverage of training samples.

[0153] As described above, the operation involves adjusting the parameters of the first model based on the difference between the second label and the predicted label to obtain the trained first model. In one possible implementation, this operation may further include the following steps: determining a first loss based on the second semantic label and the predicted semantic label; determining a second loss based on the second entity label and the predicted entity label; determining a target loss based on the first loss and the second loss; iteratively calculating the gradient of the target loss with respect to the parameters of the first model using a backpropagation algorithm, updating the parameters of the first model in the direction of the gradient, until the target loss converges, thus obtaining the trained first model.

[0154] In one example, the second loss can be the standard entity label prediction loss, as follows:

[0155]

[0156] Where n is the length of the digest token, and C represents the number of entity tags. This indicates the weight of the entity label.

[0157] The first loss can be the standard semantic label prediction loss, as follows:

[0158]

[0159] Where m is the length of the tagged token, and Y represents the number of semantic tags. This indicates the weight of the semantic label.

[0160] Combining the first and second losses above, we arrive at the final optimization objective:

[0161]

[0162] As described above, the operation involves determining a target loss based on a first loss and a second loss. In one possible implementation, this operation may further include the following steps: determining a target bag-of-words distribution based on second summary information, where the target bag-of-words distribution represents the word frequency statistics in the second summary information; determining a predicted bag-of-words distribution based on predicted labels, where the predicted bag-of-words distribution represents the word frequency statistics in the predicted labels; determining a third loss based on the target bag-of-words distribution and the predicted bag-of-words distribution; and determining the target loss based on the first loss, the second loss, and the third loss.

[0163] In one example, relying solely on local context and multi-layered GAT propagation, the model may underlearn or mispredict certain semantic or entity labels that appear infrequently in the summary but are actually important in the dialogue, and may fail to update the label count. Therefore, a bag-of-words constraint loss (BagofWordsConstraintLoss) is introduced as an auxiliary supervision signal. Given a summary of a conversation text, its word sequence is... We construct a bag-of-words vector B(w) representing the global semantic distribution to ensure that the model output is consistent with the global semantic statistics.

[0164] (1) Construct the target bag-of-words distribution B(w):

[0165] First, calculate the weighted word frequency:

[0166]

[0167] in Indicates an indicator function, Map subwords to vocabulary entries. The position weight of the r-th word:

[0168]

[0169] in, The value is usually 1; N is the number of call samples, and df(w) is the number of conversation samples containing w; Score sentences based on their importance.

[0170] The sample-level target bag-of-words distribution B(w) is obtained after smoothing and normalization:

[0171]

[0172] Where V is the defined vocabulary. is the Laplace smoothing constant.

[0173] (2) Constructing the predicted bag-of-words distribution

[0174] Then, the final node prediction results of the GAT model are used. and Let them constitute the predicted label set T, and let... The overall predicted probability of label t is:

[0175]

[0176] in, It is a node The predicted probability for label t. Therefore, when t corresponds to the PLE task, When corresponding to ALG tasks, ; This represents the set of nodes associated with t.

[0177] To map the probabilities of the "label space" to the "vocabulary space", we define a label-word association matrix A, based on the label prototype vector p from step two. t Each word has a corresponding word vector e w Using associated temperature Cosine similarity generates soft associations:

[0178]

[0179]

[0180] in, Indicates the associated temperature; The association between the calculated words and tags; The unnormalized score for word w.

[0181] right Temperature-based softmax is used to obtain the predicted bag-of-words distribution. :

[0182]

[0183] Then, the difference between the predicted bag-of-words vector and the target bag-of-words vector is calculated as the constraint loss (third loss):

[0184]

[0185] Where B(w) represents the actual bag-of-words vector, and w is the corresponding word.

[0186] Combining the first, second, and third losses above, we arrive at the final optimization objective:

[0187]

[0188] As described above, the operation involves determining the target loss based on the first loss, second loss, and third loss. In one possible implementation, this operation may further include the operations of: determining word vectors based on predicted labels; determining a fourth loss based on the label prototype vectors and word vectors; and determining the target loss based on the first loss, second loss, third loss, and fourth loss.

[0189] In one example, the fourth loss can be the standard semantic label prediction loss, as follows:

[0190]

[0191] Where m is the length of the masked token, and Y represents the number of semantic tags. This indicates the weight of the semantic label.

[0192] Combining the above losses with the two losses from step five, we arrive at the final optimization objective:

[0193] +

[0194] Wherein, γ and It is a global weight used to balance the bag-of-words consistency loss term and prototype alignment.

[0195] Understandably, during the model training phase, the bag-of-words constraint loss function and the label prototype alignment loss function are introduced to make the model prediction results fit the global semantic distribution, while enhancing the recognition performance of low-frequency and semantically key labels.

[0196] Figure 5 A schematic diagram of a text processing method according to an embodiment of the present disclosure is shown.

[0197] Reference Figure 5 The text processing method of this disclosure, while retaining the task decomposition idea of ​​the UniKeyphrase framework, proposes a complete processing flow, specifically including the following steps:

[0198] First, the input text is segmented into layers and combined with the CoT (Cooperation of Thought) prompt design to guide LLM to generate summary text with comprehensive coverage and higher semantic consistency. At the same time, based on the existing labeled dataset, pseudo-label data generated by LLM is introduced to expand the scale and diversity of training samples and enhance the generalization ability of the model.

[0199] Secondly, the obtained summary is input into a UNILM-based encoder to extract the latent state representations of semantic and entity tags, and a tag prototype vector is constructed through aggregation operations, which serves as the stable representation basis for tag semantics.

[0200] Subsequently, based on the previous work, a new Semantic Graph Attention Network (S-GAT) was innovatively built, including the following sub-steps:

[0201] (1) Based on the aforementioned tag prototypes and summary semantic information, a heterogeneous graph structure containing multiple types of nodes is constructed;

[0202] (2) A graph attention network is used for multi-hop semantic information propagation. A hop-specific gating mechanism is introduced during the propagation process. This mechanism can adaptively adjust the contribution of shallow and deep neighbor nodes to the representation of the central node, thereby effectively suppressing the over-smoothing phenomenon and noise diffusion problem.

[0203] (3) Complete the label prediction for each node in the graph, which significantly improves the accuracy and robustness of the label extraction task.

[0204] Finally, during the model training phase, a bag-of-words constraint loss function and a label prototype alignment loss function are introduced to make the model prediction results fit the global semantic distribution, while enhancing the recognition performance of low-frequency and semantically key labels.

[0205] It is understood that the text processing method (1) of this disclosure proposes to combine hierarchical input segmentation and thought chain construction prompt words to guide LLM to generate high-quality summaries. This method can efficiently process long text input, significantly alleviate the problems of input length limitation, semantic incoherence and factual illusion, and retain the core information of the text, thereby effectively solving the problem of direct processing of long dialogues. (2) It proposes a method to use the UNILM framework to explicitly distinguish the hidden states of entity label and semantic label tasks and aggregate label prototype variables. Then, it innovatively designs a semantic graph attention network (S-GAT) to explicitly model the dependency relationship between labels. The introduced Hop-Specific gating mechanism controls multi-hop information propagation, avoids noise diffusion, enhances the robustness of label prediction, and achieves accurate joint extraction. (3) It designs a joint loss of bag-of-words constraint and label prototype constraint to ensure that the prediction results are consistent with the global semantic distribution. This method enhances the ability to identify low-frequency but key labels and improves the stability and interpretability of the model output. (4) This method shows good adaptability in outbound call business and large customer list scenarios. It can flexibly adjust the graph structure construction and loss design according to multi-dimensional needs such as purchasing intention and customer attitude, thereby improving the adaptability and practicality of the model in specific business scenarios.

[0206] To facilitate the explanation of the implementation effect of the text processing method of this disclosure embodiment, the following will be explained in conjunction with test results:

[0207] Setup: This scheme reuses most of the hyperparameters from the pre-trained UNILM3. The number of layers in GAT is set to 2. γ is used when adjusting the weights of BWC. =0.99. The PLE and ALG loss weights were set to 3.0. The BatchSize was set to 256, and the MaximumLength to 384. Beamsearch was used for the ALG task during decoding, with a beamsize of 5. The model was trained on the training set for 100 epochs. Mixed-precision training was performed using eight Nvidia Tesla V100 GPUs, with each epoch taking approximately 30 minutes.

[0208] (1) Evaluation metrics: The performance of the model was evaluated using traditional text classification metrics: precision (P), recall (R), and F1 score (including F1@m, where m represents the preset number of samples), which respectively measure the extraction performance of entity tag extraction and semantic tag extraction tasks. Meanwhile, the multi-hop relation metric was used to evaluate the effect of GAT network fusion in the ablation experiment, and the robustness metric was used to evaluate the model's scalability. Specific supplementary metrics are detailed in the table below:

[0209]

[0210] (2) Test dataset: It mainly consists of two parts: one part is real interaction data, which mainly includes the actual outbound call recordings of enterprises, including customer consultation, multi-round question and answer in casual conversation, cross-topic switching and other scenarios, as well as dynamic business update datasets, including newly added business documents of enterprises and some noisy text datasets. The other part is industry standard comparison datasets, including publicly available customer service dialogue datasets (ATIS, SNIPS).

[0211] (3) Ablation experiments: The scheme mainly compared the performance of the method without using LLM to extract summaries and without GAT to construct multi-hop relations with the benchmark method on two datasets:

[0212] Model Real-world interactive entity task dataset F1@m Real-world interactive semantic task dataset F1@m Publicly spliced ​​dataset entity task F1@m Publicly spliced ​​dataset semantic task F1@m Benchmark method w / oLLM-Abstract 36.135.3 35.432.1 44.342.1 45.644.3 w / oHop-specificGAT 34.2 29.8 40.9 43.2

[0213] It can be seen that the benchmark method performs better than other methods, which also proves that using LLM to extract summaries can effectively avoid the problem of training long text networks and that GAT with the introduction of a Hop-specific mechanism can mine entity labels and semantic labels more deeply.

[0214] (4) Comparative experiment:

[0215]

[0216] It can be seen that before the introduction of the Hop-specific mechanism, the performance of different GAT networks degraded when the number of layers exceeded three, and excessive interaction would lead to a loss of task diversity. However, with the introduction of the Hop-specific mechanism, the baseline method did not experience a smooth transition as the number of layers increased, and the overall performance was better.

[0217] Furthermore, the text processing method of this disclosure belongs to an end-to-end model, which uses an LLM-related generative model to extract entity and semantic tags, and compares the quality of entity and semantic tag extraction by combining some keyword extraction and intent recognition multi-task ensemble models.

[0218] Model Real-world interactive entity task dataset P Real-world interactive semantic task dataset P Public entity splicing task dataset R Public Semantic Concatenation Task Dataset R Adding 20% ​​noise to real entity task datasets noise tolerance Adding 20% ​​noise to a real semantic task dataset with noise tolerance Benchmark method for generating models 66.163.3 57.356.4 65.462.1 54.255.1 1.121.01 1.131.03 ensemble model 64.2 56.8 59.8 50.1 1.02 1.03

[0219] The table above shows that the proposed method is not only more accurate, but also more robust, and performs better on datasets with added noise.

[0220] Figure 6 A block diagram schematically illustrates an electronic device suitable for implementing a text processing method according to an embodiment of the present disclosure.

[0221] like Figure 6 As shown, an electronic device 400 according to an embodiment of the present disclosure includes a processor 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage portion 408 into a random access memory (RAM) 403. The processor 401 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 401 may also include onboard memory for caching purposes. The processor 401 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0222] RAM 403 stores various programs and data required for the operation of electronic device 400. Processor 401, ROM 402, and RAM 403 are interconnected via bus 404. Processor 401 executes various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 402 and / or RAM 403. It should be noted that programs may also be stored in one or more memories other than ROM 402 and RAM 403. Processor 401 may also execute various operations of the method flow according to embodiments of the present disclosure by executing programs stored in one or more memories.

[0223] According to embodiments of this disclosure, the electronic device 400 may further include an input / output (I / O) interface 405, which is also connected to a bus 404. The electronic device 400 may also include one or more of the following components connected to the I / O interface 405: an input section 406 including a keyboard, mouse, etc.; an output section 407 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN card, modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. A removable medium 411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 410 as needed so that computer programs read from it can be installed into the storage section 408 as needed.

[0224] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0225] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A text processing method, comprising: The first dialogue text and the first prompt information are input into the large language model to obtain the first summary information. The first prompt information is used to guide the generation of the first summary information. Based on the first summary information, a first feature is determined using a first model, wherein the first feature represents the context information corresponding to each word in the first summary information; Based on the first feature, a first heterogeneous graph is constructed, the heterogeneous graph containing at least two nodes and edges for connecting the nodes of the two heterogeneous graphs, the nodes representing the first feature of words in the first summary information, and the edges representing that the two connected nodes are semantically related; Based on the first heterogeneous graph, a first label corresponding to the first dialogue text is determined using a multi-layer graph attention network. The first label includes a first semantic label and a first entity label.

2. According to the method of claim 1, the Nth layer graph attention network is used to process node information within N hops corresponding to each node, where the hop count represents the number of edges included in the path between two nodes, and N is an integer greater than 1. Each layer of the graph attention network embeds the multi-hop gating mechanism. Based on the first heterogeneous graph, a multi-layer graph attention network is used to determine the first tag corresponding to the first dialogue text, including for any node in the first heterogeneous graph: The first feature of any node is used as the initial input. The output of the (N-1)th layer graph attention network and the image information of the first heterogeneous graph are sequentially input into the Nth layer graph attention network to obtain the output of the Nth layer graph attention network until the second feature is output. The output of the Nth layer of the target node is the fusion of the N-hop neighbor node information of the target node. The hop count represents the number of edges included in the path between two nodes, and N is an integer greater than 1. The multi-hop gating mechanism is used to assign different weights to node information with different hop counts relative to the target node, and the second feature is the result of fusing the N-hop neighbor node information of each node. Based on the second feature, the first semantic label and the first entity label are determined.

3. The method according to claim 2, wherein determining the first semantic tag and the first entity tag based on the second feature includes: The second feature and the semantic information in the first summary information are input into the first prediction layer to obtain the first semantic label; The second feature and the entity information in the first summary information are input into the second prediction layer to obtain the first entity label.

4. The method according to claim 1, wherein the first dialogue text and the first prompt information are input into a large language model to obtain the first summary information, comprising: The first dialogue text is segmented according to the turn of the speaker to obtain at least one sub-sequence, wherein the turn of the speaker is the basic unit in which multiple speakers speak alternately. Identify the topic clusters corresponding to each subsequence; different topic clusters represent different topics. Based on the first prompt information, determine the local summary corresponding to the sub-sequence and the topic summary corresponding to the topic cluster; The first summary information is determined based on the first prompt information, the partial summary, and the topic summary.

5. The method according to claim 1, wherein the first model is obtained through the following operations: Multiple sample data are obtained, the sample data including a second dialogue text and a second tag corresponding to the second dialogue text, the second tag including a second semantic tag and a second entity tag; Based on the second dialogue text and the second prompt information, the second summary information is determined, and the second prompt information is used to guide the generation of the second summary information; The second summary information is input into the first model to obtain the third feature, which represents the context information corresponding to each word in the second summary information. Based on the third feature, a second heterogeneous graph is constructed. The second heterogeneous graph contains at least two nodes and edges for connecting the nodes of the two heterogeneous graphs. The nodes represent the third feature of words in the second summary information, and the edges represent that the two connected nodes are semantically related. Based on the second heterogeneous graph, a multi-layer graph attention network is used to determine the predicted label corresponding to the second dialogue text. The predicted label includes a predicted semantic label and a predicted entity label. The parameters of the first model are adjusted based on the difference information between the second label and the predicted label to obtain the trained first model.

6. The method according to claim 5, further comprising: Based on the third feature, a tag prototype vector is determined, which represents the vector representation of the tags contained in the second summary information.

7. The method according to claim 5, wherein the parameters of the first model are adjusted based on the difference information between the second label and the predicted label to obtain a trained first model, comprising: The first loss is determined based on the second semantic label and the predicted semantic label; The second loss is determined based on the second entity label and the predicted entity label; Based on the first loss and the second loss, determine the target loss; The gradient of the target loss with respect to the first model parameters is iteratively calculated using the backpropagation algorithm, and the first model parameters are updated according to the gradient direction until the target loss converges, thus obtaining the first model that has been trained.

8. The method according to claim 7, wherein determining the target loss based on the first loss and the second loss includes: Based on the second summary information, the target bag-of-words distribution is determined, wherein the target bag-of-words distribution characterizes the word frequency statistics results in the second summary information; Based on the predicted labels, the predicted bag-of-words distribution is determined, whereby the predicted bag-of-words distribution characterizes the word frequency statistics results in the predicted labels; The third loss is determined based on the target bag-of-words distribution and the predicted bag-of-words distribution; The target loss is determined based on the first loss, the second loss, and the third loss.

9. The method according to claim 8, wherein determining the target loss based on the first loss, the second loss, and the third loss comprises: Determine word vectors based on the predicted labels; The fourth loss is determined based on the original tag vector and the word vector; The target loss is determined based on the first loss, the second loss, the third loss, and the fourth loss.

10. An electronic device, comprising: A processor is configured to input a first dialogue text and a first prompt message into a large language model to obtain first summary information, wherein the first prompt message is used to guide the generation of the first summary information; based on the first summary information, a first feature is determined using a first model, wherein the first feature represents the context information corresponding to each word in the first summary information; based on the first feature, a first heterogeneous graph is constructed, wherein the first heterogeneous graph contains at least two nodes and edges for connecting nodes of two heterogeneous graphs, wherein the edges represent that the two nodes are semantically related; based on the first heterogeneous graph, a first label corresponding to the first dialogue text is determined using a multi-layer graph attention network, wherein the first label includes a first semantic label and a first entity label.

Images