Live semantic understanding driven intelligent automatic answering system

Abstract

This invention relates to the field of natural language processing, specifically to an intelligent automatic response system driven by live semantic understanding. The system includes a multimodal live semantic stream construction module, a dynamic semantic sparsity module, a semantic structure mapping module, a dialogue state tracking module, a dynamic knowledge fusion module, and a generative intelligent response module. Through the dynamic semantic sparsity module and the semantic structure mapping module, this invention achieves information entropy analysis, contribution evaluation, and low-dimensional structure mapping of live semantic tokens, obtaining a compressed and structured set of key semantics. This significantly reduces redundant information interference and improves the accuracy and real-time performance of semantic understanding. By using contrastive learning and slot correlation analysis methods, the system performs slot association scoring on key semantic tokens and generates dynamic dialogue states. Combined with contextual compensation, this forms an enhanced semantic representation, supporting the generative intelligent response module to output more coherent and personalized responses.

Description

Technical Field

[0001] This invention relates to the field of natural language processing, specifically to an intelligent automatic response system driven by live semantic understanding. Background Technology

[0002] With the rapid development of live-streaming e-commerce, interactive live-streaming, and online education, existing automatic response systems mainly rely on template matching or simple keyword-based searches, which presents the following problems:

[0003] (1) Insufficient semantic understanding of multi-source heterogeneous live streaming data, making it difficult to accurately capture user intent and contextual information;

[0004] (2) The dynamic tracking of the live dialogue status and the semantic slot modeling are not perfect, resulting in the lack of contextual continuity and personalization of the generated responses;

[0005] (3) It has weak ability to extract key semantic information and filter redundant information, and is prone to noise interference, which affects the quality and efficiency of response. Summary of the Invention

[0006] To address the above issues and overcome the shortcomings of existing technologies, this invention provides an intelligent automatic response system driven by live semantic understanding. This invention utilizes a dynamic semantic sparsity module and a semantic structure mapping module to perform information entropy analysis, contribution evaluation, and low-dimensional structure mapping of live semantic tokens, resulting in a compressed and structured set of key semantics. This significantly reduces redundant information interference and improves the accuracy and real-time performance of semantic understanding. Through contrastive learning and slot correlation analysis methods, key semantic tokens are scored for slot association and dynamic dialogue states are generated. Combined with contextual compensation, an enhanced semantic representation is formed, supporting the generative intelligent response module to output more coherent and personalized responses.

[0007] The technical solution adopted in this invention is as follows: This invention provides a live semantic understanding-driven intelligent automatic response system, comprising a multimodal live semantic stream construction module, a dynamic semantic sparsification module, a semantic structure mapping module, a dialogue state tracking module, a dynamic knowledge fusion module, and a generative intelligent response module, specifically including the following:

[0008] The multimodal live semantic stream construction module collects multi-source heterogeneous data in the live streaming scenario, including live barrage, anchor voice, video footage and user behavior data, and uses multimodal feature encoding and time alignment methods to obtain a unified live semantic token stream.

[0009] The dynamic semantic sparsity module obtains a set of key semantic tokens by using entropy-guided sparsity and hierarchical token pruning methods based on the information entropy and semantic contribution of the live semantic token stream.

[0010] The semantic structure mapping module calculates the probability distribution relationship based on the key semantic token set, and uses information difference measurement and nonlinear low-dimensional mapping method to obtain the live semantic structure space and live semantic clustering results.

[0011] The dialogue state tracking module obtains the current live dialogue state and key semantic slots based on the key semantic token set and live semantic clustering results using contrastive learning and slot correlation analysis methods.

[0012] The dynamic knowledge fusion module obtains an enhanced live context semantic representation by using a dynamic knowledge retrieval and prompting fusion method based on the current live dialogue state, key semantic slots, and semantic clustering results.

[0013] The generative intelligent response module uses a generative language model and a response ranking method, based on the enhanced live streaming context semantic representation, to obtain an intelligent automatic response result corresponding to the current live streaming scenario.

[0014] Furthermore, the dynamic semantic sparsity module specifically includes the following steps:

[0015] Step S1: Semantic Token Information Entropy Calculation. Based on the unified live semantic token stream, extract the semantic density features, context fluctuation features, and interaction frequency features of each semantic token. Use the information entropy calculation method to calculate the information entropy value of the corresponding semantic token.

[0016] Step S2: Dynamic sparsity prediction. Based on the information entropy value of each semantic token, a live semantic complexity index is constructed. Entropy-guided sparsification network is used to perform dynamic sparsity prediction to obtain the semantic token retention ratio corresponding to the current live streaming scenario.

[0017] Step S3: Semantic contribution evaluation. Based on the unified live semantic token stream and the semantic token retention ratio corresponding to the current live scenario, extract the association features between semantic tokens and live topic, user intent and product semantics, and use the semantic contribution evaluation model to obtain the contribution score of each semantic token.

[0018] Step S4: Hierarchical Token Pruning. Based on the contribution score of each semantic token and the retention ratio of semantic tokens, the hierarchical token pruning method is used to filter out low-contribution semantic tokens layer by layer to obtain the key semantic token set.

[0019] Step S5: Key semantic token reconstruction. Based on the set of key semantic tokens, the context relationship between key semantic tokens is reconstructed using the context association compensation method to obtain a compressed semantic token sequence.

[0020] Furthermore, step S5 specifically includes the following steps:

[0021] Step S51: Based on the set of key semantic tokens, extract the low-dimensional embedding representation and local neighborhood relationship of each key semantic token in the semantic structure mapping module;

[0022] Step S52: Based on the low-dimensional embedding representation and local neighborhood relationship of each key semantic token, calculate the contextual association strength between key semantic tokens and construct the initial semantic association graph structure;

[0023] Step S53: Based on the initial semantic association graph structure, detect semantic break regions caused by hierarchical token pruning, and mark semantic node pairs with missing contextual relationships;

[0024] Step S54: For semantic node pairs lacking contextual relationships, the neighborhood semantic information is weighted and propagated based on the contextual attention mechanism to generate compensated semantic association edges;

[0025] Step S55: Based on the compensated semantic association edges, reconstruct the topology of the key semantic token set to obtain a compressed semantic token sequence, and output it to the dialogue state tracking module.

[0026] Furthermore, the semantic structure mapping module calculates the probability distribution relationship based on the key semantic token set, and uses information difference measurement and nonlinear low-dimensional mapping method to obtain the live semantic structure space and live semantic clustering results, specifically including the following steps:

[0027] Step Q1: Based on the set of key semantic tokens, extract the context semantic representation vector of each key semantic token, calculate the high-dimensional distance between any two semantic tokens, and construct a high-dimensional semantic similarity matrix, represented as:

[0028] ;

[0029] in, Indicates the first The semantic token and the first High-dimensional semantic distance between semantic tokens Indicates the first The context semantic representation vector corresponding to each key semantic token. Indicates the first The context semantic representation vector corresponding to each key semantic token;

[0030] Step Q2: Based on the high-dimensional semantic similarity matrix, construct conditional probability relationships based on Gaussian distribution, and calculate the high-dimensional semantic probability distribution matrix. The calculation formula is expressed as follows:

[0031] ;

[0032] in, Indicates that in the first Under the condition of focusing on the semantic token, the first The conditional probability that a semantic token belongs to its neighborhood. Represents an exponential function. Indicates the first The semantic token and the first High-dimensional semantic distance between semantic tokens Indicates except the first Index of semantic tokens other than the single semantic token;

[0033] Step Q3: Based on the high-dimensional semantic probability distribution matrix, introduce an information difference measure function to perform weighted modeling of semantic distribution bias, obtaining the semantic difference distribution matrix, represented as:

[0034] ;

[0035] in, Indicates the first The semantic token and the first A measure of the information difference between semantic tokens. This represents the joint probability distribution value among semantic tokens in a high-dimensional semantic space. This represents the joint probability distribution value among corresponding semantic tokens in the low-dimensional semantic space. Represents the logarithmic function. Indicates the smoothing coefficient;

[0036] Step Q4: Based on the semantic difference distribution matrix, construct the KL divergence optimization objective function and use a nonlinear low-dimensional embedding mapping method for optimization. By minimizing the KL divergence optimization objective function, the KL divergence optimization objective function is obtained for the first... The embedding vectors of each semantic token in the live semantic structure space are iteratively updated. When the joint probability distribution values ​​converge and stabilize, the live semantic structure space is obtained, represented as:

[0037] ;

[0038] The objective function for KL divergence optimization is expressed as follows:

[0039] ;

[0040] in, Represents the first in a low-dimensional semantic space The semantic token and the first The joint probability distribution value among semantic tokens Indicates the first The embedding vector of each semantic token in the live streaming semantic structure space. Indicates the first The embedding vector of each semantic token in the live streaming semantic structure space. This represents the squared Euclidean distance between embedding vectors in the live semantic structure space. This represents the objective function value for KL divergence optimization. This indicates that the summation calculation is performed on all different semantic token pairs;

[0041] Step Q5: Based on the semantic representation distribution in the live semantic structure space, using the embedding vector in the live semantic structure space corresponding to each semantic token as input, the density clustering method is used to cluster the semantic tokens to obtain the live semantic clustering results.

[0042] Furthermore, the dialogue state tracking module, based on the key semantic token set and live semantic clustering results, uses contrastive learning and slot correlation analysis methods to obtain the current live dialogue state and key semantic slots, specifically including the following steps:

[0043] Step DS1: Key semantic token encoding. Input the key semantic token set and the live semantic clustering results, and use the Transformer encoder to generate a vector representation of each token;

[0044] Step DS2: Slot relevance calculation. Predefined slot embeddings are used to calculate the relevance score based on the vector representation of each token and the slot embedding. The importance of the vector representation of each token to the slot is quantified, and preliminary screening for dynamic slot selection is performed.

[0045] Step DS3: Contrastive learning optimization. By constructing positive and negative samples, the contrastive learning loss is minimized. The formula used is as follows:

[0046] ;

[0047] in, Indicates the first The token vector and the first The degree of matching of each slot, Indicates positive sample similarity. Indicates negative sample similarity. Indicates the contrast learning loss;

[0048] The relevance score matrix between each key semantic token and all slots is obtained, represented as follows: ;

[0049] Step DS4: Key slot selection. Combining the relevance score matrix and semantic clustering information, select slots related to the current cluster to form a set of key semantic slots.

[0050] Step DS5: Current dialogue state generation. Based on the key semantic slot set and the contextual representation of the key semantic token, the relationship between slots is adjusted through the context association compensation method to generate the current live dialogue state.

[0051] Beneficial effects

[0052] (1) This invention uses a dynamic semantic sparsification module and a semantic structure mapping module to realize information entropy analysis, contribution evaluation and low-dimensional structure mapping of live semantic tokens, so as to obtain a compressed and structured key semantic set, which significantly reduces redundant information interference and improves the accuracy and real-time performance of semantic understanding.

[0053] (2) By comparing learning and slot correlation analysis methods, the key semantic tokens are scored for slot association and dynamic dialogue state is generated. Combined with context association compensation, an enhanced semantic representation is formed, which supports the generative intelligent response module to output more coherent and personalized responses. Attached Figure Description

[0054] Figure 1 This is a schematic diagram of a live semantic understanding-driven intelligent automatic response system proposed in this invention.

[0055] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. Detailed Implementation

[0056] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0057] Example 1, see Figure 1This invention provides a live-stream semantic understanding-driven intelligent automatic response system, comprising a multimodal live-stream semantic stream construction module, a dynamic semantic sparsification module, a semantic structure mapping module, a dialogue state tracking module, a dynamic knowledge fusion module, and a generative intelligent response module, specifically including the following:

[0058] The multimodal live semantic stream construction module collects multi-source heterogeneous data in the live streaming scenario, including live barrage, anchor voice, video footage and user behavior data, and uses multimodal feature encoding and time alignment methods to obtain a unified live semantic token stream.

[0059] The dynamic semantic sparsity module obtains a set of key semantic tokens by using entropy-guided sparsity and hierarchical token pruning methods based on the information entropy and semantic contribution of the live semantic token stream.

[0060] The semantic structure mapping module calculates the probability distribution relationship based on the key semantic token set, and uses information difference measurement and nonlinear low-dimensional mapping method to obtain the live semantic structure space and live semantic clustering results.

[0061] The dialogue state tracking module obtains the current live dialogue state and key semantic slots based on the key semantic token set and live semantic clustering results using contrastive learning and slot correlation analysis methods.

[0062] The dynamic knowledge fusion module obtains an enhanced live context semantic representation by using a dynamic knowledge retrieval and prompting fusion method based on the current live dialogue state, key semantic slots, and semantic clustering results.

[0063] The generative intelligent response module uses a generative language model and a response ranking method, based on the enhanced live streaming context semantic representation, to obtain an intelligent automatic response result corresponding to the current live streaming scenario.

[0064] Example 2, based on the above examples, specifically includes the following steps in the dynamic semantic sparsity module:

[0065] Step S1: Semantic Token Information Entropy Calculation. Based on the unified live semantic token stream, extract the semantic density features, context fluctuation features, and interaction frequency features of each semantic token. Use the information entropy calculation method to calculate the information entropy value of the corresponding semantic token.

[0066] Step S2: Dynamic sparsity prediction. Based on the information entropy value of each semantic token, a live semantic complexity index is constructed. Entropy-guided sparsification network is used to perform dynamic sparsity prediction to obtain the semantic token retention ratio corresponding to the current live streaming scenario.

[0067] Step S3: Semantic contribution evaluation. Based on the unified live semantic token stream and the semantic token retention ratio corresponding to the current live scenario, extract the association features between semantic tokens and live topic, user intent and product semantics, and use the semantic contribution evaluation model to obtain the contribution score of each semantic token.

[0068] Step S4: Hierarchical Token Pruning. Based on the contribution score of each semantic token and the retention ratio of semantic tokens, the hierarchical token pruning method is used to filter out low-contribution semantic tokens layer by layer to obtain the key semantic token set.

[0069] Step S5: Key semantic token reconstruction. Based on the set of key semantic tokens, the context relationship between key semantic tokens is reconstructed using the context association compensation method to obtain a compressed semantic token sequence.

[0070] Example 3, based on the above examples, specifically includes the following steps in step S5:

[0071] Step S51: Based on the set of key semantic tokens, extract the low-dimensional embedding representation and local neighborhood relationship of each key semantic token in the semantic structure mapping module;

[0072] Step S52: Based on the low-dimensional embedding representation and local neighborhood relationship of each key semantic token, calculate the contextual association strength between key semantic tokens and construct the initial semantic association graph structure;

[0073] Step S53: Based on the initial semantic association graph structure, detect semantic break regions caused by hierarchical token pruning, and mark semantic node pairs with missing contextual relationships;

[0074] Step S54: For semantic node pairs lacking contextual relationships, the neighborhood semantic information is weighted and propagated based on the contextual attention mechanism to generate compensated semantic association edges;

[0075] Step S55: Based on the compensated semantic association edges, reconstruct the topology of the key semantic token set to obtain a compressed semantic token sequence, and output it to the dialogue state tracking module.

[0076] In this embodiment, in an e-commerce livestream room, the host is promoting a smartwatch. Viewers ask questions, like, and click on products via bullet comments. The system has already obtained the host's voice text, bullet comment content, video frame features, and user behavior data through the multimodal livestream semantic stream construction module, and then filtered out the key semantic token set through the dynamic semantic sparsity module.

[0077] • "Smartwatch";

[0078] • Heart rate monitoring;

[0079] • "Battery life";

[0080] • "Promotional discounts";

[0081] • “Insufficient inventory”;

[0082] Step S51: Low-dimensional embedding representation and local neighborhood relation extraction. The system maps each key semantic token to a low-dimensional semantic space, with "smartwatch" corresponding to a vector. It identifies neighboring tokens such as "heart rate monitoring" and "battery life" to form a preliminary local semantic neighborhood;

[0083] Step S52: Context association strength calculation and initial graph construction. The system calculates the context association strength between each token. The association degree between "smartwatch" and "heart rate monitoring" is 0.92, and the association degree between "smartwatch" and "insufficient inventory" is 0.45. Based on this, the initial semantic association graph is constructed: the nodes are tokens, and the weight of the edges is the context association strength.

[0084] Step S53: Semantic chain break detection. Due to the previous layered token pruning operation, some low-frequency words mentioning "battery life" in the bullet comments were deleted, resulting in "battery life" and "smartwatch" not being fully connected in the initial graph. The system marks the broken chain area and identifies the semantic node pairs that need to be compensated.

[0085] Step S54: Context compensation and semantic edge generation. The system uses a context attention mechanism to propagate the semantic information of the "smartwatch" neighborhood to "battery life" in a weighted manner, generate compensation edges, and reconnect the broken semantic context to ensure the continuity of the semantic flow.

[0086] Step S55: Topology reconstruction and output. Finally, the system integrates the compensated semantic association edges into the graph to form a compressed and complete semantic token sequence: "Smartwatch → Heart rate monitoring → Battery life → Promotional discount → Insufficient stock".

[0087] The sequence is output to the dialogue state tracking module, and the system can identify that the topic that the audience is most concerned about is "heart rate monitoring and battery life of smartwatches". It is ready to automatically reply to the audience's question in the generative intelligent response module: "This watch's heart rate monitoring function supports all-day tracking and has a battery life of up to 7 days. Current stock is limited, please place your order as soon as possible."

[0088] Example 4, based on the above examples, describes a semantic structure mapping module that calculates probability distribution relationships based on a set of key semantic tokens, and uses information difference measurement and nonlinear low-dimensional mapping methods to obtain the live semantic structure space and live semantic clustering results. Specifically, it includes the following steps:

[0089] Step Q1: Based on the set of key semantic tokens, extract the context semantic representation vector of each key semantic token, calculate the high-dimensional distance between any two semantic tokens, and construct a high-dimensional semantic similarity matrix, represented as:

[0090] ;

[0091] in, Indicates the first The semantic token and the first High-dimensional semantic distance between semantic tokens Indicates the first The context semantic representation vector corresponding to each key semantic token. Indicates the first The context semantic representation vector corresponding to each key semantic token;

[0092] Step Q2: Based on the high-dimensional semantic similarity matrix, construct conditional probability relationships based on Gaussian distribution, and calculate the high-dimensional semantic probability distribution matrix. The calculation formula is expressed as follows:

[0093] ;

[0094] in, Indicates that in the first Under the condition of focusing on the semantic token, the first The conditional probability that a semantic token belongs to its neighborhood. Represents an exponential function. Indicates the first The semantic token and the first High-dimensional semantic distance between semantic tokens Indicates except the first Index of semantic tokens other than the single semantic token;

[0095] Step Q3: Based on the high-dimensional semantic probability distribution matrix, introduce an information difference measure function to perform weighted modeling of semantic distribution bias, obtaining the semantic difference distribution matrix, represented as:

[0096] ;

[0097] in, Indicates the first The semantic token and the first A measure of the information difference between semantic tokens. This represents the joint probability distribution value among semantic tokens in a high-dimensional semantic space. This represents the joint probability distribution value among corresponding semantic tokens in the low-dimensional semantic space. Represents the logarithmic function. Indicates the smoothing coefficient;

[0098] Step Q4: Based on the semantic difference distribution matrix, construct the KL divergence optimization objective function and use a nonlinear low-dimensional embedding mapping method for optimization. By minimizing the KL divergence optimization objective function, the KL divergence optimization objective function is obtained for the first... The embedding vectors of each semantic token in the live semantic structure space are iteratively updated. When the joint probability distribution values ​​converge and stabilize, the live semantic structure space is obtained, represented as:

[0099] ;

[0100] The objective function for KL divergence optimization is expressed as follows:

[0101] ;

[0102] in, Represents the first in a low-dimensional semantic space The semantic token and the first The joint probability distribution value among semantic tokens Indicates the first The embedding vector of each semantic token in the live streaming semantic structure space. Indicates the first The embedding vector of each semantic token in the live streaming semantic structure space. This represents the squared Euclidean distance between embedding vectors in the live semantic structure space. This represents the objective function value for KL divergence optimization. This indicates that the summation calculation is performed on all different semantic token pairs;

[0103] Step Q5: Based on the semantic representation distribution in the live semantic structure space, using the embedding vector in the live semantic structure space corresponding to each semantic token as input, the density clustering method is used to cluster the semantic tokens to obtain the live semantic clustering results.

[0104] In this embodiment, random data is used to simulate key semantic tokens, and the core code used is as follows:

[0105] import numpy as np

[0106] from sklearn.metrics import pairwise_distances

[0107] from sklearn.cluster import DBSCAN

[0108] # -------------------------------

[0109] # Simulation Data

[0110] # -------------------------------

[0111] num_tokens = 10 # Number of key semantic tokens

[0112] high_dim = 16 # Dimension of high-dimensional semantic vector

[0113] low_dim = 2 # Low-dimensional embedding dimension

[0114] # Simulate a high-dimensional contextual semantic representation of key semantic tokens

[0115] np.random.seed(42)

[0116] x = np.random.rand(num_tokens, high_dim)

[0117] # -------------------------------

[0118] # Step Q1: Calculate the high-dimensional semantic similarity matrix

[0119] # -------------------------------

[0120] # d_ij = ||x_i - x_j||^2

[0121] d_ij = pairwise_distances(x, metric='euclidean') ** 2

[0122] # -------------------------------

[0123] # Step Q2: High-dimensional conditional probability distribution

[0124] # -------------------------------

[0125] p_ij = np.zeros_like(d_ij)

[0126] for i in range(num_tokens):

[0127] # Excluding oneself

[0128] mask = np.ones(num_tokens, dtype=bool)

[0129] mask[i] = False

[0130] p_ij[i, mask] = np.exp(-d_ij[i, mask])

[0131] p_ij[i, :] / = p_ij[i, :].sum()

[0132] # -------------------------------

[0133] # Steps Q3-Q4: KL divergence optimization, low-dimensional embedding initialization

[0134] # -------------------------------

[0135] low_emb = np.random.rand(num_tokens, low_dim) # Initialize low-dimensional embedding

[0136] q_ij = np.zeros_like(p_ij)

[0137] epsilon = 1e-8

[0138] learning_rate = 0.1

[0139] num_epochs = 500

[0140] for epoch in range(num_epochs):

[0141] # Calculate the low-dimensional joint probability q_ij

[0142] for i in range(num_tokens):

[0143] for j in range(num_tokens):

[0144] if i != j:

[0145] denom = np.sum(1 / (1 + np.sum((low_emb[:, None, :] -low_emb[None, :, :])**2, axis=2)))

[0146] q_ij[i,j] = 1 / (1 + np.sum((low_emb[i] - low_emb[j])**2)) / denom

[0147] # KL divergence loss L = sum p_ij * log(p_ij / q_ij)

[0148] L = np.sum(p_ij * np.log((p_ij + epsilon) / (q_ij + epsilon)))

[0149] # Backward gradient update low_emb

[0150] grad = np.zeros_like(low_emb)

[0151] for i in range(num_tokens):

[0152] for j in range(num_tokens):

[0153] if i != j:

[0154] grad[i] += 4 * (p_ij[i,j] - q_ij[i,j]) * (low_emb[i]- low_emb[j]) / ((1 + np.sum((low_emb[i]-low_emb[j])**2))**2)

[0155] low_emb -= learning_rate * grad

[0156] # low_emb is the obtained live semantic structure space

[0157] print("Low-dimensional embedding vector of live semantic structure space:\n", low_emb)

[0158] # -------------------------------

[0159] # Step Q5: Density clustering generates semantic clustering results

[0160] # -------------------------------

[0161] # Density clustering using DBSCAN

[0162] clustering = DBSCAN(eps=0.5, min_samples=2).fit(low_emb)

[0163] cluster_labels = clustering.labels_

[0164] print("Live semantic clustering results:", cluster_labels).

[0165] Example 5, based on the above examples, describes a dialogue state tracking module that, according to the key semantic token set and live semantic clustering results, uses contrastive learning and slot correlation analysis methods to obtain the current live dialogue state and key semantic slots. Specifically, it includes the following steps:

[0166] Step DS1: Key semantic token encoding. Input the key semantic token set and the live semantic clustering results, and use the Transformer encoder to generate a vector representation of each token;

[0167] Step DS2: Slot relevance calculation. Predefined slot embeddings are used to calculate the relevance score based on the vector representation of each token and the slot embedding. The importance of the vector representation of each token to the slot is quantified, and preliminary screening for dynamic slot selection is performed.

[0168] Step DS3: Contrastive learning optimization. By constructing positive and negative samples, the contrastive learning loss is minimized. The formula used is as follows:

[0169] ;

[0170] in, Indicates the first The token vector and the first The degree of matching of each slot, Indicates positive sample similarity. Indicates negative sample similarity. Indicates the contrast learning loss;

[0171] The relevance score matrix between each key semantic token and all slots is obtained, represented as follows: ;

[0172] Step DS4: Key slot selection. Combining the relevance score matrix and semantic clustering information, select slots related to the current cluster to form a set of key semantic slots.

[0173] Step DS5: Current dialogue state generation. Based on the key semantic slot set and the contextual representation of the key semantic token, the relationship between slots is adjusted through the context association compensation method to generate the current live dialogue state.

[0174] In this embodiment, the code used is as follows:

[0175] import torch

[0176] import torch.nn as nn

[0177] import torch.nn.functional as F

[0178] # -------------------------------

[0179] # Simulation Data

[0180] # -------------------------------

[0181] num_tokens = 10 # Number of key semantic tokens

[0182] num_slots = 5 # Number of slots

[0183] embedding_dim = 16 # Vector dimension

[0184] # Simulate the vector representation of the key semantic token set / Transformer output)

[0185] token_embeddings = torch.rand(num_tokens, embedding_dim)

[0186] # Predefined slot embedding

[0187] slot_embeddings = torch.rand(num_slots, embedding_dim)

[0188] # Simulated semantic clustering information / Cluster ID corresponding to each Token

[0189] token_cluster_ids = torch.randint(0, 3, (num_tokens,))

[0190] # -------------------------------

[0191] # Step DS1: Key Semantic Token Encoding

[0192] # -------------------------------

[0193] class TokenEncoder(nn.Module):

[0194] def __init__(self, embedding_dim):

[0195] super().__init__()

[0196] self.transformer_layer = nn.TransformerEncoderLayer(d_model=embedding_dim, nhead=2)

[0197] self.encoder = nn.TransformerEncoder(self.transformer_layer,num_layers=1)

[0198] def forward(self, x):

[0199] # The Transformer requires input (seq_len, batch, embed_dim)

[0200] x = x.unsqueeze(1) # batch=1

[0201] out = self.encoder(x)

[0202] return out.squeeze(1)

[0203] encoder = TokenEncoder(embedding_dim)

[0204] token_encoded = encoder(token_embeddings)

[0205] # -------------------------------

[0206] # Step DS2: Slot Correlation Calculation

[0207] # -------------------------------

[0208] # Calculate the relevance score of each token to each slot.

[0209] # Use cosine similarity as the degree of matching

[0210] r_ij = F.cosine_similarity(

[0211] token_encoded.unsqueeze(1), # shape: [num_tokens, 1, embedding_dim]

[0212] slot_embeddings.unsqueeze(0), # shape: [1, num_slots, embedding_dim]

[0213] dim=-1

[0214] ) # shape: [num_tokens, num_slots]

[0215] # -------------------------------

[0216] # Step DS3: Comparative Learning Optimization

[0217] # -------------------------------

[0218] # Construct positive / negative samples, simplified here: the correct slot for each token is its cluster ID % num_slots

[0219] positive_mask = torch.zeros_like(r_ij)

[0220] for i, cid in enumerate(token_cluster_ids):

[0221] positive_mask[i, cid % num_slots] = 1

[0222] # Comparison of losses

[0223] r_ij_pos = r_ij * positive_mask

[0224] r_ij_neg = r_ij * (1 - positive_mask)

[0225] # Simplified implementation: Take the maximum negative sample value in each row

[0226] r_ij_neg_max, _ = r_ij_neg.max(dim=1, keepdim=True)

[0227] L_CL = -torch.log(torch.exp(r_ij_pos + 1e-8) / (torch.exp(r_ij_pos +1e-8) + torch.exp(r_ij_neg_max + 1e-8))).sum()

[0228] print("Contrastive learning loss:", L_CL.item())

[0229] # -------------------------------

[0230] # Step DS4: Key Slot Filtering

[0231] # -------------------------------

[0232] # Select relevant slots based on the correlation matrix optimized through comparative learning.

[0233] # Simplified strategy: Select the slot with the highest relevance for each token

[0234] key_slots = r_ij.argmax(dim=1)

[0235] print("Key slot set:", key_slots.tolist())

[0236] # -------------------------------

[0237] # Step DS5: Generating the Current Dialogue State

[0238] # -------------------------------

[0239] # Assume the dialogue state is a dictionary, recording the currently active token for each slot.

[0240] dialogue_state = {slot.item(): [] for slot in torch.unique(key_slots)}

[0241] for i, slot in enumerate(key_slots):

[0242] dialogue_state[slot.item()].append(f"Token_{i}")

[0243] print("Current dialogue state DS_t:", dialogue_state).

[0244] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0245] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

[0246] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention; the actual structure is not limited thereto. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.

Claims

1. A live semantic understanding driven intelligent auto-answering system, characterized in that: It includes a multimodal live streaming semantic stream construction module, a dynamic semantic sparsity module, a semantic structure mapping module, a dialogue state tracking module, a dynamic knowledge fusion module, and a generative intelligent response module, specifically including the following: The multimodal live semantic stream construction module collects multi-source heterogeneous data in the live streaming scenario and uses multimodal feature encoding and time alignment methods to obtain a unified live semantic token stream. The dynamic semantic sparsity module obtains a set of key semantic tokens by using entropy-guided sparsity and hierarchical token pruning methods based on the information entropy and semantic contribution of the live semantic token stream. The semantic structure mapping module calculates the probability distribution relationship based on the key semantic token set, and uses information difference measurement and nonlinear low-dimensional mapping method to obtain the live semantic structure space and live semantic clustering results. The dialogue state tracking module obtains the current live dialogue state and key semantic slots based on the key semantic token set and live semantic clustering results using contrastive learning and slot correlation analysis methods. The dynamic knowledge fusion module obtains an enhanced live context semantic representation by using a dynamic knowledge retrieval and prompting fusion method based on the current live dialogue state, key semantic slots, and semantic clustering results. The generative intelligent response module uses a generative language model and a response ranking method, based on the enhanced live streaming context semantic representation, to obtain an intelligent automatic response result corresponding to the current live streaming scenario. 2.The live semantic understanding driven intelligent automatic answering system according to claim 1, characterized in that: The dynamic semantic sparsity module specifically includes the following steps: Step S1: Semantic Token Information Entropy Calculation. Based on the unified live semantic token stream, extract the semantic density features, context fluctuation features, and interaction frequency features of each semantic token. Use the information entropy calculation method to calculate the information entropy value of the corresponding semantic token. Step S2: Dynamic sparsity prediction. Based on the information entropy value of each semantic token, a live semantic complexity index is constructed. Entropy-guided sparsification network is used to perform dynamic sparsity prediction to obtain the semantic token retention ratio corresponding to the current live streaming scenario. Step S3: Semantic contribution evaluation. Based on the unified live semantic token stream and the semantic token retention ratio corresponding to the current live scenario, extract the association features between semantic tokens and live topic, user intent and product semantics, and use the semantic contribution evaluation model to obtain the contribution score of each semantic token. Step S4: Hierarchical Token Pruning. Based on the contribution score of each semantic token and the retention ratio of semantic tokens, the hierarchical token pruning method is used to filter out low-contribution semantic tokens layer by layer to obtain the key semantic token set. Step S5: Key semantic token reconstruction. Based on the set of key semantic tokens, the context relationship between key semantic tokens is reconstructed using the context association compensation method to obtain a compressed semantic token sequence. 3.The live semantic understanding driven intelligent automatic answering system of claim 1, wherein: Step S5 specifically includes the following steps: Step S51: Based on the set of key semantic tokens, extract the low-dimensional embedding representation and local neighborhood relationship of each key semantic token in the semantic structure mapping module; Step S52: Based on the low-dimensional embedding representation and local neighborhood relationship of each key semantic token, calculate the contextual association strength between key semantic tokens and construct the initial semantic association graph structure; Step S53: Based on the initial semantic association graph structure, detect semantic break regions caused by hierarchical token pruning, and mark semantic node pairs with missing contextual relationships; Step S54: For semantic node pairs lacking contextual relationships, the neighborhood semantic information is weighted and propagated based on the contextual attention mechanism to generate compensated semantic association edges; Step S55: Based on the compensated semantic association edges, reconstruct the topology of the key semantic token set to obtain a compressed semantic token sequence, and output it to the dialogue state tracking module.

4. The live semantic understanding driven intelligent auto-response system according to claim 1, wherein: The semantic structure mapping module calculates the probability distribution relationship based on the key semantic token set, and uses information difference measurement and nonlinear low-dimensional mapping method to obtain the live semantic structure space and live semantic clustering results. Specifically, it includes the following steps: Step Q1: Based on the set of key semantic tokens, extract the context semantic representation vector of each key semantic token, calculate the high-dimensional distance between any two semantic tokens, and construct a high-dimensional semantic similarity matrix, represented as: ； in, Indicates the first The semantic token and the first High-dimensional semantic distance between semantic tokens Indicates the first The context semantic representation vector corresponding to each key semantic token. Indicates the first The context semantic representation vector corresponding to each key semantic token; Step Q2: Based on the high-dimensional semantic similarity matrix, construct conditional probability relationships based on Gaussian distribution, and calculate the high-dimensional semantic probability distribution matrix. The calculation formula is expressed as follows: ； in, Indicates that in the first Under the condition of focusing on the semantic token, the first The conditional probability that a semantic token belongs to its neighborhood. Represents an exponential function. Indicates the first The semantic token and the first High-dimensional semantic distance between semantic tokens Indicates except the first Index of semantic tokens other than the single semantic token; Step Q3: Based on the high-dimensional semantic probability distribution matrix, introduce an information difference measure function to perform weighted modeling of semantic distribution bias, obtaining the semantic difference distribution matrix, represented as: ； in, Indicates the first The semantic token and the first A measure of the information difference between semantic tokens. This represents the joint probability distribution value among semantic tokens in a high-dimensional semantic space. This represents the joint probability distribution value among corresponding semantic tokens in the low-dimensional semantic space. Represents the logarithmic function. Indicates the smoothing coefficient; Step Q4: Based on the semantic difference distribution matrix, construct the KL divergence optimization objective function and use a nonlinear low-dimensional embedding mapping method for optimization. By minimizing the KL divergence optimization objective function, the KL divergence optimization objective function is obtained for the first... The embedding vectors of each semantic token in the live semantic structure space are iteratively updated. When the joint probability distribution values ​​converge and stabilize, the live semantic structure space is obtained, represented as: ； The objective function for KL divergence optimization is expressed as follows: ； in, Represents the first in a low-dimensional semantic space The semantic token and the first The joint probability distribution value among semantic tokens Indicates the first The embedding vector of each semantic token in the live streaming semantic structure space. Indicates the first The embedding vector of each semantic token in the live streaming semantic structure space. This represents the squared Euclidean distance between embedding vectors in the live semantic structure space. This represents the objective function value for KL divergence optimization. This indicates that the summation calculation is performed on all different semantic token pairs; Step Q5: Based on the semantic representation distribution in the live semantic structure space, using the embedding vector in the live semantic structure space corresponding to each semantic token as input, the density clustering method is used to cluster the semantic tokens to obtain the live semantic clustering results.

5. The live semantic understanding driven intelligent auto-response system according to claim 1, wherein: The dialogue state tracking module, based on the key semantic token set and live semantic clustering results, uses contrastive learning and slot correlation analysis methods to obtain the current live dialogue state and key semantic slots, specifically including the following steps: Step DS1: Key semantic token encoding. Input the key semantic token set and the live semantic clustering results, and use the Transformer encoder to generate a vector representation of each token; Step DS2: Slot relevance calculation. Predefined slot embeddings are used to calculate the relevance score based on the vector representation of each token and the slot embedding. The importance of the vector representation of each token to the slot is quantified, and preliminary screening for dynamic slot selection is performed. Step DS3: Contrastive learning optimization. By constructing positive and negative samples, the contrastive learning loss is minimized. The formula used is as follows: ； in, Indicates the first The token vector and the first The degree of matching of each slot, Indicates positive sample similarity. Indicates negative sample similarity. Indicates the contrast learning loss; The relevance score matrix between each key semantic token and all slots is obtained, represented as follows: ; Step DS4: Key slot selection. Combining the relevance score matrix and semantic clustering information, select slots related to the current cluster to form a set of key semantic slots. Step DS5: Current dialogue state generation. Based on the key semantic slot set and the contextual representation of the key semantic token, the relationship between slots is adjusted through the context association compensation method to generate the current live dialogue state.

Images