Intelligent agent interaction system based on semantic understanding

Abstract

The invention provides an agent interaction system based on semantic comprehension. The agent interaction system comprises a multi-mode input interface, a semantic analysis module and an intention response generator. The multi-modal input interface obtains user interaction data and preprocesses the user interaction data into a semantic input sequence; the semantic analysis module is composed of a plurality of semantic analysis units connected in series, each unit comprises a context coding unit and a plurality of interactive modeling units, the context coding unit generates a context embedding vector, and the interactive modeling units process data through a dynamic semantic focusing module and a semantic extension feedforward module and transmit the data to the next unit; and the intention response generator generates an interaction response result according to the output of the semantic analysis module. According to the method, the context understanding ability and intention recognition accuracy of the intelligent agent to the complex dialogue can be improved, and the coherence and adaptability of interaction response are enhanced.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing technology, and more specifically, to an intelligent agent interaction system based on semantic understanding. Background Art

[0002] In today's digital age, human-computer interaction technology has developed rapidly. As an important branch of this technology, intelligent agent interaction systems have gradually become an indispensable tool in people's lives and work. Most existing intelligent agent interaction systems are based on traditional natural language processing technologies, using methods such as keyword matching and template matching to understand user input and generate corresponding responses. While these systems can meet basic user needs to a certain extent, they still have many shortcomings in the depth and accuracy of semantic understanding. For example, when the user input is complex or ambiguous, the system often has difficulty accurately grasping the user's true intention, resulting in inaccurate or less relevant responses. In addition, existing intelligent agent interaction systems also face challenges when processing multimodal input. They have difficulty effectively integrating information from different modalities (such as text, voice, and images), which limits the system's ability to understand complex interaction scenarios.

[0003] From a technical perspective, traditional intelligent agent interaction systems primarily rely on rule engines or statistical machine learning methods. Rule engines require manual formulation of numerous rules to cover a wide range of possible interaction scenarios. This approach struggles with complex and changing user input and is difficult to adapt to new interaction modes. While statistical machine learning methods can learn patterns in data to a certain extent, they are highly data-dependent and prone to performance bottlenecks when processing long texts or complex semantic structures. Furthermore, these systems also struggle with dialogue coherence, making it difficult to generate natural, fluent, and logically coherent conversations.

[0004] In the process of implementing the embodiments of the present invention, there are at least the following problems or defects in the existing technology: First, the existing intelligent agent interaction system has limitations in the depth and accuracy of semantic understanding, and it is difficult to cope with scenarios with complex semantics and multimodal input; second, traditional technology performs poorly in terms of dialogue coherence and cannot generate natural and fluent dialogue content; finally, the scalability and adaptability of the existing system are poor, and it is difficult to quickly adapt to new interaction modes and data changes. Summary of the Invention

[0005] The present invention provides an intelligent agent interaction system based on semantic understanding, comprising:

[0006] Multimodal input interface, which obtains user interaction data and preprocesses it to obtain semantic input sequences;

[0007] A semantic parsing module, comprising a plurality of sequentially connected semantic parsing units, wherein the semantic input sequence is input to the front end of the first semantic parsing unit of the semantic parsing module, and the output of any semantic parsing unit serves as the input of the next semantic parsing unit connected thereto; each semantic parsing unit comprises a context encoding unit and a plurality of interaction modeling units connected thereto;

[0008] The context encoding unit generates a context embedding vector from the received data and inputs the processed data into the interaction modeling unit connected thereto;

[0009] The interactive modeling unit includes a dynamic semantic focus module and a semantic expansion feedforward module;

[0010] The intention response generator is connected to the end of the semantic parsing module, and the output data of the semantic parsing module is input into the intention response generator to generate an interactive response result.

[0011] Furthermore, the context encoding unit in the first semantic parsing unit is a shallow encoding unit, and the context encoding units in other semantic parsing units are cross-modal encoding units;

[0012] The shallow encoding unit includes processing the semantic input sequence using a causal convolution kernel to obtain an initial embedding vector, and performing dialogue flow normalization on the initial embedding vector to obtain output data of the shallow encoding unit;

[0013] The cross-modal encoding unit includes performing multi-scale convolution kernel and 1×1 cross-channel convolution processing on the output data of the previous semantic parsing unit to obtain a first time series feature and a second time series feature; performing hierarchical normalization processing on the first time series feature to obtain a third time series feature; and inputting the third time series feature and the second time series feature into the gated fusion block to obtain the output data of the cross-modal encoding unit.

[0014] Furthermore, the interaction modeling unit further includes a dialogue flow normalization layer, a first residual path, an intent level normalization layer, a second residual path, and a semantic flow normalization layer;

[0015] Processing the first interaction data input to the interaction modeling unit using a dialogue flow normalization layer to obtain second interaction data;

[0016] The dynamic semantic focusing module processes the second interaction data to obtain a focused semantic representation;

[0017] Adding the focused semantic representation to the first interaction data through the first residual path, and adding the addition result to the intent-level normalization layer;

[0018] The input data of this layer is processed using the intent-level normalization layer to obtain the third interaction data;

[0019] The semantic extension feedforward module processes the third interaction data to obtain an extended semantic representation;

[0020] The extended semantic representation is added to the addition result through a second residual path and added to the semantic stream normalization layer to obtain output data of the interaction modeling unit.

[0021] Furthermore, the semantic parsing module includes at least three semantic parsing units, the middle-layer semantic parsing unit is provided with at least three interactive modeling units, and the first and last-layer semantic parsing units are provided with at least two interactive modeling units.

[0022] Furthermore, the intention response generator includes an intention pooling layer and an intention mapping layer. The intention pooling layer is used to perform global intent aggregation on the input data of the intention response generator in the sequence dimension; the intention mapping layer is used to perform intent space transformation on the output result of the intention pooling layer to generate an interactive response result.

[0023] Furthermore, the semantic parsing module is optimized using the constructed dialogue coherence loss function, and the expression of the dialogue coherence loss function is:

[0024]

[0025] in, represents the ambiguity factor of the i-th dialogue turn, Indicates that the system correctly interprets round i as the true intention y i The confidence level, Indicates that the system predicts the true intention y for round i i The probability of C is the total number of intent categories, s i,k represents the confidence of the system in predicting round i as intention j, The dialogue coherence factor representing the intention category j, n j Indicates the number of training samples of category j, n max Indicates the number of samples of the maximum sample category.

[0026] Furthermore, the convolution kernel size of the temporal feature extraction processing is 5×1, the convolution kernel size in the gated dilated convolution is 7×1, the dilation rate is 3, the convolution kernel size in the gated linear transformation of the aggregated semantic tensor is 7×1, the convolution kernel size of the depthwise separable temporal convolution layer is 3×1, and the gated activation layer applies the GLU gating mechanism.

[0027] Furthermore, the conversation context features are subjected to deformable normalization and gated dilated convolution respectively:

[0028]

[0029] Among them, X represents the input data of the dynamic semantic focusing module, μ adaptive ,σ adaptive Represents the dynamically calculated mean and standard deviation, Gating represents the gate activation function, represents the dilated convolution operation with a convolution kernel size of 7×1 and a dilation rate of 3; DeformNorm(X) is the deformable normalization processing; GatedConv is the gated dilated convolution processing.

[0030] Furthermore, the causal convolution kernel size is 7×1, the multi-scale convolution kernel contains 3×1 and 5×1 dual-branch structures with a stride of 2, and the stride of 1×1 cross-channel convolution processing is 2.

[0031] Furthermore, the dynamic semantic focusing module includes:

[0032] Perform temporal feature extraction on the input sequence of this module to obtain conversation context features;

[0033] Performing deformable normalization processing and gated dilated convolution processing on the conversation context features, and fusing the two processing results to obtain interaction semantic features;

[0034] Performing multi-head aggregation and gated aggregation processing on the interactive semantic features respectively, and splicing the results of the two aggregations to obtain an aggregated semantic tensor;

[0035] Performing gated linear transformation and activation function processing on the aggregated semantic tensor to obtain an intention attention weight, and averaging the intention attention weight in the channel dimension to obtain a first intention weight and a second intention weight;

[0036] Channel-splitting the interaction semantic feature to obtain a first interaction feature and a second interaction feature, and processing and concatenating the first interaction feature and the second interaction feature based on the first intention weight and the second intention weight to obtain a focused semantic representation;

[0037] The semantic expansion feedforward module includes:

[0038] The input data of this module is processed in sequence by a 1×1 projection layer, a depth-wise separable temporal convolution layer, a gated activation layer, a dynamic dropout layer, and a 1×1 projection layer to obtain an extended semantic representation.

[0039] The output data of the interaction modeling unit is obtained based on the focused semantic representation and the extended semantic representation.

[0040] The above embodiments of the present invention have at least the following beneficial effects:

[0041] 1. Through the design of a multimodal input interface and a hierarchical semantic parsing module, the problem of traditional interactive systems' insufficient understanding of complex contexts is solved. It can accurately capture the multi-dimensional semantic information of user input and dynamically adjust the parsing strategy based on the context, significantly improving the system's ability to parse ambiguous expressions, implicit intentions, and multi-round conversations, making interactive responses more consistent with real conversation logic.

[0042] 2. The collaborative processing mechanism of the dynamic semantic focus module and the semantic expansion feedforward module is adopted to overcome the problems of single semantic modeling and weak long-range dependency capture in existing technologies. Through technologies such as deformable normalization, gated dilated convolution and multi-head convergence, dynamic enhancement of key semantic features and adaptive filtering of redundant information are achieved, thereby improving the accuracy and robustness of intent recognition.

[0043] 3. By optimizing the semantic parsing process through the dialogue coherence loss function and combining residual connections with hierarchical normalization technology, the problems of poor semantic consistency and intent drift in traditional models in long dialogue scenarios are solved. This ensures that the system maintains stable contextual memory and coherent response generation capabilities in multiple rounds of interaction, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily apparent by reading the following detailed description with reference to the accompanying drawings, in which several embodiments of the present invention are shown by way of example and not limitation, in which:

[0045] Figure 1 A schematic diagram of the structure of an intelligent agent interaction system based on semantic understanding provided by one embodiment of the present invention;

[0046] Figure 2 A flowchart of a semantic understanding-based agent interaction system according to an embodiment of the present invention;

[0047] Figure 3 The figure schematically shows the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The principles and spirit of the present invention will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided solely to enable those skilled in the art to better understand and implement the present invention, and are not intended to limit the scope of the present invention in any way. Rather, these embodiments are provided to make the present invention more thorough and complete, and to fully convey the scope of the present invention to those skilled in the art.

[0049] Those skilled in the art will appreciate that the embodiments of the present invention may be implemented as a system, apparatus, device, method, or computer program product. Therefore, the present invention may be implemented in the following forms: entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or in a combination of hardware and software.

[0050] It should be noted that any number of elements in the drawings is for illustration only and not for limitation, and any naming is only for distinction and does not have any limiting meaning.

[0051] Reference below Figure 1 and Figure 2 , Figure 1 A schematic diagram of the structure of an intelligent agent interaction system based on semantic understanding provided by one embodiment of the present invention; Figure 2 This is a workflow diagram of an agent interaction system based on semantic understanding provided by an embodiment of the present invention. Figure 1 and Figure 2 As shown, an intelligent agent interaction system based on semantic understanding includes:

[0052] Multimodal input interface 101, which acquires user interaction data and preprocesses it to obtain a semantic input sequence;

[0053] Semantic parsing module 102, comprising a plurality of sequentially connected semantic parsing units. The semantic input sequence is input to the front end of the first semantic parsing unit of the semantic parsing module, and the output of any semantic parsing unit serves as the input of the next semantic parsing unit connected thereto. Each semantic parsing unit comprises a context encoding unit and a plurality of interaction modeling units connected thereto.

[0054] The context encoding unit 103 generates a context embedding vector from the received data and inputs the processed data into the interaction modeling unit connected thereto;

[0055] The interaction modeling unit 104 includes a dynamic semantic focus module and a semantic expansion feedforward module;

[0056] The intention response generator 105 is connected to the end of the semantic parsing module, and the output data of the semantic parsing module is input into the intention response generator to generate an interactive response result.

[0057] It should be noted that the core of this system lies in acquiring user interaction data through a multimodal input interface and preprocessing it to obtain a semantic input sequence. A multimodal input interface here refers to an interface capable of receiving and processing multiple types of input data, such as text, voice, and images. After preprocessing, this input data is converted into a semantic input sequence that can be understood and processed by the system. The semantic input sequence transforms user input into a data sequence with a specific semantic structure, enabling further processing and analysis by the subsequent semantic parsing module. The semantic parsing module is a key component of the system, consisting of multiple semantic parsing units, which are connected in sequence to form a hierarchical processing structure. Each semantic parsing unit includes a context encoding unit and an interaction modeling unit. The context encoding unit is responsible for generating a context embedding vector from the input data, while the interaction modeling unit constructs a semantic representation through operations such as dynamic semantic focusing and semantic expansion. The intent response generator, located at the end of the semantic parsing module, receives the processed data and generates the final interaction response—the system's response or action to the user input.

[0058] Specifically, the context encoding unit is a key component of the semantic parsing unit, responsible for encoding the input data into a contextual embedding vector. In the first semantic parsing unit, the context encoding unit is a shallow encoding unit. It processes the semantic input sequence using a causal convolution kernel to obtain an initial embedding vector. This initial embedding vector is then subjected to dialogue normalization to produce the shallow encoding unit's output data. The causal convolution kernel here is a special convolution kernel that ensures that when processing sequential data, the output at the current moment depends only on the input at the previous moment, thus avoiding information leakage. Dialogue normalization is a method of normalizing data to make the data distribution more stable for subsequent processing. In other semantic parsing units, the context encoding unit is a cross-modal encoding unit. It applies a multi-scale convolution kernel and a 1×1 cross-channel convolution to the output data of the previous semantic parsing unit to obtain two temporal features. One of the temporal features is then subjected to hierarchical normalization. Both temporal features are input to the gated fusion block to produce the output data of the cross-modal encoding unit. Multi-scale convolution kernels use convolution kernels of different sizes to extract features at different scales, while 1×1 cross-channel convolutions adjust the number of channels for subsequent fusion operations. The gated fusion block is a fusion mechanism that uses a gating mechanism to control the degree of fusion of different features, resulting in a richer feature representation. The interaction modeling unit includes a dynamic semantic focus module and a semantic expansion feedforward module. The dynamic semantic focus module processes the input data to obtain a focused semantic representation, while the semantic expansion feedforward module further processes the focused semantic representation to obtain an expanded semantic representation. These modules enhance the model's learning ability and stability through a series of operations, such as residual connections and normalization. The intent response generator receives the output data from the semantic parsing module and generates an interaction response through the intent pooling layer and the intent mapping layer. The intent pooling layer globally aggregates the data in the sequence dimension, while the intent mapping layer transforms the aggregated data into the intent space to generate the final interaction response.

[0059] Preferably, for a multimodal input interface, it can be implemented as a comprehensive input processing module capable of accepting multiple input forms, including text, speech, and images. For text input, preprocessing operations such as word segmentation and part-of-speech tagging can be performed directly. For speech input, speech recognition can be performed first, converted to text, and then preprocessed. For image input, image feature vectors can be extracted and then fused with text features to generate a semantic input sequence. In the semantic parsing module, the context encoding unit and interaction modeling unit of each semantic parsing unit can be implemented as follows: the causal convolution kernel size in the shallow encoding unit can be set to 7×1 with a stride of 1 to ensure that contextual information is fully captured when processing sequential data. Multi-scale convolution kernels can include 3×1 and 5×1 kernels with a stride of 2 to extract features at different scales. The stride of the 1×1 cross-channel convolution is also 2 to adjust the number of channels. The gating mechanism in the gated fusion block can use a sigmoid function to control the degree of fusion. The temporal feature extraction in the dynamic semantic focus module can be implemented using a convolutional neural network (CNN). Specifically, a 5×1 convolution kernel is used for convolution. The resulting conversation context features are then subjected to deformable normalization and gated dilated convolution. Deformable normalization is an adaptive normalization method that dynamically adjusts the normalization parameters based on the input data distribution. Gated dilated convolution uses dilated convolution to expand the receptive field while controlling the flow of information through a gating mechanism. The 1×1 projection layer in the semantic dilation feedforward module can be used to adjust the data dimension. The depthwise separable temporal convolution layer can use a 3×1 convolution kernel for convolution. The gated activation layer can use GLU or Gated Linear Unit gating mechanisms. The dynamic dropout layer can use dropout to prevent overfitting. The intent pooling layer in the intent response generator can use methods such as global average pooling or global max pooling to aggregate intents in the sequence dimension. The intent mapping layer can use a fully connected neural network to transform the intent space. During the optimization process of the dialogue coherence loss function, the performance of the semantic parsing module can be optimized by adjusting parameters such as the ambiguity factor and confidence level, enabling the system to generate more coherent and accurate interactive response results.

[0060] In some embodiments, the context encoding unit in the first semantic parsing unit is a shallow encoding unit, and the context encoding units in other semantic parsing units are cross-modal encoding units;

[0061] The shallow encoding unit includes processing the semantic input sequence using a causal convolution kernel to obtain an initial embedding vector, and performing dialogue flow normalization on the initial embedding vector to obtain output data of the shallow encoding unit;

[0062] The cross-modal encoding unit includes performing multi-scale convolution kernel and 1×1 cross-channel convolution processing on the output data of the previous semantic parsing unit to obtain a first time series feature and a second time series feature; performing hierarchical normalization processing on the first time series feature to obtain a third time series feature; and inputting the third time series feature and the second time series feature into the gated fusion block to obtain the output data of the cross-modal encoding unit.

[0063] It should be noted that the context encoding unit of the first semantic parser in this system is a shallow encoding unit, whose primary function is to perform preliminary processing of the semantic input sequence and extract shallow semantic features. The shallow encoding unit processes the input sequence using causal convolution kernels to obtain initial embedding vectors, which are then subjected to dialogue-flow normalization to produce the shallow encoding unit's output data. This processing approach effectively captures local semantic information in the input sequence and provides foundational data for subsequent semantic parsers. In other semantic parsers, the context encoding unit is a cross-modal encoding unit, which performs deeper feature extraction and fusion on the output data of the previous semantic parser. The cross-modal encoding unit uses multi-scale convolution kernels and 1×1 cross-channel convolution to obtain first and second temporal features, respectively. The first temporal features are then hierarchically normalized to obtain third temporal features. These third and second temporal features are then fed into the gated fusion block, ultimately producing the output data of the cross-modal encoding unit. This cross-modal processing approach effectively integrates features from data of different modalities, improving the system's semantic understanding capabilities.

[0064] Specifically, the causal convolution kernel in the shallow encoding unit is a special convolution kernel with a size of 7×1 and a stride of 1. It is used to perform convolution operations on semantic input sequences, thereby extracting local semantic features within the sequence. The design of the causal convolution kernel ensures that when processing sequential data, the output at the current moment depends only on the input at the previous moment, avoiding information leakage. Dialog flow normalization is a data normalization method that normalizes the initial embedding vector to stabilize the data distribution and facilitate subsequent processing. In the cross-modal encoding unit, multi-scale convolution kernels, including 3×1 and 5×1 kernels with a stride of 2, are used to extract features at different scales. This multi-scale convolution operation captures semantic information across different ranges. 1×1 cross-channel convolution is used to adjust the number of channels for subsequent fusion operations, and its stride is also 2. Hierarchical normalization is a layer-by-layer normalization method that further stabilizes the data distribution. The gated fusion block is a fusion mechanism that uses a gating mechanism to control the degree of fusion of different features, thereby obtaining a richer feature representation. The gating mechanism here can be implemented using the sigmoid function, which dynamically adjusts the contribution of different features by calculating the weights of the input features.

[0065] Preferably, for shallow encoding units, the causal convolution kernel can be implemented using a one-dimensional convolutional neural network (1D CNN), with the input parameter being the semantic input sequence and the output being the initial embedding vector. For dialogue flow normalization, methods such as batch normalization or layer normalization can be used, with the specific choice depending on the data characteristics and system performance requirements. For cross-modal encoding units, multi-scale convolution kernels can be implemented using parallel convolution operations, convolving the input data with 3×1 and 5×1 convolution kernels, respectively, and then concatenating the resulting features. 1×1 cross-channel convolution can use a standard convolution operation, adjusting the number of channels to fuse features of different scales. Hierarchical normalization can be implemented using a layer-by-layer normalization approach, normalizing each feature layer separately. The gating mechanism in the gated fusion block can be implemented by calculating the weights of the input features. Specifically, a sigmoid function can be used to weight the input features and then fuse the weighted features. This fusion method can dynamically adjust the contribution of different features according to the characteristics of the input data, thereby obtaining a more accurate and rich feature representation.

[0066] In some embodiments, the interaction modeling unit further includes a conversation flow normalization layer, a first residual path, an intent level normalization layer, a second residual path, and a semantic flow normalization layer;

[0067] Processing the first interaction data input to the interaction modeling unit using a dialogue flow normalization layer to obtain second interaction data;

[0068] The dynamic semantic focusing module processes the second interaction data to obtain a focused semantic representation;

[0069] Adding the focused semantic representation to the first interaction data through the first residual path, and adding the addition result to the intent-level normalization layer;

[0070] The input data of this layer is processed using the intent-level normalization layer to obtain the third interaction data;

[0071] The semantic extension feedforward module processes the third interaction data to obtain an extended semantic representation;

[0072] The extended semantic representation is added to the addition result through a second residual path and added to the semantic stream normalization layer to obtain output data of the interaction modeling unit.

[0073] It should be noted that the interaction modeling unit plays a crucial role in this system, responsible for in-depth processing of semantic input sequences to generate accurate semantic representations. The interaction modeling unit comprises multiple submodules, such as the conversation flow normalization layer, the dynamic semantic focusing module, the intent-level normalization layer, and the semantic expansion feedforward module. These submodules work together to enhance the accuracy and richness of the semantic representation by performing a series of operations on the input data, such as normalization, focusing, and expansion. The conversation flow normalization layer normalizes the input data to stabilize its distribution; the dynamic semantic focusing module performs semantic focusing on the processed data to extract key semantic information; the intent-level normalization layer further normalizes the data to enhance the accuracy of intent recognition; and the semantic expansion feedforward module expands the semantic representation to generate richer semantic information. Through the collaborative work of these modules, the interaction modeling unit effectively improves the system's semantic understanding capabilities.

[0074] Specifically, the conversation flow normalization layer is a data normalization technology that processes the first interaction data input to the interaction modeling unit to produce second interaction data. This normalization can employ methods such as batch normalization or layer normalization, depending on the data characteristics and system performance requirements. The dynamic semantic focus module processes the second interaction data to produce a focused semantic representation. This module dynamically adjusts the semantic focus using a specific algorithm to extract key semantic information. The first residual path adds the focused semantic representation to the first interaction data. This residual connection technique effectively avoids the vanishing gradient problem and enhances the model's learning ability. The intent-level normalization layer processes the added result to produce third interaction data, further stabilizing the data distribution and enhancing the accuracy of intent recognition. The semantic expansion feedforward module processes the third interaction data to produce an extended semantic representation. This module expands the semantic representation using a feedforward neural network to generate richer semantic information. The second residual path adds the extended semantic representation to the added result and adds it to the semantic flow normalization layer to produce the output data of the interaction modeling unit. The collaborative work of these modules ensures the accuracy and richness of semantic representation.

[0075] Preferably, the dialogue flow normalization layer can employ layer normalization technology to normalize each feature layer of the input data separately to stabilize the data distribution. The dynamic semantic focus module can be implemented through an attention mechanism, for example, using multi-head attention to dynamically adjust the semantic focus and extract key semantic information. The first and second residual paths can be implemented using a simple addition operation, summing the semantic representations at different stages to enhance the model's learning ability and stability. The intent-level normalization layer can also employ layer normalization technology to normalize the data layer by layer. The semantic expansion feedforward module can utilize a two-layer feedforward neural network, with the first layer using the ReLU activation function and the second layer using a linear activation function to expand the semantic representation. Specific parameter settings can include the hidden layer dimension and the activation function selection, which can be adjusted based on the specific application scenario and data characteristics. For example, the hidden layer dimension can be set based on the complexity of the input data, and the activation function can be selected based on the model's performance requirements. Through these specific implementation steps and parameter settings, the interaction modeling unit can effectively enhance the system's semantic understanding capabilities and generate more accurate and rich semantic representations.

[0076] In some embodiments, the semantic parsing module includes at least three semantic parsing units, the middle-layer semantic parsing unit is provided with at least three interaction modeling units, and the first and last-layer semantic parsing units are provided with at least two interaction modeling units.

[0077] It should be noted that the semantic parsing module is a key component for achieving semantic understanding in this system. It consists of multiple semantic parsing units, which are connected in sequence to form a hierarchical processing structure. The number and structural design of the semantic parsing units have an important impact on the system's semantic understanding ability and response quality. According to the design of the system, the semantic parsing module includes at least three semantic parsing units, of which the semantic parsing units in the middle layer are equipped with at least three interaction modeling units, and the semantic parsing units in the first and last layers are equipped with at least two interaction modeling units. This design aims to gradually and deeply understand and process user input through multi-level semantic parsing and interaction modeling, thereby generating more accurate and coherent interactive responses.

[0078] Specifically, the semantic parsing module consists of multiple semantic parsing units, each of which includes a context encoding unit and an interaction modeling unit. In this system, the semantic parsing module contains at least three semantic parsing units. This design allows the system to process input data in multiple stages, gradually extracting and constructing richer semantic information. The middle-layer semantic parsing unit, located in the middle of the semantic parsing module, contains at least three interaction modeling units. These interaction modeling units further refine and enrich the semantic representation through operations such as dynamic semantic focusing and semantic expansion. The first- and last-layer semantic parsing units, located at the beginning and end of the semantic parsing module, each contain at least two interaction modeling units. The first-layer semantic parsing unit is primarily responsible for preliminary processing and semantic extraction of input data, while the last-layer semantic parsing unit is responsible for the final semantic integration and response generation of the multi-stage processed data. This hierarchical structural design enables the system to effectively process complex semantic input and generate high-quality interactive responses.

[0079] Preferably, the specific construction steps of the semantic parsing module can be as follows: First, determine the total number of semantic parsing units in the semantic parsing module. Based on the system complexity and performance requirements, three or more semantic parsing units can be selected. For the semantic parsing units in the middle layer, at least three interaction modeling units are provided. These units can be implemented by stacking multiple interaction modeling layers, each of which can include a dynamic semantic focus module and a semantic expansion feedforward module. For the semantic parsing units in the first and last layers, at least two interaction modeling units are provided. These units can adopt a structure similar to that of the middle layer, but can be appropriately simplified or optimized as needed. In each semantic parsing unit, the specific parameter settings of the context encoding unit and the interaction modeling unit can be adjusted based on the characteristics of the input data and the performance requirements of the system. For example, parameters such as the convolution kernel size and stride in the context encoding unit can be optimized based on the sequence length and semantic complexity of the input data; parameters such as the number of attention heads and hidden layer dimensions in the interaction modeling unit can be adjusted based on the performance requirements of the system. Through these specific construction steps and parameter settings, the semantic parsing module can effectively process complex semantic input and generate high-quality interactive responses.

[0080] In some embodiments, the intent response generator includes an intent pooling layer and an intent mapping layer. The intent pooling layer is used to perform global intent aggregation on the input data of the intent response generator in the sequence dimension; the intent mapping layer is used to perform intent space transformation on the output result of the intent pooling layer to generate an interactive response result.

[0081] It should be noted that the intent response generator is the core component of this system responsible for generating the final interactive response. It receives the output data processed by the semantic parsing module and generates the interactive response results through the intent pooling layer and the intent mapping layer. The purpose of the intent pooling layer is to perform global intent aggregation on the input data in the sequence dimension and extract key information related to the intent; the intent mapping layer performs intent space transformation on the aggregated results, mapping the extracted intent information to the appropriate response space, thereby generating the final interactive response. This design enables the system to generate accurate and relevant responses based on user input, improving the user experience.

[0082] Specifically, the intent pooling layer is a network layer used to extract global intent information. It operates on the input data in the sequence dimension to aggregate key features related to intent. This operation can be implemented through methods such as global average pooling or global maximum pooling, the specific choice of which depends on the distribution characteristics of the intent features and the performance requirements of the system. The intent mapping layer is a network layer used to map the aggregated intent features to the response space. It is usually implemented by a fully connected neural network, and it generates the final interactive response by learning the mapping relationship between input features and output responses. During implementation, the specific parameter settings of the intent pooling layer and the intent mapping layer need to be adjusted according to the characteristics of the input data and the performance requirements of the system. For example, the type of global pooling operation, average pooling or maximum pooling, can be selected based on the distribution characteristics of the intent features; the number of neurons and activation function in the intent mapping layer can be set according to the complexity of the response space and the performance requirements of the system.

[0083] Preferably, the intent pooling layer can employ a global average pooling approach, calculating the average value for each feature dimension of the input data to extract global intent features. This pooling approach effectively captures the average semantic information in the input data and is suitable for situations where intent features are relatively evenly distributed. The intent mapping layer can consist of one or more fully connected layers, and the number of neurons in each fully connected layer can be adjusted based on the complexity of the response space. For example, if the response space is complex, the number of neurons can be increased to improve the model's expressiveness; if the response space is simple, the number of neurons can be reduced to improve system efficiency. The activation function can be selected from ReLU or Softmax, depending on the output requirements of the system. For example, if categorical responses are required, the Softmax activation function can be used; if continuous-valued responses are required, the ReLU activation function can be used. During training, the parameters of the intent pooling and intent mapping layers can be adjusted using the backpropagation algorithm to optimize system performance. Through these specific implementation steps and parameter settings, the intent response generator can effectively generate accurate and relevant interactive responses, improving overall system performance.

[0084] In some embodiments, the semantic parsing module is optimized using a constructed dialogue coherence loss function, where the expression of the dialogue coherence loss function is:

[0085]

[0086] in, represents the ambiguity factor of the i-th dialogue turn, Indicates that the system correctly interprets round i as the true intention y i The confidence level, Indicates that the system predicts the true intention y for round i i The probability of C is the total number of intent categories, s i,j represents the confidence of the system in predicting round i as intention j, The dialogue coherence factor representing the intention category j, n j Indicates the number of training samples of category j, n max Indicates the number of samples of the maximum sample category.

[0087] It's important to note that the conversation coherence loss function is a key technical approach used to optimize the semantic parsing module in this system. By quantifying the coherence of the conversation, this loss function enables the system to learn more natural and coherent conversation patterns during training. Specifically, the conversation coherence loss function considers multiple factors, including the ambiguity factor of each conversation turn, the system's confidence in the intent of each turn, and the conversation coherence factor of each intent category, thereby balancing conversation accuracy and coherence during the optimization process. This loss function design helps improve the system's performance in multi-turn conversation scenarios, making the generated conversation more consistent with human language habits.

[0088] Specifically, the dialogue coherence loss function is a function used to measure dialogue coherence. It optimizes the semantic parsing module by calculating the difference between the system's confidence in the intent of each turn and the true intent. The ambiguity factor represents the degree of ambiguity in the i-th dialogue turn, with values ​​closer to 1 indicating higher ambiguity. The confidence factor indicates the system's confidence that turn i is correctly parsed as the true intent yi. The prediction probability represents the probability that the system predicts turn i as intent j. The total number of intent categories represents the number of possible intent categories. The dialogue coherence factor represents the degree of coherence of intent category j and is calculated based on the number of training samples for that intent category. These parameters work together to enable the loss function to comprehensively consider both dialogue accuracy and coherence. The ambiguity factor reflects the uncertainty of the dialogue turn, while the confidence factor and prediction probability measure the accuracy of the system's intent judgment. Finally, the dialogue coherence factor considers the coherence of different intent categories in the conversation.

[0089] Preferably, the process for constructing and optimizing the conversation coherence loss function can be detailed as follows: First, the ambiguity factor for each conversation turn is calculated. This can be obtained by statistically analyzing the diversity of intents in each turn within the conversation data. For example, if the interaction between the user and the system in a turn involves multiple possible intents, the ambiguity factor for that turn is higher. Second, the system's confidence and predicted probability for each turn's intent are calculated. This can be obtained from the output of the semantic parsing module. The confidence reflects the system's confidence in the true intent, while the predicted probability represents the system's predicted distribution across possible intents. Next, the conversation coherence factor is calculated based on the number of training samples for that intent category. Intent categories with more samples have smaller conversation coherence factors, indicating that the intent category is more representative in the conversation. Finally, these parameters are substituted into the expression for the conversation coherence loss function, and the parameters of the semantic parsing module are optimized by minimizing this loss function. During training, optimization algorithms such as gradient descent can be used to adjust model parameters based on the gradient information of the loss function, thereby improving the system's conversation coherence and accuracy. Through this optimization approach, the system is able to generate more natural and coherent interactive responses in multi-round conversations.

[0090] In some embodiments, the convolution kernel size of the temporal feature extraction processing is 5×1, the convolution kernel size in the gated dilated convolution is 7×1, the dilation rate is 3, the convolution kernel size in the gated linear transformation of the aggregated semantic tensor is 7×1, the convolution kernel size of the depthwise separable temporal convolution layer is 3×1, and the gated activation layer applies the GLU gating mechanism.

[0091] It should be noted that this system uses a specific convolution kernel size and gated activation mechanism in the temporal feature extraction process to enhance the model's processing capabilities for time series data and the accuracy of its semantic understanding. Specifically, the convolution kernel size for the temporal feature extraction process is set to 5×1. This size of convolution kernel can effectively capture local temporal features when processing sequence data. At the same time, the convolution kernel size in the gated dilated convolution is 7×1, and the dilation rate is 3. This setting can expand the model's receptive field, enabling the model to capture a wider range of contextual information. In addition, the convolution kernel size in the gated linear transformation of the aggregated semantic tensor is 7×1, and the convolution kernel size of the depthwise separable temporal convolution layer is 3×1. These parameter settings help further optimize the performance of the model. The gated activation layer applies the GLU, Gated Linear Unit gating mechanism, which can effectively control the flow of information and enhance the model's learning ability.

[0092] Specifically, temporal feature extraction refers to the process of extracting time series features from input data, typically achieved through convolution operations. Here, the convolution kernel size is 5×1, meaning the convolution operation covers five time steps in the temporal dimension and one feature channel in the feature dimension. This setting effectively captures local temporal features. Gated dilated convolution is a special convolution operation that expands the receptive field of the convolution kernel through the dilation rate. Here, the convolution kernel size is 7×1, with a dilation rate of 3, meaning the convolution kernel samples every three time steps in the temporal dimension, thereby capturing a wider range of contextual information. Gated linear transformation is an operation that processes aggregated semantic tensors. Its convolution kernel size is 7×1, further optimizing the model's processing of semantic information. The depthwise separable temporal convolution layer is an efficient convolution operation with a convolution kernel size of 3×1. This setting reduces computational effort while maintaining model performance. The GLU gating mechanism is an activation function that controls the flow of information through gating signals, thereby enhancing the model's learning ability and its ability to process complex semantic structures.

[0093] Preferably, the temporal feature extraction process can be implemented using a one-dimensional convolutional neural network (1D CNN), with the input parameter being a preprocessed semantic input sequence and the output being the extracted temporal features. In gated dilated convolution, the receptive field of the convolution kernel can be expanded by setting the dilation rate to 3, thereby capturing a wider range of contextual information. Gated linear transformation can be implemented through a convolution operation, with the input being an aggregated semantic tensor and the output being the transformed semantic features. The depthwise separable temporal convolution layer can be implemented using depthwise separable convolution. This convolution operation decomposes the standard convolution into two independent steps: depthwise convolution and pointwise convolution, thereby reducing computational effort and improving model efficiency. The GLU gating mechanism can be implemented by calculating a linear combination of the input data, generating a gating signal using a sigmoid function, and then multiplying the gating signal with the input data to control the flow of information. Through these specific implementation steps and parameter settings, the system can effectively process temporal data and generate accurate semantic representations.

[0094] In some embodiments, deformable normalization processing and gated dilated convolution processing are performed on the conversation context features respectively:

[0095]

[0096] Among them, X represents the input data of the dynamic semantic focusing module, μ adaptive ,σ adaptiveRepresents the dynamically calculated mean and standard deviation, Gating represents the gate activation function, represents the dilated convolution operation with a convolution kernel size of 7×1 and a dilation rate of 3; DeformNorm(X) is the deformable normalization processing; GatedConv is the gated dilated convolution processing.

[0097] It's important to note that this system uses two techniques: deformable normalization and gated dilated convolution to process conversational context features. Deformable normalization adaptively normalizes the input data based on its dynamic characteristics, better adapting to the distributional differences in input data. Gated dilated convolution, through dilated convolution and gating mechanisms, expands the model's receptive field and controls the flow of information, thereby enhancing its ability to capture long-range dependencies. The combination of these two processing methods enables the system to more effectively process complex conversational context features, improving the accuracy and coherence of semantic understanding.

[0098] Specifically, deformable normalization is an adaptive normalization technique that performs normalization based on the dynamic characteristics of the input data. This normalization method dynamically adjusts the normalization parameters based on the distribution of the input data, thereby better adapting to the distribution differences of different input data. Gated dilated convolution is a convolution operation that combines dilated convolution with a gating mechanism. Dilated convolution introduces a dilation rate into the convolution kernel, expanding the receptive field of the convolution kernel, allowing the model to capture a wider range of contextual information. The gating mechanism dynamically controls the flow of information by calculating the weights of the input data, thereby enhancing the model's ability to capture long-range dependencies. In specific implementations, deformable normalization can be achieved by calculating the mean and standard deviation of the input data and dynamically adjusting the normalization parameters based on these statistics. Gated dilated convolution can be implemented using a dilated convolution operation and a gated activation function, such as the GLU. The dilated convolution kernel size is 7×1, and the dilation rate is 3.

[0099] Preferably, the deformable normalization process can be implemented using an adaptive normalization algorithm, with the input data X of the dynamic semantic focusing module as input and the normalized data as output. The algorithm first calculates the mean and standard deviation of the input data X and then dynamically adjusts the normalization parameters based on these statistics. The gated dilated convolution process can be implemented using a dilated convolution operation, with the input being the normalized data and the output being the dilated convolution-processed data. The dilated convolution kernel size is 7×1, and the dilation rate is 3. This setting expands the model's receptive field, enabling it to capture a wider range of contextual information. The gated activation function can use a GLU, which calculates a linear combination of the input data, generates a gating signal using a sigmoid function, and then multiplies the gating signal with the input data to control the flow of information. Through these specific implementation steps and parameter settings, the system can more effectively process complex conversation context features and improve the accuracy and coherence of semantic understanding.

[0100] In some embodiments, the causal convolution kernel size is 7×1, the multi-scale convolution kernel includes 3×1 and 5×1 dual-branch structures, the stride is 2, and the stride is 2 in the 1×1 cross-channel convolution processing.

[0101] It should be noted that in the design of the context encoding unit, this system adopts different encoding strategies for different semantic parsing units. In the first semantic parsing unit, the context encoding unit is a shallow encoding unit, whose main function is to perform preliminary processing on the input semantic sequence and extract shallow semantic features. In other semantic parsing units, the context encoding unit is a cross-modal encoding unit, whose function is to perform deeper feature extraction and fusion on the output data of the previous semantic parsing unit. This design allows the system to gradually and deeply understand and integrate information from different modalities when processing multimodal input, thereby generating richer and more accurate semantic representations.

[0102] Specifically, the shallow encoding unit uses a causal convolution kernel to process the semantic input sequence to obtain an initial embedding vector. The causal convolution kernel size here is 7×1. Its design purpose is to ensure that when processing sequence data, the output at the current moment depends only on the input at the previous moment to avoid information leakage. Subsequently, the initial embedding vector is normalized according to the dialogue flow to stabilize the data distribution. In the cross-modal encoding unit, the output data of the previous semantic parsing unit is first processed with a multi-scale convolution kernel. Here, a dual-branch structure of 3×1 and 5×1 is used with a step size of 2 to extract features at different scales. Next, a 1×1 cross-channel convolution is performed, also with a step size of 2, to adjust the number of channels for subsequent fusion operations. This cross-modal processing method can effectively integrate information from different modalities, such as text, speech, and images, and improve the system's semantic understanding capabilities.

[0103] Preferably, the shallow encoding unit can be implemented using a one-dimensional convolutional neural network (1D CNN), with the input being a semantic input sequence and the output being an initial embedding vector. In the causal convolution operation, the convolution kernel size is set to 7×1 with a stride of 1 to ensure that local semantic information in the sequence is captured. Dialogue flow normalization can be performed using layer normalization, which normalizes each feature layer separately to stabilize the data distribution. For the cross-modal encoding unit, the multi-scale convolution kernel can be implemented using parallel convolution operations, convolving the input data with 3×1 and 5×1 kernels, respectively, and then concatenating the resulting features. 1×1 cross-channel convolution is used to adjust the number of channels to fuse features at different scales. Specific parameter settings include kernel size and stride, which can be adjusted based on the characteristics of the input data and the system's performance requirements. Through these specific implementation steps and parameter settings, the system can effectively process multimodal input and generate accurate semantic representations.

[0104] In some embodiments, the dynamic semantic focusing module includes:

[0105] Perform temporal feature extraction on the input sequence of this module to obtain conversation context features;

[0106] Performing deformable normalization processing and gated dilated convolution processing on the conversation context features, and fusing the two processing results to obtain interaction semantic features;

[0107] Performing multi-head aggregation and gated aggregation processing on the interactive semantic features respectively, and splicing the results of the two aggregations to obtain an aggregated semantic tensor;

[0108] Performing gated linear transformation and activation function processing on the aggregated semantic tensor to obtain an intention attention weight, and averaging the intention attention weight in the channel dimension to obtain a first intention weight and a second intention weight;

[0109] Channel-splitting the interaction semantic feature to obtain a first interaction feature and a second interaction feature, and processing and concatenating the first interaction feature and the second interaction feature based on the first intention weight and the second intention weight to obtain a focused semantic representation;

[0110] The semantic expansion feedforward module includes:

[0111] The input data of this module is processed in sequence by a 1×1 projection layer, a depth-wise separable temporal convolution layer, a gated activation layer, a dynamic dropout layer, and a 1×1 projection layer to obtain an extended semantic representation.

[0112] The output data of the interaction modeling unit is obtained based on the focused semantic representation and the extended semantic representation.

[0113] It should be noted that the dynamic semantic focus module is a key component of this system for enhancing semantic understanding. Its main function is to perform in-depth processing on the input sequence to extract more focused and accurate semantic representations. This module gradually refines and enriches the semantic information through a series of operations such as temporal feature extraction, deformable normalization, and gated dilated convolution. Finally, through multi-head convergence and gated convergence processing, the extracted semantic features are effectively integrated to generate a focused semantic representation. This design enables the system to better understand and process complex semantic inputs, improving the accuracy and coherence of semantic understanding.

[0114] Specifically, the dynamic semantic focusing module includes the following key steps: First, temporal feature extraction is performed on the input sequence to obtain conversational context features. This temporal feature extraction typically uses a convolutional neural network (CNN), which uses convolution kernels of a specific size, such as 5×1, to capture local temporal features in the sequence. Next, the conversational context features undergo deformable normalization and gated dilated convolution. Deformable normalization is an adaptive normalization technique that performs normalization based on the dynamic characteristics of the input data to better adapt to the distribution differences of different input data. Gated dilated convolution uses dilated convolution and a gating mechanism to expand the model's receptive field and control information flow, thereby enhancing the model's ability to capture long-range dependencies. Finally, multi-head and gated pooling are performed on the interaction semantic features, and the results of the two pooling processes are concatenated to form an aggregated semantic tensor. Multi-head pooling uses multiple attention heads to process data simultaneously, capturing different aspects of semantic information. Gated pooling dynamically adjusts the contribution of different features through a gating mechanism, resulting in a more accurate semantic representation.

[0115] Preferably, the dynamic semantic focusing module can be implemented as follows: First, temporal feature extraction can be performed using a one-dimensional convolutional neural network (1D CNN), with the input being a preprocessed semantic input sequence, a convolution kernel size of 5×1, and a stride of 1. Next, deformable normalization can be performed by calculating the mean and standard deviation of the input data and dynamically adjusting the normalization parameters based on these statistics. Gated dilated convolution can be performed using a dilated convolution operation with a convolution kernel size of 7×1, a dilation rate of 3, and a stride of 1. The gating mechanism can be implemented by calculating the weights of the input data, specifically by weighting the input features using a sigmoid function. Multi-head convergence can be performed using a multi-head attention mechanism, with multiple attention heads, each responsible for processing different parts of the data, thereby capturing different aspects of semantic information. Gated convergence can be implemented using a gated activation function, such as GLU, which calculates a linear combination of the input data, generates a gating signal using a sigmoid function, and then multiplies the gating signal by the input data to control the flow of information. Through these specific implementation steps and parameter settings, the dynamic semantic focusing module can effectively extract and integrate semantic features, generate focused semantic representations, and enhance the system's semantic understanding capabilities.

[0116] The above embodiments of the present invention have the following beneficial effects:

[0117] 1. Through the design of a multimodal input interface and a hierarchical semantic parsing module, the problem of traditional interactive systems' insufficient understanding of complex contexts is solved. It can accurately capture the multi-dimensional semantic information of user input and dynamically adjust the parsing strategy based on the context, significantly improving the system's ability to parse ambiguous expressions, implicit intentions, and multi-round conversations, making interactive responses more consistent with real conversation logic.

[0118] 2. The collaborative processing mechanism of the dynamic semantic focus module and the semantic expansion feedforward module is adopted to overcome the problems of single semantic modeling and weak long-range dependency capture in existing technologies. Through technologies such as deformable normalization, gated dilated convolution and multi-head convergence, dynamic enhancement of key semantic features and adaptive filtering of redundant information are achieved, thereby improving the accuracy and robustness of intent recognition.

[0119] 3. By optimizing the semantic parsing process through the dialogue coherence loss function and combining residual connections with hierarchical normalization technology, the problems of poor semantic consistency and intent drift in traditional models in long dialogue scenarios are solved. This ensures that the system maintains stable contextual memory and coherent response generation capabilities in multiple rounds of interaction, thereby improving the user experience.

[0120] Reference below Figure 3, which shows a schematic structural diagram of an electronic device 300 suitable for implementing some embodiments of the present invention. The electronic devices in some embodiments of the present invention may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 3 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0121] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. Various programs and data required for the operation of the electronic device 300 are also stored in the RAM 303. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0122] Typically, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 309. The communication device 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as needed.

[0123] Furthermore, the storage medium of the embodiment of the present application stores program instructions that can implement all the above methods, wherein the program instructions can be stored in the above storage medium in the form of a software product, including a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or a terminal device such as a computer, a server, a mobile phone, or a tablet.

[0124] The above descriptions are merely some preferred embodiments of the present invention and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present invention is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also encompass other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned inventive concept. For example, a technical solution formed by mutually replacing the above-mentioned features with (but not limited to) technical features having similar functions disclosed in the embodiments of the present invention.

Claims

1. An intelligent agent interaction system based on semantic understanding, characterized in that: include: Multimodal input interface, which obtains user interaction data and preprocesses it to obtain semantic input sequences; A semantic parsing module, comprising a plurality of sequentially connected semantic parsing units, wherein the semantic input sequence is input to the front end of the first semantic parsing unit of the semantic parsing module, and the output of any semantic parsing unit serves as the input of the next semantic parsing unit connected thereto; each semantic parsing unit comprises a context encoding unit and a plurality of interaction modeling units connected thereto; The context encoding unit generates a context embedding vector from the received data and inputs the processed data into the interaction modeling unit connected thereto; The interactive modeling unit includes a dynamic semantic focus module and a semantic expansion feedforward module; The intention response generator is connected to the end of the semantic parsing module, and the output data of the semantic parsing module is input into the intention response generator to generate an interactive response result.

2. The intelligent agent interaction system based on semantic understanding according to claim 1, characterized in that: The context encoding unit in the first semantic parsing unit is a shallow encoding unit, and the context encoding units in other semantic parsing units are cross-modal encoding units; The shallow encoding unit includes processing the semantic input sequence using a causal convolution kernel to obtain an initial embedding vector, and performing dialogue flow normalization on the initial embedding vector to obtain output data of the shallow encoding unit; The cross-modal encoding unit includes performing multi-scale convolution kernel and 1×1 cross-channel convolution processing on the output data of the previous semantic parsing unit to obtain a first time series feature and a second time series feature; Performing hierarchical normalization on the first time series feature to obtain a third time series feature; The third temporal feature and the second temporal feature are input into the gated fusion block to obtain the output data of the cross-modal encoding unit.

3. The intelligent agent interaction system based on semantic understanding according to claim 1, characterized in that: The interaction modeling unit further includes a conversation flow normalization layer, a first residual path, an intent level normalization layer, a second residual path, and a semantic flow normalization layer; Processing the first interaction data input to the interaction modeling unit using a dialogue flow normalization layer to obtain second interaction data; The dynamic semantic focusing module processes the second interaction data to obtain a focused semantic representation; Adding the focused semantic representation to the first interaction data through the first residual path, and adding the addition result to the intent-level normalization layer; The input data of this layer is processed using the intent-level normalization layer to obtain the third interaction data; The semantic extension feedforward module processes the third interaction data to obtain an extended semantic representation; The extended semantic representation is added to the addition result through a second residual path and added to the semantic stream normalization layer to obtain output data of the interaction modeling unit.

4. The intelligent agent interaction system based on semantic understanding according to claim 1, characterized in that: The semantic parsing module includes at least three semantic parsing units, the middle-layer semantic parsing unit is provided with at least three interactive modeling units, and the first and last-layer semantic parsing units are provided with at least two interactive modeling units.

5. The intelligent agent interaction system based on semantic understanding according to claim 1, characterized in that: The intention response generator includes an intention pooling layer and an intention mapping layer. The intention pooling layer is used to perform global intent aggregation on the input data of the intention response generator in the sequence dimension; the intention mapping layer is used to perform intent space transformation on the output result of the intention pooling layer to generate an interactive response result.

6. The intelligent agent interaction system based on semantic understanding according to claim 1, characterized in that: The semantic parsing module is optimized using the constructed dialogue coherence loss function. The expression of the dialogue coherence loss function is: Among them, δ i represents the ambiguity factor of the i-th dialogue turn, Indicates that the system correctly interprets round i as the true intention y i The confidence level, Indicates that the system predicts the true intention y for round i i The probability of C is the total number of intent categories, s i,j represents the confidence of the system in predicting round i as intention j, τ j The dialogue coherence factor representing the intention category j, n j Indicates the number of training samples of category j, n max represents the number of samples of the largest sample category, and N represents the total number of dialogue turns.

7. The intelligent agent interaction system based on semantic understanding according to claim 1, characterized in that: The convolution kernel size of the temporal feature extraction process is 5×1, the convolution kernel size in the gated dilated convolution is 7×1, the dilation rate is 3, the convolution kernel size in the gated linear transformation of the aggregated semantic tensor is 7×1, the convolution kernel size of the depthwise separable temporal convolution layer is 3×1, and the gated activation layer applies the GLU gating mechanism.

8. The intelligent agent interaction system based on semantic understanding according to claim 7, characterized in that: The conversation context features are subjected to deformable normalization and gated dilated convolution respectively: Among them, X represents the input data of the dynamic semantic focusing module, μ adaptive ,σ adaptive Represents the dynamically calculated mean and standard deviation, Gating represents the gate activation function, represents the dilated convolution operation with a convolution kernel size of 7×1 and a dilation rate of 3; DeformNorm(X) is the deformable normalization processing; GatedConv is the gated dilated convolution processing.

9. The intelligent agent interaction system based on semantic understanding according to claim 2, characterized in that: The causal convolution kernel size is 7×1, the multi-scale convolution kernel contains 3×1 and 5×1 dual-branch structures, the stride is 2, and the stride is 2 in the 1×1 cross-channel convolution processing.

10. The intelligent agent interaction system based on semantic understanding according to claim 1, characterized in that: The dynamic semantic focusing module includes: Perform temporal feature extraction on the input sequence of this module to obtain conversation context features; Performing deformable normalization processing and gated dilated convolution processing on the conversation context features, and fusing the two processing results to obtain interaction semantic features; Performing multi-head aggregation and gated aggregation processing on the interactive semantic features respectively, and splicing the results of the two aggregations to obtain an aggregated semantic tensor; Performing gated linear transformation and activation function processing on the aggregated semantic tensor to obtain an intention attention weight, and averaging the intention attention weight in the channel dimension to obtain a first intention weight and a second intention weight; Channel-splitting the interaction semantic feature to obtain a first interaction feature and a second interaction feature, and processing and concatenating the first interaction feature and the second interaction feature based on the first intention weight and the second intention weight to obtain a focused semantic representation; The semantic expansion feedforward module includes: The input data of this module is processed in sequence by a 1×1 projection layer, a depth-wise separable temporal convolution layer, a gated activation layer, a dynamic dropout layer, and a 1×1 projection layer to obtain an extended semantic representation. The output data of the interaction modeling unit is obtained based on the focused semantic representation and the extended semantic representation.

Images