Intelligent voice semantic understanding analysis method and system based on context

Abstract

The invention provides a context-based intelligent voice semantic understanding analysis method and system, and relates to the technical field of voice interaction, and the method comprises the steps: obtaining an input voice signal, extracting an acoustic feature, and decoding the acoustic feature to obtain a candidate text; constructing context semantic representation to carry out disambiguation processing; establishing a semantic dependency graph and carrying out multi-level correlation analysis; spreading context constraint information to perform multi-hop reasoning; and finally generating intention recognition and slot filling results and updating the session state. The semantic comprehension accuracy and the intelligent degree of the voice interaction system are improved by introducing the context information and the multi-level semantic dependency relationship.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of voice interaction, and in particular to a context-based intelligent voice semantic understanding analysis method and system. BACKGROUND

[0002] With the development of artificial intelligence technology, intelligent voice interaction systems are widely used in smart home, intelligent customer service, intelligent vehicle, etc. Intelligent voice semantic understanding, as a core component of intelligent voice interaction systems, mainly includes converting user voice into text, understanding user intent, and extracting key information, etc. Traditional voice semantic understanding methods usually adopt a pipeline mode, performing voice recognition and semantic understanding, and executing corresponding operations according to the understanding results. In recent years, with the development of deep learning technology, voice recognition and natural language processing technologies based on neural networks have made significant progress, promoting the rapid development of intelligent voice semantic understanding technology. SUMMARY

[0003] The embodiment of the present application provides a context-based intelligent voice semantic understanding analysis method and system, which can solve the problems in the prior art.

[0004] In a first aspect, the embodiment of the present application provides a context-based intelligent voice semantic understanding analysis method, comprising: obtaining an input voice signal, extracting an acoustic feature sequence of the input voice signal, inputting the acoustic feature sequence into an acoustic model for feature decoding to obtain a candidate text sequence; constructing a context semantic representation according to historical interaction information and a current conversation state, performing semantic disambiguation processing on the candidate text sequence based on the context semantic representation to obtain a disambiguated semantic unit sequence; performing correlation analysis on each semantic unit in the disambiguated semantic unit sequence using a multi-level semantic dependency relationship, establishing a cross-level semantic unit correlation mapping to obtain a semantic dependency graph; propagating constraint information of the context semantic representation in the semantic dependency graph to obtain a constraint semantic dependency graph and perform multi-hop reasoning to obtain a context-corrected semantic understanding result; generating an intent recognition result and a slot filling result according to the context-corrected semantic understanding result, and updating the current conversation state information for subsequent semantic understanding analysis of input voice signals.

[0005] extracting an acoustic feature sequence of the input voice signal, inputting the acoustic feature sequence into an acoustic model for feature decoding to obtain a candidate text sequence, comprising: performing multi-scale time-frequency analysis on the input speech signal, obtaining a multi-scale time-frequency feature set by extracting time-frequency features under different time window lengths and frequency resolutions; calculating a feature fusion weight based on energy distribution and spectral stability of each scale time-frequency feature in the multi-scale time-frequency feature set, and obtaining an acoustic feature sequence; dividing the acoustic feature sequence into a plurality of time step feature frames, for each time step feature frame, performing feature encoding on the current time step feature frame by the encoding layer of the acoustic model, and combining the hidden state information of the previous time step to perform context modeling, obtaining the encoding feature of the current time step; calculating the posterior probability distribution of each acoustic unit based on the encoding feature of the current time step, predicting the candidate acoustic unit of the current time step according to the decoding layer of the acoustic model, and combining the posterior probability distribution and the language model constraint to jointly score the candidate acoustic unit, obtaining the optimal acoustic unit of the current time step; splicing the optimal acoustic units of each time step in time sequence to obtain an acoustic unit sequence; performing unit-to-text mapping conversion on the acoustic unit sequence to obtain the candidate text sequence.

[0006] According to the historical interaction information and the current conversation state, a context semantic representation is constructed, and the candidate text sequence is processed by semantic disambiguation based on the context semantic representation to obtain a disambiguated semantic unit sequence, including: Based on the semantic integrity and conversation turning identification in the historical interaction information, the historical interaction information is divided into a plurality of interaction levels and the conversation subtopics are extracted, the semantic topic of the current conversation and the current dialogue scene are extracted; Calculate the semantic similarity between the conversation subtopics and the semantic topics, and the time sequence distance between each interaction level and the current time, select the interaction level whose semantic similarity is greater than the similarity threshold and the time sequence distance is greater than the time sequence threshold, encode the corresponding historical interaction information, and combine the current dialogue scene to obtain the context semantic representation; The candidate text sequence is segmented into an initial semantic unit sequence, and each semantic unit is matched with a polysemy word table and analyzed for context-independent semantic features to identify ambiguous semantic units with polysemy; For each ambiguous semantic unit, extract context constraint information related to the ambiguous semantic unit from the context semantic representation, and calculate the context matching degree of each candidate semantic of the ambiguous semantic unit, select the candidate semantic with the highest context matching degree as the disambiguated semantic of the ambiguous semantic unit; Replace each ambiguous semantic unit in the initial semantic unit sequence with the corresponding disambiguated semantic to obtain the disambiguated semantic unit sequence.

[0007] Multi-level semantic dependency relations are used to perform association analysis on each semantic unit in the disambiguated semantic unit sequence, establishing cross-level semantic unit association mappings to obtain a semantic dependency graph, including: Part-of-speech tagging and semantic role tagging are performed on each semantic unit in the disambiguated semantic unit sequence to construct the feature representation of each semantic unit; Based on the feature representation of each semantic unit, the lexical collocation strength between adjacent semantic units is calculated at the lexical level, the syntactic dominance and syntactic subordination relationships between semantic units are calculated at the syntactic level, and the discourse cohesion and discourse coherence relationships between cross-sentence semantic units are calculated at the discourse level, thereby obtaining the lexical dependency set, the syntactic dependency set, and the discourse dependency set, respectively. The lexical dependency relation set, the syntactic dependency relation set, and the discourse dependency relation set are hierarchically fused to establish a cross-level semantic unit association mapping. Each level dependency relation is assigned a hierarchical distinguishing identifier, and the dependency strength weight corresponding to each level dependency relation is calculated to generate a semantic dependency graph containing each semantic unit as a node, each level dependency relation as an edge, and the dependency strength weight as the edge weight.

[0008] The constraint information of the contextual semantic representation is propagated in the semantic dependency graph to obtain a constrained semantic dependency graph, and multi-hop inference is performed to obtain the context-corrected semantic understanding result, including: The dialogue intent, semantic orientation, and topic scope are extracted from the context semantic representation and encoded as constraint features. The constraint features are then injected into the semantic unit nodes associated with the context semantic representation, which serve as constraint propagation points. Starting from the constraint propagation point, constraint features are propagated. By calculating the semantic correlation and constraint compatibility between adjacent semantic unit nodes, the propagation strength and direction of constraint information during the propagation process are determined. The constraint features are propagated in multiple rounds of iteration, so that the constraint information gradually spreads from the constraint propagation point to the far semantic unit nodes in the semantic dependency graph, thus obtaining the constraint semantic dependency graph. Based on the constrained semantic dependency graph, starting from the inference start node, hop-by-hop inference expansion is performed to obtain multiple inference links. The semantic unit nodes on each inference link are aggregated to obtain the corresponding link semantic representation and then globally fused to generate the context-corrected semantic understanding result.

[0009] Based on the constrained semantic dependency graph, starting from each inference initiation node, hop-by-hop inference expansion is performed to obtain multiple inference links. Semantic unit nodes on each inference link are aggregated to obtain the corresponding link semantic representations, which are then globally fused to generate context-corrected semantic understanding results, including: In the constrained semantic dependency graph, the inference starting node is determined, and the adjacent semantic unit node with the highest edge correlation is selected as the first hop inference node. The hop inference extension is performed along the edge of the constrained semantic dependency graph. In each hop of reasoning extension, based on the semantic representation, constraint feature representation and reasoning context information passed from the reasoning start node to the current hop, an inter-hop reasoning state vector is constructed. Combined with the semantic representation of adjacent semantic unit nodes, an inter-hop extension score is calculated. The adjacent semantic unit node with the highest inter-hop extension score is selected as the next hop reasoning node and the reasoning context information is updated. The subsequent hop reasoning extension is iteratively executed until the preset reasoning depth is reached or the information entropy of the reasoning context information is lower than the convergence threshold, thus obtaining multiple reasoning links starting from each reasoning start node. On the multiple inference links, the position decay weight is calculated based on the hop count of each inference node, and the node semantic representation and node constraint features of all inference nodes are extracted and differentially aggregated to obtain the link semantic representation corresponding to each inference link. The link semantic representations corresponding to the multiple inference links are fused to obtain the semantic understanding result after context correction.

[0010] A second aspect of the present invention provides a context-based intelligent speech semantic understanding and analysis system, comprising: The first unit is used to acquire the input speech signal, extract the acoustic feature sequence of the input speech signal, input the acoustic feature sequence into the acoustic model for feature decoding, and obtain the candidate text sequence. The second unit is used to construct a contextual semantic representation based on historical interaction information and the current session state, and to perform semantic disambiguation processing on the candidate text sequence based on the contextual semantic representation to obtain a disambiguated semantic unit sequence. The third unit is used to perform association analysis on each semantic unit in the disambiguated semantic unit sequence using multi-level semantic dependency relations, establish cross-level semantic unit association mapping, and obtain a semantic dependency graph. The fourth unit is used to propagate the constraint information of the context semantic representation in the semantic dependency graph, obtain the constraint semantic dependency graph and perform multi-hop reasoning to obtain the semantic understanding result after context correction. The fifth unit is used to generate intent recognition results and slot filling results based on the semantic understanding results after context correction, and to update the current session state information for semantic understanding analysis of subsequent input voice signals.

[0011] A third aspect of the present invention, An electronic device is provided, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0012] Fourth aspect of the present invention, A computer-readable storage medium is provided, having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0013] The beneficial effects of this application are as follows: This invention performs semantic disambiguation on candidate text sequences by constructing contextual semantic representations, which effectively reduces ambiguity in the speech recognition process, improves the accuracy and reliability of speech recognition, and enables the system to more accurately understand the user's true intent.

[0014] This invention utilizes multi-level semantic dependency analysis and cross-level semantic unit association mapping to construct a complete semantic dependency graph, enabling the system to capture the complex relationships and inherent logic between semantic units, greatly enhancing the depth and breadth of semantic understanding.

[0015] This invention propagates contextual semantic constraint information in the semantic dependency graph and performs multi-hop reasoning, realizing dynamic updates of the session state. This enables the system to continuously accumulate dialogue history knowledge, maintain the coherence and consistency of semantic understanding, and thus significantly improve the interactive experience and service quality of the intelligent dialogue system. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the context-based intelligent speech semantic understanding and analysis method according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the text disambiguation process based on contextual semantics. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0019] Figure 1This is a flowchart illustrating the context-based intelligent speech semantic understanding and analysis method according to an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes: The input speech signal is acquired, the acoustic feature sequence of the input speech signal is extracted, and the acoustic feature sequence is input into the acoustic model for feature decoding to obtain the candidate text sequence. Based on historical interaction information and the current session state, a contextual semantic representation is constructed, and semantic disambiguation is performed on the candidate text sequence based on the contextual semantic representation to obtain a disambiguated semantic unit sequence. Multi-level semantic dependency relations are used to perform association analysis on each semantic unit in the disambiguated semantic unit sequence, and a cross-level semantic unit association mapping is established to obtain a semantic dependency graph. The constraint information of the context semantic representation is propagated in the semantic dependency graph to obtain the constrained semantic dependency graph and multi-hop reasoning is performed to obtain the semantic understanding result after context correction. Based on the semantic understanding results after context correction, intent recognition results and slot filling results are generated, and the current session state information is updated for semantic understanding analysis of subsequent input speech signals.

[0020] In one optional implementation, the acoustic feature sequence of the input speech signal is extracted, and the acoustic feature sequence is input into an acoustic model for feature decoding to obtain a candidate text sequence, including: Multi-scale time-frequency analysis is performed on the input speech signal. By extracting time-frequency features under different time window lengths and frequency resolutions, a multi-scale time-frequency feature set is obtained. Based on the energy distribution and spectral stability of the time-frequency features at each scale in the multi-scale time-frequency feature set, feature fusion weights are calculated to obtain an acoustic feature sequence. The acoustic feature sequence is divided into multiple time step feature frames. For each time step feature frame, the current time step feature frame is feature encoded through the coding layer of the acoustic model, and context modeling is performed by combining the hidden state information of the previous time step to obtain the encoded features of the current time step. Based on the coding features of the current time step, the posterior probability distribution of each acoustic unit is calculated. According to the decoding layer of the acoustic model, the candidate acoustic units of the current time step are predicted. The candidate acoustic units are jointly scored by combining the posterior probability distribution and language model constraints to obtain the optimal acoustic unit of the current time step. The optimal acoustic units at each time step are spliced ​​together in chronological order to obtain an acoustic unit sequence; the acoustic unit sequence is then mapped to text to obtain the candidate text sequence.

[0021] In this specific embodiment, multi-scale time-frequency analysis is performed on the input speech signal. This analysis extracts time-frequency features by setting different time window lengths and frequency resolutions. For example, short-time Fourier transform can be performed simultaneously using three window lengths: 20 milliseconds, 30 milliseconds, and 40 milliseconds, to obtain feature representations at different time-frequency resolutions. Specifically, for a speech signal with a sampling rate of 16 kHz, when using a 20-millisecond window length, each frame contains 320 sampling points with a frequency resolution of 50 Hz; when using a 30-millisecond window length, each frame contains 480 sampling points with a frequency resolution of approximately 33.3 Hz; and when using a 40-millisecond window length, each frame contains 640 sampling points with a frequency resolution of 25 Hz. This allows for the acquisition of a multi-scale time-frequency feature set containing feature matrices at three different time-frequency resolutions.

[0022] Based on the energy distribution and spectral stability of the time-frequency features at each scale in the aforementioned multi-scale time-frequency feature set, feature fusion weights are calculated. For each time period in the speech signal, the energy concentration of features at each time-frequency scale is calculated. For example, the uniformity of spectral energy distribution across frequency bands is calculated, and the degree of spectral variation between adjacent frames is analyzed to quantify spectral stability. In specific implementation, the concentration of energy distribution can be represented by calculating the entropy value of the spectrum; the lower the entropy value, the more concentrated the energy distribution. Spectral stability can be represented by calculating the cosine similarity between adjacent frames; the higher the similarity, the more stable the spectrum. For example, features with a 40-millisecond window length receive higher weights for vowel segments because vowels are generally spectrally stable; while features with a 20-millisecond window length receive higher weights for transient speech such as plosives. The calculated weights are applied to the weighted fusion of features at each scale to obtain the fused acoustic feature sequence.

[0023] The acoustic feature sequence is divided into multiple time-step feature frames, typically 10 milliseconds long, with 50% overlap between adjacent frames. For each time-step feature frame, feature encoding is performed through the encoding layer of the acoustic model. The encoding layer contains multiple convolutional neural networks and a self-attention mechanism to extract local and global features. The convolutional neural networks use a 3×3 kernel, a stride of 1, and padding of 1, stacked in 5 layers, with each layer having 256 output channels to capture local time-frequency patterns. The self-attention mechanism uses 8 attention heads with a dimension of 512 to model long-distance dependencies. The encoding process also incorporates the hidden state information of previous time steps for context modeling, employing a bidirectional long short-term memory network structure with 512 hidden units, one layer each for forward and backward passes. Historical information is stored through memory units to achieve context-aware encoding of the current frame, resulting in the encoded features of the current time step.

[0024] Calculate the posterior probability distribution of each acoustic unit based on the encoded features at the current time step. In actual implementation, the acoustic unit can be a phoneme, syllable, character, etc. Assuming that phonemes are used as acoustic units, there are a total of 42 phonemes (including English phonemes and silence markers). The encoded features are mapped to a 42-dimensional output space through a fully connected layer, and the Softmax function is applied to obtain the posterior probability distribution of each phoneme. For example, for the first time step of the character "你" in the input speech "你好", the posterior probability of the phoneme "n" is 0.85, the posterior probability of the phoneme "l" is 0.08, the posterior probability of the phoneme "i" is 0.05, and the probabilities of other phonemes are even lower.

[0025] According to the decoding layer of the acoustic model, predict the candidate acoustic units at the current time step. The decoding layer adopts a beam search-based strategy with a beam width of 10, that is, the 10 candidate phoneme sequences with the highest probabilities are retained at each time step. At the same time, the candidate acoustic units are jointly scored by combining the language model constraints, with a language model weight of 0.7 and an acoustic model weight of 0.3. For example, when recognizing "你好", if the posterior probabilities of "n i h ao" given by the acoustic model are 0.85, 0.92, 0.78, 0.89 respectively, and the probability of the sequence "n i h ao" given by the language model is 0.95, then the final score is 0.3×(0.85 + 0.92 + 0.78 + 0.89) / 4 + 0.7×0.95 = 0.8555, which is higher than other candidate sequences, so it is selected as the optimal phoneme sequence.

[0026] Concatenate the optimal acoustic units at each time step in chronological order to obtain a complete acoustic unit sequence. In actual applications, adjacent frames may predict the same phoneme. In this case, duplicate phonemes need to be merged and only one is retained. For example, if the phoneme "a" is predicted for 5 consecutive frames, only one "a" is retained in the final sequence. Through this de-duplication process, a simplified phoneme sequence such as "n i h ao" is obtained.

[0027] Perform a mapping conversion from the acoustic unit sequence to text to obtain a candidate text sequence. If the acoustic units are phonemes, the phoneme sequence needs to be converted to text through a phoneme-to-character mapping dictionary. The conversion process uses the forward maximum matching algorithm, starting from the beginning position of the acoustic unit sequence, trying to match the longest phoneme sequence. Assume that the phoneme sequence at the current position is / nihao / . Search for the longest matching item starting with this phoneme sequence in the dictionary. If a complete match is found, the corresponding text segment is added to the output result. When the phoneme sequence / nihao / matches the text "你好", "你好" is added to the candidate text sequence, and the matching position is moved to the next unprocessed phoneme.

[0028] For the case of fuzzy matching, the edit distance algorithm is used to calculate the similarity. When the phoneme combination generated by the acoustic unit sequence has no exact match in the dictionary, the edit distance between this phoneme sequence and each entry in the dictionary is calculated. The edit distance is obtained by counting the minimum number of operations required to convert the source sequence into the target sequence, including insertion, deletion, and substitution operations. Suppose the edit distance between the phoneme sequence / niha / and the dictionary entry / nihao / is 1, and the edit distance with / nihe / is 2. The entry with the minimum edit distance is selected as the matching result, that is, the text "你好" corresponding to / nihao / is selected.

[0029] During the processing, the system will handle the ambiguity of phoneme combinations. For example, the phoneme sequence "h ao l e" can be mapped to various situations such as "好了" or "号了". At this time, the most reasonable mapping result will be selected according to the context semantics and the scoring of the language model. For example, in the case where "你" is mentioned above, the probability of "好" following is higher. Therefore, the output text sequence is "你好". After processing the entire acoustic unit sequence, it is arranged in the chronological order of the original speech to form the final output result of speech recognition. For example, for a 3-second speech segment input, after 150 time steps of decoding and mapping conversion, the candidate text sequence "你好,欢迎光临" is obtained, and this text accurately reflects the semantic content of the original speech, completing the complete conversion process from acoustic signal to text representation.

[0030] Figure 2 It is a schematic diagram of the text disambiguation processing flow based on context semantics. In an optional implementation manner, according to the historical interaction information and the current session state, a context semantic representation is constructed, and semantic disambiguation processing is performed on the candidate text sequence based on the context semantic representation to obtain a disambiguated semantic unit sequence, including: Based on the semantic integrity and session transition identifiers in the historical interaction information, the historical interaction information is divided into multiple interaction levels and session sub-topics are extracted, and the semantic topic of the current session and the current dialogue scenario are extracted; Calculate the semantic similarity between the session sub-topic and the semantic topic, and the temporal distance between each interaction level and the current moment. Select the interaction levels with a semantic similarity greater than the similarity threshold and a temporal distance greater than the temporal threshold, encode the corresponding historical interaction information, and combine the current dialogue scenario to obtain the context semantic representation; Perform semantic unit segmentation on the candidate text sequence to obtain an initial semantic unit sequence, perform polysemous word list matching and context-independent semantic feature analysis on each semantic unit, and identify the ambiguous semantic units with polysemy; For each ambiguous semantic unit, context constraint information related to the semantics of the ambiguous semantic unit is extracted from the context semantic representation, and the context matching degree of each candidate semantic of the ambiguous semantic unit is calculated. The candidate semantic with the highest context matching degree is selected as the disambiguated semantic of the ambiguous semantic unit. Each ambiguous semantic unit in the initial semantic unit sequence is replaced with the corresponding disambiguated semantic unit to obtain the disambiguated semantic unit sequence.

[0031] In this implementation, historical interaction information and the current session state need to be acquired. Historical interaction information includes the historical dialogue content between the user and the system, and the current session state includes the user's current voice input and the current interaction environment information. Based on the semantic integrity and session transition markers in the historical interaction information, the historical interaction information is divided into multiple interaction levels and session sub-topics are extracted. Semantic integrity is identified by detecting complete question-and-answer pairs or semantic loops in the dialogue. For example, when a user asks "What will the weather be like tomorrow?", the answer "Tomorrow will be sunny, and the temperature will be 25°C" constitutes a semantically complete interaction unit. Session transition markers are determined by recognizing topic-switching words (such as "also" or "by the way") or semantic jumps. For example, when a user suddenly switches from asking about the weather to asking about the news, this transition point will be marked. The historical interaction information is divided into several interaction levels according to semantic integrity and session transition markers, such as recent interactions (the last 3 rounds of dialogue), mid-term interactions (the first 4-10 rounds of dialogue), and long-term interactions (more than 10 rounds of dialogue). For each interaction level, its core semantics are extracted as session sub-topics, such as "weather query," "restaurant recommendation," and "schedule."

[0032] Simultaneously, the semantic topic of the current conversation is extracted by analyzing the semantic keywords and semantic intent of the user's current input. The current dialogue scenario is determined by detecting contextual information such as the user's environment, time, and location. For example, if the user inputs "search for nearby restaurants", the extracted semantic topic is "restaurant search". The current dialogue scenario includes the user's location information and the current time (lunch or dinner time).

[0033] The system calculates the semantic similarity between conversational subtopics and semantic topics, as well as the temporal distance between each interaction level and the current moment. Semantic similarity is calculated by the cosine similarity between topic word vectors. For example, if the current semantic topic is "restaurant recommendation," and the historical conversational subtopics are "food search" and "weather forecast," the semantic similarity between "restaurant recommendation" and "food search" is calculated to be 0.85, and the semantic similarity with "weather forecast" is 0.25. Temporal distance is determined by calculating the difference between the time of the interaction and the current time. For example, the temporal distance for recent interactions is 1, for mid-term interactions it is 5, and for long-term interactions it is 15. The preset similarity threshold is 0.7, and the temporal threshold is 10. Interaction levels with semantic similarity greater than 0.7 and temporal distance less than 10 are selected, and their corresponding historical interaction information is encoded. In this example, the system selects the historical interaction information related to "food search," which has a semantic similarity of 0.85 and a temporal distance of 5.

[0034] The selected historical interaction information is encoded through a deep neural network and combined with the current dialogue scenario information to generate a contextual semantic representation. This representation contains historical information and scenario constraints related to the current semantic topic, providing a foundation for subsequent semantic disambiguation.

[0035] After obtaining the contextual semantic representation, the candidate text sequence is segmented into semantic units to obtain an initial semantic unit sequence. The segmentation is based on grammatical rules and semantic boundary recognition. For example, the input text "I want to eat spicy Sichuan cuisine" is segmented into four semantic units: "I", "want to eat", "spicy", and "Sichuan cuisine". For each semantic unit, polysemous word matching and context-independent semantic feature analysis are performed to identify ambiguous semantic units with polysemy. A polysemous word list is maintained, which contains common polysemous words and their different semantic interpretations. For example, "apple" can mean fruit or technology company. Context-independent semantic feature analysis is completed by checking the semantic ambiguity of the semantic unit itself. For example, "Sichuan cuisine" has no obvious ambiguity in context-independent cases, while "apple" has obvious polysemy.

[0036] For the identified ambiguous semantic units, contextual constraint information related to the semantics of the ambiguous semantic unit is extracted from the contextual semantic representation. For example, for the identified text "I want to buy an apple" ("apple"), relevant constraint information is extracted from the contextual semantic representation, such as whether the user has recently searched for electronic products and whether they are currently in a shopping scenario. The context matching degree of each candidate semantic (e.g., "apple-fruit" and "apple-electronic product") of the ambiguous semantic unit is calculated. The context matching degree is determined by calculating the semantic consistency score between the candidate semantic and the contextual constraint information. Assuming that the system detects from the context that the user has recently searched for electronic products and is currently in an electronic product shopping scenario, the context matching degree of "apple-electronic product" is 0.9, while the context matching degree of "apple-fruit" is 0.3. The candidate semantic with the highest context matching degree is selected as the disambiguated semantic of the ambiguous semantic unit, which in this example is "apple-electronic product".

[0037] The ambiguous semantic units in the initial semantic unit sequence are replaced with their corresponding disambiguated semantic units to obtain the disambiguated semantic unit sequence. For example, the initial semantic unit sequence of the original text "I want to buy an apple" "I", "want to buy", "one", "apple (ambiguous)" is converted to "I", "want to buy", "one", "apple (electronic product)", thus completing the semantic disambiguation process. The disambiguated semantic unit sequence will be used for subsequent semantic understanding and task execution, such as product recommendation and information retrieval.

[0038] In one optional implementation, multi-level semantic dependency relations are used to perform association analysis on each semantic unit in the disambiguated semantic unit sequence to establish a cross-level semantic unit association mapping, resulting in a semantic dependency graph, including: Part-of-speech tagging and semantic role tagging are performed on each semantic unit in the disambiguated semantic unit sequence to construct the feature representation of each semantic unit; Based on the feature representation of each semantic unit, the lexical collocation strength between adjacent semantic units is calculated at the lexical level, the syntactic dominance and syntactic subordination relationships between semantic units are calculated at the syntactic level, and the discourse cohesion and discourse coherence relationships between cross-sentence semantic units are calculated at the discourse level, thereby obtaining the lexical dependency set, the syntactic dependency set, and the discourse dependency set, respectively. The lexical dependency relation set, the syntactic dependency relation set, and the discourse dependency relation set are hierarchically fused to establish a cross-level semantic unit association mapping. Each level dependency relation is assigned a hierarchical distinguishing identifier, and the dependency strength weight corresponding to each level dependency relation is calculated to generate a semantic dependency graph containing each semantic unit as a node, each level dependency relation as an edge, and the dependency strength weight as the edge weight.

[0039] Each semantic unit in the disambiguated semantic unit sequence is subjected to part-of-speech tagging and semantic role tagging to construct a feature representation for each semantic unit. Part-of-speech tagging adopts a hybrid method integrating deep learning and rules to assign a part-of-speech label to each semantic unit, such as noun, verb, adjective, etc. For example, for the semantic unit "apple", it is labeled as a noun (n) based on the context. Semantic role tagging identifies the role played by the semantic unit in the sentence, such as agent, patient, time, and place. For the sentence "Xiaoming is reading in the classroom", "Xiaoming" is labeled as agent, "classroom" is labeled as location, and "reading" is labeled as predicate. The feature representation includes a multi-dimensional feature vector containing the semantic unit itself, part-of-speech tag, semantic role tag, location information, and context information.

[0040] Based on the feature representations of each semantic unit, the lexical collocation strength between adjacent semantic units is calculated at the lexical level. The lexical collocation strength is calculated through point mutual information and co-occurrence frequency. For example, for the adjacent semantic units "artificial" and "intelligent", their co-occurrence frequency and their independent occurrence frequency in the corpus are counted, and the collocation strength value is calculated to be 0.85. For the semantic unit sequence "deep learning technology application", the collocation strength of "deep-learning" is calculated to be 0.78, "learning-technology" is 0.62, and "technology-application" is 0.81, thus obtaining the set of lexical dependency relations.

[0041] Syntactic dominance and subordination relations between semantic units are calculated at the syntactic level. Through dependency parsing, the master-subordinate relationships between semantic units are determined. Syntactic dominance refers to a semantic unit acting as a dominating node, controlling other semantic units, such as a predicate dominating a subject and object. Syntactic subordination refers to a semantic unit depending on another semantic unit, such as an adjective modifying a noun. For the sentence "Intelligent systems process complex data," "process" is the core predicate. "System" and "process" form a subject-predicate relationship with a dominance strength of 0.9; "process" and "data" form a verb-object relationship with a dominance strength of 0.85; "intelligent" and "system" form a modifier-head relationship with a subordination strength of 0.75; and "complex" and "data" form a modifier-head relationship with a subordination strength of 0.8. These relationships constitute the set of syntactic dependency relations.

[0042] At the discourse level, we calculate discourse cohesion and coherence relationships between semantic units across sentences. Discourse cohesion relationships include referential, substitution, ellipsis, conjunction, and lexical cohesion relationships. For example, in the adjacent sentences "Machine learning models need a lot of data, and this data must be preprocessed," we identify a referential relationship between "this data" and "data" in the preceding sentence, with a cohesion strength of 0.95. Discourse coherence relationships focus on semantic logical relationships, such as causal, adversative, and parallel relationships. For example, between the sentences "The system malfunctioned, and the technicians immediately repaired it," we identify an implicit causal relationship, with a coherence strength of 0.82. By analyzing various relationships at the discourse level, we construct a set of discourse dependency relationships.

[0043] The sets of lexical, syntactic, and discourse dependencies are hierarchically fused to establish a cross-level semantic unit association mapping. The hierarchical fusion adopts a layered weighting mechanism, assigning hierarchical distinguishing identifiers to different levels of dependency relations: the lexical level identifier is "L", the syntactic level identifier is "S", and the discourse level identifier is "D". For multi-level dependency relations between the same pair of semantic units, a comprehensive dependency strength weight is calculated. For example, for the semantic units "intelligent" and "system", they have a collocation relationship at the lexical level (L identifier) ​​with a weight of 0.8; they have a nominative-heading relationship at the syntactic level (S identifier) ​​with a weight of 0.75; the comprehensive dependency strength weight is the weighted average of the two, resulting in 0.78.

[0044] Through the above processing, a semantic dependency graph is generated, containing semantic units as nodes, hierarchical dependency relationships as edges, and dependency strength weights as edge weights. In this graph, nodes contain semantic units and their feature information, and edges contain dependency type, hierarchical identifier, and weight information. For example, for the text "Intelligent systems can effectively process complex data, and this processing method improves efficiency," the generated semantic dependency graph contains the nodes: "intelligent," "system," "can," "effective," "process," "complex," "data," "this," "processing method," "improves," and "efficiency"; the edges include: "intelligent" - "system" (S: attributive, 0.75), and "system" - "process" (S: subject-verb, 0.9). "Able to" - "process" (S: auxiliary verb, 0.7), "Effective" - ​​"process" (S: adverbial, 0.68), "Process" - "data" (S: verb object, 0.85), "Complex" - "data" (S: attributive, 0.8), "Process" - "processing method" (D: pronoun, 0.92), "Processing method" - "improve" (S: subject verb, 0.87), "Improve" - ​​"efficiency" (S: verb object, 0.83), etc.

[0045] This semantic dependency graph can capture multi-level semantic relationships in text, supporting more accurate semantic understanding and reasoning. By analyzing the connectivity paths and weight distribution between nodes, core semantic relationships and key information points can be identified, providing basic support for subsequent applications such as text summarization, question answering systems, and knowledge graph construction.

[0046] In one optional implementation, the constraint information of the contextual semantic representation is propagated in the semantic dependency graph to obtain a constrained semantic dependency graph, and multi-hop reasoning is performed to obtain a context-corrected semantic understanding result, including: The dialogue intent, semantic orientation, and topic scope are extracted from the context semantic representation and encoded as constraint features. The constraint features are then injected into the semantic unit nodes associated with the context semantic representation, which serve as constraint propagation points. Starting from the constraint propagation point, constraint features are propagated. By calculating the semantic correlation and constraint compatibility between adjacent semantic unit nodes, the propagation strength and direction of constraint information during the propagation process are determined. The constraint features are propagated in multiple rounds of iteration, so that the constraint information gradually spreads from the constraint propagation point to the far semantic unit nodes in the semantic dependency graph, thus obtaining the constraint semantic dependency graph. Based on the constrained semantic dependency graph, starting from the inference start node, hop-by-hop inference expansion is performed to obtain multiple inference links. The semantic unit nodes on each inference link are aggregated to obtain the corresponding link semantic representation and then globally fused to generate the context-corrected semantic understanding result.

[0047] In this specific embodiment, the input dialogue text sequence is received, and a semantic dependency graph is constructed through a semantic parser. The semantic dependency graph contains multiple semantic unit nodes and dependency relationship edges between nodes, where semantic unit nodes represent the core semantic components in the text, and dependency relationship edges represent the semantic association types between nodes. For example, for the recognized speech text "I want to check yesterday's weather", the semantic dependency graph contains semantic unit nodes such as "I", "want to check", "yesterday", and "weather", as well as dependency relationship edges such as "subject-behavior", "time-behavior", and "behavior-object".

[0048] When extracting constraint features from the contextual semantic representation, the dialogue history is encoded to obtain the contextual semantic representation, which contains global information about the dialogue. The feature extraction module identifies the dialogue intent, semantic orientation, and topic scope from the contextual semantic representation. The dialogue intent represents the user's current interaction purpose, such as "query" or "booking"; the semantic orientation represents the current semantic focus, such as "weather" or "flight"; and the topic scope represents the domain boundaries involved in the dialogue, such as "travel" or "entertainment". These three types of information are fused and encoded into a constraint feature vector with a dimension of 256. This vector contains the constraints of the context on the current semantic understanding.

[0049] During the constraint feature injection process, based on the correlation between the current context semantic representation and the semantic unit nodes, the 3-5 nodes with the highest correlation are selected as constraint propagation points. The correlation is calculated based on semantic similarity, with a value ranging from 0 to 1. Nodes with a correlation greater than 0.75 are selected as propagation points. In the example above, the "want to query" and "weather conditions" nodes are identified as constraint propagation points, and constraint feature vectors are injected into these nodes, making them the starting point for the propagation of constraint information.

[0050] The constraint feature propagation process employs an iterative diffusion mechanism, calculating the semantic correlation and constraint compatibility between adjacent semantic unit nodes to determine the propagation strength and direction. Semantic correlation represents the semantic coherence between nodes, calculated based on dependency relationship type and node representation. Constraint compatibility represents the degree of adaptation of a node to receive constraint information, calculated based on the matching degree between node semantic type and constraint features. The propagation strength is derived by weighting the two factors, with weights of 0.6 and 0.4 respectively. Five rounds of iterative propagation are performed. In each round, nodes update their semantic representations by fusing the incoming constraint information. For example, the propagation strength from the "want to query" node to the "weather conditions" node is 0.85, indicating that the constraint information can propagate strongly from the former to the latter. After propagation, each node in the semantic dependency graph incorporates constraint information from the context, forming a constraint semantic dependency graph.

[0051] When performing multi-hop inference based on constraint semantic dependency graphs, the node most relevant to the user's current input is selected as the starting node for inference. A breadth-first search approach is used, expanding hop by hop from the starting node, with a maximum number of hops set to 3. During each hop expansion, the system selects the most reasonable expansion direction based on the dependency relationship edges between nodes and the constraint semantic information, generating multiple inference links. In the example above, starting from "weather conditions," an inference link is formed as "weather conditions → yesterday → specific date (May 10, 2023)."

[0052] During the link aggregation process, information is integrated for each node on each inference link, the semantic representation of each node on the link is extracted, the weight of the dependency relationship between nodes is considered, and the node representations are weighted and summed to obtain the link semantic representation. The node weight is determined based on its position and semantic importance in the link, with higher weights for nodes closer to the starting node and correspondingly higher weights for key semantic nodes.

[0053] In the global fusion phase, the semantic representations of all inference links are adaptively fused to obtain the final semantic understanding result. The fusion process uses an attention mechanism to assign different weights to different links. The weight calculation is based on the relevance of the link to the current query and the semantic consistency within the link. The final output is the fused semantic understanding result, including information such as identified entities, intents, and slot values. In the example above, the understood user intent is "query the weather," and the time is "yesterday (May 10, 2023)," even if the user does not explicitly provide complete date information.

[0054] The above methods can effectively integrate contextual constraint information for semantic understanding, improve the ability of dialogue systems to handle complex interactions, and significantly enhance the processing performance of scenarios involving omission, reference, and multi-turn interactions.

[0055] In one optional implementation, based on the constrained semantic dependency graph, starting from each inference initiation node, hop-by-hop inference expansion is performed to obtain multiple inference links. Semantic unit nodes on each inference link are aggregated to obtain the corresponding link semantic representations, which are then globally fused to generate a context-corrected semantic understanding result, including: In the constrained semantic dependency graph, the inference starting node is determined, and the adjacent semantic unit node with the highest edge correlation is selected as the first hop inference node. The hop inference extension is performed along the edge of the constrained semantic dependency graph. In each hop of reasoning extension, based on the semantic representation, constraint feature representation and reasoning context information passed from the reasoning start node to the current hop, an inter-hop reasoning state vector is constructed. Combined with the semantic representation of adjacent semantic unit nodes, an inter-hop extension score is calculated. The adjacent semantic unit node with the highest inter-hop extension score is selected as the next hop reasoning node and the reasoning context information is updated. The subsequent hop reasoning extension is iteratively executed until the preset reasoning depth is reached or the information entropy of the reasoning context information is lower than the convergence threshold, thus obtaining multiple reasoning links starting from each reasoning start node. On the multiple inference links, the position decay weight is calculated based on the hop count of each inference node, and the node semantic representation and node constraint features of all inference nodes are extracted and differentially aggregated to obtain the link semantic representation corresponding to each inference link. The link semantic representations corresponding to the multiple inference links are fused to obtain the semantic understanding result after context correction.

[0056] In this embodiment, the constrained semantic dependency graph is a directed graph structure composed of semantic unit nodes and relational edges between nodes. In practical application scenarios, semantic unit nodes represent semantic units in the text, such as entities, attributes, events, etc. Each semantic unit node contains a node semantic representation and a node constraint feature. The node semantic representation is usually a vector, such as a vector with a dimension of 768, used to express the semantic information of the node. The node constraint feature is also in vector form, used to express the constraint conditions and restriction information of the node. The relational edge represents the semantic relationship between two semantic unit nodes. Each edge has an edge correlation attribute, which represents the strength of the semantic association between two nodes, usually a real value between 0 and 1.

[0057] In the stage of determining the starting node for inference, key nodes in the constrained semantic dependency graph are selected as the starting points for inference based on the importance scores of the nodes. The importance score can be calculated based on factors such as the in-degree, semantic features, and query relevance of the node. For example, in a question-answering system, the entity node most relevant to the user's question can be selected as the starting node for inference; in a knowledge reasoning task, a concept node with high centrality can be selected as the starting point.

[0058] After determining the starting node for inference, the system performs a first-hop inference extension starting from each starting node. The system checks all nodes directly connected to the starting node, calculates the edge correlation degree, and selects the adjacent node with the highest edge correlation degree as the first-hop inference node. For example, if the starting node A is connected to nodes B, C, and D, and the edge correlation degrees are 0.8, 0.6, and 0.5 respectively, then node B is selected as the first-hop inference node.

[0059] In each subsequent hop of inference expansion, an inter-hop inference state vector is constructed. This vector integrates three parts of information: the semantic representation of the current hop inference node, the constraint feature representation, and the inference context information passed from the inference start node to the current hop. The inference context information includes the semantic and path information of the nodes visited during the inference process, which is used to guide the direction of subsequent inference.

[0060] Specifically, an attention mechanism is used to fuse the semantic representation of the current node, the constraint feature representation, and the inference context information to generate an inter-jump inference state vector. Combined with the semantic representation of adjacent semantic unit nodes, an inter-jump expansion score is calculated. The inter-jump expansion score comprehensively considers factors such as edge correlation, node semantic relevance, and path consistency to evaluate the rationality of expanding to the node.

[0061] For example, suppose the current inference node is P, and its neighboring nodes are Q, R, and S. The hop expansion scores from P to these three nodes are calculated to be 0.75, 0.68, and 0.82, respectively. Then, node S is selected as the next hop inference node. At the same time, the inference context information is updated, and the semantic information and constraint features of node S are integrated into the inference context.

[0062] The above reasoning expansion process is iteratively executed until a preset reasoning depth (such as 3 or 5 hops) is reached or the information entropy of the reasoning context is lower than the convergence threshold (such as 0.05). Information entropy is used to measure the uncertainty of the reasoning context. When the information entropy drops below a certain threshold, it indicates that the reasoning has become stable and the expansion can be terminated.

[0063] Through the above process, multiple different inference links are formed starting from each inference starting node. For example, two different paths can be formed from the starting node X: X→Y→Z→W and X→A→B→C.

[0064] For the obtained multiple inference links, link aggregation is performed. For each inference link, the system calculates a position decay weight based on the hop count of each inference node. The position decay weight typically decreases as the hop count increases; for example, an exponential decay method can be used, with the weight of the i-th hop node being α. (i-1) , where α is the attenuation factor, with a value such as 0.8.

[0065] After calculating the position decay weight, the semantic representations and constraint features of all inference nodes on the link are extracted, and differentiated aggregation is performed based on the position decay weight. Specifically, the semantic representations of nodes are aggregated using a weighted average method, and the constraint features of nodes are aggregated using a weighted max pooling method, thereby preserving the salient features of the constraint information. Through this differentiated aggregation strategy, the semantic representation of each inference link is obtained.

[0066] The semantic representations of each link corresponding to multiple inference links are globally fused. During the fusion process, the system weights each link according to its credibility score. The credibility score is based on a comprehensive evaluation of link length, node importance, and edge correlation. Through global fusion, a context-corrected semantic understanding result is generated. This result comprehensively considers the semantic information and constraint features of multiple inference paths, thereby more accurately understanding and expressing the semantic content of the input text.

[0067] In practical applications, this method can be used for tasks such as complex question-answering systems, knowledge reasoning, and text understanding. By performing multi-hop reasoning on constrained semantic dependency graphs, it captures deep semantic relationships, effectively handles complex semantic understanding problems, and improves the accuracy and completeness of semantic understanding.

[0068] This invention also provides a specific embodiment whereby a voice assistant receives a user's spoken voice signal, extracts an acoustic feature sequence from it, and inputs the acoustic feature sequence into an acoustic model containing a 6-layer encoder and a 4-layer decoder. Each layer of the encoder contains 512 hidden units and uses a bidirectional loop structure to capture contextual information. The decoder outputs one acoustic unit at each time step and maintains 5 candidate paths through a beam search strategy, obtaining the candidate text sequence with the highest probability, "Book me a meeting room for tomorrow afternoon at 3 pm". At this point, the system has processed 3 rounds of dialogue. Historical interaction information includes the user saying "Check today's schedule" in the first round, receiving the reply "You have a project review meeting at 10 am today", and the user saying "Cancel this meeting" in the second round, receiving the reply "Your project review meeting at 10 am has been canceled". The current session state is recorded as dialogue round 3, the user's intention in the previous round was to cancel the meeting, and the relevant slots include a time slot value of 10 am and an event type slot value of project review meeting.

[0069] When constructing the contextual semantic representation, the text in the historical interaction information is converted into word vector representations, with each word vector having a dimension of 300. For the candidate text "Help me book a meeting room at 3 PM tomorrow," it is segmented into the word sequence "help, me, book, tomorrow, afternoon, 3 PM, of, meeting room," totaling 8 word units. Keywords extracted from the historical dialogue include "query, schedule, cancel, meeting, 10 AM, project review meeting," totaling 6 keywords. The 8 word vectors of the current candidate text and the 6 word vectors of the historical keywords are input into an attention mechanism network, which calculates the association strength between the current word and the historical words. The association strength between the word "book" and the historical word "cancel" is 0.72, and the association strength between "book" and the historical word "meeting" is 0.85. The context enhancement vector of the word "book" is obtained by weighted summation. The same operation is performed on each word in the candidate text to obtain 8 context enhancement word vectors. These vectors are then fused into a 512-dimensional contextual semantic representation vector through average pooling.

[0070] The word "three points" in the candidate text sequence is semantically ambiguous. Disambiguation is performed using contextual semantic representation. The cosine similarity between the contextual representation vector and the candidate semantic vector of the word "three points" is calculated. The similarity between the time semantic vector and the contextual representation is 0.81, and the similarity between the quantity semantic vector and the contextual representation is 0.34. The time semantic vector with higher similarity is selected as the disambiguation result. The word "tomorrow" is parsed into a specific date using timestamps. Assuming the current date is October 15th, "tomorrow" is parsed as October 16th. The disambiguated semantic unit sequence contains the action semantic unit "booking", the time semantic unit "15:00 on October 16th", and the object semantic unit "meeting room". These three semantic units constitute the disambiguated semantic unit sequence.

[0071] Multi-level semantic dependency analysis was performed on the disambiguated semantic unit sequence. At the lexical level, a time modification relationship was identified between "afternoon" and "3 o'clock," and a date constraint relationship was identified between "tomorrow" and "3 o'clock in the afternoon." At the syntactic level, "book" was identified as the core predicate governing the object "meeting room," and "tomorrow afternoon 3 o'clock" was identified as a time adverbial modifying the predicate "book." At the semantic level, the verb-object relationship between the booking action and the meeting room object, and the constraint relationship between the time attribute and the booking action were identified. When establishing cross-level associations, the time modification relationship at the lexical level was mapped to the semantic level, forming a dependency edge between the time semantic unit "15:00 on October 16th" and the action semantic unit "book," with the edge weight set to 0.9 to indicate a strong dependency relationship. A dependency edge was also established between the object semantic unit "meeting room" and the action semantic unit "book," with a weight set to 0.95. The three semantic units served as nodes, and the two dependency edges served as connections, forming a semantic dependency graph containing 3 nodes and 2 edges.

[0072] When propagating the constraint information of contextual semantic representation in the semantic dependency graph, the 512-dimensional contextual representation vector is decomposed into three sub-vectors, each with a dimension of 170, which are associated with three semantic unit nodes. The context sub-vector received by the action node "booking" contains the reverse semantic information of the "cancel" action in the historical dialogue. This information is used as a constraint to adjust the semantic weight of the action node. The context sub-vector received by the time node contains the comparative information of the historical time "10:00 AM", strengthening the distinction between the current time "3:00 PM" and the historical time. The context sub-vector received by the object node contains the semantic information of the historical object "project review meeting", establishing the association between the current object "meeting room" and the historical object. The constraint information is propagated in the graph through dependency edges. The constraint information of the action node is passed to the time node and the object node along the dependency edges. During the propagation process, the information intensity decays according to the edge weight. After three rounds of propagation iterations, the semantic representation of each node is integrated with the constraint information of its neighboring nodes, forming a constraint semantic dependency graph.

[0073] When performing multi-hop inference on a constrained semantic dependency graph, starting from the action node "Booking," the process jumps along the dependency edge to the time node "15:00 on October 16th." The semantic representation of this node contains time constraint information, indicating that the booking action must be associated with a specific time. The process continues from the time node back to the action node, which updates its semantic representation, reinforcing the time-sensitive attribute of the booking action. The process then jumps to the object node "Meeting Room," whose semantic representation contains resource type constraints, indicating that the booking object is a venue resource. The process jumps back from the object node to the action node, which updates its semantic representation again, incorporating the object type information. After two hops of inference, the semantic representation of the action node simultaneously contains both time and object constraints, forming a complete semantic understanding. The semantic vectors of the three nodes, after context correction, are concatenated to obtain a 510-dimensional comprehensive semantic vector, which serves as the result of the context-corrected semantic understanding.

[0074] When generating intent recognition results based on semantic understanding, a 510-dimensional semantic vector is input into an intent classifier. This classifier contains 12 intent categories, including booking resources, canceling bookings, querying information, and modifying settings. The classifier outputs the probability distribution of each intent category. The probability of booking resources is 0.89, the probability of canceling bookings is 0.04, the probability of querying information is 0.03, and the probabilities of the remaining intents are all below 0.02. The booking resource with the highest probability is selected as the intent recognition result. Slot filling is achieved through sequence labeling. Each word in the candidate text "Book me a meeting room at 3 PM tomorrow" is labeled with a slot label. The word "3 PM tomorrow" is labeled as a time slot with the filling value "October 16th, 3:00 PM". The word "meeting room" is labeled as an object slot with the filling value "meeting room resource". The slot filling result contains two slots: a time slot and an object slot, each corresponding to a specific filling value.

[0075] When updating the current session state information, the number of dialogue rounds is increased from 3 to 4. The user intent of the current round is recorded as "reserving resources," involving slots including a time slot with a value of 15:00 on October 16th and an object slot with a value of "meeting room resources." The intent and slot information of the current round are appended to the historical interaction information list, which now contains a complete record of 4 rounds of dialogue. The updated session state information is stored in a memory cache with a cache capacity set to retain the most recent 10 rounds of dialogue records. When the user continues to input the next round of voice signal "it needs to be able to accommodate ten people," the updated session state is read. The reservation intent and meeting room object slot in the current session state are used as historical context to understand the association between "ten people" in the new input voice and the meeting room capacity requirement, thus achieving contextual continuation in continuous dialogue.

[0076] The present invention provides a context-based intelligent speech semantic understanding and analysis system, comprising: The first unit is used to acquire the input speech signal, extract the acoustic feature sequence of the input speech signal, input the acoustic feature sequence into the acoustic model for feature decoding, and obtain the candidate text sequence. The second unit is used to construct a contextual semantic representation based on historical interaction information and the current session state, and to perform semantic disambiguation processing on the candidate text sequence based on the contextual semantic representation to obtain a disambiguated semantic unit sequence. The third unit is used to perform association analysis on each semantic unit in the disambiguated semantic unit sequence using multi-level semantic dependency relations, establish cross-level semantic unit association mapping, and obtain a semantic dependency graph. The fourth unit is used to propagate the constraint information of the context semantic representation in the semantic dependency graph, obtain the constraint semantic dependency graph and perform multi-hop reasoning to obtain the semantic understanding result after context correction. The fifth unit is used to generate intent recognition results and slot filling results based on the semantic understanding results after context correction, and to update the current session state information for semantic understanding analysis of subsequent input voice signals.

[0077] A third aspect of the present invention provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0078] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0079] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A context-based intelligent speech semantic understanding and analysis method, characterized in that, include: The input speech signal is acquired, the acoustic feature sequence of the input speech signal is extracted, and the acoustic feature sequence is input into the acoustic model for feature decoding to obtain the candidate text sequence. Based on historical interaction information and the current session state, a contextual semantic representation is constructed, and semantic disambiguation is performed on the candidate text sequence based on the contextual semantic representation to obtain a disambiguated semantic unit sequence. Multi-level semantic dependency relations are used to perform association analysis on each semantic unit in the disambiguated semantic unit sequence, and a cross-level semantic unit association mapping is established to obtain a semantic dependency graph. The constraint information of the context semantic representation is propagated in the semantic dependency graph to obtain the constrained semantic dependency graph and multi-hop reasoning is performed to obtain the semantic understanding result after context correction. Based on the semantic understanding results after context correction, intent recognition results and slot filling results are generated, and the current session state information is updated for semantic understanding analysis of subsequent input speech signals.

2. The method according to claim 1, characterized in that, Extract the acoustic feature sequence of the input speech signal, input the acoustic feature sequence into the acoustic model for feature decoding, and obtain a candidate text sequence, including: Multi-scale time-frequency analysis is performed on the input speech signal. By extracting time-frequency features under different time window lengths and frequency resolutions, a multi-scale time-frequency feature set is obtained. Based on the energy distribution and spectral stability of the time-frequency features at each scale in the multi-scale time-frequency feature set, feature fusion weights are calculated to obtain an acoustic feature sequence. The acoustic feature sequence is divided into multiple time step feature frames. For each time step feature frame, the current time step feature frame is feature encoded through the coding layer of the acoustic model, and context modeling is performed by combining the hidden state information of the previous time step to obtain the encoded features of the current time step. Based on the coding features of the current time step, the posterior probability distribution of each acoustic unit is calculated. According to the decoding layer of the acoustic model, the candidate acoustic units of the current time step are predicted. The candidate acoustic units are jointly scored by combining the posterior probability distribution and language model constraints to obtain the optimal acoustic unit of the current time step. The optimal acoustic units at each time step are spliced ​​together in chronological order to obtain an acoustic unit sequence; the acoustic unit sequence is then mapped to text to obtain the candidate text sequence.

3. The method according to claim 1, characterized in that, Based on historical interaction information and the current session state, a contextual semantic representation is constructed. Semantic disambiguation is then performed on the candidate text sequence based on this contextual semantic representation to obtain a disambiguated semantic unit sequence, including: Based on the semantic integrity and conversation transition markers in the historical interaction information, the historical interaction information is divided into multiple interaction levels and conversation subtopics are extracted, and the semantic topic of the current conversation and the current dialogue scenario are extracted. Calculate the semantic similarity between the conversation subtopic and the semantic topic, as well as the temporal distance between each interaction level and the current time. Select the interaction levels with semantic similarity greater than the similarity threshold and temporal distance greater than the temporal threshold, encode their corresponding historical interaction information, and combine them with the current dialogue scenario to obtain the contextual semantic representation. The candidate text sequence is segmented into semantic units to obtain an initial semantic unit sequence. Each semantic unit is then subjected to polysemous word matching and context-independent semantic feature analysis to identify ambiguous semantic units with polysemy. For each ambiguous semantic unit, context constraint information related to the semantics of the ambiguous semantic unit is extracted from the context semantic representation, and the context matching degree of each candidate semantic of the ambiguous semantic unit is calculated. The candidate semantic with the highest context matching degree is selected as the disambiguated semantic of the ambiguous semantic unit. Each ambiguous semantic unit in the initial semantic unit sequence is replaced with the corresponding disambiguated semantic unit to obtain the disambiguated semantic unit sequence.

4. The method according to claim 1, characterized in that, Multi-level semantic dependency relations are used to perform association analysis on each semantic unit in the disambiguated semantic unit sequence, establishing cross-level semantic unit association mappings to obtain a semantic dependency graph, including: Part-of-speech tagging and semantic role tagging are performed on each semantic unit in the disambiguated semantic unit sequence to construct the feature representation of each semantic unit; Based on the feature representation of each semantic unit, the lexical collocation strength between adjacent semantic units is calculated at the lexical level, the syntactic dominance and syntactic subordination relationships between semantic units are calculated at the syntactic level, and the discourse cohesion and discourse coherence relationships between cross-sentence semantic units are calculated at the discourse level, thereby obtaining the lexical dependency set, the syntactic dependency set, and the discourse dependency set, respectively. The lexical dependency relation set, the syntactic dependency relation set, and the discourse dependency relation set are hierarchically fused to establish a cross-level semantic unit association mapping. Each level dependency relation is assigned a hierarchical distinguishing identifier, and the dependency strength weight corresponding to each level dependency relation is calculated to generate a semantic dependency graph containing each semantic unit as a node, each level dependency relation as an edge, and the dependency strength weight as the edge weight.

5. The method according to claim 1, characterized in that, The constraint information of the contextual semantic representation is propagated in the semantic dependency graph to obtain a constrained semantic dependency graph, and multi-hop inference is performed to obtain the context-corrected semantic understanding result, including: The dialogue intent, semantic orientation, and topic scope are extracted from the context semantic representation and encoded as constraint features. The constraint features are then injected into the semantic unit nodes associated with the context semantic representation, which serve as constraint propagation points. Starting from the constraint propagation point, constraint features are propagated. By calculating the semantic correlation and constraint compatibility between adjacent semantic unit nodes, the propagation strength and direction of constraint information during the propagation process are determined. The constraint features are propagated in multiple rounds of iteration, so that the constraint information gradually spreads from the constraint propagation point to the far semantic unit nodes in the semantic dependency graph, thus obtaining the constraint semantic dependency graph. Based on the constrained semantic dependency graph, starting from the inference start node, hop-by-hop inference expansion is performed to obtain multiple inference links. The semantic unit nodes on each inference link are aggregated to obtain the corresponding link semantic representation and then globally fused to generate the context-corrected semantic understanding result.

6. The method according to claim 5, characterized in that, Based on the constrained semantic dependency graph, starting from each inference initiation node, hop-by-hop inference expansion is performed to obtain multiple inference links. Semantic unit nodes on each inference link are aggregated to obtain the corresponding link semantic representations, which are then globally fused to generate context-corrected semantic understanding results, including: In the constrained semantic dependency graph, the inference starting node is determined, and the adjacent semantic unit node with the highest edge correlation is selected as the first hop inference node. The hop inference extension is performed along the edge of the constrained semantic dependency graph. In each hop of reasoning extension, based on the semantic representation, constraint feature representation and reasoning context information passed from the reasoning start node to the current hop, an inter-hop reasoning state vector is constructed. Combined with the semantic representation of adjacent semantic unit nodes, an inter-hop extension score is calculated. The adjacent semantic unit node with the highest inter-hop extension score is selected as the next hop reasoning node and the reasoning context information is updated. The subsequent hop reasoning extension is iteratively executed until the preset reasoning depth is reached or the information entropy of the reasoning context information is lower than the convergence threshold, thus obtaining multiple reasoning links starting from each reasoning start node. On the multiple inference links, the position decay weight is calculated based on the hop count of each inference node, and the node semantic representation and node constraint features of all inference nodes are extracted and differentially aggregated to obtain the link semantic representation corresponding to each inference link. The link semantic representations corresponding to the multiple inference links are fused to obtain the semantic understanding result after context correction.

7. A context-based intelligent speech and semantic understanding and analysis system, used to implement the method as described in any one of claims 1-6, characterized in that, include: The first unit is used to acquire the input speech signal, extract the acoustic feature sequence of the input speech signal, input the acoustic feature sequence into the acoustic model for feature decoding, and obtain the candidate text sequence. The second unit is used to construct a contextual semantic representation based on historical interaction information and the current session state, and to perform semantic disambiguation processing on the candidate text sequence based on the contextual semantic representation to obtain a disambiguated semantic unit sequence. The third unit is used to perform association analysis on each semantic unit in the disambiguated semantic unit sequence using multi-level semantic dependency relations, establish cross-level semantic unit association mapping, and obtain a semantic dependency graph. The fourth unit is used to propagate the constraint information of the context semantic representation in the semantic dependency graph, obtain the constraint semantic dependency graph and perform multi-hop reasoning to obtain the semantic understanding result after context correction. The fifth unit is used to generate intent recognition results and slot filling results based on the semantic understanding results after context correction, and to update the current session state information for semantic understanding analysis of subsequent input voice signals.

8. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 6.

Images