An interactive method and system for LED displays supporting voice interaction

Abstract

This invention provides an interactive method and system for LED displays that supports voice interaction. It involves acquiring a continuous voice data stream from a user, extracting voice features to obtain an acoustic feature sequence and a semantic association feature set for each voice segment, and then using a pre-trained voice semantic understanding model to perform joint semantic parsing on the acoustic feature sequence and semantic association feature set. This generates a semantic understanding result for the voice segment, which determines the user's interaction intent type corresponding to the continuous voice data stream and the key information location features of that intent within the voice content. Based on the user's interaction intent type and key information location features, an LED display control command containing an interactive content identifier is generated and sent to the target LED display to execute the interactive display operation. This invention can accurately understand the true intent and key information in a user's continuous voice, significantly improving the accuracy and naturalness of the voice interaction between the LED display and the user, and effectively enhancing the user's interactive experience.

Description

Technical Field

[0001] This invention relates to the field of speech recognition, and more specifically, to an interactive method and system for LED displays that supports voice interaction. Background Technology

[0002] With the development of intelligent interaction technology, voice-interactive LED display screens are widely used in information display, public services, and other scenarios. These methods control the content displayed on the LED screen by recognizing user voice commands, thus achieving more convenient human-computer interaction. Currently, the common implementation method is to acquire the user's single-sentence voice input, extract keywords or simple semantic features, and then generate corresponding display commands based on preset rules. However, this processing method, which only addresses discrete voice commands, is difficult to effectively handle the interaction needs of continuous voice data streams in natural user conversations. Continuous voice often contains multiple temporally continuous voice segments with semantic continuity and intent changes between segments. Existing methods, unable to capture the semantic connections and intent evolution between segments, easily lead to a mismatch between the content displayed on the LED screen and the core information of the user's true intent, reducing the accuracy and naturalness of the interaction and affecting the user experience. Summary of the Invention

[0003] This invention provides an interactive method and system for LED displays that supports voice interaction.

[0004] In a first aspect, embodiments of the present invention provide an interactive method for an LED display screen that supports voice interaction, the method comprising:

[0005] Acquire a continuous voice data stream sent by the user, the continuous voice data stream containing multiple voice segments with timestamps;

[0006] The continuous speech data stream is subjected to speech feature extraction processing to obtain the acoustic feature sequence and semantic association feature set of the speech segment;

[0007] A pre-trained speech semantic understanding model is invoked to perform joint semantic parsing processing on the acoustic feature sequence and the semantic association feature set to generate the semantic understanding result of the speech segment;

[0008] Based on the semantic understanding results, determine the user interaction intent type corresponding to the continuous voice data stream and the key information location features of the user interaction intent in the voice content.

[0009] Based on the user interaction intent type and the key information location features, an LED display control command containing an interactive content identifier is generated, and the LED display control command is sent to the target LED display screen to execute the interactive display operation.

[0010] In a second aspect, embodiments of the present invention provide a computer system, comprising:

[0011] A memory, wherein a computer program is stored;

[0012] A processor is used to load the computer program to implement the LED display interaction method that supports voice interaction as described above.

[0013] This invention provides an interactive LED display screen method supporting voice interaction. By acquiring a continuous voice data stream emitted by the user and segmenting it into voice segments with timestamps, it can effectively handle the temporal continuity of continuous voice, providing a structured foundation for subsequent semantic analysis. It performs voice feature extraction processing on the continuous voice data stream, simultaneously acquiring acoustic feature sequences and semantic association feature sets. This allows subsequent semantic analysis to overcome the limitations of traditional solutions by simultaneously utilizing the acoustic expression features of the voice and the semantic association features between segments. It calls a pre-trained voice semantic understanding model to perform joint semantic analysis on the dual-modal features, generating a semantic understanding result containing the intent evolution trajectory and core semantic content. This captures the intent change trend between continuous voice segments and the core semantics in colloquial expressions. Based on the semantic understanding result, it determines the user's interaction intent type and key information location features, clarifying both the user's overall intent and locating the core information region strongly related to the intent within the voice content, solving the problem of traditional solutions only recognizing intent but failing to match display focus. Based on the intent type and key information, it generates LED display control commands containing interactive content identifiers, enabling the display screen to accurately adjust the displayed content according to the core intent and key information continuously expressed by the user, rather than simply generating commands based on the intent type. Through the synergistic effect of the above steps, this method can more accurately understand the true intentions and key information in the user's continuous speech, significantly improve the accuracy and naturalness of the LED display screen's voice interaction with the user, and effectively improve the user's interactive experience. Attached Figure Description

[0014] Figure 1 This is a flowchart of an LED display screen interaction method that supports voice interaction, provided by an embodiment of the present invention.

[0015] Figure 2 This is a schematic diagram of the composition of a computer system provided in an embodiment of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Please see Figure 1 , Figure 1 The flowchart illustrates an interactive method for an LED display screen supporting voice interaction, provided in an embodiment of the present invention. This method can be executed by a computer system, which may be embedded in the LED display screen. The method provided in this embodiment of the present invention includes the following steps:

[0018] Step S100: Obtain a continuous voice data stream sent by the user, the continuous voice data stream containing multiple voice segments with timestamps.

[0019] A continuous speech data stream is a data stream formed by the continuous speech signals emitted by a user during speech expression; it is a continuous, uninterrupted sequence of speech information. A timestamp is an identifier added to a speech segment to indicate its temporal position within the continuous speech data stream. Timestamps allow for precise location of each speech segment within the entire speech expression process. Acquiring continuous speech data streams emitted by users can be done using audio acquisition devices such as microphones. For example, in a conference room setting, a high-performance microphone mounted on a conference table can capture the continuous speech emitted by participants in real time. During acquisition, the audio acquisition device converts the received speech signals into digital signals and records them chronologically, while simultaneously adding a timestamp to each speech segment. These timestamped speech segments constitute a continuous speech data stream, facilitating subsequent processing and analysis.

[0020] Step S200: Perform speech feature extraction processing on the continuous speech data stream to obtain the acoustic feature sequence and semantic association feature set of the speech segment.

[0021] Speech feature extraction is the process of extracting information that reflects the essential characteristics of speech from a continuous speech data stream. An acoustic feature sequence is a set of features describing the acoustic properties of speech segments, including stress distribution patterns, intonation trajectories, and other information that helps in understanding the emotion and emphasis of speech. A semantic association feature set represents the semantic relationships between speech segments, including information such as cross-segment semantic continuity and intra-segment semantic concentration, reflecting the semantic coherence and tightness of the speech content. By performing speech feature extraction on a continuous speech data stream, both acoustic and semantic information can be comprehensively utilized to more accurately understand the user's speech intent.

[0022] As one implementation method, step S200 involves performing speech feature extraction processing on the continuous speech data stream to obtain the acoustic feature sequence and semantic association feature set of the speech segments. Specifically, this can be implemented as the following steps S210 to S240:

[0023] Step S210: Perform boundary detection processing on the continuous speech data stream, determine the natural segmentation boundary of the continuous speech data stream based on the abrupt change points of speech energy and semantic pause markers, and segment the continuous speech data stream into speech segments with semantic integrity.

[0024] Boundary detection processing is the process of finding suitable segmentation points in a continuous speech data stream to divide it into different speech segments. Abrupt changes in speech energy are locations where the speech signal energy changes significantly within a short period, usually related to pauses, stress, etc. Semantic pause markers are pause locations determined based on the semantic logic of the speech content, such as pauses between sentences or paragraphs. Natural segmentation boundaries are segmentation locations determined based on speech energy abrupt changes and semantic pause markers, conforming to natural speech expression and semantic understanding. Speech segments with semantic integrity are speech units that can semantically express a relatively complete meaning, such as a complete sentence or an independent topic. When performing boundary detection processing on a continuous speech data stream, energy calculation can be performed on the speech signal first to identify energy abrupt changes. For example, the sliding window method can be used to calculate the energy of the speech signal within each window; when the energy change exceeds a certain threshold, the location is marked as an energy abrupt change. Simultaneously, semantic pause markers are determined by analyzing the speech text using natural language processing techniques. By combining energy mutation points and semantic pause markers, natural segmentation boundaries are determined, thereby segmenting continuous speech data streams into speech segments with semantic integrity.

[0025] Step S220: Perform intent-oriented acoustic feature focusing processing on each speech segment, and extract the acoustic feature subset that is strongly correlated with the user's intent in the speech segment as an acoustic feature sequence. The acoustic feature subset includes the stress distribution pattern and the pitch rise and fall trajectory.

[0026] Intent-driven acoustic feature focusing is a process that, guided by the user's interactive intent, filters acoustic features from speech segments to identify those closely related to that intent. The subset of acoustic features strongly correlated with user intent is the set of features that directly reflect the user's intent. Stress distribution patterns, the location and distribution of stress in speech, can indicate the user's emphasis on certain words or content. Pitch trajectory, the temporal variation of speech intonation, can convey different tones such as emotion, question, and statement. When performing intent-driven acoustic feature focusing on each speech segment, it is first necessary to clarify the user's interactive intent, and then filter relevant acoustic features based on that intent. For example, in a speech querying information, the user may emphasize key information; in this case, the key information can be identified by analyzing the stress distribution pattern. Simultaneously, pitch rises and falls can also reflect the user's tone; for example, a questioning tone usually has a rising pitch. By extracting acoustic features strongly correlated with user intent, such as stress distribution patterns and pitch trajectories, an acoustic feature sequence is formed.

[0027] As one implementation method, step S220 involves performing intent-oriented acoustic feature focusing processing on each speech segment, extracting a subset of acoustic features strongly correlated with the user's intent from the speech segment as an acoustic feature sequence. Specifically, this can be implemented as the following steps S221 to S224:

[0028] Step S221: Perform multi-resolution time-frequency analysis on the speech segments to generate a set of time-spectrum graphs containing different time resolutions.

[0029] Multi-resolution time-frequency analysis (MTA) is the process of analyzing speech signals in both time and frequency dimensions, using different resolutions to obtain more comprehensive time-frequency information. A time-spectrum graph is a graphical representation of the time and frequency distribution of a speech signal, visually reflecting the changes in frequency components over time. A collection of time-spectrum graphs at different time resolutions is a set of time-spectrum graphs obtained by performing time-frequency analysis on a speech segment using different time resolutions. Methods such as wavelet transform can be used to perform MTA on speech segments. For example, for a single speech segment, wavelet transforms can be performed at coarse and fine resolutions to obtain two time-spectrum graphs with different time resolutions. The coarse-resolution time-spectrum graph displays the overall time-frequency characteristics of the speech signal, while the fine-resolution time-spectrum graph shows the local time-frequency characteristics of the speech signal in more detail. By generating a collection of time-spectrum graphs containing different time resolutions, the time-frequency characteristics of the speech signal can be observed from multiple perspectives.

[0030] Step S222: Extract the energy concentration points of the stressed regions from the time-spectrum graph set, statistically analyze the positional distribution pattern of the energy concentration points in the speech segment, and generate a stressed distribution pattern.

[0031] The energy concentration points of accented regions are the locations corresponding to relatively concentrated energy areas in the time-spectrum graph. These locations are usually related to the stressed parts of speech. Statistical analysis of the distribution patterns of energy concentration points in speech segments involves statistically analyzing the locations of these energy concentration points to identify their distribution characteristics and patterns. The accent distribution pattern is the pattern of accent distribution in a speech segment formed based on the distribution patterns of energy concentration points. When extracting energy concentration points of accented regions from a set of time-spectrum graphs, a thresholding process can be performed on the time-spectrum graphs to filter out regions with energy values ​​below a certain threshold, retaining regions with higher energy. Then, energy concentration points are identified within these high-energy regions. For example, a local maximum detection algorithm can be used to find the energy maximum point in each local region of the time-spectrum graph and use these points as energy concentration points. When statistically analyzing the distribution patterns of energy concentration points in speech segments, statistical characteristics such as the frequency and distribution density of energy concentration points at different time positions can be calculated. Based on these statistical characteristics, an accent distribution pattern can be generated, which can be represented graphically or numerically to show the distribution of accents in a speech segment.

[0032] Step S223: Perform trend analysis on the fundamental frequency curve of the speech segment, identify the rising and falling inflection points in the fundamental frequency curve, calculate the slope change rate between adjacent inflection points, and generate the pitch rise and fall trajectory.

[0033] The fundamental frequency (FFM) curve is the curve showing the change of the fundamental frequency in a speech signal over time. The fundamental frequency is the basic frequency of the speech signal and is related to the pitch of the speech. Trend analysis processing is the process of analyzing the changing trend of the FFM curve. Rising and falling inflection points are points on the FFM curve where the slope changes from positive to negative or vice versa; these points are related to the rise and fall of speech intonation. The slope change rate between adjacent inflection points reflects the change in the slope of the FFM curve between two adjacent inflection points, indicating the speed and amplitude of pitch rise and fall. The intonation trajectory is the trajectory of pitch change over time formed by the rising, falling, and slope change rates of the FFM curve. When performing trend analysis processing on the FFM curve of a speech segment, the slope of the FFM curve can be calculated using methods such as numerical differentiation. Rising and falling inflection points are identified by comparing the slopes of adjacent points. For example, when the slope of the FFM curve changes from positive to negative, that point is a falling inflection point; when the slope changes from negative to positive, that point is a rising inflection point. When calculating the rate of change of slope between adjacent inflection points, the change in fundamental frequency between two adjacent inflection points can be divided by the time interval. The pitch trajectory can be generated based on the rising inflection point, falling inflection point, and rate of change of slope; for example, the pitch changes within a speech segment can be represented as a curve or data.

[0034] Step S224: Based on the stress distribution pattern and intonation trajectory, the part that overlaps with general speech features is filtered out through the pre-trained intention-acoustic feature mapping relationship, and the feature components that are strongly associated with the target interaction intention are retained to form an acoustic feature sequence.

[0035] Pre-trained intent-acoustic feature mappings are pre-trained models or rules that describe the correspondence between different acoustic features and specific interactive intents. General speech features refer to features that are ubiquitous in various speech patterns and do not have a specific interactive intent. Targeted interactive intent is the user's intention, expressed through speech, to achieve a specific interaction, such as querying information or controlling a device.

[0036] In this step, the first step is to obtain the stress distribution pattern and intonation trajectory generated in the previous steps. For example, in a user's voice scenario of querying the weather, after processing in steps S221-S223, the stress distribution pattern of the voice segment has been obtained, revealing that the location of the word "weather" is a concentrated stress area; simultaneously, the intonation trajectory is obtained, showing that when asking "How's the weather today?", the intonation rises at the "How's" position. Then, the stress distribution pattern and intonation trajectory are analyzed using a pre-trained intent-acoustic feature mapping relationship. The pre-trained intent-acoustic feature mapping relationship can be implemented using a deep learning model, such as a Multilayer Perceptron (MLP). This model can have an input layer, a hidden layer, and an output layer. The input layer receives the feature vectors of the stress distribution pattern and intonation trajectory, the hidden layer performs non-linear transformations on the input features through multiple neurons, and the output layer outputs the probability corresponding to each possible interactive intent. Training this MLP model requires a large amount of labeled data. This data includes the stress distribution patterns, intonation trajectories, and corresponding interactive intent labels of different voice segments. The training steps are as follows: First, the labeled data is divided into training, validation, and test sets. Then, the parameters of the MLP model are initialized, such as weights and biases. Next, the model is iteratively trained using the training set data, calculating the gradient of the loss function through backpropagation to update the model parameters and minimize the error between the predicted interaction intent and the true label. During training, the validation set is used to adjust the model's hyperparameters, such as the learning rate and the number of hidden layer neurons, to avoid overfitting. Finally, the trained model is evaluated using the test set to ensure it has good generalization ability.

[0037] In this embodiment, the stress distribution pattern and intonation trajectory are input into a trained MLP model, which outputs the probability of each interactive intent. Based on this probability, and combined with knowledge of general speech features, the model filters out parts that overlap with these general speech features. General speech features can be obtained through statistical analysis of a large amount of speech data without specific intent, such as speech rhythm at normal speaking speed and common speech pauses.

[0038] For example, subtle pauses, common in speech, appear in many speech patterns and lack specific interactive intent; they are considered general speech features. The model can determine which features belong to general speech features based on their output probabilities and filter them out. After filtering out general speech features, the model retains feature components strongly correlated with the target interactive intent. Finally, the retained feature components are combined to form an acoustic feature sequence, which more accurately reflects the user's target interactive intent. In this way, the accuracy of the voice interaction system's understanding of user intent can be improved, misjudgments caused by interference from general speech features can be reduced, and the robustness and reliability of the system can be enhanced.

[0039] Step S230: Perform cross-segment semantic association modeling for each speech segment, and perform contextual fusion of the text conversion results of the speech segment and the preceding speech segment to generate an association feature matrix containing semantic continuity.

[0040] Cross-segment semantic association modeling is the process of modeling and analyzing the semantic associations between different speech segments. Text conversion results in the conversion of speech segments into text form, which can be achieved using general speech recognition techniques. Context fusion integrates the text information of the current speech segment with that of the preceding speech segment, considering their semantic relationships and contextual information. The association feature matrix is ​​a matrix representing the semantic associations between speech segments, containing information such as semantic continuity. When performing cross-segment semantic association modeling for each speech segment, the speech segment is first converted into text form. Then, the text conversion result of the current speech segment is fused with the text conversion result of the preceding speech segment. Methods such as semantic vector encoding in natural language processing can be used to convert text information into vector representations, and then context fusion is achieved through vector operations and fusion. An association feature matrix is ​​generated based on the context fusion result. The elements in the matrix can represent information such as the strength of semantic associations and semantic continuity between different speech segments. For example, a row and a column in the matrix may correspond to two different speech segments, and the values ​​of the matrix elements represent the degree of semantic association between these two speech segments.

[0041] In one implementation, step S230 involves performing cross-segment semantic association modeling on each speech segment, fusing the text conversion results of the speech segment with those of the preceding speech segment in context, and generating an association feature matrix that includes semantic continuity. Specifically, this can be implemented as the following steps S231 to S235:

[0042] Step S231: Perform semantic vector encoding on the text conversion result of the current speech segment and the text conversion result of the previous speech segment to generate the semantic vector of the current segment and the semantic vector of the previous segment.

[0043] Semantic vector encoding converts text information into vector form, allowing the semantic information of the text to be represented in a vector space. The semantic vector of the current segment is the vector obtained after semantic vector encoding of the text conversion result of the current speech segment. The semantic vector of the preceding segment is the vector obtained after semantic vector encoding of the text conversion result of the preceding speech segment. When performing semantic vector encoding on the text conversion results of the current and preceding speech segments, word embedding models such as Word2Vec and GloVe can be used. These models can convert each word into a fixed-length vector, and then obtain the semantic vector of the entire text by performing operations such as summing and averaging the word vectors in the text. The same method can be used to obtain the semantic vector of the preceding segment. Semantic vectors can represent the semantic information of text in vector form, facilitating subsequent semantic association analysis and calculation.

[0044] Step S232: Calculate the cosine similarity between the semantic vector of the current segment and the semantic vector of the preceding segment, and generate a cross-segment semantic association strength value.

[0045] Cosine similarity is a metric used to measure the similarity between two vectors. It represents the degree of similarity by calculating the cosine of the angle between the two vectors. The cross-segment semantic association strength value is obtained based on the cosine similarity between the semantic vector of the current segment and the semantic vector of the preceding segment, representing the semantic association strength between the current and preceding speech segments. The resulting cosine similarity value is the cross-segment semantic association strength value; the closer the value is to 1, the stronger the semantic association between the two speech segments, and the closer the value is to 0, the weaker the semantic association.

[0046] Step S233: Adjust the fusion weight between the current segment semantic vector and the preceding segment semantic vector based on the cross-segment semantic association strength value. The fusion weight is positively correlated with the semantic association strength value.

[0047] The fusion weight is the weight assigned to each vector when fusing the semantic vector of the current segment with the semantic vector of the preceding segment. The semantic association strength is the cosine similarity between the semantic vector of the current segment and the semantic vector of the preceding segment. The fusion weight and the semantic association strength are positively correlated: the larger the semantic association strength, the larger the corresponding fusion weight, and vice versa. When adjusting the fusion weight of the current segment semantic vector and the semantic vector of the preceding segment based on the cross-segment semantic association strength, a linear mapping method can be used. For example, a weight adjustment function f(x) = ax + b can be set, where x represents the cross-segment semantic association strength, a and b are pre-set parameters, and a > 0. When the cross-segment semantic association strength is x1, the calculated fusion weight is f(x1). If the semantic association strength is large, it indicates a strong semantic association between the current speech segment and the preceding speech segment. Therefore, a larger weight should be assigned to the semantic vector of the preceding segment during fusion to better reflect the contextual information. By adjusting the fusion weight, context fusion can be performed more reasonably, generating more accurate association features.

[0048] Step S234: Perform context fusion processing on the semantic vector of the current segment and the semantic vector of the preceding segment through a weighted summation operation to generate a fused semantic vector that includes semantic continuity.

[0049] The weighted summation operation involves multiplying the semantic vector of the current segment and the semantic vector of the preceding segment by their respective fusion weights, and then summing them. Contextual fusion processing integrates the textual information of the current speech segment with that of the preceding speech segment, considering their semantic relationships and contextual information. The fused semantic vector is the vector obtained by fusing the semantic vectors of the current segment and the preceding segment through the weighted summation operation, and it includes contextual information such as semantic continuity.

[0050] Step S235: Perform dimensional concatenation processing on the fused semantic vector, the current segment semantic vector, and the preceding segment semantic vector to form an associated feature matrix.

[0051] Dimensional concatenation is an operation that concatenates multiple vectors in a specific dimension, arranging their elements sequentially to form a higher-dimensional vector or matrix. The association feature matrix, formed by dimensionally concatenating the fused semantic vector with the semantic vectors of the current and preceding segments, represents the semantic relationships between speech segments. Through dimensional concatenation, the fused semantic vector is combined with the semantic vectors of the current and preceding segments to form the association feature matrix, which contains more semantic relationship information.

[0052] Step S240: Perform semantic intensity decay calculation on the associated feature matrix, adjust the weight values ​​of the associated features according to the time interval between speech segments, and generate a semantic associated feature set. The semantic associated feature set includes cross-segment semantic continuity and intra-segment semantic concentration.

[0053] Semantic strength attenuation calculation involves attenuating the associated features in the associated feature matrix, considering the impact of time intervals between speech segments on the semantic association strength. The time interval between speech segments is the temporal distance between two adjacent segments; a longer time interval may indicate a weaker semantic association. The weights of the associated features are adjusted based on the time interval length, reducing the weight of associated features between speech segments with longer time intervals. The semantic association feature set is the feature set obtained after semantic strength attenuation calculation on the associated feature matrix. It includes information such as cross-segment semantic continuity and intra-segment semantic concentration. Cross-segment semantic continuity reflects the degree of semantic continuity between different speech segments, indicating the semantic coherence of the speech content. Intra-segment semantic concentration reflects the concentration of semantics within a single speech segment, indicating the semantic tightness of the speech segments. When calculating semantic strength attenuation on the associated feature matrix, the time interval between speech segments is first calculated, and then a preset attenuation function table is consulted to obtain the corresponding weight attenuation coefficient. Each element in the associated feature matrix is ​​multiplied by the weight attenuation coefficient to obtain the attenuated associated feature vector. Based on the decayed associated feature vectors, calculate the cross-segment semantic continuity and intra-segment semantic concentration, and arrange them in the temporal order of the speech segments to form a set of semantic associated features.

[0054] As one implementation method, step S240 involves calculating semantic intensity decay on the associated feature matrix, adjusting the weight values ​​of the associated features according to the time interval between speech segments, and generating a semantic associated feature set. Specifically, this can be implemented as follows: steps S241 to S246.

[0055] Step S241: Calculate the time interval length between the current speech segment and the preceding speech segment, and generate the time interval feature value.

[0056] The time interval length is the temporal difference between the current speech segment and the preceding speech segment, reflecting the temporal distance between the two segments. The time interval feature value is a feature value generated based on the time interval length to represent the time interval. The time interval length between the current speech segment and the preceding speech segment can be calculated by comparing their timestamp information.

[0057] Step S242: Query the preset attenuation function table based on the time interval feature value to obtain the corresponding weight attenuation coefficient. The attenuation function table defines the negative correlation between the time interval length and the weight attenuation coefficient.

[0058] The predefined attenuation function table represents the relationship between the time interval length and the weight attenuation coefficient. The weight attenuation coefficient is used to attenuate the associated features in the association feature matrix; the longer the time interval, the smaller the weight attenuation coefficient, reflecting the attenuation effect of the time interval on the semantic association strength. When querying the predefined attenuation function table based on a time interval feature value, the time interval feature value is used as an index to find the corresponding weight attenuation coefficient in the table.

[0059] Step S243: Apply a weight decay coefficient to each correlation feature dimension in the correlation feature matrix to generate a decayed correlation feature vector.

[0060] Applying the weight decay factor involves multiplying each element of the association feature matrix by its corresponding weight decay factor. The decayed association feature vector is the vector obtained by applying the weight decay factor to the association feature dimensions of the association feature matrix. For example, suppose the association feature matrix is ​​M = (m ij Let i represent the row index, j represent the column index, and k represent the weight decay coefficient. Then, the decayed associated feature vector V = (k × m) ij By applying a weight decay coefficient to each dimension of the association feature matrix, the weights of association features between speech segments with longer time intervals are relatively smaller, reflecting the attenuating effect of time intervals on the semantic association strength. The generated decayed association feature vector more accurately reflects the semantic association relationships between speech segments, providing more reasonable data for subsequent semantic analysis.

[0061] Step S244: Calculate the semantic concentration of the decayed associated feature vectors, statistically analyze the distribution density of high-weight dimensions in the associated feature vectors, and generate the semantic concentration within the segment.

[0062] Semantic concentration calculation involves analyzing the attenuated associated feature vectors to determine the degree of semantic concentration. High-weight dimensions are those with relatively high weight values ​​within the associated feature vectors; these dimensions typically contain more important semantic information. Distribution density represents the distribution of high-weight dimensions within the associated feature vectors, such as their frequency of occurrence and degree of clustering. Intra-segment semantic concentration is an index generated based on the distribution density of high-weight dimensions, representing the degree of semantic concentration within a speech segment. When calculating semantic concentration on attenuated associated feature vectors, a weight threshold can be set first, considering dimensions with weight values ​​greater than this threshold as high-weight dimensions. Then, the distribution density of high-weight dimensions within the associated feature vectors is statistically analyzed. For example, statistical indicators such as the length of consecutive occurrences of high-weight dimensions in the vector and the proportion of high-weight dimensions to the total number of dimensions can be calculated. Intra-segment semantic concentration is generated based on these statistical indicators, reflecting the tightness and concentration of semantics within a speech segment.

[0063] Step S245: Perform cross-segment continuity evaluation on the decayed associated feature vector, compare the high-weight dimension overlap between the current speech segment and the preceding speech segment, and generate cross-segment semantic continuity.

[0064] Cross-segment continuity assessment involves analyzing the attenuated associated feature vectors to evaluate the degree of semantic continuity between different speech segments. High-weighted dimension overlap refers to the overlap between the high-weighted dimensions in the attenuated associated feature vector of the current speech segment and those in the attenuated associated feature vector of the preceding speech segment. Higher overlap indicates stronger semantic continuity between the two speech segments. Cross-segment semantic continuity is an index generated based on the high-weighted dimension overlap, representing the degree of semantic continuity between different speech segments. When performing cross-segment continuity assessment on the attenuated associated feature vectors, the high-weighted dimensions must first be determined. A weight threshold can be set, and dimensions with weight values ​​greater than this threshold are considered high-weighted dimensions. Then, the high-weighted dimensions of the current speech segment and the preceding speech segment are compared to calculate their overlap. For example, the ratio of intersection to union can be used to calculate the overlap, i.e., Where A represents the set of high-weight dimensions of the current speech segment, and B represents the set of high-weight dimensions of the preceding speech segments. The cross-segment semantic continuity is generated based on the calculated high-weight dimension overlap, and this metric reflects the semantic coherence of speech content across different segments.

[0065] Step S246: Arrange the semantic concentration within a segment and the semantic continuity across segments according to the time order of the speech segments to form a set of semantic association features.

[0066] The semantic concentration within a segment and the semantic continuity across segments are arranged according to the temporal order of the speech segments. This involves arranging the semantic concentration within a segment and the semantic continuity across segments for each speech segment according to their chronological order within the continuous speech data stream. The semantic association feature set is a set of features formed through this arrangement, containing information such as the semantic continuity across segments and the semantic concentration within segments.

[0067] Step S300: Call the pre-trained speech semantic understanding model to perform joint semantic parsing processing on the acoustic feature sequence and the semantic association feature set to generate the semantic understanding result of the speech segment.

[0068] The pre-trained speech semantic understanding model is a model trained on a large amount of data beforehand to understand speech semantics. This model can learn the mapping relationship between the acoustic features and semantic features of speech and their semantic meaning. Joint semantic parsing combines the acoustic feature sequence and the set of semantically related features, and analyzes and parses them through the pre-trained speech semantic understanding model to extract the semantic information of speech segments. The semantic understanding result is the result obtained through joint semantic parsing, containing information such as the evolution trajectory of the speech segment's intent and its core semantic content. When calling the pre-trained speech semantic understanding model to perform joint semantic parsing on the acoustic feature sequence and the set of semantically related features, the acoustic feature sequence and the set of semantically related features are input into the pre-trained speech semantic understanding model. The model analyzes and processes the input features based on its internal structure and the learned mapping relationship, and outputs the semantic understanding result of the speech segment. For example, the pre-trained speech semantic understanding model can use deep learning models, such as recurrent neural networks (RNNs) and long short-term memory networks (LSTMs). These models can process sequential data and effectively capture the time-series features and semantic association information of speech. By combining semantic parsing processing and taking into account both acoustic and semantic features, more accurate semantic understanding results for speech segments are generated.

[0069] As one implementation method, step S300 involves calling a pre-trained speech semantic understanding model to perform joint semantic parsing processing on the acoustic feature sequence and the semantic association feature set to generate the semantic understanding result of the speech segment. Specifically, this can be implemented as follows: steps S310 to S340.

[0070] Step S310: Input the acoustic feature sequence and the semantic association feature set into the context fusion module of the speech semantic understanding model, determine the context window length based on the cross-segment semantic continuity in the semantic association feature set, and generate an extended feature sequence containing historical segment association information.

[0071] The context fusion module is a component of the speech semantic understanding model, used to fuse the features of the current speech segment with those of historical speech segments, taking contextual information into account. Cross-segment semantic continuity refers to the degree of semantic continuity between different speech segments, reflecting the strength of the semantic association between the current and historical segments. The context window length determines the number or time range of historical speech segments considered during context fusion. The extended feature sequence is the feature sequence containing more historical segment association information obtained after fusing the features of the current and historical speech segments through the context fusion module. When the acoustic feature sequence and the semantic association feature set are input into the context fusion module of the speech semantic understanding model, the cross-segment semantic continuity is first extracted from the semantic association feature set. The context window length is determined based on the strength of the cross-segment semantic continuity; a stronger continuity indicates a closer semantic association between the current and historical segments, allowing for a larger context window length to consider more historical speech segments; a weaker continuity allows for a smaller context window length to consider fewer historical speech segments. For example, assuming cross-segment semantic continuity is represented numerically, when the cross-segment semantic continuity is greater than 0.8, the context window length is set to 3, meaning it considers the three historical speech segments preceding the current speech segment; when the cross-segment semantic continuity is between 0.5 and 0.8, the context window length is set to 2; and when the cross-segment semantic continuity is less than 0.5, the context window length is set to 1. Based on the determined context window length, the acoustic feature sequence and semantic association feature set of the selected historical speech segments are concatenated with the corresponding features of the current speech segment to generate a cross-segment feature concatenation vector. The cross-segment feature concatenation vector is then aligned in terms of feature dimensions to unify the feature representation dimensions of speech segments with different timestamps, generating an extended feature sequence containing historical segment association information.

[0072] In practical implementation, the context fusion module can be implemented using a neural network architecture based on an attention mechanism. Taking the Transformer architecture as an example, it contains multiple encoder layers, each consisting of a multi-head self-attention mechanism and a feedforward neural network. The multi-head self-attention mechanism can capture the dependencies between different positions in the input sequence, while the feedforward neural network performs a non-linear transformation on the output of the attention mechanism.

[0073] As one implementation method, step S310 involves inputting the acoustic feature sequence and the semantic association feature set into the context fusion module of the speech semantic understanding model, determining the context window length based on the cross-segment semantic continuity in the semantic association feature set, and generating an extended feature sequence containing historical segment association information. Specifically, this can be implemented as follows: steps S311 to S314.

[0074] Step S311: Extract the cross - segment semantic continuity degree between the current speech segment and the historical speech segment from the semantic association feature set. The cross - segment semantic continuity degree reflects the strength of the semantic association between the historical segment and the current segment.

[0075] When extracting the cross - segment semantic continuity degree from the semantic association feature set, according to the storage structure and organization method of the semantic association feature set, find the cross - segment semantic continuity degree information corresponding to the current speech segment and the historical speech segment. For example, if the semantic association feature set is a list, where each element contains the intra - segment semantic concentration degree of a speech segment and the cross - segment semantic continuity degree with the adjacent speech segment, then according to the index position of the current speech segment, extract the cross - segment semantic continuity degree with the historical speech segment from the list. The cross - segment semantic continuity degree reflects the degree of semantic association between the historical segment and the current segment. The larger its value, the stronger the semantic association between the historical segment and the current segment; the smaller the value, the weaker the semantic association.

[0076] Step S312: Select the historical speech segments semantically associated with the current segment according to the strength of the cross - segment semantic continuity degree. The stronger the continuity degree, the more associated historical segments are selected; the weaker the continuity degree, the fewer associated historical segments are selected.

[0077] When selecting historical speech segments according to the strength of the cross - segment semantic continuity degree, different thresholds can be set for judgment. For example, set three thresholds T1, T2 (T1 < T2). When the cross - segment semantic continuity degree is greater than T2, select three historical speech segments before the current speech segment; when the cross - segment semantic continuity degree is between T1 and T2, select two historical speech segments before the current speech segment; when the cross - segment semantic continuity degree is less than T1, only select one historical speech segment before the current speech segment. The stronger the continuity degree, the closer the semantic association between the historical segment and the current segment. Selecting more historical speech segments can better utilize the context information and improve the accuracy of semantic understanding.

[0078] Step S313: Cascade and splice the acoustic feature sequences and semantic association feature sets of the selected historical speech segments with the corresponding features of the current speech segment to generate a cross - segment feature splicing vector.

[0079] Cascaded concatenation is an operation that concatenates multiple vectors according to their dimensions. It involves sequentially arranging the acoustic feature sequence and semantic association feature set of a selected historical speech segment with the corresponding features of the current speech segment to form a higher-dimensional vector. The cross-segment feature concatenation vector is a vector containing feature information from multiple speech segments, obtained through cascaded concatenation. By combining the features of multiple speech segments through cascaded concatenation, a cross-segment feature concatenation vector is formed. This vector contains more historical segment association information, providing richer data for subsequent semantic analysis.

[0080] Step S314: Perform feature dimension alignment processing on the cross-segment feature splicing vector to unify the feature representation dimensions of speech segments with different timestamps and generate an extended feature sequence containing historical segment association information.

[0081] Feature dimension alignment is an operation that processes different vectors in the cross-segment feature concatenation vector to make their dimensions consistent. The feature representation dimensions of speech segments with different timestamps may differ. For example, differences in speech segment length and feature extraction methods may lead to inconsistencies in the dimensions of the acoustic feature sequences and semantic association feature sets of different speech segments. Unifying the feature representation dimensions ensures consistency in the data within the cross-segment feature concatenation vector, facilitating subsequent model processing. Feature dimension alignment can be performed using padding or truncation methods. For example, if the dimension of a speech segment's feature vector is smaller than that of other vectors, zero values ​​can be padded to make its dimension consistent with the others; conversely, if the dimension of a speech segment's feature vector is larger than that of other vectors, some elements can be truncated to make its dimension the same as the others. After feature dimension alignment, an extended feature sequence containing historical segment association information is generated. This sequence is more suitable as input to a speech semantic understanding model, improving the model's ability to understand speech semantics.

[0082] Step S320: The extended feature sequence is processed by time dependency modeling through the intent evolution modeling layer of the speech semantic understanding model to capture the intent change trend between consecutive speech segments and generate intent evolution feature vectors.

[0083] The intent evolution modeling layer is a network layer in the speech semantic understanding model used to model and analyze the changes in intent among speech segments. Temporal dependency modeling considers the temporal order and relationships between speech segments, modeling the extended feature sequence to extract temporal dependency information. The intent change trend between consecutive speech segments represents how the user's interactive intent changes over time in continuous speech expression, such as gradual changes, abrupt changes, or stabilization. The intent evolution feature vector is a vector obtained after modeling the temporal dependency of the extended feature sequence, containing feature information about the changes in intent between consecutive speech segments. When modeling the temporal dependency of the extended feature sequence using the intent evolution modeling layer of the speech semantic understanding model, the intent evolution modeling layer can employ recurrent neural networks (RNNs) or their variants, such as Long Short-Term Memory (LSTM) networks and Gated Recurrent Units (GRUs). These network structures can process sequential data and effectively capture the temporal dependency information within the sequence. Taking LSTM as an example, LSTM includes input gates, forget gates, and output gates. Through these gating mechanisms, LSTM can selectively retain or forget historical information, thereby better capturing the trend of intent changes between consecutive speech segments. The extended feature sequence is input into the LSTM, which updates the hidden state at the current time step based on the current input and the hidden state at the previous time step. The final output hidden state is the intent evolution feature vector, which reflects the change in intent between consecutive speech segments.

[0084] As one implementation method, step S320 involves performing time dependency modeling on the extended feature sequence through the intent evolution modeling layer of the speech semantic understanding model to capture the intent change trend between consecutive speech segments and generate an intent evolution feature vector. Specifically, this can be implemented as the following steps S321 to S325:

[0085] Step S321: Input the extended feature sequence into the temporal gating unit of the intention evolution modeling layer. The temporal gating unit generates forget gate and update gate parameters based on the semantic association strength between historical segments and current segments in the extended feature sequence.

[0086] The temporal gating unit (TGU) is a unit within the intent evolution modeling layer used to control the flow and updating of information. It generates forget and update gate parameters based on input feature information. These parameters are crucial; the forget gate controls whether historical information is forgotten, while the update gate controls whether current information is updated. When an extended feature sequence is input into the TGU, it generates these parameters based on the semantic strength of the connection between historical and current segments. A strong semantic connection between the two segments results in a smaller forget gate parameter, indicating more historical information is retained; conversely, a weaker connection results in a larger forget gate parameter, suggesting more historical information is forgotten. By generating these parameters, the TGU can better manage the flow of information and capture the intent change trends between consecutive speech segments.

[0087] Step S322: Filter historical feature information in the extended feature sequence that is irrelevant to the current intent by using the forget gate parameter, and retain key feature components that are related to the evolution of intent.

[0088] When filtering historical feature information in an expanded feature sequence using a forget gate parameter, the forget gate parameter is multiplied element-wise with the historical feature information in the expanded feature sequence. The forget gate parameter is a vector with values ​​between [0,1]. The closer its value is to 0, the more historical feature information should be forgotten; the closer its value is to 1, the more historical feature information should be retained. In this way, historical feature information irrelevant to the current intent is filtered out, and only key feature components related to the evolution of intent are retained. These key feature components better reflect the trend of intent changes between consecutive speech segments, providing more valuable information for subsequent intent analysis and processing.

[0089] Step S323: By updating the gate parameters, the core features of the current speech segment are weighted and fused with the retained historical key feature components to generate the intent state update vector.

[0090] The update gate parameter is a vector with values ​​between [0,1], used to control whether the current intent state is updated. The core feature of the current speech segment is the feature extracted from the current speech segment that best represents its intent. The weighted fusion process is an operation that sums the core feature of the current speech segment with the retained historical key feature components according to the update gate parameter. The intent state update vector is the vector obtained after the weighted fusion process, which reflects the intent state at the current moment. When the core feature of the current speech segment and the retained historical key feature components are weighted and fused through the update gate parameter, the update gate parameter determines the weight of the core feature of the current speech segment and the retained historical key feature components in the intent state update vector. If the value of the update gate parameter is larger, it means that the core feature of the current speech segment is given more importance; if the value of the update gate parameter is smaller, it means that the retained historical key feature components are given more importance. The intent state update vector generated by the weighted fusion process integrates the intent information of the current speech segment and historical speech segments, better reflecting the changes in intent between consecutive speech segments.

[0091] Step S324: Perform trend analysis on the intent state update vector to identify the change pattern of intent category between consecutive speech segments and generate evolution trend features that reflect the direction of intent change.

[0092] Trend analysis involves analyzing the intent state update vector to identify its changing trends and patterns. The change pattern of intent categories between consecutive speech segments refers to the change in the user's interactive intent category over time during continuous speech expression, such as gradual change patterns, abrupt change patterns, and stable change patterns. Evolutionary trend features are features generated based on the trend analysis results of the intent state update vector, reflecting the direction and degree of intent change. Various methods can be used for trend analysis of the intent state update vector. For example, time series differencing can be performed on the intent state update vector to calculate the difference in intent state between adjacent speech segments, generating a sequence of intent change amounts. This sequence of intent change amounts is then matched against a pre-defined library of typical intent change patterns, including gradual change patterns, abrupt change patterns, and stable change patterns. The specific change pattern of intent categories between consecutive speech segments is determined based on the matching results. The sign and amplitude features of the intent change amount sequence are extracted to generate directional features reflecting the direction of intent change and amplitude features reflecting the degree of change. Finally, the specific change patterns, directional features, and amplitude features are fused to form the evolutionary trend features. By using trend analysis, we can more accurately identify the changing patterns of intent categories between consecutive speech segments and generate evolutionary trend features that reflect the direction of intent change.

[0093] As one implementation method, step S324 involves performing trend analysis on the intent state update vector to identify the change pattern of intent categories between consecutive speech segments and generate evolution trend features that reflect the direction of intent change. Specifically, this can be implemented as the following steps S3241 to S3245:

[0094] Step S3241: Perform time series differencing on the intent state update vector, calculate the difference in intent state between adjacent speech segments, and generate a sequence of intent change.

[0095] Time series differencing involves calculating the difference between the intent state update vectors over time. The difference in intent state between adjacent speech segments is the difference between the corresponding elements of the intent state update vector of the current speech segment and the intent state update vector of the previous speech segment. The intent change sequence, obtained through time series differencing, reflects the changes in intent state between adjacent speech segments. The intent change sequence can help analyze the trend of intent changes between consecutive speech segments; for example, a large value in the intent change sequence indicates a significant change in intent, while a small value indicates a relatively stable intent.

[0096] Step S3242: Match the sequence of intention changes with a preset library of typical intention change patterns. The library of typical intention change patterns includes gradual change patterns, abrupt change patterns, and stable change patterns.

[0097] The predefined library of typical intent change patterns is a collection of various typical intent change patterns. Matching involves comparing the intent change sequence with patterns in the library to find the most similar pattern. Gradual change patterns represent intents that change gradually over a period of time, with relatively small and stable changes. Abrupt change patterns represent intents that change drastically in a short period, with large and sudden changes. Stable change patterns represent intents that remain largely unchanged over a period of time, with changes close to zero. When matching the intent change sequence with the predefined library of typical intent change patterns, similarity calculation methods can be used. For example, the Euclidean distance or cosine similarity between the intent change sequence and each typical intent change pattern can be calculated. The smaller the Euclidean distance or the larger the cosine similarity, the more similar the intent change sequence is to that typical intent change pattern. Through matching, the specific change patterns of intent categories between consecutive speech segments are determined, providing a basis for subsequent intent analysis and processing.

[0098] Step S3243: Determine the specific change pattern of intent category between consecutive speech segments based on the matching results.

[0099] When determining the specific change pattern of intent category between consecutive speech segments based on matching results, if the similarity between the intent change sequence and the gradual change pattern is the highest, then the intent change pattern between consecutive speech segments is determined to be gradual; if the similarity with the abrupt change pattern is the highest, then it is determined to be abrupt; and if the similarity with the stable change pattern is the highest, then it is determined to be stable. For example, assuming that by calculating similarity, the cosine similarity between the intent change sequence and the gradual change pattern is 0.9, the cosine similarity with the abrupt change pattern is 0.2, and the cosine similarity with the stable change pattern is 0.3, then the specific change pattern of intent category between consecutive speech segments is determined to be gradual. Determining the specific intent change pattern helps to better understand how the user's interactive intent changes over time, providing more accurate information for subsequent intent analysis and processing.

[0100] Step S3244: Extract the sign and amplitude features of the intention change sequence, and generate directional features reflecting the direction of intention change and amplitude features reflecting the degree of change.

[0101] The sign feature of the intention change sequence is the sign of each element in the sequence; a positive sign indicates an increase in intention, and a negative sign indicates a decrease. The direction feature, generated based on the sign feature, reflects the direction of the intention change. The amplitude feature is the absolute value of each element in the sequence, representing the degree of intention change. The magnitude feature, generated based on the amplitude feature, reflects the magnitude of the intention change. When extracting the sign and amplitude features of the intention change sequence, it is assumed that the intention change sequence is... Then symbolic features Where sgn(x) is the sign function, sgn(x) = 1 when x > 0; sgn(x) = 0 when x = 0; and sgn(x) = -1 when x < 0. Amplitude characteristics. Directional features are generated based on symbolic features, and amplitude features are generated based on amplitude features.

[0102] Step S3245: Perform feature fusion processing on the specific change patterns, directional features and amplitude features to form evolution trend features.

[0103] Feature fusion processing integrates specific change patterns, directional features, and amplitude features, combining them into a more comprehensive feature. Evolutionary trend features, obtained through feature fusion, contain information such as the pattern, direction, and degree of intent change between consecutive speech segments. When fusing specific change patterns, directional features, and amplitude features, concatenation or weighted combination methods can be used. For example, if the specific change pattern is represented by a one-hot encoded vector, and the directional and amplitude features are represented by separate vectors, these three vectors can be concatenated sequentially to form a new vector as the evolutionary trend feature. Through feature fusion processing, multiple features are integrated to form the evolutionary trend feature, which more comprehensively reflects the trend of intent change between consecutive speech segments.

[0104] Step S325: Perform dimensional concatenation processing on the intention state update vector and the evolution trend feature to form the intention evolution feature vector.

[0105] Dimensional concatenation is an operation that concatenates the intent state update vector and evolution trend features in a specific dimension, arranging their elements sequentially to form a higher-dimensional vector. The intent evolution feature vector, obtained through dimensional concatenation, contains information from both the intent state update vector and the evolution trend features, providing a more comprehensive reflection of intent changes between consecutive speech segments. By combining the intent state update vector and evolution trend features through dimensional concatenation, the intent evolution feature vector is formed. This vector provides richer information for subsequent speech semantic understanding and interactive intent analysis, helping to more accurately grasp the user's interactive intent.

[0106] Step S330: The speech semantic understanding model’s colloquial interference calibration module performs expression normation correction on the intention evolution feature vector to eliminate non-normative expression interference in the speech content and generate calibrated semantic core features.

[0107] The colloquialism interference calibration module is a component of the speech semantic understanding model used to correct and process non-standard expressions in speech. Expression standardization correction involves analyzing and adjusting the intent evolution feature vector, removing interference information caused by colloquialisms to make the speech content more standardized and accurate. Non-standard expression interference refers to situations in speech that do not conform to standard grammar or vocabulary usage due to spoken habits, casual expressions, etc., such as filler words, repetitive words, and mispronunciations. The calibrated semantic core features are the features obtained after processing the intent evolution feature vector through the colloquialism interference calibration module; they more accurately reflect the semantic core of the speech content. When performing expression standardization correction on the intent evolution feature vector through the colloquialism interference calibration module of the speech semantic understanding model, the module can employ a combination of rule matching and machine learning. For example, rules for some common filler words and non-standard expressions can be predefined; when these rules match in the speech content corresponding to the intent evolution feature vector, appropriate corrections are made. Simultaneously, machine learning models, such as neural networks, are trained on a large amount of standardized and non-standard speech data, allowing the model to learn the mapping relationship between non-standard and standardized expressions. The intent evolution feature vector is then input into the colloquialization interference calibration module. The module processes the vector according to rules and the trained model to eliminate interference from non-standard expressions and generate calibrated semantic core features.

[0108] Step S340: Bind and associate the calibrated semantic core features with the timestamp information of the speech segments to generate a semantic understanding result containing the intent evolution trajectory and semantic core content.

[0109] The binding and association process combines the calibrated semantic core features with the timestamp information of speech segments, linking them together. The timestamp information identifies the temporal position of each speech segment in the continuous speech data stream, accurately recording when the segment occurs. The intent evolution trajectory is the trajectory of the user's interactive intent changing over time in continuous speech expression, which can be obtained through the previous intent evolution modeling layer. The semantic core content is the most crucial and semantically expressive information extracted from the speech content, and the calibrated semantic core features reflect this information. When binding and associating the calibrated semantic core features with the timestamp information of speech segments, the timestamp information can be added as a dimension of the calibrated semantic core features, forming a new vector. In this way, temporal and semantic information are combined to generate a semantic understanding result that includes the intent evolution trajectory and semantic core content. This semantic understanding result not only contains the core semantics of the speech content but also records the temporal position of that semantics, providing more comprehensive information for subsequent determination of user interactive intent types and key information location based on the semantic understanding result.

[0110] Step S400: Determine the user interaction intent type corresponding to the continuous speech data stream and the key information location features of the user interaction intent in the speech content based on the semantic understanding results.

[0111] Semantic understanding results include the intent evolution trajectory and core semantic content. Further analysis of this information allows us to determine the user interaction intent type corresponding to the continuous speech data stream and the key information location features of that intent within the speech content. The user interaction intent type refers to the interactive goal the user hopes to achieve through speech expression, such as querying information, issuing commands, or expressing emotions. Key information location features are used to identify the location of key information related to the user interaction intent within the speech content; they help to accurately extract and understand important information related to the intent. Determining this information based on semantic understanding results requires comprehensive consideration of changes in the intent evolution trajectory, the meaning of the core semantic content, and their matching relationships with different interaction intent types.

[0112] As one implementation method, step S400, determining the user interaction intent type corresponding to the continuous speech data stream and the key information location features of the user interaction intent in the speech content based on the semantic understanding results, can be specifically implemented as the following steps S410 to S440:

[0113] Step S410: Extract the time series features of the intent evolution trajectory from the semantic understanding results and generate evolution parameters that reflect the rate of change of intent.

[0114] The time-series features of the intent evolution trajectory represent the changes in intent over time, including its trend and magnitude. Evolutionary parameters reflecting the rate of intent change are obtained through analysis of these time-series features and describe the speed of intent change. Various methods can be used to analyze the intent evolution trajectory when extracting its time-series features from semantic understanding results. For example, the trajectory can be smoothed to remove noise and fluctuations, making the trend more apparent. Then, the amount of change in intent at different time points is calculated to reflect its evolution. When generating evolutionary parameters reflecting the rate of intent change, the average rate of change can be calculated based on the ratio of the change in intent to the time interval, or factors such as acceleration can be considered. Generating evolutionary parameters allows for a more accurate understanding of the rate of intent change, providing crucial information for determining the type of user interaction intent.

[0115] As one implementation method, step S410: extracting the time-series features of the intent evolution trajectory from the semantic understanding results, and generating evolution parameters reflecting the rate of change of intent, can be specifically implemented as the following steps S411 to S415:

[0116] Step S411: Perform time window segmentation processing on the intention evolution trajectory to divide the intention evolution trajectory into multiple sub-trajectory units with temporal continuity.

[0117] The time window segmentation process involves dividing the time axis of the intent evolution trajectory into multiple windows at regular time intervals. Each window forms a sub-trajectory unit. A temporally continuous sub-trajectory unit represents a segment of the intent evolution trajectory that is continuous in time, containing information about changes in intent within that time period. When segmenting the intent evolution trajectory into time windows, an appropriate time window size can be selected based on actual needs and the characteristics of the speech data. For example, if the intent of the speech content changes frequently, a smaller time window can be chosen to observe the changes in intent more closely; if the intent changes relatively slowly, a larger time window can be chosen to reduce computational load. By dividing the intent evolution trajectory into multiple sub-trajectory units, it is easier to analyze the changes in intent within each time period, providing a foundation for subsequent calculations of the intent change rate.

[0118] Step S412: Calculate the slope for each sub-trajectory unit, extract the rate of change of the intended evolution value over time in the sub-trajectory unit, and generate the sub-trajectory change rate.

[0119] The slope calculation involves analyzing the relationship between the intent evolution value and time in each sub-trajectory unit and calculating its slope. The rate of change of the intent evolution value with time is the proportion of change of the intent evolution value with time within the sub-trajectory unit, reflecting the speed of intent change within that time period. The sub-trajectory change rate is the rate of change obtained by calculating the slope for each sub-trajectory unit. When calculating the slope for each sub-trajectory unit, the intent evolution value in the sub-trajectory unit can be regarded as a function of time, and its slope can be calculated using numerical differentiation. For example, for a sub-trajectory unit, if two adjacent time points t1 and t2 are selected, and the corresponding intent evolution values ​​are y1 and y2 respectively, then the slope of the sub-trajectory unit between these two time points can be approximately calculated as follows: By performing such calculations on each sub-trajectory unit, the rate of change of the sub-trajectory of each sub-trajectory unit can be obtained. These rates of change can reflect the changes in the intention over different time periods.

[0120] Step S413: Statistically analyze the distribution characteristics of the sub-trajectory change rate over time, including the mean, variance, and extreme values ​​of the change rate, to generate statistical characteristics of the intended change rate.

[0121] The statistical distribution characteristics of the sub-trajectory change rate over time are obtained by statistically analyzing the change rates of all sub-trajectory units to understand their temporal distribution patterns. The mean of the change rate is the average of the change rates of all sub-trajectory units, reflecting the overall level of the intention change rate. The variance represents the dispersion of the sub-trajectory change rate relative to the mean; a larger variance indicates greater fluctuation in the change rate. Extreme values ​​are the maximum and minimum values ​​among the sub-trajectory change rates, reflecting the maximum and minimum amplitude of the intention change. To generate the statistical characteristics of the intention change rate, first, the sum of the change rates of all sub-trajectory units is calculated, then divided by the number of sub-trajectory units to obtain the mean. Next, the square of the difference between each sub-trajectory change rate and the mean is calculated, and the average of these squares is obtained to obtain the variance. Finally, the maximum and minimum values ​​among all sub-trajectory change rates are identified as extreme values. These statistical characteristics can more comprehensively describe the distribution of the intention change rate.

[0122] Step S414: Perform frequency domain transformation on the intention evolution trajectory, convert the time domain intention evolution value into a frequency domain energy distribution map, and extract the energy proportion of the main frequency components as the periodic feature of intention change.

[0123] Frequency domain transformation converts a time-domain signal into a frequency-domain representation, allowing analysis of the signal's frequency components and energy distribution. The frequency-domain energy distribution map shows how the signal's energy is distributed with frequency. The energy percentage of the dominant frequency component represents the proportion of the higher-energy frequency component within the total energy. The periodicity of intent changes, obtained through frequency domain transformation of the intent evolution trajectory, describes whether the intent change is periodic and its periodic characteristics. Methods such as Fourier transform can be used when performing frequency domain transformation on the intent evolution trajectory. The Fourier transform decomposes the time-domain signal into combinations of sine and cosine waves of different frequencies, thus obtaining the signal's frequency-domain representation. After performing a Fourier transform on the intent evolution trajectory, the frequency-domain energy distribution map is obtained. Then, the higher-energy dominant frequency components are identified, and their proportion of the total energy is calculated. This proportion reflects whether the intent change is periodic; if a distinct dominant frequency component exists with a high energy percentage, the intent change may exhibit some periodicity. By extracting this periodic feature, we can further understand the changing patterns of intent and provide more evidence for determining the types of user interaction intents.

[0124] Step S415: Combine statistical features and periodic features in a dimensionality-based manner to generate evolution parameters that reflect the rate of change of intent.

[0125] Dimensional combination processing combines statistical and periodic features to form a feature vector containing more information. The evolution parameter reflecting the rate of change of intent is obtained through this combination, providing a more comprehensive description of the rate of change of intent. When performing dimensional combination processing on statistical and periodic features, the statistical features (mean, variance, extreme values, etc.) and the periodic features (energy proportions of major frequency components) can be arranged sequentially into a vector. For example, assuming the statistical feature vector is S = (s1, s2, s3) and the periodic feature is p, the combined evolution parameter vector is E = (s1, s2, s3, p). Through this dimensional combination processing, multiple features reflecting changes in intent are integrated to form a more comprehensive evolution parameter, which can more accurately reflect the rate and pattern of change in intent.

[0126] Step S420: Perform correlation analysis on the evolution parameters and the cross-segment semantic continuity in the semantic association feature set to generate an intent stability evaluation vector.

[0127] Association analysis involves analyzing evolution parameters and cross-segment semantic continuity to identify their correlations. Cross-segment semantic continuity reflects the degree of semantic continuity between different speech segments, indicating the semantic coherence of the speech content. The intent stability assessment vector, obtained through association analysis, is used to evaluate the stability of user interaction intent. When performing association analysis on evolution parameters and cross-segment semantic continuity, both parameters and the degree of semantic continuity are first normalized to ensure they fall within the same numerical range. Then, their correlation is calculated, for example, using the Pearson correlation coefficient. Based on the correlation results, and combined with the values ​​of evolution parameters and cross-segment semantic continuity, the intent stability assessment vector is generated. If the evolution parameters indicate a low rate of intent change and a high degree of cross-segment semantic continuity, the intent is relatively stable, and the corresponding element value in the intent stability assessment vector may be large. Conversely, if the rate of intent change is high and the degree of cross-segment semantic continuity is low, the intent is unstable, and the corresponding element value in the intent stability assessment vector may be small. By generating the intent stability assessment vector, the stability of user interaction intent can be evaluated more accurately, providing an important reference for determining the intent type.

[0128] Step S430: Perform intent type matching processing based on the intent stability evaluation vector and the semantic core content in the semantic understanding result to determine the overall user interaction intent type of the continuous voice data stream.

[0129] Intent type matching involves matching the intent stability assessment vector and the semantic core content from the semantic understanding results with different preset user interaction intent types to find the most suitable intent type. The overall user interaction intent type of the continuous speech data stream is the main interaction goal that the user expects to achieve throughout the speech expression process, determined through this matching process. When performing intent type matching based on the intent stability assessment vector and the semantic core content from the semantic understanding results, a preset intent type library needs to be established first, containing various common user interaction intent types and their feature descriptions. Then, the intent stability assessment vector and the semantic core content are compared with each intent type in the intent type library. Similarity calculation methods can be used, such as calculating the semantic similarity between the semantic core content and the description of each intent type, while also considering the degree of matching between the intent stability assessment vector and the intent type stability requirements. Based on the similarity and matching results, the most matching intent type is selected as the overall user interaction intent type of the continuous speech data stream.

[0130] As one implementation method, step S430 involves performing intent type matching processing based on the intent stability evaluation vector and the semantic core content in the semantic understanding result to determine the overall user interaction intent type of the continuous voice data stream. Specifically, this can be implemented as follows: steps S431 to S435.

[0131] Step S431: Input the intent stability assessment vector into the preset intent confidence calculation model. The model generates a confidence score for the user interaction intent type based on the correlation between intent evolution parameters and cross-segment semantic continuity.

[0132] The pre-trained intent confidence calculation model is used to calculate the confidence score of user interaction intent types based on the intent stability assessment vector and related relationships. The relationship between intent evolution parameters and cross-segment semantic continuity is analyzed in previous steps, showing their correlation and mutual influence. The confidence score of user interaction intent types is a score of credibility for each user interaction intent type calculated by this model. When the intent stability assessment vector is input into the pre-trained intent confidence calculation model, the model will perform calculations based on its internal structure and trained parameters, combined with the relationship between intent evolution parameters and cross-segment semantic continuity. For example, the model can use a neural network structure, using the intent stability assessment vector as input to the input layer, performing feature transformation and calculations through intermediate hidden layers, and finally outputting the confidence score for each user interaction intent type at the output layer. The higher the confidence score, the better the intent type matches the user's speech expression, and the higher the credibility.

[0133] Step S432: Extract keywords from the semantic core content in the semantic understanding results to obtain a set of intent keywords that are strongly associated with the semantic core content.

[0134] Keyword extraction is the process of extracting the keywords that best represent the meaning of the semantic core content from the semantic understanding results. The set of intent keywords strongly associated with the semantic core content is the set of keywords closely related to the user's interaction intent, obtained through this extraction. Various methods can be used to extract keywords from the semantic core content of the semantic understanding results. For example, statistical methods can be used to count the frequency of each word in the semantic core content and select words with high frequencies as keywords. Semantic methods can also be used, such as using word vector models to calculate the semantic similarity between words and identifying words with high semantic similarity to the overall semantic core content as keywords. Domain knowledge and grammatical rules can also be combined to filter out important words related to intent. Through keyword extraction, an intent keyword set is obtained, which can more accurately reflect the key information of the semantic core content.

[0135] Step S433: Perform semantic similarity calculation between the intent keyword set and the preset intent type tag library to generate intent tag matching degree.

[0136] The predefined intent type tag library is a collection of various user interaction intent types and their corresponding tag keywords. Semantic similarity calculation involves determining the semantic similarity between keywords in the intent keyword set and tag keywords in the intent type tag library. Intent tag matching score is a parameter obtained through this semantic similarity calculation, used to measure the matching degree between the intent keyword set and each intent type tag library. When calculating the semantic similarity between the intent keyword set and the predefined intent type tag library, methods such as word vector models can be used. For example, both the intent keyword set and the tag keywords in the intent type tag library can be converted into word vector representations, and then the cosine similarity between them can be calculated. For each intent type tag library, the average cosine similarity between all keywords in the intent keyword set and all tag keywords in that tag library is calculated as the intent tag matching score. The higher the matching score, the more closely the intent keyword set matches the intent type tag library, and the more likely that intent type is the user's interaction intent type.

[0137] Step S434: Integrate the confidence score and intent tag matching degree to generate a comprehensive matching index.

[0138] The fusion of confidence score and intent tag matching score involves comprehensively processing both to generate a more comprehensive matching metric. This comprehensive matching metric, obtained through this fusion, is a parameter used to more accurately assess the degree of matching between each user's interaction intent type and their voice expression. A weighted summation method can be used when fusing confidence score and intent tag matching score. For example, different weights can be assigned to the confidence score and intent tag matching score respectively, and then multiplied by their respective weights and summed to obtain the comprehensive matching metric. The weight allocation can be adjusted based on actual circumstances and experience; if the confidence score is considered more important, it can be assigned a larger weight; if the intent tag matching score is considered more crucial, it can be assigned a larger weight. By generating a comprehensive matching metric, the stability of intent and the degree of semantic matching can be considered more comprehensively, providing a more accurate basis for determining the type of user interaction intent.

[0139] Step S435: Select the intent type with the highest comprehensive matching index as the overall user interaction intent type of the continuous voice data stream.

[0140] After obtaining the comprehensive matching index for each user interaction intent type, the intent type with the highest comprehensive matching index is selected as the overall user interaction intent type for the continuous voice data stream by comparing the magnitudes of these indices. The highest comprehensive matching index indicates that the intent type best matches the user's voice expression in terms of intent stability and semantic matching degree, and is therefore most likely to be the user's actual interaction intent.

[0141] Step S440: Perform dual-modal saliency joint analysis on the semantic core content in the semantic understanding result and the stress distribution pattern in the acoustic feature sequence to identify the overlapping areas in the speech content that simultaneously meet the semantic concentration threshold and the acoustic saliency threshold, and generate key information localization features.

[0142] The bimodal saliency joint analysis process integrates information from two different modalities: the semantic core content in the semantic understanding results and the stress distribution pattern in the acoustic feature sequence, considering their saliency features. A pre-set semantic concentration threshold is used to determine the degree of semantic concentration in the semantic core content; regions exceeding this threshold are considered to have high semantic concentration. A pre-set acoustic saliency threshold is used to determine the acoustic saliency in the stress distribution pattern; regions exceeding this threshold are considered acoustically prominent. The overlapping region is the intersection of regions in the semantic core content where semantic concentration exceeds the threshold and regions in the stress distribution pattern where acoustic saliency exceeds the threshold. Key information location features are obtained through this joint analysis and are used to identify the location of key information related to user interaction intent within the speech content. When performing bimodal saliency joint analysis on the semantic core content and stress distribution pattern in the acoustic feature sequence, saliency analysis is first performed separately for the semantic core content and stress distribution pattern. For the semantic core content, the semantic concentration of each region is calculated, and regions exceeding the semantic concentration threshold are identified. For the accent distribution pattern, the acoustic saliency of each region is calculated, and regions exceeding the acoustic saliency threshold are identified. Then, the overlapping portion of these two regions is determined. Finally, based on the location and extent of the overlapping region, key information localization features are generated. These features can be represented by parameters such as location coordinates and length, and they can help accurately locate and extract key information related to the user's interaction intent from the speech content.

[0143] Step S500: Generate an LED display control command containing an interactive content identifier based on the user's interaction intent type and key information location features, and send the LED display control command to the target LED display screen to execute the interactive display operation.

[0144] When generating LED display control commands based on user interaction intent types and key information location characteristics, it is necessary to select appropriate interactive content according to the user's intent and determine the display method and timing of this content on the LED display screen. Interactive content identifiers are tags or numbers used to uniquely identify different interactive content, helping to accurately select and display the corresponding content. LED display control commands are instructions containing interactive content identifiers and display control parameters, which can control the target LED display screen to perform interactive display operations as required. After the generated LED display control commands are sent to the target LED display screen, the display screen will execute the corresponding display operations according to the content of the commands, realizing interaction with the user.

[0145] As one implementation method, step S500, generating an LED display control instruction containing interactive content identifiers based on the user's interaction intent type and key information location features, can be specifically implemented as the following steps S510 to S560:

[0146] Step S510: Query the preset interactive content knowledge graph according to the user's interaction intent type to obtain the candidate content chain associated with the intent type. The candidate content chain contains multiple content identifiers sorted by semantic relevance.

[0147] The pre-built interactive content knowledge graph is a pre-constructed knowledge graph containing the relationships between various user interaction intent types and related interactive content. It is represented as a graph, where nodes represent interaction intent types and interactive content, and edges represent the relationships between them. The candidate content chain is a sequence of content identifiers related to user interaction intent types, obtained by querying the knowledge graph. Semantic relevance is the degree of semantic relevance between interactive content and user interaction intent types; higher relevance indicates a better match between the content and the intent. When querying the pre-built interactive content knowledge graph based on a user interaction intent type, the node corresponding to that intent type is found in the knowledge graph. Then, along the associated edges, related interactive content nodes are found, and the content identifiers corresponding to these nodes are extracted. These content identifiers are sorted from high to low semantic relevance to form the candidate content chain.

[0148] Step S520: Extract the temporal distribution features of overlapping regions in the speech content from the key information localization features, and generate a temporal weight sequence of the information region.

[0149] The overlapping regions in the key information location features are those regions in the speech content that simultaneously meet the semantic concentration threshold and acoustic saliency threshold, obtained through bimodal saliency joint analysis in the previous steps. Temporal distribution features include the temporal location and duration of these overlapping regions within the speech content. The temporal weight sequence of information regions is a weight sequence used to represent the importance of information regions at different time points, obtained through analysis of these temporal distribution features. When extracting the temporal distribution features of overlapping regions in the speech content from the key information location features, the start and end times of the overlapping regions in the speech content are first determined. Then, the duration of each overlapping region is calculated. Based on this temporal information, a temporal weight sequence of information regions is generated. For example, time points corresponding to overlapping regions with longer durations can be assigned higher weights, as these regions may contain more important information. By generating a temporal weight sequence, the temporal distribution of key information in the speech content can be grasped more accurately, providing a basis for subsequent content display priority adjustments.

[0150] Step S533: Align the candidate content chain with the time weight sequence, and adjust the display priority of the content identifiers in the candidate content chain according to the time order of appearance of the information area.

[0151] Alignment involves mapping content identifiers in the candidate content chain to time points in the time-weighted sequence, making them temporally related. Display priority refers to the order and importance of each content identifier in the candidate content chain when displayed on the LED display. When adjusting the display priority of content identifiers in the candidate content chain based on the chronological order of information appearance, the content identifiers are first matched with time points in the time-weighted sequence. For example, if a content identifier is associated with a time point with a high time weight, it means that the content corresponds to the time when key information appears in the voice content, and thus the display priority of that content identifier can be increased. Then, the candidate content chain is reordered according to the adjusted priority. In this way, when displaying interactive content on the LED display, content related to key information can be displayed first, according to the chronological order of appearance of key information in the voice, improving the relevance and effectiveness of the display.

[0152] Step S540: Perform redundant content filtering on the adjusted candidate content chain and retain the target content identifier with the highest semantic relevance to the key information location features.

[0153] Redundant content filtering involves screening the content in the adjusted candidate content chain, removing content with low semantic relevance to the key information positioning features or content that is repetitive. The target content identifier with the highest semantic relevance to the key information positioning features is the one obtained through this filtering, most accurately reflecting the user's interaction intent and key information. When performing redundant content filtering on the adjusted candidate content chain, semantic similarity calculation can be used. The semantic similarity between the content corresponding to each content identifier and the key information identified by the key information positioning features is calculated, and the content identifier with higher semantic similarity is selected as the target content identifier. Alternatively, filtering can be based on the importance and uniqueness of the content, removing duplicate or unimportant content. Through redundant content filtering, the most relevant target content identifiers are retained, unnecessary displayed content is reduced, and the accuracy and simplicity of the LED display screen are improved.

[0154] Step S550: Bind and associate the target content identifier with the peak time point in the time weight sequence to generate display control parameters that include the content display sequence and the key highlight position.

[0155] The binding and association process involves combining target content identifiers with peak time points in the time-weighted sequence, thus linking them together. Peak time points are the time points with the highest weight values ​​in the time-weighted sequence, typically corresponding to the moments when key information in the audio content is most prominent. Content display sequence refers to the chronological order in which target content identifiers are displayed on the LED display screen, which can be determined based on their association with peak time points. Emphasis positions are the parts of the displayed content that need to be highlighted, determined based on key information location features. When generating display control parameters that include content display sequence and emphasis positions, each target content identifier is first bound to its corresponding peak time point. For example, if a target content identifier is associated with a peak time point, then that content will be displayed on the LED display screen at the time corresponding to that peak time point. Then, the parts of the content that need to be highlighted are determined based on key information location features. Finally, this information is combined to form display control parameters. These parameters can be represented using timestamps, location coordinates, font size, etc., allowing for precise control over the display method and timing of the target content on the LED display screen.

[0156] Step S560: Encapsulate the display control parameters into LED display control instructions.

[0157] Encapsulation is the process of combining display control parameters according to a specific format and protocol to form a complete instruction. LED display control instructions, obtained through this encapsulation, are used to control the target LED display screen for interactive display operations. When encapsulating display control parameters into LED display control instructions, the control protocol and interface specifications of the LED display screen must be followed. For example, the instruction may include information such as the identifier of the display content, the start time of the display, the duration of the display, the display position, font size, and color. This information is encoded according to the format specified in the protocol to form a binary or text-based instruction. Finally, the encapsulated LED display control instruction is sent to the target LED display screen, which will then execute the corresponding interactive display operations based on the content of the instruction, enabling voice interaction with the user.

[0158] It is understood that the various algorithms involved in the above descriptions of the embodiments of the present invention, such as the Euclidean distance algorithm, the cosine distance algorithm, the Pearson correlation coefficient algorithm, the local maximum detection algorithm, etc., can all be obtained from relevant content in the prior art. To save space, they will not be elaborated on in the embodiments of the present invention. In addition, those skilled in the art can supplement the details based on common knowledge in the art when implementing the solutions of the present invention. For example, based on common knowledge in the art, normalization can be used to eliminate dimensional conflicts before feature fusion, interpolation can be used to eliminate dimensional differences, thresholds can be reasonably set in combination with historical data, experience or business scenario requirements, the model can be trained based on a general model training method, the number of layers in the model structure can be set according to actual needs, the activation function can be selected, etc. The present invention will not provide redundant descriptions of overly detailed implementation processes.

[0159] Please see Figure 2 , Figure 2This is a schematic diagram of a computer system provided in an embodiment of the present invention. The computer system includes at least a processor 101, a communication interface 102, and a memory 103. The processor 101, communication interface 102, and memory 103 can be connected via a bus or other means. The processor 101 (or Central Processing Unit, CPU) is the computing and control core of the computer system, capable of parsing various instructions and processing various data within the computer system. The communication interface 102 may optionally include a standard wired interface or a wireless interface (such as Wi-Fi, mobile communication interface, etc.), and can be used to send and receive data under the control of the processor 101; the communication interface 102 can also be used for data transmission and interaction within the computer system. The memory 103 is a storage device in the computer system used to store programs and data. It is understood that the memory 103 here can include the computer system's built-in memory, or it can include extended memory supported by the computer system. The memory 103 provides storage space, which stores the computer system's operating system; this invention does not limit this storage space. In one embodiment, the processor 101 executes the LED display interaction method supporting voice interaction provided above in the embodiments of the present invention by running a computer program in the memory 103.

Claims

1. An interactive method for an LED display screen supporting voice interaction, characterized in that, The method includes: Acquire a continuous voice data stream sent by the user, the continuous voice data stream containing multiple voice segments with timestamps; wherein, the timestamp is an identifier added to the voice segment to indicate the time position of the voice segment in the continuous voice data stream, and the time point of each voice segment in the entire voice expression process can be located through the timestamp; The continuous speech data stream is subjected to speech feature extraction processing to obtain the acoustic feature sequence and semantic association feature set of the speech segment; A pre-trained speech semantic understanding model is invoked to perform joint semantic parsing processing on the acoustic feature sequence and the semantic association feature set to generate the semantic understanding result of the speech segment; Based on the semantic understanding results, the user interaction intent type corresponding to the continuous voice data stream and the key information location features of the user interaction intent in the voice content are determined; wherein, the key information location features are features used to identify the location of key information of the user interaction intent in the voice content. Based on the user interaction intent type and the key information location features, an LED display control command containing an interactive content identifier is generated, and the LED display control command is sent to the target LED display screen to perform the interactive display operation. Specifically, this includes: querying a preset interactive content knowledge graph according to the user interaction intent type to obtain a candidate content chain associated with the intent type, wherein the candidate content chain contains multiple content identifiers sorted by semantic relevance. Extract the temporal distribution features of overlapping regions in the speech content from the key information location features, and generate a temporal weight sequence of the information regions; Align the candidate content chain with the time weight sequence, and adjust the display priority of the content identifiers in the candidate content chain according to the time order of the information area. Redundant content filtering is performed on the adjusted candidate content chain, retaining the target content identifier with the highest semantic correlation to the key information location feature. The target content identifier is bound and associated with the peak time point in the time weight sequence to generate display control parameters that include the content display sequence and the key highlight position; The display control parameters are encapsulated into LED display control instructions.

2. The LED display screen interaction method supporting voice interaction as described in claim 1, characterized in that, The step of performing speech feature extraction processing on the continuous speech data stream to obtain the acoustic feature sequence and semantic association feature set of the speech segment includes: Boundary detection processing is performed on the continuous speech data stream. The natural segmentation boundary of the continuous speech data stream is determined based on the abrupt change points of speech energy and semantic pause markers, and the continuous speech data stream is segmented into speech segments with semantic integrity. For each of the speech segments, intent-oriented acoustic feature focusing processing is performed to extract a subset of acoustic features that are strongly correlated with the user's intent from the speech segment as an acoustic feature sequence. The acoustic feature subset includes stress distribution patterns and intonation rise and fall trajectories. For each of the speech segments, cross-segment semantic association modeling is performed, and the text conversion results of the speech segment and the preceding speech segment are fused together to generate an association feature matrix containing semantic continuity. The semantic intensity decay calculation is performed on the associated feature matrix, and the weight values ​​of the associated features are adjusted according to the time interval between speech segments to generate the semantic associated feature set, which includes cross-segment semantic continuity and intra-segment semantic concentration.

3. The LED display screen interaction method supporting voice interaction as described in claim 2, characterized in that, The step of performing intent-guided acoustic feature focusing processing on each of the speech segments, and extracting a subset of acoustic features strongly correlated with the user's intent from the speech segments as an acoustic feature sequence, includes: The speech segment is subjected to multi-resolution time-frequency analysis to generate a set of time-spectrum graphs containing different time resolutions; Extract the energy concentration points of the stressed regions from the set of time-spectrum graphs, statistically analyze the positional distribution pattern of the energy concentration points in the speech segment, and generate the stressed distribution pattern. The fundamental frequency curve of the speech segment is subjected to trend analysis processing to identify the rising and falling inflection points in the fundamental frequency curve, calculate the slope change rate between adjacent inflection points, and generate the pitch rise and fall trajectory. Based on the stress distribution pattern and the pitch rise and fall trajectory, the part that overlaps with general speech features is filtered out through the pre-trained intention-acoustic feature mapping relationship, and the feature components that are strongly associated with the target interaction intention are retained to form the acoustic feature sequence. The step of performing semantic intensity attenuation calculation on the associated feature matrix and adjusting the weight values ​​of the associated features according to the time interval between speech segments to generate the semantic associated feature set includes: Calculate the time interval between the current speech segment and the preceding speech segment, and generate time interval feature values; The corresponding weight attenuation coefficient is obtained by querying a preset attenuation function table based on the time interval feature value. The attenuation function table defines a negative correlation between the time interval length and the weight attenuation coefficient. The weight decay coefficient is applied to each dimension of the associated feature matrix to generate a decayed associated feature vector; The semantic concentration of the decayed associated feature vector is calculated, and the distribution density of the high-weight dimension in the associated feature vector is statistically analyzed to generate the semantic concentration within the segment. The attenuated associated feature vectors are subjected to cross-segment continuity evaluation processing. The high-weight dimension overlap between the current speech segment and the preceding speech segment is compared to generate the cross-segment semantic continuity. The semantic concentration within the segment and the semantic continuity across segments are arranged in chronological order of the speech segments to form the semantic association feature set.

4. The LED display screen interaction method supporting voice interaction as described in claim 1, characterized in that, The method of calling a pre-trained speech semantic understanding model to perform joint semantic parsing processing on the acoustic feature sequence and the semantic association feature set to generate the semantic understanding result of the speech segment includes: The acoustic feature sequence and the semantic association feature set are input into the context fusion module of the speech semantic understanding model. The context window length is determined based on the cross-segment semantic continuity in the semantic association feature set, and an extended feature sequence containing historical segment association information is generated. The extended feature sequence is modeled using the intent evolution modeling layer of the speech semantic understanding model to capture the intent change trend between consecutive speech segments and generate intent evolution feature vectors. The speech semantic understanding model uses a colloquial interference calibration module to perform expression normative correction on the intention evolution feature vector, eliminating non-normative expression interference in the speech content and generating calibrated semantic core features. The calibrated semantic core features are bound and associated with the timestamp information of the speech segment to generate a semantic understanding result containing the intent evolution trajectory and semantic core content.

5. The LED display screen interaction method supporting voice interaction as described in claim 4, characterized in that, The step of inputting the acoustic feature sequence and the semantic association feature set into the context fusion module of the speech semantic understanding model, determining the context window length based on the cross-segment semantic continuity in the semantic association feature set, and generating an extended feature sequence containing historical segment association information includes: Extract the cross-segment semantic continuity between the current speech segment and historical speech segments from the semantic association feature set. The cross-segment semantic continuity reflects the strength of the semantic association between the historical segment and the current segment. Based on the strength of the cross-segment semantic continuity, historical speech segments that are semantically related to the current segment are selected. The stronger the continuity, the more related historical segments are selected, and the weaker the continuity, the fewer related historical segments are selected. The acoustic feature sequence and semantic association feature set of the selected historical speech segment are concatenated and spliced ​​with the corresponding features of the current speech segment to generate a cross-segment feature splicing vector. The cross-segment feature splicing vector is subjected to feature dimension alignment processing to unify the feature representation dimensions of speech segments with different timestamps, thereby generating the extended feature sequence containing historical segment association information.

6. The LED display screen interaction method supporting voice interaction as described in claim 4, characterized in that, The process of modeling the temporal dependency relationship of the extended feature sequence through the intent evolution modeling layer of the speech semantic understanding model, capturing the intent change trend between consecutive speech segments, and generating an intent evolution feature vector includes: The extended feature sequence is input into the temporal gating unit of the intent evolution modeling layer, and the temporal gating unit generates forget gate and update gate parameters based on the semantic association strength between historical segments and current segments in the extended feature sequence; The forget gate parameter filters out historical feature information in the extended feature sequence that is irrelevant to the current intent, while retaining key feature components that are related to the evolution of intent. The core features of the current speech segment are weighted and fused with the retained historical key feature components through the update gate parameters to generate an intent state update vector. The intent state update vector is subjected to trend analysis processing to identify the change pattern of intent category between consecutive speech segments and generate evolution trend features that reflect the direction of intent change. The intention state update vector and the evolution trend feature are concatenated dimensionally to form the intention evolution feature vector.

7. The LED display screen interaction method supporting voice interaction as described in claim 1, characterized in that, The step of determining the user interaction intent type corresponding to the continuous speech data stream and the key information location features of the user interaction intent in the speech content based on the semantic understanding result includes: Extract the time-series features of the intent evolution trajectory from the semantic understanding results to generate evolution parameters that reflect the rate of change of intent; The evolution parameters are correlated with the cross-segment semantic continuity in the semantic association feature set to generate an intent stability evaluation vector. Based on the intent stability evaluation vector and the semantic core content in the semantic understanding result, intent type matching processing is performed to determine the overall user interaction intent type of the continuous voice data stream. The semantic core content in the semantic understanding result and the accent distribution pattern in the acoustic feature sequence are subjected to dual-modal saliency joint analysis to identify overlapping regions in the speech content that simultaneously meet the semantic concentration threshold and the acoustic saliency threshold, thereby generating the key information localization features.

8. The LED display screen interaction method supporting voice interaction as described in claim 7, characterized in that, The process of matching intent types based on the intent stability evaluation vector and the semantic core content in the semantic understanding result to determine the overall user interaction intent type of the continuous voice data stream includes: The intent stability evaluation vector is input into a preset intent confidence calculation model, which generates a confidence score for the user interaction intent type based on the correlation between intent evolution parameters and cross-segment semantic continuity. The semantic core content in the semantic understanding result is subjected to keyword extraction processing to obtain a set of intent keywords that are strongly associated with the semantic core content; The semantic similarity of the set of intent keywords with the preset intent type tag library is calculated to generate the intent tag matching degree; By combining the confidence score and the intent tag matching degree, a comprehensive matching index is generated; The intent type with the highest comprehensive matching index is selected as the overall user interaction intent type of the continuous voice data stream.

9. A computer system, characterized in that, include: A memory, wherein a computer program is stored; A processor for loading the computer program to implement the LED display interaction method supporting voice interaction as described in any one of claims 1-8.

Images