A speech emotion recognition method of dynamic window boundary patching and multi-scale feature extraction

Abstract

This invention discloses a speech emotion recognition method based on dynamic window boundary repair and multi-scale feature extraction. It extracts MFCC features from the input speech and maps them to a unified space. Shared features are obtained through multi-scale front-end, deep convolution, and macro-temporal modeling. These shared features are then fed into a global context modeling branch to extract sentence-level emotional semantics, and a dynamic window modeling branch to obtain local-global collaborative features. Boundary masks are constructed based on the start and end positions of the dynamic window, and latent space diffusion is used to repair the boundary tokens. The prediction results from the two branches are then gated and fused to output the emotion classification result. This invention solves problems such as insufficient multi-scale emotion feature extraction, fragmented dynamic window boundary context, distorted boundary representation, and high model deployment overhead. It achieves collaborative representation of global semantics, local salient emotional cues, and boundary transition features, enhancing the discriminative ability of emotional transition regions. It is suitable for real-time, lightweight speech emotion recognition scenarios.

Description

Technical Field

[0001] This invention relates to the field of speech emotion recognition, specifically to a speech emotion recognition method that uses dynamic window boundary repair and multi-scale feature extraction. Background Technology

[0002] With the rapid development of telecommunications networks, smart terminals, and artificial intelligence technologies, human-computer voice interaction has been widely applied in scenarios such as telecommunications customer service, intelligent voice assistants, online psychological counseling, and intelligent robots. In these scenarios, simply "understanding what is said" is far from enough; the system also needs to "understand the emotional state of the speaker," especially those subtle and easily overlooked emotions and those that are deliberately hidden or disguised.

[0003] Despite significant progress in deep learning-based speech emotion recognition technology, existing methods still face several key scientific and technological challenges that urgently need to be addressed when dealing with highly non-stationary, multi-granular, and acoustically complex speech signals in real-world scenarios.

[0004] The existing models mainly face the following problems: (1) Speech emotion cues have multi-scale characteristics. Short-term emotion changes may be concentrated in local high-energy regions, while long-term emotion trends may be distributed over a wider time range. Single-scale feature extraction methods are difficult to capture complementary emotion cues in short-term, medium-term, long-term and frequency directions at the same time. (2) Fixed window or ordinary local window attention is difficult to adapt to the problem of inconsistent lengths of emotion segments in different speech samples. Fixed windows may generate redundant calculations on emotionless segments or silent segments, and may also cut off emotion cues distributed across windows. (3) Although existing dynamic window modeling methods can adaptively divide windows according to the saliency of speech content, dynamic windows are essentially still a hard segmentation operation. When emotion information crosses adjacent windows, tokens at the window boundaries are prone to contextual fragmentation, unstable representation and feature distortion, thus affecting the model's ability to discriminate emotion transition regions. (4) Although some methods based on large-scale pre-trained speech models have strong representation capabilities, the number of model parameters and inference overhead are large, making it difficult to deploy in practical application scenarios that require real-time performance and lightweight design. Summary of the Invention

[0005] In view of this, this application provides a speech emotion recognition method based on multi-scale feature extraction and dynamic window boundary repair, which solves the problems of insufficient multi-scale emotion feature extraction, contextual fragmentation of dynamic window boundaries, distortion of boundary token representation, and high model deployment cost in existing speech emotion recognition methods.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows:

[0007] A speech emotion recognition method with dynamic window boundary repair and multi-scale feature extraction includes the following steps:

[0008] S1: Preprocess the input speech signal, extract the Mel frequency cepstral coefficients (MFCC) features, map the MFCC features to a unified representation space through linear projection, and then extract shared acoustic features through multi-scale front-end, deep convolution and macroscopic temporal modeling.

[0009] S2: Input the shared acoustic features into the global context modeling branch to extract sentence-level emotional semantics;

[0010] S3: Input the shared acoustic features into the dynamic window modeling branch to obtain local-global collaborative features;

[0011] S4: Construct a boundary mask based on the start and end positions of the dynamic window, perform latent space diffusion repair on the boundary tokens indicated by the boundary mask, and keep the non-boundary tokens unchanged as context.

[0012] S5: Gated fusion of the prediction results of the global context modeling branch and the dynamic window boundary repair branch;

[0013] S6: Output the final speech emotion classification result based on the fusion result obtained in S5.

[0014] Furthermore, in step S1 of the present invention, the multi-scale front end extracts multi-scale features through four parallel depthwise separable convolutional branches. The four branches are respectively used to capture short-term local time-frequency patterns, medium-term temporal changes, long-term sentiment trends, and discriminative cues along the frequency dimension. The outputs of the four branches are concatenated along the channel dimension and fused through convolution. Then, high-level local time-frequency representations are extracted through multiple two-dimensional convolutional layers and max pooling operations. Finally, significant temporal regions, frequency bands, and discriminative channels are enhanced through a spatial channel attention module to generate shared feature representations.

[0015] Furthermore, in step S2 of the present invention, the global context modeling branch flattens the shared features into spatial token dimensions, and then performs global feature transformation through a gated MLP.

[0016] Furthermore, in steps S3 and S4 of the present invention, the dynamic window modeling branch includes boundary-guided dynamic window partitioning, intra-window local attention, inter-window global attention, and boundary latent space diffusion repair; wherein, dynamic window partitioning adaptively generates windows of variable length according to the halting score; intra-window local attention captures fine-grained sentiment patterns within each window; inter-window global attention models long-distance dependencies between different windows; and boundary latent space diffusion repair selectively denoises and restores features of boundary markers.

[0017] Furthermore, in the dynamic window partitioning stage of this invention, the model identifies high-confidence boundary positions based on pause scores and constructs a token-to-window assignment matrix accordingly. Based on this assignment matrix, a local attention mask is generated, allowing each token to primarily interact with other tokens within the same dynamic window. Then, the model introduces a feature importance filtering mechanism, obtains the feature label importance scores calculated by the previous network module, extracts the median of all importance scores as a dynamic threshold, and adaptively groups the feature labels into strong sentiment association windows and weak sentiment association windows in chronological order.

[0018] Furthermore, the feature label importance score of the present invention is calculated as follows: a local attention mask is generated based on the label-to-window assignment relationship. Under the mask constraint, the importance score of each feature label is calculated using the attention weights obtained from the Transformer multi-head attention module. The calculation formula is as follows:

[0019] ;

[0020] in, This represents the calculated feature label importance score. This represents the total number of attention heads in a multi-head attention mechanism. This represents the attention weight from the s-th attention head. This represents the length of the input sequence, i.e., the size of one side dimension of the average attention matrix. The model accumulates this matrix along the sequence length dimension to reduce its dimensionality. This represents the normalized exponential function, used to map the accumulated weight values ​​to... The interval is transformed into a probability distribution.

[0021] Furthermore, in the local attention stage within a window, the model performs local multi-head self-attention within each dynamic window; in the global attention stage between windows, the model weights and aggregates local token features based on local attention scores to form a window-level representation, and then applies global window attention to obtain window-level context and window attention scores; finally, the local token features are fused with the global context so that each token contains both local information within the window and global information between windows.

[0022] Further, in step S4 of the present invention, the latent space diffusion repair includes: during the training process, the model adds Gaussian noise to the dynamic window boundary tokens, and the forward diffusion process is described as follows:

[0023] ;

[0024] in, For noise scheduling parameters, Represents the potential features of the original boundary. Indicates the first Features after noise injection Gaussian noise is represented; subsequently, a conditional denoiser, conditioned on noisy boundary features, non-boundary context, boundary mask, stopping score, and time step, predicts the noise components in the boundary region and progressively recovers their latent representations; the repaired boundary tokens are merged back into the complete token sequence, while the non-boundary tokens retain their original representations; finally, the repaired token features are normalized and average pooled, and input into the classification head to obtain the predicted logit of the branch.

[0025] Furthermore, in step S5 of the present invention, the gated fusion generates fusion weights through learnable parameters, calculates the weighted sum of logits generated by the two branches, and adaptively balances the contributions of global semantic information and local boundary enhancement information.

[0026] Furthermore, the preprocessing of the present invention includes: uniformly setting the speech sampling rate to 16kHz, performing peak normalization processing on the speech waveform, uniformly adjusting the speech length to 2 seconds, randomly pruning speech longer than 2 seconds, and zero-padding for speech shorter than 2 seconds, and then extracting the MFCC acoustic characterization.

[0027] The beneficial effects of this invention are: it effectively alleviates the context discontinuity and boundary representation distortion at the dynamic window boundary in speech emotion recognition, and realizes the collaborative representation of global semantics, local salient emotional cues and boundary transition features. Thus, while maintaining a lightweight architecture with low inference overhead, it makes up for the shortcomings of traditional local window methods in cross-window dependency modeling, and can capture the speech emotion change process more comprehensively, effectively enhancing the discrimination ability of emotional transition regions. Attached Figure Description

[0028] Figure 1 This is a flowchart of a speech emotion recognition model based on multi-scale features and dynamic window boundary repair provided by one or more embodiments of the present invention;

[0029] Figure 2 This is a flowchart of multi-scale time-frequency feature extraction provided by one or more embodiments of the present invention;

[0030] Figure 3 This is a dynamic window frame diagram based on boundary repair provided in one or more embodiments of the present invention;

[0031] Figure 4 This is a dynamic window segmentation diagram provided in one or more embodiments of the present invention;

[0032] Figure 5This is a confusion matrix diagram constructed using the IEMOCAP dataset, provided by one or more embodiments of the present invention. Detailed Implementation

[0033] This invention provides a speech emotion recognition method based on multi-scale features and dynamic window boundary repair. It can be applied in a network environment where terminals and servers interact collaboratively, or it can run independently on a single device with computing capabilities. In a typical network interaction scenario, the terminal establishes a connection with the server through a communication network. The relevant data storage system can be deployed independently, integrated on the server, or deployed in the cloud, to store feature data such as audio sequences, Mel-frequency cepstral coefficients (MFCCs), and model weights. The terminal can send the collected target audio data, or the acoustic features initially extracted locally, to the server. After receiving the data, the server calls a pre-trained speech emotion recognition model (e.g., a neural network including a boundary latent diffusion repair module) to predict the emotion type of the target audio and feeds back the identified emotion label to the terminal.

[0034] Furthermore, in some embodiments, this multi-scale speech emotion recognition method can also be implemented independently on a local terminal or a cloud server. For example, if the terminal's computing power allows, the terminal can directly perform emotion recognition of the target audio locally, or the backend server can directly read audio data in batches from the database and perform offline emotion type recognition based on the multi-scale features of the audio.

[0035] like Figure 1 As shown, it is a flowchart of the overall model of the method described in this invention; Figure 2 A detailed flowchart of multi-scale time-frequency feature extraction is presented; Figure 3 A framework diagram illustrating dynamic window segmentation and boundary repair is provided. Figure 4 A more detailed dynamic window split diagram is displayed; Figure 5 This illustrates the confusion matrix constructed using the IEMOCAP dataset. The following five figures will be used to explain the specific implementation steps of this embodiment in detail.

[0036] A speech emotion recognition method based on multi-scale features and dynamic window boundary repair, the training steps are as follows: Figure 1 As shown:

[0037] Step S1: Preprocess the input speech signal and extract the Mel frequency cepstral coefficient (MFCC) features. Then, map the features to a unified representation space through linear projection and extract shared features through a multi-scale front-end, deep convolution, and macroscopic temporal modeling.

[0038] Step S2: Input the shared acoustic features into the global context modeling branch to extract sentence-level emotional semantics.

[0039] Step S3: Input the shared acoustic features into the dynamic window modeling branch to obtain local-global collaborative features.

[0040] Step S4: Construct a boundary mask based on the start and end positions of the dynamic window, perform latent space diffusion repair on the boundary tokens indicated by the boundary mask, and leave the non-boundary tokens unchanged as context.

[0041] Step S5: Gated fusion of the prediction results from the global context modeling branch and the dynamic window boundary repair branch.

[0042] Step S6: Obtain the final speech emotion classification result based on S5.

[0043] First, in step S1, the input speech signal is acquired, the speech sampling rate is uniformly set to 16 kHz, peak normalization is performed on the speech waveform, and the speech length is uniformly adjusted to 2 seconds. For speech longer than 2 seconds, random pruning is used; for speech shorter than 2 seconds, zero-padding is used for completion. Then, MFCC acoustic representation is extracted. Since speech signals have complex local and long-range correlations in the time and frequency domains, this invention designs a multi-branch structure to capture time-frequency features at different scales.

[0044] Input features The initial speech feature map has a single-channel input dimension.

[0045] Short-term local feature extraction: The first branch uses a kernel size of The void ratio is Depth-separable convolution This is used to capture local time-frequency patterns and obtain short-time local features. .

[0046] Medium-scale feature extraction: The second branch uses a kernel size of void ratio Depth-separable convolution By increasing the convolution kernel and dilation rate in the temporal direction, the receptive field in the temporal dimension is increased, resulting in features with a medium time span. .

[0047] Long-term feature extraction: The third branch uses a kernel size of void ratio Depth-separable convolution This is used to capture long-range dynamic contextual information of speech, resulting in long-time scale features L.

[0048] Cross-band feature extraction: The fourth branch uses a kernel size of void ratio Depth-separable convolution This branch has a wider receptive field in the frequency direction, which is used to capture cross-band correlation features and obtain frequency characteristics. .

[0049] After obtaining the multi-scale features of the four branches mentioned above, they will be stitched together along the channel dimensions S, M, L, and F. To reduce dimensionality and promote information interaction between features of different scales, a [further steps are needed]. Convolutional processing fuses the concatenated features, which are then sequentially processed by batch normalization (BN) and the GELU activation function to obtain multi-scale fused features. .

[0050] The multi-scale fusion features The backbone network, composed of multiple convolutional layers and max pooling, performs feature downsampling and channel upscaling to obtain deep spatial features. ,in , High after pooling, The width after pooling.

[0051] To capture the macroscopic dependencies of speech features across time-frame sequences, this invention introduces a gated attention mechanism based on a bidirectional long short-term memory (BiLSTM) network: First, for Average pooling compression is performed along the channel dimension to obtain a two-dimensional feature map. :

[0052] ;

[0053] Will The width direction (time dimension) is used as the sequence length and input into the BiLSTM network to calculate the hidden layer state. :

[0054] ;

[0055] Subsequently, a macroscopic spatial weight matrix is ​​generated through linear projection and the Sigmoid activation function:

[0056] ;

[0057] The weight matrix is ​​broadcast along the channel dimension, and the original features are modulated element-wise:

[0058] ;

[0059] Among them, here This indicates element-wise multiplication. This operation gives the model the ability to focus on key emotional vocal segments over a long time span.

[0060] To further eliminate redundancy between local features, the modulated features It will enter the spatial-channel hybrid attention module.

[0061] Specifically, it is divided along the channel into The nth group. For any nth group. Features of each group Divide its channel into channel branches for input. and spatial branch input .

[0062] For channel branches, adaptive average pooling is used to obtain the global response of the channel and generate channel recalibration features:

[0063] ;

[0064] in and These are learnable affine parameters.

[0065] For spatial branches, group normalization (GroupNorm) is applied to generate spatial augmentation features:

[0066] ;

[0067] in and These are learnable affine parameters. Then, the outputs of the two branches are reassembled to obtain:

[0068] ;

[0069] In order to break down the information isolation caused by grouping, for all The output of each group undergoes a channel shuffle operation to obtain the fused and enhanced features:

[0070] .

[0071] In step S3, the shared acoustic features obtained in step S2 are... Extract sentence-level sentiment semantics from the input global context modeling branch.

[0072] In the global context modeling branch, it is first flattened into spatial token dimensions, and then global feature transformation is performed through a gated MLP to obtain... .

[0073] In step S4, the shared features obtained in S2 are input into the dynamic window modeling branch. The architecture of this module is as follows: Figure 3 and Figure 4 It consists of four parts: boundary-guided dynamic window partitioning, local attention within the window, global attention between windows, and local-global fusion with boundary potential spatial diffusion repair.

[0074] Specifically, dynamic window partitioning adaptively generates windows of variable length based on the stop score.

[0075] Local attention within a window captures fine-grained sentiment patterns within each window, global attention between windows models long-distance dependencies between different windows, and boundary latent space diffusion remedy selectively denoises and restores features from boundary markers.

[0076] First, the pause scores of the input sequence are evaluated to accurately identify speech or semantic boundary locations with high confidence. Based on the identified boundaries, the model adaptively divides the continuous feature sequence into multiple variable-length windows and constructs an assignment mapping matrix from feature labels to their respective windows.

[0077] After the initial segmentation is completed, a feature importance filtering mechanism is introduced.

[0078] Specifically, the feature label importance scores calculated by the previous network module are obtained, and the median of all importance scores is extracted as a dynamic threshold. Based on this median threshold, the feature labels are adaptively grouped into two types of windows with strong sentiment association and weak sentiment association in chronological order. Strong sentiment association windows contain feature labels with importance scores above the threshold, representing key segments with dense sentiment expression. Weak sentiment association windows contain feature labels with importance scores below the threshold, representing segments with weak or transitional sentiment expression. Dynamic window partitioning adaptively generates variable-length windows based on the halting score. Intra-window local attention captures fine-grained sentiment patterns within each window. Inter-window global attention models long-distance dependencies between different windows.

[0079] Based on the above label-to-window assignment relationship, a specific local attention mask is generated. Under the constraint of the mask, the importance score of each feature label is calculated using the attention weights obtained from the Transformer multi-head attention module.

[0080] ;

[0081] in, This represents the calculated feature label importance score. This represents the total number of attention heads in a multi-head attention mechanism. This represents the attention weight from the s-th attention head. The length of the input sequence is represented by the dimension of one side of the average attention matrix. This matrix is ​​then reduced in dimensionality by accumulating along the sequence length dimension. This represents the normalized exponential function, used to map the accumulated weight values ​​to... The interval is transformed into a probability distribution form.

[0082] Intra-window local attention forces each token to primarily interact with other tokens within the same dynamic window through local masking. This not only effectively suppresses invalid attentional connections across independent semantic boundaries, significantly reducing global computational costs, but also accurately captures fine-grained sentiment patterns within each independent window.

[0083] Global attention between windows, after completing the fusion of local features, further performs global attention modeling between windows, establishes long-distance contextual dependencies across different variable-length windows, and thus grasps the macro-emotional direction of the entire speech.

[0084] Specifically, local features are first weighted and aggregated based on local attention scores to form a window-level representation. Then, global window attention is applied to obtain the window-level context and window attention score.

[0085] Boundary latent space diffusion repair selectively denoises and restores features from boundary markers.

[0086] In the boundary space diffusion repair stage, to address the problem of boundary context truncation or information damage that may be caused by dynamic hard segmentation operations, this invention introduces a boundary space diffusion repair module after attention interaction is completed.

[0087] By leveraging the generative capabilities of the diffusion model, denoising and feature recovery are selectively performed on feature labels located at the window division boundaries.

[0088] This invention effectively compensates for the loss of boundary information, ensuring the coherence and complete expression of emotional features between adjacent semantic segments.

[0089] The core idea of ​​this invention is to perform targeted repairs only on the features of the boundary region, while retaining the features of other regions as a reference.

[0090] In practice, boundary feature markers are located based on pre-generated boundary masks. Conditional diffusion repair is performed only on the indicated boundary feature markers, while feature markers in non-boundary areas are left untouched and used as contextual references during the repair process.

[0091] During the training phase, the repair module of this invention needs to perform a forward diffusion process, that is, gradually adding Gaussian noise only to the boundary feature markers.

[0092] In step S5, a gating fusion mechanism is used to adaptively fuse the two types of prediction results from the global context modeling branch and the dynamic window boundary repair branch.

[0093] Specifically, fusion weights are generated through learnable parameters, and a weighted sum of the logits generated by the two branches is calculated to adaptively balance the contributions of global semantic information and local boundary enhancement information.

[0094] In step S5, the final speech emotion classification result is output based on the result obtained from the fusion in S4.

[0095] To further investigate the classification details, this embodiment plots a confusion matrix on the IEMOCAP dataset, as shown below. Figure 5 As shown, the confusion matrix is ​​used to analyze the specific classification performance of different emotion categories. Rows represent the true classification categories, columns represent the categories predicted by the model, and the values ​​on the diagonal represent the number of correctly classified samples. Analysis of the confusion matrix shows that the model has a full distribution of diagonal values ​​across all categories. Especially in distinguishing easily confused similar emotions (such as sadness and neutral emotions), the misclassification rate is low, and the model can capture emotional features relatively accurately.

[0096] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that adjustments or equivalent substitutions can still be made to the technical solutions of the present invention (such as network layer architecture, feature extraction methods, etc.). All modifications that do not depart from the spirit and scope of the technical solutions of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A speech emotion recognition method with dynamic window boundary repair and multi-scale feature extraction, characterized in that, Includes the following steps: S1: Preprocess the input speech signal, extract the Mel frequency cepstral coefficients (MFCC) features, map the MFCC features to a unified representation space through linear projection, and then extract shared acoustic features through multi-scale front-end, deep convolution and macroscopic temporal modeling. S2: Input the shared acoustic features into the global context modeling branch to extract sentence-level emotional semantics; S3: Input the shared acoustic features into the dynamic window modeling branch to obtain local-global collaborative features; S4: Construct a boundary mask based on the start and end positions of the dynamic window, perform latent space diffusion repair on the boundary tokens indicated by the boundary mask, and keep the non-boundary tokens unchanged as context. S5: Gated fusion of the prediction results of the global context modeling branch and the dynamic window boundary repair branch; S6: Output the final speech emotion classification result based on the fusion result obtained in S5.

2. The method according to claim 1, characterized in that, In step S1, the multi-scale front end extracts multi-scale features through four parallel depthwise separable convolutional branches. The four branches are used to capture short-term local time-frequency patterns, medium-term temporal changes, long-term sentiment trends, and discriminative cues along the frequency dimension, respectively. The outputs of the four branches are concatenated along the channel dimension and fused through convolution. Then, high-level local time-frequency representations are extracted through multiple two-dimensional convolutional layers and max pooling operations. Finally, a spatial channel attention module is used to enhance significant time regions, frequency bands, and discriminative channels to generate shared feature representations.

3. The method according to claim 1, characterized in that, In step S2, the global context modeling branch flattens the shared features into spatial token dimensions, and then performs global feature transformation through a gated MLP.

4. The method according to claim 1, characterized in that, In steps S3 and S4, the dynamic window modeling branch includes boundary-guided dynamic window partitioning, intra-window local attention, inter-window global attention, and boundary latent space diffusion repair. Among them, dynamic window partitioning adaptively generates windows of variable length based on the halting score; intra-window local attention captures fine-grained sentiment patterns within each window; inter-window global attention models long-distance dependencies between different windows; and boundary latent space diffusion repair selectively denoises and restores features from boundary markers.

5. The method according to claim 4, characterized in that, In the dynamic window partitioning stage, the model identifies high-confidence boundary positions based on pause scores and constructs a token-to-window assignment matrix accordingly. Based on this assignment matrix, a local attention mask is generated, allowing each token to primarily interact with other tokens within the same dynamic window. Then, the model introduces a feature importance filtering mechanism, obtaining the feature label importance scores calculated by the previous network module, extracting the median of all importance scores as a dynamic threshold, and adaptively grouping the feature labels into strong sentiment association windows and weak sentiment association windows in chronological order.

6. The method according to claim 5, characterized in that, The feature label importance score is calculated as follows: a local attention mask is generated based on the label-to-window assignment relationship. Under the mask constraint, the importance score of each feature label is calculated using the attention weights obtained from the Transformer multi-head attention module. The calculation formula is as follows: ； in, This represents the calculated feature label importance score. This represents the total number of attention heads in a multi-head attention mechanism. This represents the attention weight from the s-th attention head. This represents the length of the input sequence, i.e., the size of one side dimension of the average attention matrix. The model accumulates this matrix along the sequence length dimension to reduce its dimensionality. This represents the normalized exponential function, used to map the accumulated weight values ​​to... The interval is transformed into a probability distribution.

7. The method according to claim 4, characterized in that, In the local attention phase within a window, the model performs local multi-head self-attention within each dynamic window. In the global attention phase between windows, the model weights and aggregates local token features based on local attention scores to form a window-level representation, and then applies global window attention to obtain window-level context and window attention scores. Finally, the local token features are fused with the global context so that each token contains both local information within the window and global information between windows.

8. The method according to claim 1, characterized in that, In step S4, the latent space diffusion repair includes: during training, the model adds Gaussian noise to the dynamic window boundary tokens, and the forward diffusion process is described as follows: ； in, For noise scheduling parameters, Represents the potential features of the original boundary. Indicates the first Features after noise injection Gaussian noise is represented; subsequently, a conditional denoiser, conditioned on noisy boundary features, non-boundary context, boundary mask, stopping score, and time step, predicts the noise components in the boundary region and progressively recovers their latent representations; the repaired boundary tokens are merged back into the complete token sequence, while the non-boundary tokens retain their original representations; finally, the repaired token features are normalized and average pooled, and input into the classification head to obtain the predicted logit of the branch.

9. The method according to claim 1, characterized in that, In step S5, the gated fusion generates fusion weights through learnable parameters, calculates the weighted sum of logits generated by the two branches, and adaptively balances the contributions of global semantic information and local boundary enhancement information.

10. The method according to any one of claims 1 to 9, characterized in that, In step S1, the preprocessing includes: setting the speech sampling rate to a uniform 16kHz, performing peak normalization on the speech waveform, adjusting the speech length to a uniform 2 seconds, randomly pruning speech longer than 2 seconds, and zero-padding for speech shorter than 2 seconds, and then extracting the MFCC acoustic characterization.

Images