Speech context semantic extraction method based on deep learning

Abstract

The invention discloses a voice context semantic extraction method based on deep learning, and the method comprises the following steps: S1, collecting a voice signal, and carrying out the pre-emphasis, framing and windowing processing, and obtaining a frame-level voice signal; s2, extracting MFCC and differential features, and constructing an input feature tensor after normalization; s3, inputting the feature tensor into a modeling network formed by a plurality of layers of Transform encoders, and outputting a semantic embedding vector sequence; s4, fusing local pooling and global attention, and generating context enhanced semantic representation; s5, outputting an intention, a keyword, an emotion and a slot tag through a decoder; s6, constructing a joint loss function, and adopting an Adam optimizer to perform end-to-end training on model parameters; s7, deploying the model to an equipment end, executing S1 to S5 in real time, and writing the model into a semantic cache; and S8, outputting a structured semantic result through an interface. According to the method, a multilayer semantic modeling structure is constructed, and voice context understanding and structured semantic tag extraction are realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of speech processing, and particularly relates to a speech context semantic extraction method based on deep learning. BACKGROUND

[0002] With the development of human-computer voice interaction technology, speech recognition systems have been widely applied in voice assistants, customer service robots, vehicle navigation, intelligent terminal control and other fields. Traditional speech recognition technology mainly relies on the cooperation of acoustic models, language models and decoders to complete the conversion process from speech to text, among which the acoustic model often uses a combination of hidden Markov models (HMM) and deep neural networks (DNN), and the language model uses n-gram probability statistical methods or rule-based syntax modeling methods. However, such systems only achieve one-way mapping of speech to text, and cannot deeply understand the semantic structure in the speech, especially cannot capture the context-dependent information between cross-sentences, resulting in a significant decrease in accuracy when processing multi-turn dialogues, colloquial instructions and omitted expressions.

[0003] In recent years, with the rapid development of deep learning technology, end-to-end speech recognition models have gradually replaced traditional modular systems, using long short-term memory networks (LSTM), convolutional neural networks (CNN) or Transformer structures to directly model speech. Although significant improvements have been made in recognition accuracy, most models still focus on the text reconstruction of speech content, lacking the ability to recognize deeper semantic information such as semantic intent, keyword structure, and emotional attitude. Although existing research has introduced a speech understanding module based on intent recognition, it often uses text input as a prerequisite and relies on ASR transcribed text for language understanding, forming a "recognition-understanding" two-level structure. This method has the problems of decoupling, error accumulation, and inability to train uniformly, making it difficult to achieve context-consistent semantic extraction.

[0004] In addition, in existing systems, most speech processing methods lack the ability to model long-distance context semantics, and cannot effectively capture cross-frame semantic cues in speech conversations, especially in multi-turn dialogue and paragraph-level input scenarios. The model lacks understanding of language phenomena such as reference, topic continuation, and emotional turning. At the same time, although some models have semantic label prediction functions, their feature extraction and decoding processes do not form a unified end-to-end optimization process, and cannot control the overall intent recognition, slot filling and semantic consistency through a joint loss function, resulting in fragmented and unstable semantic label output, which seriously affects the overall semantic understanding performance of the system.

[0005] The prior art still has significant deficiencies in processing deep semantic structure, context relationship maintenance and multi-task structured semantic output of speech, and there is an urgent need for a deep speech understanding method supporting end-to-end optimization, having context modeling and semantic fusion capabilities, and being capable of realizing joint extraction of multiple types of semantic labels, so as to comprehensively improve the intelligent understanding capability and semantic output quality of a speech interaction system.

[0006] Therefore, how to provide a speech context semantic extraction method based on deep learning is a problem to be solved by those skilled in the art. SUMMARY

[0007] An object of the present application is to provide a speech context semantic extraction method based on deep learning. The present application constructs a context semantic modeling network based on Transformer, combines multi-scale semantic fusion and structured decoding mechanism, and realizes joint extraction of multiple types of semantic labels such as speech intent, keyword, emotion and slot. Through end-to-end optimization training, the semantic recognition accuracy and context consistency are improved, and the method is suitable for multi-round speech understanding and intelligent interaction scenarios.

[0008] According to the speech context semantic extraction method based on deep learning, the method comprises the following steps:

[0009] S1, collecting and preliminarily processing the original speech signal, the processing comprising collecting continuous speech stream by using a microphone array, suppressing environmental noise by using a band-pass filter, pre-emphasizing the speech data, and performing frame division and Hamming window operation according to a fixed window to obtain a pre-processed frame-level speech signal;

[0010] S2, performing acoustic feature extraction operation on the frame-level speech signal, calculating MFCC features for each frame of speech by using a mel-frequency cepstral coefficient extraction algorithm, simultaneously obtaining first-order difference and second-order difference features, performing mean normalization processing on all features, and constructing an input feature tensor in the form of frame sequence;

[0011] S3, inputting the input feature tensor into a context semantic modeling network, constructing a deep neural network structure composed of multiple layers of Transformer encoders, the encoders comprising a self-attention mechanism, a feedforward neural network and a residual connection structure, for modeling context semantic dependency relationship between frames, and outputting a semantic embedding vector sequence;

[0012] S4, performing multi-scale feature fusion on the semantic embedding vector sequence, extracting short-time semantic features by using local sliding window average pooling, simultaneously extracting long-distance dependency information across paragraphs by using global self-attention, fusing to form context-enhanced semantic representation, and uniformly mapping the semantic space through full connection;

[0013] S5, input the context enhanced semantic representation into a structured semantic decoder, the decoder comprising a bidirectional gated recurrent unit network, a fully connected layer and a Softmax output layer, predict speech semantic labels, the semantic labels comprising intent classification, keyword boundary recognition, sentiment orientation and slot filling information;

[0014] S6, construct a joint loss function based on the supervision signal, calculate the cross-entropy loss between the predicted results of the semantic labels and the manually labeled labels, construct a total loss function comprising intent recognition loss, slot filling loss and context consistency regularization term, perform end-to-end backpropagation training on the context semantic modeling network using the Adam optimizer, and update all model parameters;

[0015] S7, after the training is completed, deploy the converged deep learning model to a server environment or an edge device end, receive real-time speech input data, and repeatedly execute steps S1 to S5 to obtain structured semantic understanding results corresponding to each round of speech conversation, and write to an online semantic cache module;

[0016] S8, provide structured semantic result output to external systems through a semantic service interface, the interface supporting standard HTTP or WebSocket protocol, for voice question and answer systems, human-computer interaction systems or multi-round dialogue engines to call, realize context-based speech understanding and intelligent semantic interaction function.

[0017] Optionally, the S1 specifically comprises:

[0018] S11, synchronously collect natural speech signals through a microphone array, the microphone array being composed of a plurality of digital acoustic sensors arranged at equal intervals, having directional sound collection and multi-channel input capability, and being used to acquire original speech stream containing continuous semantic content;

[0019] S12, convert the collected original speech signals into a standard digital audio format, set the sampling rate and quantization bit depth, and form continuous audio stream data for subsequent processing modules to call;

[0020] S13, perform band pass filtering on the audio stream data, set the lower and upper limit frequencies of the filter, suppress high-frequency background noise and low-frequency device interference, and retain the main speech frequency band;

[0021] S14, perform a silence detection operation, identify and eliminate non-speech segments using a frame energy analysis method, and only retain valid speech segments to form an uninterrupted speech content sequence;

[0022] S15, pre-emphasize each valid speech segment to enhance high-frequency information, improve spectral feature distribution, and ensure stable frequency band characteristics;

[0023] S16, frame the speech segment using fixed frame length and frame shift to generate a sequence of frame-level speech signals that are continuous in time and overlap, each frame containing waveform data of multiple sampling points;

[0024] S17, apply a Hamming window function to each frame of speech signal to improve spectral smoothness and reduce spectral leakage effects caused by window edges;

[0025] S18, combine and output all windowed frame-level speech signals to form a sequence of preprocessed speech frames arranged in time sequence as input to the acoustic feature extraction module.

[0026] Optionally, the S2 specifically comprises:

[0027] S21, receive the output frame-level speech signal sequence, and input each frame of speech signal to the acoustic feature extraction module to perform frequency domain analysis processing to generate a short-time power spectrum corresponding to the frame;

[0028] S22, apply a Mel filter bank to the power spectrum of each frame to calculate the Mel frequency energy distribution result and construct a feature base for semantic modeling;

[0029] S23, perform logarithmic compression and discrete transform processing on the Mel frequency energy vector to extract the Mel frequency cepstrum coefficient feature corresponding to the frame, and construct a frame-level MFCC feature sequence;

[0030] S24, based on the MFCC feature sequence, calculate the inter-frame change rate between adjacent frames to generate a first-order difference feature sequence; and again calculate the change trend to obtain a second-order difference feature sequence;

[0031] S25, concatenate the static MFCC feature, the first-order difference feature and the second-order difference feature in the channel dimension to construct a three-channel frame-level acoustic feature tensor for representing the inter-frame semantic content and the time sequence change trend;

[0032] S26, perform mean normalization processing on the three-channel feature tensor to maintain uniform distribution of features in each channel within the entire speech segment, thereby improving modeling stability;

[0033] S27, form a feature sequence by arranging the normalized feature tensor in time frame order, and input the feature sequence as input data into the contextual semantic modeling network.

[0034] Optionally, the S3 specifically comprises:

[0035] S31, receive the input feature tensor and input it to the contextual semantic modeling network in time frame order, wherein the modeling network is composed of multiple layers of Transformer encoder stacks and has the ability to model cross-frame context;

[0036] S32, in each layer of the Transformer encoder, first perform multi-head self-attention mechanism on the input features, calculate the semantic dependency weight between frames in parallel through multiple attention heads, and generate a weighted combined frame-level context representation;

[0037] S33, input the context representation output by the self-attention module into a feedforward neural network sublayer, the feedforward network comprising two linear transformation structures for nonlinear feature mapping and dimension adjustment;

[0038] S34, introduce a residual connection mechanism between the output of each sublayer and the input, add the sublayer input and the transformed result, and enhance the gradient flow capacity in the deep network;

[0039] S35, perform layer normalization operation on the residual connection result of each sublayer to standardize the activation distribution of different channel features and improve the stability of model training;

[0040] S36, the output after layer normalization is input into the next layer of the Transformer encoder to sequentially complete the context modeling process of multiple stacked layers;

[0041] S37, obtain the context semantic embedding vector sequence of all frames in the output of the last layer of the Transformer encoder, and the embedding sequence is used to represent the semantic meaning of each frame in the whole context in the speech;

[0042] S38, output the semantic embedding vector sequence as the input of the multi-scale semantic fusion module and enter the subsequent step S4 for further semantic expression processing.

[0043] Optionally, the S4 specifically includes:

[0044] S41, input the semantic embedding vector sequence into a multi-scale feature fusion module, the module being composed of a local semantic extraction branch and a global semantic modeling branch, and being used for jointly modeling short-time and long-time semantic dependency relationships;

[0045] S42, in the local semantic extraction branch, a fixed-length sliding window is set, and time-ordered sliding average pooling is performed on the semantic embedding vector sequence to extract local context semantic features in each time period;

[0046] S43, in the global semantic modeling branch, a full-sequence range attention mechanism is used to calculate the long-distance dependency relationship between frames to extract global context semantic features covering the entire speech segment;

[0047] S44, concatenate the local semantic features and the global semantic features in the feature dimension to construct a multi-scale semantic feature fusion representation, and form a comprehensive representation vector with fine-grained and overall semantic information;

[0048] S45, input the fusion representation into the fully connected mapping network for dimension alignment and nonlinear transformation, unify the local and global information into the same semantic space, and obtain a context-enhanced semantic representation;

[0049] S46, perform a channel weighting mechanism on the context-enhanced semantic representation, dynamically adjust the weight coefficients according to the importance of different semantic channels, and realize selective enhancement of multi-scale features;

[0050] S47, perform layer normalization operation on the weighted and fused semantic representation, and output a context-enhanced semantic representation with stable structure and consistent scale, which is used for subsequent structured semantic label prediction and decoding processing.

[0051] Optionally, the S5 specifically comprises:

[0052] S51, input the context-enhanced semantic representation into a structured semantic decoder, the decoder being composed of a bidirectional gated recurrent unit network, a fully connected mapping layer and a multi-task output layer;

[0053] S52, in the bidirectional gated recurrent unit network, respectively pass the state of the semantic representation sequence along the time in the forward and reverse order to generate the forward and reverse context state sequences, and splice the two in the time dimension to form a joint context state representation;

[0054] S53, input the joint context state representation into the fully connected mapping layer to perform nonlinear activation transformation and dimension compression operation, and extract an intermediate semantic vector suitable for semantic classification tasks;

[0055] S54, input the intermediate semantic vector into the multi-task output layer, the output layer being divided into multiple decoding branches according to the task categories, respectively corresponding to an intent classification branch, a keyword boundary recognition branch, a sentiment orientation recognition branch and a slot filling branch;

[0056] S55, in the intent classification branch, use classification logic to perform category mapping on the global vector of the entire speech segment, and output the corresponding intent label;

[0057] S56, in the keyword boundary recognition branch, time sequence labeling is performed on each frame of intermediate semantic vector, and frame-level keyword start and end boundary label is outputted;

[0058] S57, in the sentiment orientation recognition branch, an emotion feature vector of the entire speech segment is extracted, a multi-class sentiment classification operation is performed, and a sentiment category label is outputted;

[0059] S58, in the slot filling branch, combined with the inter-frame context feature, slot structure labeling is performed on each frame, and a semantic filling label sequence is outputted;

[0060] S59: Reorganize the semantic tags output by the above branches in chronological order to generate a structured semantic tag set as the final output result of speech semantic understanding.

[0061] Optionally, the S6 specifically includes:

[0062] S61: Construct a supervisory signal set, wherein the supervisory signal set includes manually annotated labels corresponding to the speech data, and the labels include intent recognition labels, slot filling labels, and contextual semantic relationship labels;

[0063] S62, aligning the semantic label prediction results output by the structured semantic decoder with the manually annotated labels one by one, and constructing a cross entropy loss function for the classification task based on the label comparison results;

[0064] S63. Output the corresponding supervision label to construct the intent recognition loss function, use the slot filling output supervision label to construct the slot filling loss function, and construct the context consistency regularization term based on the context label continuity;

[0065] S64. Intent recognition loss, slot filling loss, and context consistency regularization term are weighted and combined according to the set ratio to construct a joint loss function, which is used to uniformly measure the overall deviation of the model on each semantic task.

[0066] S65. Select the Adam optimizer as the optimization algorithm for the modeling network, initialize the optimizer parameters, and set the learning rate, regularization factor, and gradient update strategy;

[0067] S66. In each training batch, input the input feature tensor generated in step S2 and the corresponding supervision signal, and perform end-to-end forward propagation and joint loss calculation;

[0068] S67. Use the backpropagation method based on the Adam optimizer to perform gradient backpropagation and parameter update on the contextual semantic modeling network to optimize all parameters in the Transformer encoder and structured semantic decoder;

[0069] S68. Repeat steps S66 and S67 until all training rounds are completed or the joint loss function meets the preset convergence conditions, and output the model parameter set after training convergence as the final deep semantic extraction model.

[0070] Optionally, the S7 specifically includes:

[0071] S71. Export the deep learning model after training convergence to a deployable model file, and load the model file in a deployment environment, which includes a server system or an edge computing device;

[0072] S72, configure a voice input receiving module, the module is used for receiving user real-time voice stream input, and sending voice data into the processing flow constituted by steps S1 to S5, and performing voice signal collection, feature extraction, context modeling, semantic fusion and decoding operation;

[0073] S73, at the beginning of each round of voice conversation, performing real-time preprocessing and feature conversion on the received voice stream, and dynamically generating corresponding frame-level voice signal and input feature tensor;

[0074] S74, inputting the generated input feature tensor into the deployed deep semantic modeling model, performing forward inference operation, and outputting a set of structured semantic labels corresponding to the current voice data;

[0075] S75, constructing a semantic cache module, the module performs cache management on the structured semantic labels in each round of voice interaction, and archives and stores according to time sequence or conversation identifier;

[0076] S76, performing identification update and index operation on the label set stored in the semantic cache module, and generating a semantic understanding result document or data object that can be called;

[0077] S77, after the voice conversation ends, keeping the data in the semantic cache accessible, supporting context association reference in voice interaction, and ensuring that the semantic processing has historical dependence ability;

[0078] S78, binding the structured semantic understanding result of each round of voice conversation with meta information as a data source output by local cache data or service interface, and providing stable semantic input for external semantic service module.

[0079] Optionally, the S8 specifically comprises:

[0080] S81, constructing a semantic service interface module, the interface module includes a standard HTTP protocol interface and a WebSocket communication interface, and is used for providing real-time access ability of semantic understanding result to external system;

[0081] S82, extracting a set of structured semantic labels of the current round from the semantic cache module generated in step S7, the label set includes voice intent label, keyword boundary label, sentiment tendency label and slot filling label;

[0082] S83, serializing and packaging the semantic label set into a standard data response structure according to a predefined semantic data format, the structure supports JSON, XML or custom binary format;

[0083] S84, listen to the request event of the external system, when detecting the service call request from the voice question and answer system, the man-machine interaction system or the multi-round dialogue engine, analyze the request content and establish a communication channel;

[0084] S85, return the structured semantic result corresponding to the current voice conversation to the calling system through the semantic service interface, ensure the integrity, time sequence consistency and multi-task label cooperation of the semantic information;

[0085] S86, support the interface asynchronous push mode, when the voice input is still in the receiving process, the semantic output content can be updated in real time based on the phased analysis result, and the interaction logic such as conversation interruption, completion, redirection is realized;

[0086] S87, record the request parameters, response content and calling time information of each semantic service delivery process, build an interface call log system, and use it for subsequent task scheduling, behavior auditing and system performance evaluation;

[0087] S88, after the semantic service output is completed, the conversation context state is closed and the cache resource is released, and an independent running environment is provided for the structured semantic understanding task of the next voice conversation round.

[0088] The beneficial effects of the present application are:

[0089] The voice context semantic extraction method based on deep learning provided by the present application can significantly improve the understanding ability of the voice interaction system to multi-level semantic information, overcome the limitations of the prior art in context perception modeling, semantic structure extraction and joint training optimization, and has good intelligent interaction adaptability and actual deployment effect.

[0090] Firstly, the present application constructs a context semantic modeling network based on a multi-layer Transformer encoder, which significantly enhances the modeling ability of the model to long-term dependencies between voice segments. Compared with the traditional frame-independent processing or local window mechanism, the present application can capture the deep semantic logic across sentences and paragraphs in the voice conversation, realize the structured semantic recognition of complex language phenomena such as omission, reference, tone change, etc., and provide accurate context semantic basis for subsequent multi-round voice understanding.

[0091] Secondly, the present application introduces a multi-scale semantic fusion mechanism, which combines local sliding window pooling and global attention mechanism in parallel, and through feature channel weighted fusion, it takes into account short-time semantic details and global context expression, effectively improves the integrity and discriminability of the final semantic representation, and makes the task output results such as intent recognition, keyword positioning, emotion perception and slot filling more accurate and stable.

[0092] In addition, the application adopts a structured semantic decoder to decode the context-enhanced semantic representation in a multi-task manner, introduces a supervision signal to construct a joint loss function, integrates an intent recognition loss, a slot filling loss and a context consistency regular term, combines an Adam optimizer to perform end-to-end training, ensures that the model can cooperatively optimize multiple semantic tasks in the overall architecture, avoids interference between tasks, reduces error accumulation, and significantly improves the robustness and generalization ability of overall semantic understanding.

[0093] Finally, the application supports deploying the trained model on a server or an edge device end, can efficiently respond to voice input requests in actual application, and provides structured semantic results in a standard communication protocol through a semantic service interface, adapts to a voice question and answer system, a man-machine interaction system or a multi-turn dialogue engine, has good engineering deployment and system compatibility, and meets the comprehensive requirements of real-time performance, accuracy and semantic integrity in an intelligent voice interaction scene. BRIEF DESCRIPTION OF DRAWINGS

[0094] The accompanying drawings are included to provide a further understanding of the application and constitute a part of the specification, illustrate the application, and are used together with the application examples to explain the application, and do not constitute a limitation on the application. In the drawings:

[0095] Fig. 1 A whole flowchart of a voice context semantic extraction method based on deep learning is provided for the application;

[0096] Fig. 2 A structural schematic diagram of a voice preprocessing and acoustic feature extraction module is provided for the application;

[0097] Fig. 3 A loss function construction and optimization mechanism diagram of an end-to-end training flow is provided for the application. DETAILED DESCRIPTION

[0098] The application will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, and only illustrate the basic structure of the application in a schematic manner, and therefore only show the components related to the application.

[0099] REFERENCE Figs. 1-3 A voice context semantic extraction method based on deep learning includes the following steps:

[0100] S1, collect and preliminarily process the original voice signal, the processing includes collecting continuous voice stream by using a microphone array, suppressing environmental noise by using a band-pass filter, pre-emphasizing the voice data, and performing frame division and Hamming window operation according to a fixed window to obtain a preprocessed frame-level voice signal;

[0101] S2. Perform acoustic feature extraction on the frame-level speech signal. Calculate the MFCC features for each frame of speech using the Mel-frequency cepstral coefficient extraction algorithm. Obtain first-order and second-order difference features simultaneously. Perform mean normalization on all features to construct an input feature tensor in the form of a frame sequence.

[0102] S3. Input the input feature tensor into the contextual semantic modeling network to build a deep neural network structure composed of a multi-layer Transformer encoder. The encoder includes a self-attention mechanism, a feedforward neural network, and a residual connection structure to model the contextual semantic dependencies between frames and output a sequence of semantic embedding vectors.

[0103] S4. Perform multi-scale feature fusion on the semantic embedding vector sequence, using local sliding window average pooling to extract short-term semantic features, while using global self-attention to extract long-range dependency information across paragraphs, fusion to form context-enhanced semantic representation, and unify the semantic space through fully connected mapping;

[0104] S5. Input the context-enhanced semantic representation into a structured semantic decoder, which includes a bidirectional gated recurrent unit network, a fully connected layer, and a softmax output layer to predict speech semantic labels. The semantic labels include intent classification, keyword boundary identification, sentiment tendency, and slot filling information.

[0105] S6. Construct a joint loss function based on the supervisory signal, calculate the cross entropy loss for the predicted results of the semantic labels and the manually annotated labels, construct a total loss function that includes intent recognition loss, slot filling loss, and context consistency regularization term, use the Adam optimizer to perform end-to-end backpropagation training on the contextual semantic modeling network, and update all model parameters;

[0106] S7. After training is completed, the converged deep learning model is deployed to a server environment or edge device, which receives voice input data in real time. Steps S1 to S5 are repeated to obtain the structured semantic understanding results corresponding to each round of voice conversation and write them into the online semantic cache module.

[0107] S8. Provide structured semantic result output to external systems through a semantic service interface. The interface supports standard HTTP or WebSocket protocols and can be called by voice question-answering systems, human-computer interaction systems, or multi-round dialogue engines to achieve context-based speech understanding and intelligent semantic interaction functions.

[0108] The application can effectively realize the structured extraction of multi-dimensional semantic information such as intention, emotion, keyword and slot in the speech stream by constructing a speech context semantic extraction method based on deep learning, covering the whole process from speech acquisition preprocessing, acoustic feature extraction, context modeling, multi-scale semantic fusion, structured semantic decoding, end-to-end optimization training, to model deployment and service interface output. Compared with traditional speech recognition or speech transcription methods, the method not only can understand the speech content, but also can identify the multi-round semantic relationship and context dependence hidden in the speech, greatly improving the intelligent level, response accuracy and scene adaptability of the speech interaction system, and having stronger engineering practicability and platform compatibility.

[0109] In the embodiment, S1 specifically includes:

[0110] S11, synchronously collecting natural speech signals through a microphone array, the microphone array being composed of a plurality of digital acoustic sensors arranged at equal intervals and having directional sound collection and multi-channel input capabilities, and being used for acquiring original speech streams containing continuous semantic content;

[0111] S12, converting the collected original speech signals into a standard digital audio format, setting a sampling rate and a quantization bit depth, and forming continuous audio stream data for calling by subsequent processing modules;

[0112] S13, performing band-pass filtering processing on the audio stream data, setting a lower limit frequency and an upper limit frequency, suppressing high-frequency background noise and low-frequency device interference, and retaining main speech frequency bands;

[0113] S14, performing a silence detection operation, identifying and eliminating non-speech segments by using a frame energy analysis method, retaining only valid speech segments, and constituting an uninterrupted speech content sequence;

[0114] S15, performing pre-emphasis processing on each valid speech segment, enhancing high-frequency information, improving spectral feature distribution, and ensuring that the frequency band characteristics remain stable;

[0115] S16, performing frame processing on the speech segments by using a fixed frame length and a frame shift, generating a sequence of frame-level speech signals that are continuous and overlapped in time, each frame containing waveform data of a plurality of sampling points;

[0116] S17, applying a Hamming window function to each frame of speech signals to improve spectral smoothness and reduce spectral leakage effects caused by window edges;

[0117] S18, combining and outputting all the windowed frame-level speech signals to form a preprocessed speech frame sequence arranged in time sequence as the input of an acoustic feature extraction module.

[0118] By adopting a multi-stage speech preprocessing technology combination, including microphone array acquisition, band-pass filtering, silence detection, pre-emphasis, framing and windowing operations, the quality of the speech data can be significantly improved, and the density of useful information in the original signal is enhanced. This step not only eliminates environmental noise and invalid speech segment interference, but also enhances high-frequency speech features, providing more stable and representative frame-level speech data input for subsequent acoustic modeling. Compared with the traditional preprocessing process, the method has higher optimization capability in the synchronization of multi-channel input, the accuracy of valid speech segment identification and spectral fidelity, ensuring that the speech signal has a good time-frequency structure basis before entering the deep model.

[0119] In the embodiment, S2 specifically comprises:

[0120] S21, receiving the output frame-level speech signal sequence, inputting each frame of speech signal to an acoustic feature extraction module, performing frequency domain analysis processing, and generating a short-time power spectrum corresponding to the frame;

[0121] S22, applying a Mel filter bank on the power spectrum of each frame, calculating the Mel frequency energy distribution result, and constructing a feature base for semantic modeling;

[0122] S23, performing logarithmic compression and discrete transform processing on the Mel frequency energy vector, extracting the Mel frequency cepstrum coefficient feature corresponding to the frame, and constructing a frame-level MFCC feature sequence;

[0123] S24, based on the MFCC feature sequence, calculating the inter-frame change rate between adjacent frames to generate a first-order difference feature sequence; and again calculating the change trend to obtain a second-order difference feature sequence;

[0124] S25, concatenating the static MFCC feature, the first-order difference feature and the second-order difference feature in the channel dimension to construct a three-channel frame-level acoustic feature tensor for representing the inter-frame semantic content and the time sequence change trend;

[0125] S26, performing mean normalization processing on the three-channel feature tensor to make the features of each channel maintain uniform distribution within the entire speech segment range, and improving the modeling stability;

[0126] S27, grouping the normalized feature tensor into a feature sequence in time frame order, and sending it into a context semantic modeling network as input data.

[0127] The step is to construct a three-dimensional frame-level acoustic feature tensor by MFCC extraction and multi-channel fusion of its first and second order differences, and to comprehensively retain the static and dynamic timing characteristics of the speech signal by combining normalization processing. Compared with the traditional method of using only static acoustic features, this scheme can more accurately capture the change trend and instantaneous semantic signal of the speech, and improve the robustness of the model to factors such as speaking style, speech rate change and pitch drift. The normalization operation further eliminates the feature scale difference between frames, improves the model convergence speed and training stability, and provides consistent input for subsequent context modeling.

[0128] In the embodiment, the S3 specifically includes:

[0129] S31, receiving an input feature tensor, which is input to a context semantic modeling network in time frame order, the modeling network is composed of a multi-layer Transformer encoder stack and has a cross-frame context modeling capability;

[0130] S32, in each layer of the Transformer encoder, first perform multi-head self-attention mechanism on the input features, calculate the semantic dependency weight between frames through multiple attention heads in parallel, and generate the weighted combined frame-level context representation;

[0131] S33, input the context representation output by the self-attention module to a feedforward neural network sublayer, the feedforward network includes two linear transformation structures for nonlinear feature mapping and dimension adjustment;

[0132] S34, introduce a residual connection mechanism between the output and the input of each sublayer, add the sublayer input and the transformed result to enhance the gradient flow capacity in the deep network;

[0133] S35, perform layer normalization operation on each sublayer residual connection result to regulate the activation distribution of different channel features and improve the model training stability;

[0134] S36, the output after layer normalization is the input of the next layer of Transformer encoder, and the context modeling process of multi-layer stack is completed in turn;

[0135] S37, obtain the context semantic embedding vector sequence of all frames in the output of the last layer of the Transformer encoder, the embedding sequence is used to represent the semantic meaning of each frame in the whole context in the speech;

[0136] S38, output the semantic embedding vector sequence as the input of the multi-scale semantic fusion module and enter the subsequent step S4 for further semantic expression processing.

[0137] The application constructs a context semantic modeling network with a multi-layer Transformer encoder as the core, effectively solving the problem of insufficient long-term dependency modeling capability of existing speech models. By introducing a multi-head self-attention mechanism, residual connection and layer normalization operation, the network can automatically focus on the semantic dependency path between different time frames and stably transmit deep semantic features. This structure has high scalability and parallel computing capability, can simultaneously process long text speech and complex semantic relationships, and realizes embedding modeling of frame-level semantic vectors in the overall context, providing a solid foundation for accurately understanding user intent.

[0138] In the embodiment, the S4 specifically includes:

[0139] S41, input the semantic embedding vector sequence to a multi-scale feature fusion module, which is composed of a local semantic extraction branch and a global semantic modeling branch, for joint modeling of short-time and long-time semantic dependency relationships;

[0140] S42, in the local semantic extraction branch, a fixed-length sliding window is set, and sliding average pooling is performed on the semantic embedding vector sequence in time sequence to extract local context semantic features in each time period;

[0141] S43, in the global semantic modeling branch, a long-distance dependency relationship between frames is calculated using an attention mechanism with a full sequence range, and global context semantic features covering the entire speech segment are extracted;

[0142] S44, the local semantic features and the global semantic features are spliced in the feature dimension to construct a multi-scale semantic feature fusion representation, forming a comprehensive representation vector with fine-grained and overall semantic information;

[0143] S45, input the fusion representation to a fully connected mapping network for dimension alignment and nonlinear transformation, and uniformly map the local and global information to the same semantic space to obtain a context-enhanced semantic representation;

[0144] S46, perform a channel weighting mechanism on the context-enhanced semantic representation, dynamically adjust the weight coefficients according to the importance of different semantic channels, and realize selective enhancement of multi-scale features;

[0145] S47, perform layer normalization operation on the weighted and fused semantic representation, output a context-enhanced semantic representation with stable structure and consistent scale, and use it for subsequent structured semantic label prediction and decoding processing.

[0146] By introducing a parallel structure of local sliding window pooling and global self-attention mechanism, multi-scale feature information is fused, so that the model has sharp semantic extraction ability at both phrase level and paragraph level. This module can finely capture the keyword changes in a short time, and also identify the theme continuation and emotion changes in the whole speech paragraph, enhancing the consistency and integrity of semantic representation. Channel weighting and fully connected mapping operations ensure the fusion of features at different scales in a unified semantic space, providing high-precision and medium-granularity input basis for semantic decoding.

[0147] In the embodiment, the S5 specifically includes:

[0148] S51, input the context-enhanced semantic representation to a structured semantic decoder, the decoder being composed of a bidirectional gated recurrent unit network, a fully connected mapping layer and a multi-task output layer;

[0149] S52, in the bidirectional gated recurrent unit network, respectively along the time positive order and inverse order, the state transmission of the semantic representation sequence is performed, the forward and backward context state sequences are generated, and the two are spliced in the time dimension to form a joint context state representation;

[0150] S53, input the joint context state representation to the fully connected mapping layer, perform nonlinear activation transformation and dimension compression operation, and extract the intermediate semantic vector suitable for semantic classification task;

[0151] S54, input the intermediate semantic vector to the multi-task output layer, the output layer is divided into multiple decoding branches according to the task category, respectively corresponding to the intent classification branch, the keyword boundary recognition branch, the sentiment orientation recognition branch and the slot filling branch;

[0152] S55, in the intent classification branch, the classification logic is used to perform category mapping on the global vector of the whole speech segment, and the corresponding intent label is output;

[0153] S56, in the keyword boundary recognition branch, the time sequence labeling is performed on each frame intermediate semantic vector, and the frame-level keyword start and end boundary label is output;

[0154] S57, in the sentiment orientation recognition branch, the emotion feature vector of the whole speech segment is extracted, the multi-class sentiment classification operation is performed, and the sentiment category label is output;

[0155] S58, in the slot filling branch, combined with the inter-frame context feature, the slot structure labeling is performed on each frame, and the semantic filling label sequence is output;

[0156] S59, reorganize the semantic labels output by the above branches in time sequence to generate a structured semantic label set as the final output result of speech semantic understanding.

[0157] The application designs a structured semantic decoder and sets a multi-task output structure to support simultaneous completion of multiple semantic tasks such as intent classification, keyword boundary detection, emotion recognition and slot filling in a single model architecture. The structure avoids the semantic fragmentation problem caused by independent decoding of multiple modules in traditional methods, realizes unified prediction and synchronous optimization of semantic labels, and improves the prediction accuracy and the integrity of the output structure. The bidirectional gated recurrent network has a good memory mechanism in time series modeling, further enhancing the decoder's understanding of the depth of context information.

[0158] In the embodiment, the S6 specifically includes:

[0159] S61, a supervised signal set is constructed, the supervised signal set including artificial labeling labels corresponding to voice data, the labels including intent recognition labels, slot filling labels and context semantic relationship labels;

[0160] S62, the semantic label prediction results output by the structured semantic decoder are aligned with the artificial labeling labels item by item, and a cross-entropy loss function for a classification task is constructed based on the label comparison results;

[0161] S63, an intent recognition loss function is constructed by outputting corresponding supervised labels, a slot filling loss function is constructed by outputting supervised labels for slot filling, and a context consistency regularization term is constructed based on the continuity of context labels;

[0162] S64, the intent recognition loss, the slot filling loss and the context consistency regularization term are weighted and combined according to a set proportion to construct a joint loss function for uniformly measuring the overall deviation of the model on each semantic task;

[0163] S65, an Adam optimizer is selected as the optimization algorithm of the modeling network, the optimizer parameters are initialized, and the learning rate, the regularization factor and the gradient update strategy are set;

[0164] S66, in each training batch, the input feature tensor generated in step S2 and the corresponding supervised signal are input, and end-to-end forward propagation and joint loss calculation are performed;

[0165] S67, a gradient backpropagation method based on the Adam optimizer is used to perform gradient backpropagation and parameter update on the context semantic modeling network, and all parameters in the Transformer encoder and the structured semantic decoder are optimized;

[0166] S68, steps S66 and S67 are repeatedly executed until all training rounds are completed or the joint loss function meets the preset convergence condition, and the model parameter set after training convergence is output as the final deep semantic extraction model.

[0167] By constructing multiple supervised loss functions and integrating them into a joint loss function system, and introducing a context consistency regularization term, the collaborative optimization among multiple tasks can be realized. The Adam optimizer is used for back propagation training, which not only speeds up the network convergence speed, but also dynamically adjusts the parameter update rate, improves the overall model training efficiency and result stability. Compared with the traditional single task optimization strategy, the scheme significantly improves the consistency, accuracy and scalability of the semantic label output, and is a key technology link to improve the multi-dimensional semantic understanding ability in complex speech scenarios.

[0168] In the embodiment, the S7 specifically includes:

[0169] S71, export the deep learning model after training convergence to a deployable model file, and load the model file in a deployment environment including a server-side system or an edge computing device;

[0170] S72, configure a voice input receiving module, which is used to receive real-time voice stream input of a user and send voice data to the processing flow constituted by steps S1 to S5 to perform voice signal collection, feature extraction, context modeling, semantic fusion and decoding operations;

[0171] S73, at the beginning of each round of voice conversation, perform real-time preprocessing and feature conversion on the received voice stream, and dynamically generate corresponding frame-level voice signal and input feature tensor;

[0172] S74, input the generated input feature tensor to the deployed deep semantic modeling model to perform forward inference operation, and output a structured semantic label set corresponding to the current voice data;

[0173] S75, construct a semantic cache module, which performs cache management on the structured semantic labels in each round of voice interaction, and archives and stores them in time sequence or conversation identifier;

[0174] S76, perform identification update and indexing operation on the label set stored in the semantic cache module to generate a semantic understanding result document or data object that can be called;

[0175] S77, after the voice conversation ends, keep the data in the semantic cache accessible, support context association reference in voice interaction, and ensure that the semantic processing has historical dependence ability;

[0176] S78, bind the structured semantic understanding result of each round of voice conversation with meta information as a data source output by local cache data or service interface, and provide stable semantic input for external semantic service modules.

[0177] The trained deep semantic model is deployed to a server or an edge device end, and combined with a semantic cache module to realize structured storage and index management of semantic results, so that efficient online semantic prediction services can be realized. The deployment architecture supports low-delay voice processing, context semantic backtracking and multi-round conversation continuous interaction, and enhances the support capability of the system for real-time and context continuity. Through turn identification and cache state control, the application has good task isolation and resource release mechanism, and is suitable for various application scenarios such as intelligent voice assistants, voice customer service, voice interaction terminals and the like.

[0178] In the embodiment, the S8 specifically comprises:

[0179] S81, a semantic service interface module is constructed, the interface module includes a standard HTTP protocol interface and a WebSocket communication interface, and is used for providing real-time access capability of semantic understanding results to an external system;

[0180] S82, a structured semantic label set of a current turn is extracted from the semantic cache module generated in the step S7, and the label set includes a voice intent label, a keyword boundary label, an emotion tendency label and a slot filling label;

[0181] S83, the semantic label set is serialized and packaged into a standard data response structure according to a predefined semantic data format, and the structure supports JSON, XML or a custom binary format;

[0182] S84, a request event of an external system is listened to, when a service call request from a voice question and answer system, a man-machine interaction system or a multi-round dialogue engine is detected, the request content is analyzed and a communication channel is established;

[0183] S85, the structured semantic result corresponding to the current voice conversation is returned to the calling system through the semantic service interface, so that the integrity, time sequence consistency and multi-task label cooperation of the semantic information are ensured;

[0184] S86, an asynchronous push mode of the interface is supported, when voice input is still in a receiving process, the semantic output content can be updated in real time based on a phased analysis result, and interaction logics such as conversation interruption, completion and redirection are realized;

[0185] S87, request parameters, response content and calling time information of each semantic service delivery process are recorded, an interface calling log system is constructed, and is used for subsequent task scheduling, behavior auditing and system performance evaluation;

[0186] S88, after the semantic service output is completed, the conversation context state is closed and the cache resource is released, so as to provide an independent running environment for a structured semantic understanding task of a next voice conversation turn.

[0187] By constructing a semantic service interface supporting HTTP and WebSocket protocols, the voice understanding result can be output in a structured format to an external system in real time, ensuring the stability and standardization of semantic invocation. The interface design supports synchronous and asynchronous data delivery, has the ability to interface with various voice question and answer systems, human-computer interaction engines or decision execution modules, and can meet the semantic communication needs in high concurrency, high consistency and multi-task scenarios. The interface call log recording mechanism further improves the controllability and traceability of the system, which is beneficial to subsequent analysis and optimization and operation and maintenance.

[0188] Embodiment 1:

[0189] In order to verify the feasibility of the application in implementation, the application is applied to an intelligent voice assistant system online on a call center of a large communication operator as a test platform, and the system processes about 85,000 voice interaction tasks per day. Taking the "user query bill" task as the main line, the system needs to identify the user's intention, key entities such as "this month", "bill", "upper limit", "package details", emotion categories such as "neutral" and "unhappy", and intent slot structures such as "month = this month" and "type = traffic", and complete structured output.

[0190] During the voice signal collection process, the system uses a 16kHz sampling rate microphone array to collect user voice data, and the on-site environment is a 50-60dB regular office noise background. A band-pass filter is used for band suppression to effectively filter out noise below 200Hz and above 8kHz. The system is configured with a silence detection module to automatically remove non-speech segments, with an average removal ratio of 15.2%. After pre-emphasis and Hamming window function processing, the voice signal is divided into 25ms frame width and 10ms frame shift frame-level voice sequences.

[0191] In the feature extraction stage, 40-dimensional MFCC coefficients are used, and first-order and second-order difference features are calculated respectively, forming a 120-dimensional three-channel feature tensor. After performing channel-by-channel mean normalization on the tensor, it is sent to the modeling network. The modeling network is a stack of 4-layer Transformer encoder, each layer contains 8 heads of self-attention mechanism and 2048-dimensional feedforward neural network, the input sequence length is dynamically adjusted, and the maximum is not more than 512 frames. The context window of the model input feature is ±3 frames, and the long and short dependency relationship between voice segments is learned through multi-head attention mechanism.

[0192] In the semantic fusion link, a local feature extraction module with a sliding average pooling window size of 5 frames and a global attention context modeling module covering the entire voice segment are run in parallel, and after fusion, they are compressed into a 256-dimensional context enhanced representation through full connection mapping.

[0193] The decoder part configures a bidirectional GRU network, 2 layers per direction, 256 hidden units per layer, and accesses multiple Softmax branch output structured semantic labels, including intent (10 categories), emotion (4 categories), keyword boundary (BIO annotation), and slot (12 slot types). The training adopts a cross-entropy loss function, adds a context consistency regularization term to constrain the smoothness of the label sequence, and the loss weighting ratio is as follows: intent recognition loss 0.4, slot filling loss 0.4, and regularization term loss 0.2. The training adopts an Adam optimizer, the initial learning rate is 0.0008, the batch size is 32, the total training rounds are 30 rounds, and the convergence time is about 8 hours.

[0194] The model training data is derived from historical voice dialogue data of a call center, containing 58,276 artificially annotated voice samples, and the test set contains 12,000 samples, covering 30 common query scenarios. After being deployed online, compared with the original ASR+NLU combination structure, the context semantic understanding model of the application performs excellently in the overall recognition effect: the intent recognition accuracy is improved from 92.8% to 96.3%; the slot filling F1 value is improved from 84.5% to 90.1%; the emotion recognition accuracy is improved from 76.2% to 83.7%; the user complete semantic structure recognition rate is improved from 69.4% to 82.5%; the multi-round context indication resolution accuracy is improved from 41.6% to 71.8%; the average semantic response time delay is controlled within 180 ms, meeting the real-time requirement. The actual business feedback shows that the model can accurately understand complex semantic instructions. For example, in the user's continuous voice "I want to check how much traffic I used this month, and also send me the call records of last month", the system can correctly recognize two parallel intents, double time parameters and query types, and structure the output as two independent task items, which is obviously better than the traditional sentence-by-sentence recognition system.

[0195] The embodiment fully demonstrates the effectiveness and practicality of the application in the actual voice understanding scene. By constructing a multi-layer context modeling network and a multi-task structured decoding structure, and assisted by a joint loss optimization mechanism, the problems of semantic understanding fragmentation and context disconnection of traditional voice systems are solved, and the accuracy, completeness and system running efficiency of complex voice semantic processing are significantly improved, which has wide application value.

[0196] The above describes only the preferred embodiments of the application, but the protection scope of the application is not limited thereto, and any person skilled in the art can make equivalent replacements or changes to the technical solutions and inventive concepts of the application within the technical scope disclosed by the application, which should be covered within the protection scope of the application.

Claims

1. A speech context semantic extraction method based on deep learning, characterized in that: The steps include: S1. Collect and preliminarily process the original speech signal to obtain a preprocessed frame-level speech signal; S2. Perform acoustic feature extraction on the frame-level speech signal. Calculate the MFCC features for each frame of speech using the Mel-frequency cepstral coefficient extraction algorithm. Obtain first-order and second-order difference features simultaneously. Perform mean normalization on all features to construct an input feature tensor in the form of a frame sequence. S3. Input the input feature tensor into the contextual semantic modeling network, build a deep neural network structure composed of multiple layers of Transformer encoders, and output a sequence of semantic embedding vectors; S4. Perform multi-scale feature fusion on the semantic embedding vector sequence, using local sliding window average pooling to extract short-term semantic features, while using global self-attention to extract long-range dependency information across paragraphs, fusion to form context-enhanced semantic representation, and unify the semantic space through fully connected mapping; S5, input the context-enhanced semantic representation into the structured semantic decoder to predict the speech semantic label; S6. Constructing a joint loss function based on the supervisory signal, calculating the cross entropy loss between the predicted results of the semantic labels and the manually labeled labels, constructing a total loss function, and performing end-to-end backpropagation training on the contextual semantic modeling network using the Adam optimizer to update the parameters; S7: Receive voice input data in real time, repeat steps S1 to S5, obtain structured semantic understanding results corresponding to each round of voice conversation, and write them into the online semantic cache module; S8. Provide structured semantic result output to external systems through semantic service interfaces.

2. The method for extracting speech context semantics based on deep learning according to claim 1, characterized in that: Said S1 specifically includes: S11. Synchronously collecting natural speech signals through a microphone array, wherein the microphone array is composed of multiple equally spaced digital acoustic sensors and has directional sound reception and multi-channel input capabilities; S12, converting the collected original voice signal into a standard digital audio format, setting the sampling rate and quantization bit depth, and forming continuous audio stream data; S13, performing bandpass filtering on the audio stream data, setting a lower limit frequency and an upper limit frequency of the filter, suppressing high-frequency background noise and low-frequency device interference, and retaining the main voice and audio frequency band; S14, performing a silence detection operation, using a frame energy analysis method to identify and remove non-speech segments, retaining only valid speech segments to form an uninterrupted speech content sequence; S15, performing pre-emphasis processing on each valid speech segment to enhance high-frequency information and improve spectral feature distribution; S16, using a fixed frame length and frame shift to perform frame processing on the voice segment to generate a temporally continuous and overlapping frame-level voice signal sequence, where each frame contains waveform data of multiple sampling points; S17, applying a Hamming window function to each frame of speech signal to improve spectrum smoothness and reduce spectrum leakage effects caused by window edges; S18: Combining and outputting all the windowed frame-level speech signals to form a time-sequentially arranged pre-processed speech frame sequence.

3. The method for extracting speech context semantics based on deep learning according to claim 1, characterized in that: The S2 specifically includes: S21, receiving the output frame-level speech signal sequence, inputting each frame of speech signal into the acoustic feature extraction module, performing frequency domain analysis processing, and generating a short-time power spectrum of the corresponding frame; S22. Apply a Mel filter bank to the power spectrum of each frame, calculate the Mel frequency energy distribution result, and construct a feature basis for semantic modeling; S23, performing logarithmic compression and discrete transformation processing on the Mel-frequency energy vector, extracting the Mel-frequency cepstral coefficient features corresponding to the frame, and forming a frame-level MFCC feature sequence; S24, based on the MFCC feature sequence, calculating the inter-frame change rate between adjacent frames to generate a first-order difference feature sequence; calculating the change trend again to obtain a second-order difference feature sequence; S25, concatenating the static MFCC features, the first-order difference features, and the second-order difference features in the channel dimension to construct a three-channel frame-level acoustic feature tensor; S26, performing mean normalization processing on the three-channel feature tensor to ensure that the features of each channel are uniformly distributed throughout the entire speech segment, thereby improving modeling stability; S27. The normalized feature tensors are organized into a feature sequence in the order of time frames.

4. The method for extracting speech context semantics based on deep learning according to claim 1, characterized in that: The S3 specifically includes: S31, receiving input feature tensors and inputting them into a contextual semantic modeling network in time frame order, wherein the modeling network is composed of a multi-layer Transformer encoder stack and has cross-frame context modeling capabilities; S32. In each layer of the Transformer encoder, a multi-head self-attention mechanism is first performed on the input features. The semantic dependency weights between frames are calculated in parallel through multiple attention heads, and a weighted combined frame-level context representation is generated. S33, inputting the context representation output by the self-attention module into a feedforward neural network sublayer, wherein the feedforward network includes a two-layer linear transformation structure; S34. Introduce a residual connection mechanism between the output and input of each sub-layer, add the sub-layer input to the transformed result, and enhance the gradient flow ability in the deep network; S35. Perform layer normalization on the residual connection results of each sub-layer to standardize the activation distribution of different channel features and improve the stability of model training; S36: The normalized output of the layer is used as the input of the next layer Transformer encoder, completing the multi-layer stacking context modeling process in sequence; S37, obtain the contextual semantic embedding vector sequence of all frames in the output of the last layer Transformer encoder; S38. Output the semantic embedding vector sequence.

5. The method for extracting speech context semantics based on deep learning according to claim 1, characterized in that: The S4 specifically includes: S41, inputting the semantic embedding vector sequence into a multi-scale feature fusion module, wherein the module is composed of a local semantic extraction branch and a global semantic modeling branch; S42. In the local semantic extraction branch, a sliding window of fixed length is set, and sliding average pooling is performed on the semantic embedding vector sequence in chronological order to extract local context semantic features within each time period; S43. In the global semantic modeling branch, the full-sequence attention mechanism is used to calculate the long-range dependencies between frames and extract global contextual semantic features covering the entire speech segment; S44. Concatenate local semantic features with global semantic features in the feature dimension to construct a multi-scale semantic feature fusion representation, forming a comprehensive representation vector that has both fine-grained and overall semantic information. S45. Input the fused representation into a fully connected mapping network for dimension alignment and nonlinear transformation, mapping local and global information into the same semantic space to obtain a context-enhanced semantic representation. S46, performing a channel weighting mechanism on the context-enhanced semantic representation, dynamically adjusting the weight coefficients according to the importance of different semantic channels; S47. Perform layer normalization on the weighted fused semantic representation to output a context-enhanced semantic representation with stable structure and consistent scale.

6. The method for extracting speech context semantics based on deep learning according to claim 1, characterized in that: The S5 specifically includes: S51, inputting the context-enhanced semantic representation into a structured semantic decoder, wherein the decoder is composed of a bidirectional gated recurrent unit network, a fully connected mapping layer, and a multi-task output layer; S52. In the bidirectional gated recurrent unit network, the semantic representation sequence is transferred along the forward and reverse time order to generate forward and backward context state sequences, and the two are concatenated in the time dimension to form a joint context state representation; S53, inputting the joint context state representation into the fully connected mapping layer, performing nonlinear activation transformation and dimension compression operations, and extracting intermediate semantic vectors suitable for semantic classification tasks; S54, inputting the intermediate semantic vector into a multi-task output layer, wherein the output layer is divided into multiple decoding branches according to task categories, corresponding to an intent classification branch, a keyword boundary recognition branch, a sentiment tendency recognition branch, and a slot filling branch respectively; S55. In the intent classification branch, use classification logic to perform category mapping on the global vector of the entire speech segment and output the corresponding intent label; S56. In the keyword boundary identification branch, perform temporal annotation on the semantic vectors in each frame and output the start and end boundary labels of the keyword at the frame level; S57, in the emotion tendency recognition branch, extracting the emotion feature vector of the entire speech segment, performing a multi-class emotion classification operation, and outputting an emotion category label; S58. In the slot filling branch, combining inter-frame context features, performing slot structure annotation on each frame, and outputting a semantic filling label sequence; S59: Reorganize the semantic tags output by the above branches in chronological order to generate a structured semantic tag set.

7. The method for extracting speech context semantics based on deep learning according to claim 1, characterized in that: The S6 specifically includes: S61: Construct a supervisory signal set, wherein the supervisory signal set includes manually annotated labels corresponding to the speech data, and the labels include intent recognition labels, slot filling labels, and contextual semantic relationship labels; S62, aligning the semantic label prediction results output by the structured semantic decoder with the manually annotated labels one by one, and constructing a cross entropy loss function for the classification task based on the label comparison results; S63. Output the corresponding supervision label to construct the intent recognition loss function, fill the slot with the output supervision label to construct the slot filling loss function, and construct the context consistency regularization term based on the continuity of the context label; S64. Intent recognition loss, slot filling loss, and context consistency regularization term are weighted and combined according to a set ratio to construct a joint loss function. S65. Select the Adam optimizer as the optimization algorithm for the modeling network, initialize the optimizer parameters, and set the learning rate, regularization factor, and gradient update strategy; S66. In each training batch, input the input feature tensor generated in step S2 and the corresponding supervision signal, and perform end-to-end forward propagation and joint loss calculation; S67. Use the backpropagation method based on the Adam optimizer to perform gradient backpropagation and parameter update on the contextual semantic modeling network to optimize all parameters in the Transformer encoder and structured semantic decoder; S68. Repeat steps S66 and S67 until all training rounds are completed or the joint loss function meets the preset convergence conditions, and output the model parameter set after training convergence as the final deep semantic extraction model.

8. The method for extracting speech context semantics based on deep learning according to claim 1, characterized in that: The S7 specifically includes: S71. Export the deep learning model after training convergence to a deployable model file, and load the model file in a deployment environment, which includes a server system or an edge computing device; S72: Configure a voice input receiving module, which is used to receive real-time voice stream input from the user and send the voice data into the processing flow consisting of steps S1 to S5 to perform voice signal acquisition, feature extraction, context modeling, semantic fusion and decoding operations; S73. At the beginning of each round of voice conversation, perform real-time preprocessing and feature conversion on the received voice stream to dynamically generate corresponding frame-level voice signals and input feature tensors; S74: Input the generated input feature tensor into the deployed deep semantic modeling model, perform forward reasoning operations, and output a set of structured semantic labels corresponding to the current speech data; S75: Build a semantic cache module, which caches and manages the structured semantic tags in each round of voice interaction, and archives and stores them in chronological order or by session identifier; S76: Perform identification update and index operations on the tag set stored in the semantic cache module to generate a semantic understanding result document or data object that can be called; S77. After the voice conversation ends, keep the data in the semantic cache accessible; S78. Bind the structured semantic understanding results of each round of voice conversation with the meta-information.

9. The method for extracting speech context semantics based on deep learning according to claim 1, characterized in that: The S8 specifically includes: S81. Build a semantic service interface module, where the interface module includes a standard HTTP protocol interface and a WebSocket communication interface. S82: extracting a structured semantic tag set of the current round from the semantic cache module generated in step S7, wherein the tag set includes a speech intent tag, a keyword boundary tag, a sentiment tendency tag, and a slot filling tag; S83. Serialize and encapsulate the semantic tag set and package it into a standard data response structure according to a predefined semantic data format. The structure supports JSON, XML, or a custom binary format. S84. Monitor request events from external systems. When a service call request from a voice question-answering system, a human-computer interaction system, or a multi-round dialogue engine is detected, parse the request content and establish a communication channel. S85. Return the structured semantic result corresponding to the current voice conversation to the calling system through the semantic service interface; S86, support interface asynchronous push mode, when the voice input is still in the process of receiving, the semantic output content can be updated in real time based on the stage analysis results; S87. Record the request parameters, response content, and call time information of each semantic service delivery process, and build an interface call log system; S88. After the semantic service output is completed, the session context state is closed and the cache resources are released to provide an independent running environment for the structured semantic understanding task of the next voice conversation round.

Images