Voice interaction and data processing method for social security service question and answer digital employees

Abstract

The invention relates to a voice interaction and data processing method for social security service question and answer digital employees, and belongs to the technical field of artificial intelligence and data processing. The method comprises the following steps: acquiring a voice signal and text data, and dynamically framing the voice signal to obtain a time-frequency characteristic matrix; the matrix generates a hidden state sequence through an acoustic feature extraction model, the hidden state sequence is mapped into phoneme probability distribution through a full connection layer, and a Pinyin sequence is obtained through a dynamic programming algorithm; constructing a social security term pinyin prefix tree, limiting paths to match effective branches and rewarding matching items during beam search decoding, correcting the effective branches and rewarding the matching items into complete terms during partial matching, and obtaining a speech recognition text; performing lexical and dependency syntactic analysis on the speech recognition text, and constructing a dependency graph; aggregating node information by using a graph attention mechanism to obtain a node representation sequence; and finally, an intention classification result and an entity label sequence are obtained through processing of a classifier and a sequence annotation device. According to the invention, the accuracy of speech recognition can be improved, and accurate intention classification is realized.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and data processing technology, and specifically relates to a voice interaction and data processing method for a digital employee who answers questions about social security business. Background Art

[0002] Against the backdrop of the increasingly robust social security system, the frequency of social security-related consultations and transactions has increased significantly, encompassing areas such as pension insurance, medical reimbursement, contribution bases, and insurance records. In AI digital employee applications, accurately recognizing speech content and identifying intent is crucial. Social security-related question-and-answer scenarios often feature complex natural language input, highly specialized terminology, pronounced dialects, and significant polysemy, placing higher demands on speech recognition and understanding systems. Existing general-purpose speech recognition systems, often based on fixed-frame segmentation and general dictionary models, struggle to handle the rapid pronunciation, dialect inflections, and low-frequency terminology found in social security contexts. This leads to frequent recognition errors, misjudgment of intent, and missing terminology in real-world applications, severely impacting user experience and system service capabilities. Furthermore, while the social security sector boasts a clear knowledge structure and highly standardized terminology, current speech recognition systems fail to effectively integrate business knowledge and lack terminology awareness and contextual guidance. This results in inaccurate responses during critical business processes and fails to meet the high accuracy, stability, and interpretability requirements of intelligent question-and-answer scenarios. Therefore, there is an urgent need for a voice interaction and data processing method that can be oriented towards the characteristics of social security business and take into account both voice processing accuracy and professional knowledge comprehension capabilities, so as to achieve full-link optimization from acoustic modeling to intent recognition, from term decoding to knowledge reasoning, and improve the overall performance and application value of the social security intelligent question-and-answer system. Summary of the Invention

[0003] In order to solve the above problems, the present invention provides a voice interaction and data processing method for social security business question and answer digital employees.

[0004] To achieve the above-mentioned purpose, the present invention is implemented through the following technical solutions: The present invention provides a voice interaction and data processing method for a social security business question-and-answer digital employee, comprising the following steps: S1. Obtain natural language speech signals and text data input by the user; S2. Dynamically frame the natural language speech signal to obtain a time-frequency feature matrix after framing; S3. The time-frequency feature matrix after the frame is processed by the acoustic feature extraction model to obtain a hidden state sequence; the acoustic feature extraction model includes a one-dimensional convolution layer, a local attention mechanism, a layer normalization layer, and a bidirectional gated recurrent network encoder; S4. The hidden state sequence is mapped to a phoneme probability distribution through a fully connected layer, and the optimal phoneme path is searched by a dynamic programming algorithm to obtain a pinyin sequence; S5. Construct a phonetic prefix tree for social security terms, with each node corresponding to the phonetic sequence of the term. During the beam search decoding process, the generated path is constrained to match valid branches in the prefix tree, and weighted rewards are assigned to terminology library matches in the path. When the phonetic sequence output by the model partially matches the terminology library, it is forcibly corrected to the complete term, resulting in the text after speech recognition. S6. Perform lexical analysis and dependency syntactic analysis on the text after speech recognition to construct a dependency graph. Use the graph attention mechanism to aggregate information for each node in the dependency graph to obtain a node representation sequence after the graph attention network is fused. The node representation sequence after the graph attention network is processed by the classifier and sequence annotator to obtain the intent classification result and entity label sequence.

[0005] Furthermore, step S2 specifically includes: Dynamically divide the continuous speech signal into frames, dynamically adjust the frame start time and frame length according to the speech energy peak, use wavelet packet decomposition to divide each frame signal into multiple sub-bands, dynamically assign frequency band weights by calculating the difference between the energy of each sub-band and the global energy, and use the preset wavelet basis function to perform time-frequency localization on the original waveform, retaining the key frequency band characteristics of business terms in the speech. The time-frequency feature matrix after framing is obtained, which is expressed as follows: , in, Indicates the Speech No. The time-frequency feature matrix of the frame, represents the total number of subbands of wavelet packet decomposition, is a positive integer, Indicates the Speech No. The band weight of the sub-band, Represents the original waveform of the i-th segment of speech; represents the residual term or numerical disturbance term in the wavelet packet decomposition process, represents the Daubechies8 wavelet basis function, Indicates the dynamically adjusted frame start time, Indicates the dynamically adjusted bandwidth, Indicates the frame start time A local time window centered on .

[0006] Furthermore, the Speech No. The frequency band weight of each sub-band is dynamically calculated by the difference between the sub-band energy and the global average energy to enhance the energy of the frequency band where the social security term is located.

[0007] Furthermore, the dynamically adjusted frame start time and the dynamically adjusted frequency bandwidth are determined by speech energy peak detection.

[0008] Furthermore, step S3 specifically includes: The time-frequency feature matrix after frame division is subjected to a multi-scale one-dimensional convolution operation through a one-dimensional convolution layer, and convolution kernels of different step sizes are used to extract syllable-level acoustic boundary features. Then, in the local attention mechanism, each frame is restricted to focus only on the context frames within a fixed range before and after it. The word-level information is associated by calculating the similarity between frames in the window. The features output by the one-dimensional convolution layer and the local attention mechanism are superimposed after a layer normalization layer to form a hidden state that integrates the local syllable features and the word context; the hidden state is processed by a bidirectional gated recurrent network encoder to obtain a hidden state sequence.

[0009] Furthermore, the bidirectional gated recurrent network encoder is composed of two symmetrical gated recurrent units (GRUs), which run independently in the forward and reverse directions along the time axis respectively. Each gated recurrent unit (GRU) contains two gating structures: an update gate and a reset gate.

[0010] Furthermore, in step S4, the hidden state sequence is mapped to a phoneme probability distribution through a fully connected layer, and the optimal phoneme path is searched through a dynamic programming method. At the same time, the difference constraint of the hidden states of adjacent frames is adopted in the loss function. When adjacent phoneme labels are the same, their hidden states are forced to be similar, thereby reducing the alignment jitter of consecutive similar phonemes. During the alignment process, blank labels are allowed to skip redundant frames, and finally the frame sequence is mapped to a pinyin sequence; the dynamic programming method is the Viterbi algorithm.

[0011] Furthermore, the quality of the training effect of the acoustic feature extraction model is measured by the quality of the pronunciation-text alignment. During the training process of the acoustic feature extraction model, the gradient descent method is used to optimize the training parameters of the acoustic feature extraction model.

[0012] Furthermore, step S5 specifically includes: A prefix tree is constructed based on the phonetic sequence of each term in the domain terminology library. During the beam search decoding process, the generated paths are restricted to matching valid branches in the prefix tree. The paths are scored using a scoring function to enhance term matching and optimize path selection. The formula is as follows: , in, Indicates that the The phonetic sequence of the steps; Indicates that the Step path score; Indicates the The hidden state of the frame; Represents a given hidden state Next, the current frame phoneme label probability; is the term matching reward factor; Represents the set of all legal term phonetic sequences in the term prefix tree; Indicates that when the deadline Pinyin sequence of steps If it is a legal term pinyin sequence in the term prefix tree, the value is 1, otherwise it is 0; when the pinyin sequence output by beam search decoding partially matches the term library, it is forced to be corrected to a complete term to obtain the text after speech recognition.

[0013] Furthermore, step S6 specifically includes: Text after speech recognition Perform lexical analysis and dependency syntactic analysis to build a dependency graph ;in, Indicates the node corresponding to each term; is a dependency edge, indicating the syntactic dependency between words; each edge With dependency tags ; Indicates the terms, Indicates the nodes, Indicates from the term To term The dependency syntactic relation label of Indicates correspondence The dependency graph nodes of Represents Nodes with dependency relationships; a graph attention mechanism is used to aggregate information on each node in the dependency graph to obtain a node representation sequence after the graph attention network is fused; the node representation sequence after the graph attention network is fused is input into a classifier for processing to obtain a graph classification result; the node representation sequence after the graph attention network is fused is input into a sequence tagger, and entity recognition is performed on each word using the BIO tagging form to obtain an entity label sequence; the sequence tagger uses a conditional random field sequence tagger.

[0014] The advantages of the present invention are: The present invention combines energy peak-driven dynamic framing with wavelet packet decomposition to adaptively adjust frame length and frequency band weights, effectively enhancing the speech features of the frequency band where social security terminology resides, and overcoming the problem of signal drift under speech rate changes and dialect interference. Compared with the global attention model, which is overly sensitive to irrelevant words, the present invention uses multi-scale one-dimensional convolution to extract syllable boundary features, and combines it with the attention mechanism within the local window to effectively capture the word context and achieve accurate distinction between homophones. The present invention uses hidden state similarity constraints in CTC to enhance the model's alignment robustness to phenomena such as dialect linking and tone shifting, and maintains a stable mapping between speech frames and phoneme sequences under strong accent conditions. To address the problem of fixed and highly specialized terminology in social security services, the present invention constructs a term pinyin prefix tree and combines path constraints with term reward functions in the beam search decoding process to improve the recognition accuracy and completeness of long-tail professional terminology at low confidence levels. The present invention adopts a graph attention mechanism and integrates dependencies for semantic enhancement modeling, significantly improving the ability to handle syntactic nesting, omissions, and ambiguity, and achieving accurate intent classification and entity parameter extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0016] Figure 1 is a flow chart of the steps of the method of the present invention; Figure 2 Comparison of the performance of different framing methods under varying speech rates; Figure 3 The time-frequency feature distribution of the traditional framing method and the dynamic framing method of the present invention; Figure 4 This is a structural diagram of the acoustic feature extraction model of the present invention; Figure 5 Comparison of the performance of different model structures in social security speech recognition tasks; Figure 6 Confusion matrix comparison between the method of the present invention and the traditional method; Figure 7 The alignment stability of different CTC loss functions under dialect linking; Figure 8 Comparison of the recognition effects of different decoding methods on long terms. DETAILED DESCRIPTION

[0017] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments derived by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0018] Example 1 In this embodiment, Figure 1 As shown, the present invention provides a voice interaction and data processing method for a digital employee who answers questions about social security business, and the specific steps include: S1. Obtain natural language speech signals and text data input by the user.

[0019] S2. Dynamically divide the natural language speech signal into frames to obtain a time-frequency feature matrix after the frame division.

[0020] Specifically, in social security services, speech data contains non-stationary signals and narrowband frequency characteristics of business terms. For example, non-stationary signals are generated by changes in speech rate and pauses, and the high-frequency consonant band of the business term "pension" belongs to the narrowband frequency band. Traditional MFCC (Mel-Frequency-Cepstral Coefficient) features use fixed 20-40ms frames, which results in loss of inter-frame information during fast speech and fails to highlight key frequency bands for social security terminology. The present invention dynamically divides continuous speech signals into frames, dynamically adjusts the frame start time and frame length according to the speech energy peak, and avoids the speech rate sensitivity problem caused by fixed framing. At the same time, wavelet packet decomposition is used to divide each frame signal into multiple sub-bands. The frequency band weights are dynamically assigned by calculating the difference between the energy of each sub-band and the global energy. The original waveform is processed in time-frequency localization using a preset wavelet basis function, retaining the key frequency band characteristics of business terms in the speech, which can be expressed as: , Where, For the Speech No. The time-frequency feature matrix of the frame represents the frequency domain characteristics of the speech signal in a specific frame; The total number of sub-bands decomposed by wavelet packets, which decomposes the speech signal into multiple frequency bands. ; is a positive integer; For the Speech No. The frequency band weight of each sub-band is dynamically calculated by the difference between the sub-band energy and the global average energy, strengthening the energy of the frequency band where the social security term is located, such as 200-4000Hz, and suppressing irrelevant frequency bands. Speech No. Band weight of sub-band The calculation method is expressed as: , is an exponential function with a natural constant as its base; is the energy smoothing coefficient, such as, ; For the The voice in The instantaneous energy of a sub-band is calculated as: , is a positive integer; is the half window width, which controls the local energy calculation range, such as ; For the The global average energy of all sub-bands of the speech segment, emphasizing the frequency band where the business terms are located, is calculated as follows: , is the original waveform of the i-th speech segment; for Represents the residual term or numerical perturbation term in the wavelet packet decomposition process, which is used to improve numerical stability and noise resistance, such as obeying the mean , Gaussian noise with a standard deviation of 0.1; The Daubechies8 wavelet basis function covers the key frequency bands of social security terms. This wavelet basis function was chosen because of its high resolution in the 200-4000 Hz frequency band, which is suitable for the frequency distribution of social security terms, such as the high-frequency characteristics of the initial consonant of "payment base"; The frame start time is dynamically adjusted and is determined by energy peak detection. The calculation method is expressed as: , in, The maximum allowed frame length, such as 50ms; It is a preset lightweight meta-learning network; The end time of the previous frame. Indicates the time interval Find the time point where a specific function reaches its maximum value; The bandwidth is dynamically adjusted and is determined by energy peak detection. The calculation method is expressed as: , Adaptively expand or contract the frequency band based on sub-band energy differences, enhancing the time-frequency focus of energy-significant areas, such as the long-tail syllable in the term "deemed payment years"; For the The preset lower bound of the frequency band of the sub-band; For the The preset upper bound of the frequency band of the sub-band; is the hyperbolic tangent function; Indicates the frame start time A local time window centered on It should be noted that Characterization The voice in In each frequency band, based on the frame start time The local time-domain weighted value of can achieve focused amplification of key time-frequency areas. For example, it can retain and focus on the frequency bands of terms such as "pension" and "contribution base" in speech. It should also be noted that conventional dynamic framing relies on energy peak detection and is not adaptable enough to non-stationary signals, such as non-stationary signals such as intermittent pronunciation of dialects. Before framing, the preset lightweight meta-learning network uses the pre-trained meta-network to predict the initial value of the optimal frame length and frequency band weight in real time. That is, the input of the pre-trained meta-network is the short-time Fourier transform spectrum of the original speech signal, and the output includes the frame length. ms, first Speech No. Initial value of the band weight of the subband , thereby achieving adaptive dialect and speech speed changes and reducing false triggering of energy detection; It should also be noted that the pre-trained meta-network can be a combination of a 3-layer CNN and a Gated-Recurrent-Unit (GRU). Using meta-learning for frame parameter prediction replaces traditional signal processing methods, which helps improve the contextual perception ability of dynamic framing.

[0021] S3. The time-frequency feature matrix after the frame is processed by the acoustic feature extraction model to obtain a hidden state sequence; Figure 4 As shown in FIG, the acoustic feature extraction model includes a one-dimensional convolutional layer, a local attention mechanism, a layer normalization layer, and a bidirectional gated recurrent network encoder.

[0022] Specifically, the framed time-frequency feature matrix undergoes multi-scale one-dimensional convolution operations through a one-dimensional convolutional layer. Convolution kernels with different strides are used to extract syllable-level acoustic boundary features. Then, in the local attention mechanism, each frame is restricted to only focus on the context frames within a fixed range before and after it. By calculating the similarity between frames within the window, word-level information is associated. The features output by the one-dimensional convolutional layer and the local attention mechanism are stacked after passing through a layer normalization layer to form a hidden state that fuses local syllable features and word context, which is expressed by the formula: , In the formula, is the hidden state of the th segment of speech at the th layer. Among them, for the hidden state of the th segment of speech at the first layer, it is the framed time-frequency feature matrix; is the hidden state of the th segment of speech at the th layer; represents the layer normalization operation; is a positive integer, indicates that the strides of the convolution kernels are 1, 2, and 4 respectively, with the unit being frames. By fusing features at different granularities, homophonic terms are distinguished, such as the acoustic differences between "supplementary payment" and "补交"; represents the one-dimensional convolutional layer of length at the th layer, which is used to extract syllable boundary features. For example, d = 4 captures the starting point of "养" in "养老金". Moreover, the total number of layers of the one-dimensional convolutional layer can be set to 3; ... represents the local attention mechanism at the th layer, which limits the context window to 10 frames before and after the current frame. The calculation method is expressed as: , The total number of layers of the local attention mechanism can be set to 3; is the value vector; is the query vector; is the key vector; is the transpose of; is the dimension of the attention mechanism; is the mask matrix, and its element , which forces the model to only focus on local context, such as the cross-word dependency in "养老保险"; is the Softmax function; is the query vector at the th layer, and the calculation method is expressed as: , is the The layer's trainable query weight matrix; For the The key vector of the layer is calculated as: , For the The layer's trainable key weight matrix; For the The value vector of the layer is calculated as follows: , For the The layer's trainable weight matrix.

[0023] It should be noted that The representation is used to extract local syllable features. Multi-scale convolution kernels are used to capture acoustic boundaries of different granularities, such as the starting point of a syllable, to achieve acoustic differentiation of homophones, such as the distinction between "bujiao" and "bujiao"; It should also be noted that Representing word-level attention enhancement, by calculating similarity within a local window and associating word-level context, such as the cross-word dependency in "cumulative payment years"; It should also be noted that the collaboration between multi-scale convolution and local attention is not a simple stacking, but rather a complementary enhancement of syllable-level to word-level features through step size selection and window limitation.

[0024] After the convolution and local attention are fused, the hidden state is input into a bidirectional gated recurrent network encoder, which outputs a hidden state sequence to enhance the context modeling capability. The bidirectional gated recurrent network encoder consists of two symmetrical GRUs, which run independently in the forward and reverse directions along the time axis to process the input sequence at each time step. Each GRU unit contains two gating structures, an update gate and a reset gate, to control the retention and forgetting of information.

[0025] S4. The hidden state sequence is mapped to a phoneme probability distribution through a fully connected layer, and the optimal phoneme path is searched by a dynamic programming algorithm to obtain a pinyin sequence; Social security terminology often suffers from problems such as connected reading and dialectal intonation. For example, "personal account" is pronounced as "personal bank account," requiring a robust alignment mechanism to align the pronunciation with the translated text. Traditional CTC (Connectionist-Temporal-Classification) assumes independence between frames, resulting in a high error rate for aligning consecutive similar phonemes, such as the similar pronunciations of "sh" and "s".

[0026] The present invention maps the hidden state sequence output by the bidirectional gated recurrent network encoder into a phoneme probability distribution, searches for the optimal phoneme path through the dynamic programming algorithm, and at the same time adopts the difference constraint of adjacent frame hidden states in the loss function. When the adjacent phoneme labels are the same, it forces their hidden states to be similar, reducing the alignment jitter of consecutive similar phonemes. During the alignment process, blank labels are allowed to skip redundant frames, and finally the frame sequence is mapped into a pinyin sequence. For example, "[j][i][a][o]" is aligned to "jiao". The training effect of the acoustic feature extraction model is measured by the quality of pronunciation-text alignment. During the training process of the acoustic feature extraction model, the gradient descent method is used to optimize the training parameters of the acoustic feature extraction model. The loss calculation method in each iteration of the gradient descent method is expressed as: , wherein, is the CTC loss function of the dynamic transition constraint. By minimizing this loss function, the acoustic feature extraction model learns to map continuous speech frames into accurate and stable phoneme sequences, and improves the alignment robustness to phenomena such as connected speech and tone sandhi; is the total number of speech frames; is the phoneme label of the th frame; is the phoneme label of the th frame; is the encoder output hidden state sequence of the th segment of speech, output by the bidirectional gated recurrent network encoder; represents the probability of the phoneme label at the th frame under the given hidden state sequence ; represents the logarithmic function, with the default base of 10; is the transition penalty coefficient. For example, ; is the indicator function, which is 1 when the adjacent phoneme labels are different and 0 otherwise; is the L2 norm; is the hidden state of the tth frame, representing the acoustic features of the current frame; is the hidden state of the t th frame.

[0027] It should be noted that, the role of

[0028] Based on the CTC loss function with dynamic transition constraints, the specific steps for pronunciation-text alignment are: 1) Generate phoneme probability distribution Hidden State Through the fully connected layer, each hidden state is mapped to the output phoneme probability distribution , covering all phoneme categories, including 20 phonemes and blank labels.

[0029] 2) Dynamic programming to search for the optimal alignment path Use the dynamic programming method to calculate the optimal path and find the optimal mapping path from the frame sequence to the phoneme sequence, solve the problem of mismatch between pronunciation and text length, and allow blank labels to skip repeated frames. The dynamic programming method can use the Viterbi algorithm. The specific steps are as follows: 2.1) Initialize the dynamic programming matrix Create a two-dimensional matrix where the rows represent the positions of the target phonemes and the columns represent the time steps of the speech frames. Each position in the matrix stores the maximum cumulative probability from the starting point to that position. Initially, only blank labels or the first phoneme are allowed to start, and the probabilities of other positions are set to invalid values; 2.2) Recursively calculate path scores Continue the current phoneme, move to a new phoneme, or insert a blank label. If the speech features of the current frame belong to the same phoneme as those of the previous frames, the probability path of the current phoneme is continued, allowing phoneme repetition; If a change in speech features is detected, it moves to a new phoneme, that is, moves to the next phoneme and updates the path probability; Insert blank tags: If the current frame is silent or redundant, such as a pause at the beginning or end of a sentence, insert a blank tag to skip the frame without occupying the phoneme position.

[0030] 3) Hidden state continuity constraints The hidden state continuity constraint includes different penalties for adjacent phonemes and tolerance for the same adjacent phonemes, specifically: The penalty for different adjacent phonemes means that if adjacent frames are assigned to different phonemes, the difference in the acoustic features (hidden states) of the two frames is checked. If the difference is too large, the score of the path is reduced, and the path with smooth transition of acoustic features is preferred; Tolerance for identical adjacent phonemes means that if adjacent frames contain the same phoneme, no penalty is imposed and reasonable repetition is allowed.

[0031] It should be noted that the way to check the difference in acoustic features (hidden states) between two frames is calculated through the CTC loss function with dynamic transition constraints. That is, if the CTC loss function with dynamic transition constraints is greater than the preset threshold, it indicates that the difference is too large; otherwise, it indicates that there is no excessive difference.

[0032] It should also be noted that the CTC loss function with dynamic transition constraints During the alignment process, by adopting the difference constraint term of adjacent frame hidden states , when the adjacent phoneme labels are the same, force their hidden states to be similar, thereby reducing the alignment jitter of consecutive similar phonemes, making the alignment process more robust, and being able to effectively handle problems such as connected speech and dialect tone sandhi.

[0033] 4) Backtracking to generate the optimal path Starting from the final frame of the last phoneme, trace back the path with the highest score in the entire matrix in reverse; Furthermore, merge the repeated phonemes and blank labels to generate the final phoneme sequence.

[0034] In one embodiment, a speech frame may be mis-mapped to "ge - yin - zhang - hu", but through the hidden state constraint, force the acoustic feature difference between "ge" and "yin" to decrease, and the path is corrected to "ge - ren - zhang - hu", and finally aligned to the standard term "personal account".

[0035] In this embodiment, for the speech frame sequence "[sil][j][i][a][o][sil]", it may be aligned to the pinyin "jiao".

[0036] S5. Construct a pinyin prefix tree for social security terms, where each node corresponds to the pinyin sequence of the term. During the beam search decoding process, restrict the generated path to match the valid branches in the prefix tree, and give weight rewards to the term library matching items in the path. When the pinyin sequence output by the model partially matches the term library, force it to be corrected to the complete term to obtain the text after speech recognition.

[0037] Specifically, social security services have strong domain dictionary constraints. For example, "视同缴费年限" (equivalent contribution years) is a fixed term, and it is necessary to avoid interference from general vocabulary; conventional beam search methods tend to favor high-frequency general words, which are prone to incorrect recognition. For example, "参保" (participate in insurance) is mis-recognized as "餐包" (meal package); The present invention pre-constructs a pinyin prefix tree for social security terms, where each node corresponds to the pinyin sequence of the term. During the beam search decoding process, restrict the generated path to match the valid branches in the prefix tree, and give weight rewards to the term library matching items in the path. When the pinyin sequence output by the model partially matches the term library, force it to be corrected to the complete term, such as correcting "yang - lao - jin" to "养老金" (pension), specifically including: 1) Term prefix tree construction A prefix tree is constructed for the pinyin sequence of each term in the domain term library, such as "pension" and "contribution base". The nodes store prefix information, such as "yang→lao→jin" represents "pension".

[0038] 2) Path constraints during beam search decoding The expansion of each path in the beam search is limited to the valid pinyin paths on the prefix tree. For example, if the current path is "yang-lao", it is only allowed to continue to expand to legal branches such as "jin"; At the same time, the paths are scored by the scoring function to enhance term matching and optimize path selection, which is expressed as: , Where, To the end of The phonetic sequence of the steps; Indicates that the The path score of the step is a weighted sum of the model log probability and the term reward; For the The hidden state of the frame; Represents a given hidden state Next, the current frame phoneme label probability; is a term matching reward factor that increases the probability of retaining a term in low confidence situations, e.g. , so that even if the confidence level of “payment base” is low, the standard term is still output first; For the set of all legal term pinyin sequences in the term prefix tree, pre-construct the social security term pinyin sequence, such as "yang-lao-jin" → "pension", and constrain the decoding path; Indicates that when the deadline Pinyin sequence of steps If it is a legal term phonetic sequence in the term prefix tree, the value is 1, otherwise it is 0.

[0039] 3) Mandatory correction of terminology If the output pinyin sequence is "yang-lao-jin", even if the model confidence is 0.7, the sequence will still be corrected to "pension" output because it matches the term "pension" in the prefix tree.

[0040] In one embodiment, taking the 3-second voice message "What is the pension insurance payment base for flexible employment personnel?" as an example, the processing method is as follows: 1) 3 seconds of speech is divided into 150 dynamic frames, using dynamic framing, with frame lengths ranging from 20 to 50 ms. Then, after 80-band feature extraction and 512-dimensional encoding, it is mapped into a pinyin sequence and output as a standard term through a prefix tree; 2) Generate time-frequency matrix , the dimension is , i.e. 80 frequency bands; 3) Process the acoustic feature extraction model and output the hidden state , the dimension is ; 4) Perform CTC alignment to map consecutive frames into pinyin sequences, such as "qing-wen-ling-huo-jiu-ye-ren-yuan-yang-lao-jin-jiao-fei-ji-shu-shi-duo-shao"; 5) Decode the term and correct it to "pension insurance contribution base for flexible employment personnel" through prefix tree.

[0041] S6. Perform lexical analysis and dependency syntactic analysis on the text after speech recognition to construct a dependency graph. Use the graph attention mechanism to aggregate information for each node in the dependency graph to obtain a node representation sequence after the graph attention network is fused. The node representation sequence after the graph attention network is processed by the classifier and sequence annotator to obtain the intent classification result and entity label sequence.

[0042] Specifically, Step 1: Syntactic dependency graph construction and embedding initialization Suppose the text after speech recognition is , perform lexical analysis and dependency syntax analysis on the text, and build a dependency graph ;in, Indicates the node corresponding to each term; is a dependency edge, which indicates the syntactic dependency between words, such as subject-predicate, verb-object, and modification relationships; each edge With dependency tags . For the The first term in the text A word, indicating the original form of a word; For the nodes, representing terms One-to-one correspondence between nodes in the graph is used to build the graph attention structure; For the word arrive Dependency syntactic relationship labels, such as "subject-predicate", "modification", "object-complement", etc. For the Term, which is an element in the original recognition text; To correspond The dependency graph nodes of For Nodes with dependency relationships, i.e. edges The target node in For the The initial representation of each node in the graph attention network is the word embedding vector, such as obtained by the Word2Vec method.

[0043] Furthermore, for each word , initialize its representation vector as word embedding ,The present invention uses pre-trained word vectors, such as Word2Vec or BERT embedding.

[0044] Step 2: Syntactically enhanced graph attention calculation A graph attention mechanism is used to aggregate information at each node in the dependency graph, allowing each node to integrate its syntactic context features. This allows the model to not only focus on the local context but also incorporate important information about dependency grammar in social security terms, such as the "noun-modifier" structure in "pension insurance" and "contribution base," improving the ability to eliminate ambiguity and identify intent, as expressed as: , Where, For the Word pair The attention weight of each word; for Activation function; is a learnable attention vector; for The transpose of is the shared linear mapping matrix; For the Layer 1 graph attention network The representation of nodes; For the Layer 1 graph attention network The representation of nodes; For the Layer 1 graph attention network The representation of nodes; Represents vector concatenation operation; Dependency tags Embedded representation of is the ReLU activation function; For the All nodes adjacent to a node.

[0045] It should be noted that the syntax-enhanced graph attention mechanism combined with dependency relations can improve the accuracy of intent recognition. Compared with the traditional Transformer structure, it can retain the explicit representation of the syntactic dependency structure and solve common problems such as phrase nesting, omission, and ambiguity in social security business statements. For example, it can distinguish the intention differences between "query payment" and "query payment history" and accurately identify parameter entities before and after the word order.

[0046] Step 3: Intent Classification and Parameter Identification The node representation sequence after fusion of the graph attention network Input classifier and sequence tagger, where For the graph attention network The embedding representation of the first word in the layer, used for the first word of the sentence or CLS marker, Graph Attention Network The embedding representation of the second term in the layer, For the graph attention network Layer The intent classification is obtained by pooling the sentence vectors and is expressed as: , Where, Intention classification results, such as querying payment years, changing insurance type information, verifying account status, etc. Represents the node representation sequence after the graph attention network fusion; is the weight parameter of the trainable neural network classification layer; is the bias parameter of the classification layer of the trainable neural network.

[0047] Furthermore, parameter extraction adopts a conditional random field sequence tagger, and uses the BIO tagging form to perform entity recognition on each word. The output result is an entity label sequence, such as name, ID number, payment year, insurance type, etc.

[0048] In one embodiment, for the input text "I want to query the pension insurance contribution base for flexible employment personnel in 2022", the intent classification result is: "Query the pension contribution base", and the parameter extraction results are: time parameter "2022", insurance type parameter "pension insurance", and population type "flexible employment personnel".

[0049] Example 2 In this embodiment, in order to verify the robustness of the dynamic framing technology in different speech speed scenarios, the phoneme recognition accuracy of the traditional fixed framing method (red dotted line) and the dynamic framing method of the present invention (blue solid line) at different speech speeds is compared. Figure 2As shown in the figure, the experimental results show that as the speaking speed increases, the accuracy of the fixed framing method drops significantly due to the loss of inter-frame information, especially in the high-speed speech segment. The dynamic framing makes the accuracy curve smoother through adaptive frame length adjustment and frequency band weight distribution, and remains highly stable in the high-speed segment. The changing trend of the accuracy on the vertical axis in the figure intuitively reflects that this method effectively alleviates the problem of speech speed sensitivity through the dynamic framing strategy. The advantage lies in the joint optimization of the speech energy peak and the wavelet packet subband energy, which avoids the feature loss of the traditional method under non-stationary signals.

[0050] Example 3 In this embodiment, if Figure 3 The heat map shows a comparison of the time-frequency feature distribution of the traditional framing method and the dynamic framing of the present invention, verifying the dynamic framing's ability to focus on the key frequency bands of social security business terms. The vertical axis of the heat map represents the frequency band number, corresponding to the actual frequency band range, and the horizontal axis is the speech frame number. The color depth reflects the energy intensity. The traditional method (a) causes energy distribution dispersion due to fixed framing. Especially at fast speaking speeds, the energy of the key frequency band (the 200-4000Hz area marked by the red dotted line) is blurred and the boundaries overflow. However, the present method (b) forms a high-energy focus area (dark block) within the key frequency band by dynamically adjusting the frame length and frequency band weight, and significantly reduces the residual noise (light area). Experimental results show that the present invention effectively enhances the feature extraction of term-related frequency bands through wavelet packet decomposition and dynamic frequency band allocation, solving the problem of feature loss in traditional methods when the speaking speed changes.

[0051] Example 4 In this embodiment, Figure 5 As shown in the figure, by comparing the performance differences of different model structures in the social security speech recognition task, the four models in the bar graph are, from left to right, recurrent neural network, global attention model, multi-scale convolution combined with attention model and this method. The traditional recurrent neural network has the lowest performance due to its weak long-term dependency modeling ability (the first red column on the left). Although the global attention model introduces context perception, it is easily disturbed by irrelevant modal particles (the second blue column on the left). This method captures syllable boundary features through multi-scale one-dimensional convolution and combines the local attention mechanism to constrain the context window. It appears as the highest column in the bar graph (the first gold column on the right) and has the smallest error bar range, indicating that the model has the best stability. It proves that the fusion of multi-granular features and the coordination of local attention can significantly distinguish homophones and improve the accuracy of acoustic modeling in social security scenarios.

[0052] Example 5 In this embodiment, Figure 6As shown, a confusion matrix is ​​used to evaluate the model's ability to distinguish homophones. This compares a traditional recurrent neural network with the multi-scale convolution-attention fusion model of our invention. Each row of the matrix represents the true term category, and each column is the model's prediction. The dark blocks on the diagonal represent the correct recognition rate. The traditional model (a) exhibits large crossover errors (off-diagonal dark blocks) between easily confused terms, especially for pairs of terms with similar syllables. Our method (b) significantly improves the diagonal accuracy and nearly eliminates crossover errors by capturing syllable boundary features through multi-scale convolution and combining local attention with word-level context. Experimental results demonstrate that our method's feature fusion mechanism decouples acoustic details from semantic context, effectively resolving homophone ambiguity.

[0053] Example 6 In this embodiment, the alignment stability of the traditional connection time series classification loss function and the constraint loss function of the present invention in the dialect linking scenario is compared through the error range diagram, as shown in FIG. Figure 7 As shown in the figure, the horizontal axis is the dialect accent intensity level, and the vertical axis is the mean square error of the hidden states of adjacent speech frames, which reflects the degree of alignment jitter. The traditional method (orange curve) has an error range (shaded area) that expands rapidly with the increase of dialect intensity, indicating its sensitivity to tone changes and connected readings. However, this method (blue curve) significantly reduces the error mean and fluctuation range by constraining the similarity of the hidden states of adjacent identical phonemes. The blue area in the figure is always narrow and close to the horizontal axis, indicating that even under strong dialect interference, this method can still maintain the smoothness of the alignment path, verifying the key role of dynamic transition constraints in improving the robustness of pronunciation-text alignment.

[0054] Example 7 In the embodiment, the recognition effect of term-enhanced decoding on long terms is analyzed by combining box plots and line graphs, and the conventional beam search is compared with the domain term constraint decoding method of the present invention, such as Figure 8 As shown in the figure, the box height of the box plot reflects the stability of the recognition rate distribution, and the broken line trend indicates the performance change as the term length increases. The box of the traditional method (orange) significantly expands and shifts downward as the term length increases, indicating that its recognition rate fluctuates greatly and is easily interfered with by common words. However, this method (blue), through the term prefix tree constraint and score reward mechanism, maintains a compact box and a gently decreasing broken line, maintaining a high recognition rate especially for long terms. The upper edge of the blue box in the figure almost coincides with the broken line, indicating that this technology can stably correct low-confidence paths and force the output of terms that meet domain standards. Experimental results show that enhanced term decoding significantly improves the recognition reliability of long-tail professional expressions in social security business.

[0055] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A voice interaction and data processing method for digital employees who answer questions about social security business, characterized by: The following steps are involved: S1. Obtain natural language speech signals and text data input by the user; S2. Dynamically frame the natural language speech signal to obtain a time-frequency feature matrix after framing; S3. The time-frequency feature matrix after the frame is processed by the acoustic feature extraction model to obtain a hidden state sequence; the acoustic feature extraction model includes a one-dimensional convolution layer, a local attention mechanism, a layer normalization layer, and a bidirectional gated recurrent network encoder; S4. The hidden state sequence is mapped to a phoneme probability distribution through a fully connected layer, and the optimal phoneme path is searched by a dynamic programming algorithm to obtain a pinyin sequence; S5. Construct a phonetic prefix tree for social security terms, with each node corresponding to the phonetic sequence of the term. During the beam search decoding process, the generated path is constrained to match valid branches in the prefix tree, and weighted rewards are assigned to terminology library matches in the path. When the phonetic sequence output by the model partially matches the terminology library, it is forcibly corrected to the complete term, resulting in the text after speech recognition. S6. Perform lexical analysis and dependency syntactic analysis on the text after speech recognition to construct a dependency graph. Use the graph attention mechanism to aggregate information for each node in the dependency graph to obtain a node representation sequence after the graph attention network is fused. The node representation sequence after the graph attention network is processed by the classifier and sequence annotator to obtain the intent classification result and entity label sequence.

2. The voice interaction and data processing method for social security business question-answering digital employees according to claim 1 is characterized in that: Step S2 specifically includes: Dynamically divide the continuous speech signal into frames, dynamically adjust the frame start time and frame length according to the speech energy peak, use wavelet packet decomposition to divide each frame signal into multiple sub-bands, dynamically assign frequency band weights by calculating the difference between the energy of each sub-band and the global energy, and use the preset wavelet basis function to perform time-frequency localization on the original waveform, retaining the key frequency band characteristics of business terms in the speech. The time-frequency feature matrix after framing is obtained, which is expressed as follows: ， in, Indicates the Speech No. The time-frequency feature matrix of the frame, represents the total number of subbands of wavelet packet decomposition, is a positive integer, Indicates the Speech No. The band weight of the sub-band, Represents the original waveform of the i-th segment of speech; represents the residual term or numerical disturbance term in the wavelet packet decomposition process, represents the Daubechies8 wavelet basis function, Indicates the dynamically adjusted frame start time, Indicates the dynamically adjusted bandwidth, Indicates the frame start time A local time window centered on .

3. The voice interaction and data processing method for social security business question-answering digital employees according to claim 2 is characterized in that: Step S3 specifically includes: The time-frequency feature matrix after frame division is subjected to a multi-scale one-dimensional convolution operation through a one-dimensional convolution layer, and convolution kernels of different step sizes are used to extract syllable-level acoustic boundary features. Then, in the local attention mechanism, each frame is restricted to focus only on the context frames within a fixed range before and after it. The word-level information is associated by calculating the similarity between frames in the window. The features output by the one-dimensional convolution layer and the local attention mechanism are superimposed after a layer normalization layer to form a hidden state that integrates the local syllable features and the word context; the hidden state is processed by a bidirectional gated recurrent network encoder to obtain a hidden state sequence.

4. The voice interaction and data processing method for social security business question-answering digital employees according to claim 3 is characterized in that: In step S4, the hidden state sequence is mapped to a phoneme probability distribution through a fully connected layer, and the optimal phoneme path is searched through a dynamic programming method. At the same time, the difference constraint of the hidden states of adjacent frames is adopted in the loss function. When adjacent phoneme labels are the same, their hidden states are forced to be similar, reducing the alignment jitter of consecutive similar phonemes. During the alignment process, blank labels are allowed to skip redundant frames, and finally the frame sequence is mapped to a pinyin sequence; the dynamic programming method is the Viterbi algorithm.

5. The voice interaction and data processing method for social security business question-answering digital employees according to claim 4 is characterized in that: Step S5 specifically includes: A prefix tree is constructed based on the phonetic sequence of each term in the domain terminology library. During the beam search decoding process, the generated paths are restricted to matching valid branches in the prefix tree. The paths are scored using a scoring function to enhance term matching and optimize path selection. The formula is as follows: ， in, Indicates that the The phonetic sequence of the steps; Indicates that the Step path score; Indicates the The hidden state of the frame; Represents a given hidden state Next, the current frame phoneme label probability; is the term matching reward factor; Represents the set of all legal term phonetic sequences in the term prefix tree; Indicates that when the deadline Pinyin sequence of steps If it is a legal term pinyin sequence in the term prefix tree, the value is 1, otherwise it is 0; when the pinyin sequence output by beam search decoding partially matches the term library, it is forced to be corrected to a complete term to obtain the text after speech recognition.

6. The voice interaction and data processing method for social security business question-answering digital employees according to claim 5 is characterized in that: Step S6 specifically includes: Text after speech recognition Perform lexical analysis and dependency syntactic analysis to build a dependency graph ;in, Indicates the node corresponding to each term; is a dependency edge, indicating the syntactic dependency between words; each edge With dependency tags ; Indicates the terms, Indicates the nodes, Indicates from the term To term The dependency syntactic relation label of Indicates correspondence The dependency graph nodes of Represents Nodes with dependency relationships; a graph attention mechanism is used to aggregate information on each node in the dependency graph to obtain a node representation sequence after the graph attention network is fused; the node representation sequence after the graph attention network is fused is input into a classifier for processing to obtain a graph classification result; the node representation sequence after the graph attention network is fused is input into a sequence tagger, and entity recognition is performed on each word using the BIO tagging form to obtain an entity label sequence; the sequence tagger uses a conditional random field sequence tagger.

7. The voice interaction and data processing method for social security business question-answering digital employees according to claim 6 is characterized in that: The step S2 Speech No. The frequency band weight of each sub-band is dynamically calculated by the difference between the sub-band energy and the global average energy to enhance the energy of the frequency band where the social security term is located.

8. The voice interaction and data processing method for social security business question-answering digital employees according to claim 7 is characterized in that: The dynamically adjusted frame start time and the dynamically adjusted frequency bandwidth in step S2 are determined by speech energy peak detection.

9. The voice interaction and data processing method for social security business question-answering digital employees according to claim 8 is characterized in that: In step S3, the bidirectional gated recurrent network encoder is composed of two symmetrical gated recurrent units (GRUs), which run independently in the forward and reverse directions along the time axis respectively. Each gated recurrent unit (GRU) contains two gating structures: an update gate and a reset gate.

10. The voice interaction and data processing method for social security business question-answering digital employees according to claim 9 is characterized in that: In step S4, the quality of the training effect of the acoustic feature extraction model is measured by the quality of the pronunciation-text alignment. During the training process of the acoustic feature extraction model, the training parameters of the acoustic feature extraction model are optimized using the gradient descent method.

Images