Voice interaction and data processing method for social security business question and answer digital staff

Abstract

The application relates to a voice interaction and data processing method for a social security business question and answer digital staff, and belongs to the technical field of artificial intelligence and data processing. The method comprises the following steps: acquiring a voice signal and text data, dynamically framing the voice signal to obtain a time-frequency feature matrix; the matrix is subjected to an acoustic feature extraction model to generate a hidden state sequence, and then is mapped into a phoneme probability distribution through a full connection layer, and a dynamic programming algorithm is used to obtain a pinyin sequence; a social security term pinyin prefix tree is constructed, a path matching effective branch is limited during bundle search decoding, a matching item is rewarded, and when a partial match is achieved, the partial match is corrected into a complete term to obtain a voice recognition text; the voice recognition text is subjected to morphological and dependency syntax analysis to construct a dependency graph; a graph attention mechanism is used to aggregate node information to obtain a node representation sequence; finally, a classifier and a sequence labeler are used for processing to obtain an intention classification result and an entity label sequence. The application can improve the accuracy of voice recognition and realize accurate intention classification.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence and data processing, and particularly relates to a voice interaction and data processing method for social security business question and answer digital staff. BACKGROUND

[0002] Under the background of the increasingly perfect social security system, the frequency of consultation and handling of social security related business has increased significantly, involving multiple fields such as endowment insurance, medical reimbursement, payment base, and insurance record. In the application scenario of AI digital staff, how to accurately identify voice content and perform intent recognition is crucial. The question and answer scenario related to social security business generally presents the characteristics of complex natural language input, strong professional terminology, significant dialect voice, and large ambiguity of polysemous words, which puts higher requirements on the voice recognition and understanding system. The existing general voice recognition system is based on fixed framing and general dictionary model, which is difficult to handle the recognition problems of fast pronunciation, dialect tone change and low-frequency terminology in the social security context, resulting in inaccurate recognition, misjudgment of intent, loss of terminology and other situations in real applications, which seriously affects user experience and system service capability. At the same time, the social security field has the characteristics of clear knowledge structure and high standardization of terminology, but the current voice recognition system fails to effectively integrate business knowledge and lacks terminology perception and context guidance mechanism, resulting in inaccurate response in key business links and failing to meet the high requirements of accuracy, stability and explainability in intelligent question and answer scenarios. Therefore, there is an urgent need for a voice interaction and data processing method that can face the characteristics of social security business, taking into account the accuracy of voice processing and the understanding ability of professional knowledge, to realize the whole-link optimization from acoustic modeling to intent recognition, from terminology decoding to knowledge reasoning, and to improve the overall performance and application value of the social security intelligent question and answer system. SUMMARY

[0003] The application provides a voice interaction and data processing method for social security business question and answer digital staff.

[0004] To achieve the above purpose, the application realizes the following technical scheme:

[0005] The application provides a voice interaction and data processing method for social security business question and answer digital staff, comprising the following steps:

[0006] S1. Obtain the natural language voice signal and text data input by the user;

[0007] S2. Perform dynamic framing processing on the natural language voice signal to obtain a framed time-frequency feature matrix;

[0008] S3. The time-frequency feature matrix after framing is processed by an acoustic feature extraction model to obtain a hidden state sequence; the acoustic feature extraction model includes a one-dimensional convolutional layer, a local attention mechanism, a layer normalization layer, and a bidirectional gated recurrent network encoder.

[0009] S4. The hidden state sequence is mapped to a phoneme probability distribution through a fully connected layer, and the optimal phoneme path is searched through a dynamic programming algorithm to finally obtain the pinyin sequence;

[0010] S5. Construct a prefix tree for the pinyin of social security terms. Each node corresponds to the pinyin sequence of the term. During the bundle search decoding process, the generated path must match the valid branch in the prefix tree. Weight rewards are given to the term matching items in the term library that exist in the path. When the pinyin sequence output by the model partially matches the term library, it is forced to be corrected to a complete term to obtain the text after speech recognition.

[0011] S6. Perform lexical analysis and dependency parsing on the text after speech recognition to construct a dependency graph; use a graph attention mechanism to aggregate information for each node in the dependency graph to obtain the node representation sequence after graph attention network fusion; the node representation sequence after graph attention network fusion is processed by a classifier and a sequence labeler to obtain the intent classification result and entity label sequence.

[0012] Furthermore, step S2 specifically includes:

[0013] Dynamic framing processing is performed on continuous speech signals. The frame start time and frame length are dynamically adjusted according to the speech energy peak. Wavelet packet decomposition is used to divide each frame signal into multiple sub-bands. Frequency band weights are dynamically allocated by calculating the difference between the energy of each sub-band and the global energy. Time-frequency localization processing of the original waveform is performed using a preset wavelet basis function to preserve the key frequency band features of business terms in the speech, resulting in the time-frequency feature matrix after framing. The formula is as follows:

[0014] ,

[0015] in, Indicates the first Segment of speech The time-frequency feature matrix of the frame, This represents the total number of subbands in the wavelet packet decomposition. It is a positive integer. Indicates the first Segment of speech The frequency band weight of each sub-band This represents the original waveform of the i-th speech segment; This represents the residual term or numerical perturbation term in the wavelet packet decomposition process. This represents the Daubechies8 wavelet basis functions. a frame start time dynamically adjusted, a frequency band width dynamically adjusted, a frame start time dynamically adjusted, a local time window centered on the frame start time.

[0016] Further, the first segment of speech is divided into a frequency band weight of the first sub-band is dynamically calculated by a difference between a sub-band energy and a global average energy, and energy of a frequency band in which the social insurance term is located is strengthened.

[0017] Further, the dynamically adjusted frame start time and the dynamically adjusted frequency band width are determined by speech energy peak detection.

[0018] Further, step S3 specifically comprises:

[0019] The time-frequency feature matrix after the framing is subjected to a one-dimensional convolution layer for a multi-scale one-dimensional convolution operation, different step lengths of convolution kernels are used to extract syllable-level acoustic boundary features, then in a local attention mechanism, each frame is only allowed to pay attention to context frames within a fixed range before and after the frame, word-level information is associated by calculating similarity between frames within a window, and features output by the one-dimensional convolution layer and the local attention mechanism are superimposed after a layer normalization layer to form a hidden state that fuses local syllable features and word context; the hidden state is processed by a bidirectional gated recurrent network encoder to obtain a hidden state sequence.

[0020] Further, the bidirectional gated recurrent network encoder is composed of two symmetrical gated recurrent units GRU, which independently run along the positive direction and the reverse direction of the time axis respectively, and each gated recurrent unit GRU internally contains two gating structures of an update gate and a reset gate.

[0021] Further, in step S4, the hidden state sequence is mapped to a phoneme probability distribution through a fully connected layer, an optimal phoneme path is searched through a dynamic programming method, and a difference constraint of adjacent frame hidden states is adopted in a loss function, so that when adjacent phoneme labels are the same, the hidden states are forced to be similar, alignment jitter of continuous similar phonemes is reduced, and a blank label is allowed to skip redundant frames in the alignment process, and finally a frame sequence is mapped to a pinyin sequence; the dynamic programming method is a Viterbi algorithm.

[0022] Further, the training effect of the acoustic feature extraction model is measured by prosody-text alignment, and in the training process of the acoustic feature extraction model, a gradient descent method is used to optimize the training parameters of the acoustic feature extraction model.

[0023] Further, step S5 specifically comprises:

[0024] A prefix tree is constructed according to the pinyin sequence of each term in the domain term library; during the beam search decoding process, the generated path must match the valid branch in the prefix tree, and the path is scored by a score function, so as to realize the enhancement of term matching and the optimization of path selection, and the formula is as follows:

[0025] ,

[0026] represents the pinyin sequence up to the step i; represents the path score up to the step i; represents the hidden state of the i th frame; represents the probability of the current frame phoneme label under the given hidden state is a term matching reward factor; represents a set composed of all legal term pinyin sequences in the term prefix tree; represents 1 when the pinyin sequence up to the step i is a legal term pinyin sequence in the term prefix tree, otherwise 0; when the pinyin sequence output by the beam search decoding partially matches the term library, it is forced to be corrected to the complete term, and the text after speech recognition is obtained.

[0027] Further, the step S6 specifically comprises:

[0028] performing morphological analysis and dependency syntax analysis on the text after speech recognition to construct a dependency graph ; wherein, represents a node corresponding to each word item; is a dependency edge, representing the syntactic dependency between words; each edge is accompanied by a dependency label ; represents the i th word item, represents the i th node, represents the dependency syntax relationship label from the word item to the word item , represents the dependency graph node corresponding to , represents the dependency graph node corresponding to , represents the dependency graph node corresponding to ​​​​​​Nodes with dependency relationship; a graph attention mechanism is used to aggregate information of each node in the dependency graph to obtain a node representation sequence fused by the graph attention network; the node representation sequence fused by the graph attention network is input into a classifier for processing to obtain a graph classification result; the node representation sequence fused by the graph attention network is input into a sequence labeler, and each word is identified using a BIO label form to obtain an entity label sequence; the sequence labeler uses a conditional random field sequence labeler.

[0029] The present application has the advantages that:

[0030] The present application combines dynamic frame division driven by energy peak value and wavelet packet decomposition, adaptively adjusts frame length and frequency band weight, effectively enhances the speech features in the frequency band where the social security terms are located, and overcomes the signal drift problem under the influence of speech rate change and dialect; compared with the global attention model which is too sensitive to irrelevant words, the present application extracts syllable boundary features using multi-scale one-dimensional convolution, and combines the attention mechanism in the local window to effectively capture the word context, and realizes accurate differentiation of homophonic terms; the present application uses a hidden state similarity constraint term in CTC to enhance the alignment robustness of the model to dialectal reading and tone change, and still maintains the stable mapping of speech frames and phoneme sequences under strong accent conditions; in view of the problem that the terms in the social security business are fixed and professional, the present application constructs a term pinyin prefix tree, combines path constraints and term reward functions in the beam search decoding process, and improves the recognition accuracy and integrity of long-tail professional terms under low confidence; the present application uses a graph attention mechanism to fuse dependency relationships for semantic enhancement modeling, significantly improves the processing capacity for syntactic nesting, omission and ambiguity, and realizes accurate intent classification and entity parameter extraction. BRIEF DESCRIPTION OF DRAWINGS

[0031] The accompanying drawings are included to provide a further understanding of the present application, and constitute a part of the specification, illustrate the present application together with the embodiments thereof, and explain the present application, but do not constitute a limitation of the present application.

[0032] Figure 1 A step flowchart of the method of the present application;

[0033] Figure 2 Performance comparison of different frame division methods under speech rate change;

[0034] Figure 3 Time-frequency feature distribution of the traditional frame division method and the dynamic frame division of the present application;

[0035] Figure 4 Acoustic feature extraction model structure diagram of the present application;

[0036] Figure 5 Performance comparison of different model structures in the social security speech recognition task;

[0037] Figure 6 Confusion matrix comparison of the method of the present application and the traditional method;

[0038] Figure 7 Alignment stability of different CTC loss functions under dialectal connection;

[0039] Figure 8 Comparison of recognition effects of different decoding methods on long terms. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0041] Embodiment 1

[0042] In this embodiment, as shown in the accompanying drawings, Figure 1 The present application provides a voice interaction and data processing method for social security business question and answer digital staff, and the specific steps include:

[0043] S1. Obtain the natural language voice signal and text data input by the user.

[0044] S2. Perform dynamic frame processing on the natural language voice signal to obtain a frame processed time-frequency feature matrix.

[0045] Specifically, in the social security business, the voice data contains non-stationary signals and narrow-band features of business terms, such as non-stationary signals caused by changes in speech rate and pauses, and the high-frequency band of the consonant of the business term “pension” belongs to a narrow band;

[0046] The traditional MFCC (Mel-Frequency-Cepstral-Coefficients, Mel-frequency cepstral coefficients) feature uses fixed 20-40 ms frame processing, which causes loss of inter-frame information when the speech rate is fast, and cannot highlight the key frequency band of the social security term;

[0047] The present application performs dynamic frame processing on the continuous voice signal, dynamically adjusts the frame start time and frame length according to the voice energy peak, avoids the speech rate sensitivity problem caused by fixed frame processing, and uses wavelet packet decomposition to divide each frame signal into multiple sub-bands, dynamically allocates the frequency band weight by calculating the difference between the energy of each sub-band and the global energy, and uses a preset wavelet basis function to perform time-frequency localization processing on the original waveform, thereby retaining the key frequency band features of the business term in the voice, and is expressed as:

[0048] ,

[0049] In the formula, is the i-th segment voice The time-frequency feature matrix of the i-th segment voice characterizes the frequency domain features of the voice signal in a specific frame; is the total number of subbands of wavelet packet decomposition, which decomposes the voice signal into multiple frequency bands, and the default ; is a positive integer; is the frequency band weight of the i-th segment voice subband , which is dynamically calculated by the difference between the subband energy and the global average energy, and strengthens the energy of the frequency band where the social security term is located, such as 200-4000Hz, and suppresses irrelevant frequency bands, and the calculation method of the frequency band weight of the i-th segment voice

[0050] subband is expressed as:

[0051] is an exponential function with a natural constant as the base; is an energy smoothing coefficient, such as ; is the instantaneous energy of the i-th segment voice in the i-th subband

[0052] , and the calculation method is expressed as:

[0053] , is a positive integer; is the half-window width, which controls the range of local energy calculation, such as ; is the global average energy of all subbands of the i-th

[0054] segment voice , which strengthens the frequency band where the business term is located, and the calculation method is expressed as:

[0055] is the original waveform of the i-th segment voice ; is a residual term or a numerical perturbation term in the wavelet packet decomposition process, which is used to improve numerical stability and noise resistance, such as, subject to Gaussian noise with mean is Daubechies 8 wavelet base function, covering the key frequency band of social security terms, the reason for selecting this wavelet base function is its high resolution in the frequency band of 200-4000Hz, which is suitable for the frequency band distribution of social security terms, such as the initial high frequency characteristics of "premium base"; is the dynamically adjusted frame start time, determined by energy peak detection, and the calculation method is represented as:

[0056] ,

[0057] wherein, is the maximum allowed frame length, such as 50ms; is the preset lightweight meta-learning network; is the end time of the previous frame. represents finding the time point that makes a certain function reach the maximum value in the time interval ; is the dynamically adjusted frequency band width, determined by energy peak detection, and the calculation method is represented as:

[0058] ,

[0059] According to the adaptive expansion or contraction of the frequency band according to the energy difference of the sub-band, the time-frequency focusing of the energy significant area is enhanced, such as the long tail syllable in the term "deemed premium payment period"; is the preset lower boundary of the frequency band boundary of the th sub-band; is the preset upper boundary of the frequency band boundary of the th sub-band; is the hyperbolic tangent function; represents the local time window centered on the frame start time ;

[0060] It should be noted that, characterizes the th segment of speech in the th frequency band, and the local time domain weighting value based on the frame start time achieves focusing and amplification of the key time-frequency area, such as preserving and focusing and amplifying the characteristics of the frequency band where terms such as "pension" and "premium base" are located in the speech;

[0061] It should also be noted that the conventional dynamic framing relies on energy peak detection, which is not adaptive to non-stationary signals, such as dialect intermittent pronunciation, and the preset lightweight meta-learning network predicts the initial values of the optimal frame length and frequency band weight in real time before framing through the pre-trained meta-network, i.e., the input of the pre-trained meta-network is the short-time Fourier transform spectrum of the original speech signal, and the output includes the frame length ms, the initial value of the frequency band weight of the th segment of speech, and the initial value of the frequency band weight of the Initial value of the band weight of a sub - band , thereby achieving adaptive dialect and speech rate changes, and reducing false triggers of energy detection;

[0062] It should also be noted that the pre - trained meta - network can be a combined network of a 3 - layer CNN and GRU (Gated - Recurrent - Unit). Applying meta - learning to frame parameter prediction and replacing traditional signal processing methods helps to improve the context - awareness ability of dynamic framing.

[0063] S3. The framed time - frequency feature matrix is processed by an acoustic feature extraction model to obtain a hidden state sequence; as Figure 4 shown, the acoustic feature extraction model includes a one - dimensional convolutional layer, a local attention mechanism, a layer normalization layer, and a bidirectional gated recurrent network encoder.

[0064] Specifically, the framed time - frequency feature matrix undergoes multi - scale one - dimensional convolution operations through a one - dimensional convolutional layer, using convolutional kernels with different strides to extract syllable - level acoustic boundary features. Then, in the local attention mechanism, each frame is restricted to only focus on the context frames within a fixed range before and after it, and word - level information is associated by calculating the similarity between frames within the window. The features output by the one - dimensional convolutional layer and the local attention mechanism are superimposed after passing through the layer normalization layer to form a hidden state that combines local syllable features and word context, and the formula is expressed as:<00,00249>

[0065] ,

[0066] In the formula, is the hidden state of the th segment of speech at the th layer. Among them, for the hidden state of the​​​​​​​​​​​​​​​​​​​​​​​Layer-wise local attention mechanism, the context window of 10 frames before and after the current frame is limited, and the calculation method is represented as:

[0067] ,

[0068] The total number of layers of the local attention mechanism can be set to 3; is a value vector; is a query vector; is a key vector; is the transpose of; is the dimension of the attention mechanism; is a mask matrix, the elements of which force the model to only pay attention to the local context, such as cross-word dependencies in "endowment insurance"; is a Softmax function; is the query vector of the first layer, and the calculation method is represented as:

[0069] ,

[0070] is the trainable query weight matrix of the first layer; is the key vector of the first layer, and the calculation method is represented as:

[0071] ,

[0072] is the trainable key weight matrix of the first layer; is the value vector of the first layer, and the calculation method is represented as:

[0073] ,

[0074] is the trainable value weight matrix of the first layer.

[0075] It should be noted that represents a local syllable feature extraction method, which captures acoustic boundaries of different granularities such as syllable starting points through multi-scale convolution kernels, and realizes acoustic differentiation of homophonic terms such as differentiation of "supplement" and "supplement";

[0076] It should also be noted that represents a word-level attention enhancement method, which correlates word-level context such as cross-word dependencies in "cumulative payment years" through similarity calculation within a local window;

[0077] It also needs to be explained that the synergy of multi-scale convolution and local attention is not simply stacked, but through step selection and window restriction, the complementary enhancement of syllable-level to word-level features is realized.

[0078] After the fusion of convolution and local attention, the hidden state is input into the bidirectional gated recurrent network encoder, and the hidden state sequence is output to enhance the context modeling capability;The bidirectional gated recurrent network encoder is composed of two symmetrical GRUs, which run independently along the positive and negative directions of the time axis, process the input sequence of each time step, and each GRU unit contains update gate and reset gate two gating structures for controlling information retention and forgetting.

[0079] S4. The hidden state sequence is mapped to a phoneme probability distribution through a fully connected layer, and the optimal phoneme path is searched through a dynamic programming algorithm to finally obtain a pinyin sequence;

[0080] Social security terms usually have problems such as connected reading and dialectal tone change, such as "personal account" pronunciation "individual silver account", which needs a robust alignment mechanism to align the pronunciation with the translated text;

[0081] The traditional CTC (Connectionist-Temporal-Classification) assumes independence between frames, resulting in high alignment error rate for consecutive similar phonemes, such as the similar pronunciation of "sh" and "s".

[0082] The application maps the hidden state sequence output by the bidirectional gated recurrent network encoder to a phoneme probability distribution, searches for the optimal phoneme path through a dynamic programming algorithm, and adopts a difference constraint of adjacent frame hidden states in the loss function, forces the hidden states to be similar when the adjacent phoneme labels are the same, reduces the alignment jitter of consecutive similar phonemes, and allows the blank label to skip the redundant frame during the alignment process. Finally, the frame sequence is mapped to a pinyin sequence, such as aligning "[j][i][a][o]" to "jiao";

[0083] The training effect of the acoustic feature extraction model is measured by the pros and cons of pronunciation-text alignment, and the training parameters of the acoustic feature extraction model are optimized by using gradient descent method in the training process of the acoustic feature extraction model. The loss calculation method of gradient descent method in each iteration is represented as:

[0084] ,

[0085] In the formula, The CTC loss function of dynamic transition constraint is used to minimize the loss function, so that the acoustic feature extraction model learns to map consecutive speech frames to accurate and stable phoneme sequences, and improves the alignment robustness of connected reading, tone change and other phenomena; total number of speech frames; total number of speech frames; phoneme label of the t-th frame; total number of speech frames; phoneme label of the t-th frame; total number of speech frames; encoder output hidden state sequence of the segment speech, output by a bidirectional gated recurrent network encoder; denotes a given hidden state sequence the t-th frame frame phoneme label probability; denotes a logarithm function, with a default base of 10; is a transition penalty coefficient, such as ; is an indicator function, denotes 1 if the adjacent phoneme labels are different, and 0 otherwise; is an L2 norm; is a hidden state of the t-th frame, representing the acoustic feature of the current frame; is a hidden state of the t-th frame.

[0086] It should be noted that, the role of is to force the hidden states of adjacent same labels to be similar, such as constraining the difference between the hidden states of “ge” and “ren” in the connected reading “personal account”, solving the error alignment caused by dialectal tone, such as the tone confusion of “ji” and “ji”.

[0087] The CTC loss function based on dynamic transition constraint, the specific steps of pronunciation-text alignment are as follows:

[0088] 1) generating phoneme probability distribution

[0089] hidden state mapping each hidden state to output phoneme probability distribution covering all phoneme categories, including 20 phonemes and a blank label, through a fully connected layer.

[0090] 2) dynamic programming search for optimal alignment path

[0091] The optimal path is calculated using a dynamic programming method to find the optimal mapping path from the frame sequence to the phoneme sequence, solving the problem of mismatch between pronunciation and text length, and allowing the blank label to skip repeated frames, the dynamic programming method can use Viterbi algorithm, the specific steps are as follows:

[0092] 2.1) initialize the dynamic programming matrix

[0093] A two-dimensional matrix is created, with rows representing the positions of target phonemes and columns representing the time steps of speech frames, and each position in the matrix stores the maximum cumulative probability from the starting point to that position.

[0094] Initially, only starting from a blank label or the first phoneme is allowed, and the probabilities of the remaining positions are set to invalid values.

[0095] 2.2) Recursively calculate path scores

[0096] Perform operations of continuing the current phoneme, transferring to a new phoneme, or inserting a blank label, specifically:

[0097] If the speech features of the current frame belong to the same phoneme as the previous frames, the probability path of continuing the current phoneme is allowed, allowing phoneme repetition.

[0098] If a speech feature change is detected, transfer to a new phoneme, i.e., transfer to the next phoneme and update the path probability.

[0099] Insert a blank label: If the current frame is silent or redundant, such as a pause at the beginning or end of a sentence, insert a blank label to skip the frame and not occupy the phoneme position.

[0100] 3) Hidden state continuity constraint

[0101] The hidden state continuity constraint includes a penalty for adjacent phonemes being different and a tolerance for adjacent phonemes being the same, specifically:

[0102] The penalty for adjacent phonemes being different means that if adjacent frames are assigned to different phonemes, check the difference in acoustic features (hidden state) between the two frames. If the difference is too large, reduce the score of the path and prefer the path with smooth acoustic feature transition.

[0103] The tolerance for adjacent phonemes being the same means that if adjacent frames are the same phoneme, no penalty is applied and reasonable repetition is allowed.

[0104] It should be noted that the way to check the difference in acoustic features (hidden state) between the two frames is calculated by the dynamic transition constrained CTC loss function, i.e., if the dynamic transition constrained CTC loss function is greater than a preset threshold, it means that the difference is too large, otherwise, it means that there is no difference.

[0105] It should also be noted that the dynamic transition constrained CTC loss function In the alignment process, the difference constraint term of adjacent frame hidden states is used When the adjacent phoneme labels are the same, the hidden states are forced to be similar, thereby reducing the alignment jitter of consecutive similar phonemes and making the alignment process more robust, effectively handling problems such as connected reading and dialectal tone variation.

[0106] 4) Backtracking to generate the optimal path

[0107] Starting from the final frame of the last phoneme, the highest scoring path in the entire matrix is traced back in reverse;

[0108] Further, the repeated phonemes and blank labels are merged to generate the final phoneme sequence.

[0109] In one embodiment, the speech frame may be incorrectly mapped as "ge-yin-zheng-hu", but through the hidden state constraint, the acoustic feature difference between "ge" and "yin" is forced to decrease, and the path is corrected to "ge-ren-zheng-hu", and finally aligned to the standard term "personal account".

[0110] In this embodiment, for the speech frame sequence "[sil][j][i][a][o][sil]", it can be aligned to the pinyin "jiao".

[0111] S5. Construct a social security term pinyin prefix tree, each node corresponds to the pinyin sequence of the term, in the beam search decoding process, limit the generated path to match the valid branch in the prefix tree, and reward the weight of the term library matching item existing in the path, when the pinyin sequence output by the model partially matches the term library, force it to be corrected to the complete term, and get the text after speech recognition.

[0112] Specifically, the social security business has strong domain dictionary constraints, such as "viewed as payment years" as a fixed term, which needs to avoid interference from general words; the conventional beam search method tends to high-frequency general words, which is easy to cause incorrect recognition, such as "participate" being misrecognized as "lunch bag";

[0113] The present application pre-constructs a social security term pinyin prefix tree, each node corresponds to the pinyin sequence of the term, in the beam search decoding process, limit the generated path to match the valid branch in the prefix tree, and reward the weight of the term library matching item existing in the path, when the pinyin sequence output by the model partially matches the term library, force it to be corrected to the complete term, such as "yang-lao-jin" to "pension", specifically including:

[0114] 1) Term prefix tree construction

[0115] The prefix tree is constructed for the pinyin sequence of each term in the domain term library, such as domain terms "pension", "payment base", etc. The node stores prefix information, such as "yang→lao→jin" representing "pension".

[0116] 2) Path constraint during beam search decoding

[0117] The extension of each path in the beam search is limited to the valid pinyin path on the prefix tree, for example, the current path is "yang-lao", which is only allowed to continue to extend to "jin" and other legal branches;

[0118] At the same time, the path is scored by the scoring function to realize the enhancement of term matching and the optimization of path selection, which is expressed as:

[0119] ,

[0120] In the formula, is the pinyin sequence of the first step; is the path score of the first step, which is the weighted sum of the model log probability and the term reward; is the hidden state of the first frame; represents the probability of the current frame phoneme label under the given hidden state ; is the term matching reward factor, which enhances the retention probability of the term in the case of low confidence, such as , so that the "premium base" is still preferentially output as the standard term even if the confidence is low; is a set composed of all legal term pinyin sequences in the term prefix tree, and the social security term pinyin sequence is pre-constructed, such as "yang-lao-jin"→"old-age pension", which constrains the decoding path; represents that the value is 1 when the pinyin sequence of the first step is a legal term pinyin sequence in the term prefix tree, otherwise it is 0.

[0121] 3) Term forced correction

[0122] If the output pinyin sequence is "yang-lao-jin", even if the model confidence is 0.7, the sequence will be corrected to "old-age pension" due to the matching of the term "old-age pension" in the prefix tree.

[0123] In one embodiment, taking a 3-second speech "May I ask how much is the premium base of the flexible employment personnel old-age insurance?" as an example, the processing manner is as follows:

[0124] 1) The 3-second speech is divided into 150 dynamic frames, and the dynamic framing manner is adopted, and the frame length is 20-50 ms;

[0125] Then, after 80 frequency band feature extraction and 512-dimensional coding processing, it is mapped into a pinyin sequence, and the standard term is output through the prefix tree;

[0126] 2) Generate a time-frequency matrix with a dimension of , i.e. 80 frequency bands;

[0127] 3) Output hidden state by acoustic feature extraction model processing , dimension ;

[0128] 4) Perform CTC alignment to map consecutive frames to a sequence of pinyin, such as "qing-wen-ling-huo-jiu-ye-ren-yuan-yang-lao-jin-jiao-fei-ji-shu-shi-duo-shao";

[0129] 5) Perform term decoding and correct to "flexible employment personnel endowment insurance premium base" through a prefix tree.

[0130] S6. Perform morphological analysis and dependency syntax analysis on the text after speech recognition, and construct a dependency graph; use a graph attention mechanism to aggregate information for each node in the dependency graph to obtain a node representation sequence after graph attention network fusion; process the node representation sequence after graph attention network fusion through a classifier and a sequence labeler to obtain an intent classification result and an entity label sequence.

[0131] Specifically,

[0132] Step 1, syntax dependency graph construction and embedding initialization

[0133] Let the text after speech recognition be , perform morphological analysis and dependency syntax analysis on the text to construct a dependency graph ; wherein, represents a node corresponding to each word; is a dependency edge, representing the syntactic dependency between words, such as subject-predicate, verb-object, and modification relationships; each edge is associated with a dependency label. . is the th word item, i.e., the th word in the text, representing the original form of the word; is the th node, representing the node in the graph corresponding to the word item , used to construct a graph attention structure; is a dependency syntax relationship label from word to , such as "subject-predicate", "modification", "object complement", etc.; is the th word item, which is an element in the original recognized text; is the dependency graph node corresponding to ; is a node that has a dependency relationship with , i.e., edge The target node in; For the first The initial representation of each node in the graph attention network, i.e., the word embedding vector, is obtained, for example, through the Word2Vec method.

[0134] Furthermore, for each word Initialize its representation vector as word embedding This invention uses pre-trained word vectors, such as Word2Vec or BERT embeddings.

[0135] Step 2: Attention Calculation for Syntactic Augmentation Graph

[0136] A graph attention mechanism is employed to aggregate information at each node in the dependency graph, enabling each node to incorporate its syntactic context features. This allows the model to not only focus on local context but also integrate important information from dependency syntax within social security terminology, such as the noun-modifier structure in "pension insurance" and "contribution base," thereby enhancing ambiguity resolution and intent recognition capabilities. This can be represented as follows:

[0137] ,

[0138] In the formula, For the first The word for the first Attention weights for each word; for Activation function; This is a learnable attention vector; for Transpose of; For a shared linear mapping matrix; For the first The first layer in the graph attention network Representation of each node; For the first The first layer in the graph attention network Representation of each node; For the first The first layer in the graph attention network Representation of each node; This represents a vector concatenation operation; For Dependency Tags Embedded representation; It is the ReLU activation function; In order to be with the first All nodes adjacent to a given node.

[0139] It should be noted that the syntax enhanced graph attention mechanism combined with the dependency relationship can improve the intent recognition accuracy. Compared with the traditional Transformer structure, the explicit representation of the syntax dependency structure can be retained, and the problems such as common phrase nesting, omission, ambiguity and the like in the social security business sentence can be solved, such as distinguishing the intent difference between “query payment” and “payment query history”, and accurately identifying the parameter entities before and after the sequence.

[0140] Step 3, intent classification and parameter identification

[0141] The node representation sequence fused by the graph attention network The input classifier and the sequence labeler, wherein is the embedding representation of the first word in the first layer of the graph attention network, used for the first word or the CLS identifier, is the embedding representation of the second word in the first layer of the graph attention network, is the embedding representation of the second word in the first layer of the graph attention network, is the embedding representation of the second word in the first layer of the graph attention network, is the embedding representation of the second word in the first layer of the graph attention network. The intent classification is obtained by pooling the sentence vector, and is represented as:

[0142] ,

[0143] In the formula, is the intent classification result, such as query payment period, change of insurance information, verification of account status and the like; is the node representation sequence fused by the graph attention network; is the weight parameter of the trainable neural network classification layer; is the bias parameter of the trainable neural network classification layer.

[0144] Further, the parameter extraction adopts a conditional random field sequence labeler, and uses a BIO label form to identify each word as an entity, and the output result is an entity label sequence, such as name, ID number, payment year, insurance type and the like.

[0145] In one embodiment, for the input text “I want to query the 2022 flexible employment personnel old-age insurance payment base”, the intent classification result is “query old-age payment base”, and the parameter extraction result is: time parameter “2022”, insurance parameter “old-age insurance”, and crowd type “flexible employment personnel”.

[0146] Embodiment 2

[0147] In this embodiment, in order to verify the robustness of the dynamic framing technology in different speech speed scenes, the phoneme recognition accuracy of the traditional fixed framing method (red dashed line) and the dynamic framing method (blue solid line) of the present application in different speech speeds is compared, such as​​Figure 2 As shown in the figure, the experimental results show that as the speech rate increases, the accuracy of the fixed framing method drops significantly due to the loss of inter-frame information, especially in high-speed speech segments. In contrast, dynamic framing, through adaptive frame length adjustment and frequency band weight allocation, makes the accuracy curve smoother and maintains high stability in high-speed segments. The trend of accuracy change on the vertical axis of the figure intuitively reflects that the method effectively alleviates the speech rate sensitivity problem through the dynamic framing strategy. Its advantage lies in the joint optimization of speech energy peak and wavelet packet energy, avoiding feature loss of traditional methods under non-stationary signals.

[0148] Example 3

[0149] In this embodiment, as Figure 3 The heatmap compares the time-frequency feature distribution of traditional framing methods with that of the present invention's dynamic framing, verifying the ability of dynamic framing to focus on key frequency bands of social security business terms. The vertical axis of the heatmap represents the frequency band number, corresponding to the actual frequency band range, while the horizontal axis represents the speech frame number, and the color depth reflects the energy intensity. Traditional method (a) suffers from dispersed energy distribution due to fixed framing, especially at fast speech rates, where the key frequency band (the 200-4000Hz region marked by the red dashed line) has blurred energy and overflow boundaries. In contrast, the present method (b) dynamically adjusts the frame length and frequency band weights to form a high-energy focusing area (dark block) within the key frequency band, and significantly reduces residual noise (light-colored area). Experimental results show that the present invention effectively enhances the feature extraction of terminology-related frequency bands through wavelet packet decomposition and dynamic frequency band allocation, solving the feature loss problem of traditional methods when the speech rate changes.

[0150] Example 4

[0151] In this embodiment, as Figure 5 As shown in the bar chart, by comparing the performance differences of different model structures in the social security speech recognition task, the four models from left to right are recurrent neural networks, global attention models, multi-scale convolution combined with attention models, and the proposed method. Traditional recurrent neural networks have the lowest performance due to their weak long-term dependency modeling ability (first red bar on the left). Although the global attention model introduces context awareness, it is easily interfered with by irrelevant interjections (second blue bar on the left). The proposed method captures syllable boundary features through multi-scale one-dimensional convolution and combines local attention mechanisms to constrain the context window, which is represented by the highest bar in the bar chart (first gold bar on the right). Moreover, the error bar range is the smallest, indicating that the model has the best stability. This proves that multi-granularity feature fusion and local attention collaboration can significantly distinguish homophones and improve the acoustic modeling accuracy in social security scenarios.

[0152] Example 5

[0153] In this embodiment, as Figure 6As shown in the figure, the confusion matrix is used to evaluate the ability of the model to distinguish homophonic terms. Compared with the traditional recurrent neural network and the multi-scale convolution-attention fusion model of the present application, each row of the matrix represents the true term category, each column represents the model prediction result, and the dark block on the diagonal line represents the correct recognition rate. The traditional model (a) has a large area of cross error (non-diagonal dark block) between easily confused terms, especially for terms with similar syllables. The present method (b) captures the syllable boundary features through multi-scale convolution and combines local attention with word-level context, which significantly improves the diagonal line accuracy and almost eliminates the cross error. The experimental results show that the feature fusion mechanism of the present method can decouple acoustic details and semantic context, effectively solving the ambiguity problem of homophonic terms.

[0154] Embodiment 6

[0155] In this embodiment, the alignment stability of the traditional connection time sequence classification loss function and the constraint loss function of the present application in the dialect connected reading scene is compared through the error range diagram, as shown in Figure 7 As shown in the figure, the horizontal axis is the dialect accent intensity level, and the vertical axis is the mean square error of the hidden state of the adjacent speech frame, reflecting the alignment jitter degree. The error range (shaded area) of the traditional method (orange curve) rapidly expands with the increase of dialect intensity, indicating its sensitivity to tonal variation and connected reading. However, the error mean and fluctuation range of the present method (blue curve) are significantly reduced by constraining the similarity of the hidden states of adjacent same phonemes. The blue area in the figure is always narrow and close to the horizontal axis, indicating that the present method can maintain the smoothness of the alignment path even under strong dialect interference, verifying the key role of dynamic transition constraints in improving the robustness of pronunciation-text alignment.

[0156] Embodiment 7

[0157] In this embodiment, the recognition effect of term enhancement decoding on long terms is analyzed by combining box plots and line graphs, and the field term constraint decoding method of the present application is compared with the ordinary beam search, as shown in Figure 8 As shown in the figure, the box height of the box plot reflects the stability of the recognition rate distribution, and the line trend represents the performance change as the term length increases. The box of the traditional method (orange) significantly expands and moves downward as the term length increases, indicating that its recognition rate fluctuates greatly and is easily affected by general words. However, the box of the present method (blue) is always compact and the line is gently downward, especially in long terms, it still maintains high recognition rate. The upper edge of the blue box in the figure is almost coincided with the line, indicating that the present technology can stably correct low confidence paths and force to output terms that meet the field specifications. The experimental results show that term enhancement decoding significantly improves the recognition reliability of long-tail professional expressions in social security business.

[0158] Finally, it should be noted that the above only describes the preferred embodiments of the present application and is not intended to limit the present application. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art will appreciate that the technical solutions described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalent ones. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A voice interaction and data processing method for a social security business-oriented question and answer digital staff, characterized in that, The method comprises the following steps: S1. obtaining natural language voice signals and text data input by a user; S2. performing dynamic frame processing on the natural language voice signals to obtain a time-frequency feature matrix after frame processing; S3. the time-frequency feature matrix after frame processing is processed by an acoustic feature extraction model to obtain a hidden state sequence; the acoustic feature extraction model comprises a one-dimensional convolution layer, a local attention mechanism, a layer normalization layer, and a bidirectional gated recurrent network encoder; S4. the hidden state sequence is mapped to a phoneme probability distribution through a full connection layer, and an optimal phoneme path is searched through a dynamic programming algorithm, and finally a pinyin sequence is obtained; S5. a social security term pinyin prefix tree is constructed, each node corresponds to a pinyin sequence of a term, in the beam search decoding process, the generated path must match the valid branch in the prefix tree, and the term library matching items existing in the path are rewarded, when the pinyin sequence output by the model partially matches the term library, the model is forced to correct it to a complete term, and a text after voice recognition is obtained; S6. performing morphological analysis and dependency syntax analysis on the text after voice recognition to construct a dependency graph; a graph attention mechanism is used to aggregate information of each node in the dependency graph to obtain a node representation sequence after graph attention network fusion; the node representation sequence after graph attention network fusion is processed by a classifier and a sequence labeler to obtain an intent classification result and an entity label sequence.

2. The voice interaction and data processing method for social security business-oriented Q&A digital staff according to claim 1, characterized in that, Step S2 specifically comprises: The continuous voice signal is dynamically frame processed, the frame start time and the frame length are dynamically adjusted according to the voice energy peak value, each frame signal is divided into multiple subbands by wavelet packet decomposition, the frequency band weight is dynamically allocated by calculating the difference between the energy of each subband and the global energy, the original waveform is processed by using a preset wavelet basis function to realize time-frequency localization, the key frequency band features of the business terms in the voice are reserved, and a time-frequency feature matrix after frame processing is obtained, and the formula is as follows: ， wherein, represents the segment of speech the time-frequency feature matrix of the frame, represents the total number of subbands of the wavelet packet decomposition, is a positive integer, represents the segment of speech the bandwidth weight of the i-th subband, represents the original waveform of the i-th segment of speech; represents the residual term or numerical perturbation term in the wavelet packet decomposition process, represents the Daubechies 8 wavelet basis function, represents the dynamically adjusted frame start time, represents the dynamically adjusted frequency band width, represents the local time window centered at the frame start time .

3. The voice interaction and data processing method for social security business-oriented Q&A digital staff according to claim 2, characterized in that, Step S3 specifically comprises: The time-frequency feature matrix after frame processing is subjected to a multi-scale one-dimensional convolution operation through a one-dimensional convolution layer, different step length convolution kernels are used to extract syllable level acoustic boundary features, then in the local attention mechanism, each frame can only pay attention to the context frames within a fixed range before and after it, the word level information is associated by calculating the similarity between the frames in the window, the features output by the one-dimensional convolution layer and the local attention mechanism are superimposed after layer normalization to form hidden states that fuse local syllable features and word context; the hidden states are processed by a bidirectional gated recurrent network encoder to obtain a hidden state sequence.

4. The voice interaction and data processing method for social security business-oriented Q&A digital staff according to claim 3, characterized in that, In step S4, the hidden state sequence is mapped to a phoneme probability distribution through a full connection layer, an optimal phoneme path is searched through a dynamic programming method, and at the same time, a difference constraint of adjacent frame hidden states is adopted in a loss function, when the adjacent phoneme labels are the same, the hidden states are forced to be similar, the alignment jitter of continuous similar phonemes is reduced, and in the alignment process, a blank label is allowed to skip redundant frames, finally the frame sequence is mapped to a pinyin sequence; the dynamic programming method is a Viterbi algorithm.

5. The voice interaction and data processing method for social security business-oriented Q&A digital staff according to claim 4, characterized in that, Step S5 specifically comprises: According to the pinyin sequence of each term in the domain term library, a prefix tree is constructed; in the beam search decoding process, the generated path must match the valid branch in the prefix tree, the path is scored by a score function, the enhancement of term matching and the optimization of path selection are realized, and the formula is expressed as follows: ， in, Indicates up to the number The pinyin sequence of steps; Indicates up to the number Path score for each step; Indicates the first The hidden state of a frame; Represents a given hidden state Below, current frame phoneme tag The probability of; Match reward factors to terms; This represents the set of all legal term pinyin sequences in the term prefix tree; This indicates that as of the end of the period The pinyin sequence of steps The value is 1 if it is a valid term pinyin sequence in the term prefix tree, and 0 otherwise. When the pinyin sequence output by the beam search decoding partially matches the term library, it is forcibly corrected to a complete term to obtain the text after speech recognition.

6. The voice interaction and data processing method for social security business-oriented Q&A digital staff according to claim 5, characterized in that, The step S6 specifically comprises: Text after speech recognition Conduct morphological analysis and dependency syntax analysis to construct a dependency graph ; wherein, represents the node corresponding to each word item; is a dependency edge, representing the syntactic dependency between words; each edge is accompanied by a dependency label ; represents the th word item, represents the th node, represents the dependency syntax relationship label from the word item to the word item , and represents the dependency graph node corresponding to , and represents the node having a dependency relationship with ; a graph attention mechanism is used to aggregate information of each node in the dependency graph to obtain a node representation sequence after graph attention network fusion; the node representation sequence after graph attention network fusion is input to a classifier for processing to obtain a graph classification result; the node representation sequence after graph attention network fusion is input to a sequence labeler, and each word is recognized as an entity using a BIO label form to obtain an entity label sequence; the sequence labeler uses a conditional random field sequence labeler.

7. The voice interaction and data processing method for social security business-oriented Q&A digital staff according to claim 6, characterized in that, The first The band weight of the first The band weight of the first sub-band is dynamically calculated by the difference between the sub-band energy and the global average energy, and the energy of the frequency band where the social security term is located is strengthened.

8. The voice interaction and data processing method for social security business-oriented Q&A digital staff according to claim 7, characterized in that, The dynamic adjustment of the frame start time and the dynamic adjustment of the frequency band width in the step S2 are determined by speech energy peak detection.

9. The voice interaction and data processing method for social security business-oriented Q&A digital staff according to claim 8, characterized in that, In the step S3, the bidirectional gated recurrent network encoder is composed of two symmetrical gated recurrent units GRU, which are independently operated along the positive direction and the reverse direction of the time axis respectively, and each gated recurrent unit GRU internally contains two kinds of gating structures, i.e., an update gate and a reset gate.

10. The voice interaction and data processing method for social security business-oriented Q&A digital staff according to claim 9, characterized in that, In the step S4, the training effect of the pronunciation-text alignment-based acoustic feature extraction model is measured, and in the training process of the acoustic feature extraction model, the training parameters of the acoustic feature extraction model are optimized by using the gradient descent method.

Images