An AI algorithm-based optimized sound recognition acceleration method

Abstract

The application discloses an AI algorithm optimization-based sound recognition acceleration method, and particularly relates to the technical field of speech recognition, and comprises instantaneous audio frame composite feature extraction, dynamic inference configuration strategy generation, instantaneous reconstruction inference network construction, accelerated forward inference and decoding lattice generation, and optimal recognition text sequence output. The application dynamically specifies a macroscopic calculation path for a pickup input signal by calculating the instantaneous features of audio frames, and establishes shielding rules according to real-time speech speed. During inference, the network is instantaneously reconstructed according to the hierarchical regulation mechanism, shallow paths that skip the encoder layer are used for simple signals such as silence or vowels, and deep paths are called for complex consonants. The strategy double-binds the calculation resources and real-time features of signals, thereby significantly reducing the average calculation load and delay of a terminal device when processing real speech streams without sacrificing the processing capacity of complex signals.

Description

Technical Field

[0001] This invention relates to the field of speech recognition technology, and in particular to a method for accelerating speech recognition based on AI algorithm optimization. Background Technology

[0002] The field of speech recognition technology encompasses the process of converting human speech into text or commands. The systematic process begins with capturing the raw audio waveform from a microphone. This waveform signal is preprocessed to extract acoustic features such as Mel-frequency cepstral coefficients, which are then sequentially fed into an acoustic model and a language model. Finally, a decoder combines the results of both models to search for and output the most probable text sequence. With the application of deep neural networks, the accuracy of speech recognition has been greatly improved, but this has also led to a significant increase in the computational complexity of the models.

[0003] In many applications, microphones and their subsequent processing circuits are often integrated into terminal devices with limited computing power, memory bandwidth, and power consumption, such as smart home controllers, wearable devices, or in-vehicle systems. Deploying large recognition models directly on such devices results in unacceptable inference latency and power consumption. Therefore, AI-based methods for accelerating voice recognition have emerged, with the core objective of shortening the inference time of the model on the terminal device. Existing technologies primarily employ static optimization schemes, such as model quantization, model pruning, or knowledge distillation, to build a compact model with a fixed structure before deployment.

[0004] However, such static optimization schemes have inherent limitations. The audio streams captured by microphones in real-world environments have dynamically changing information density, containing numerous silences, simple vowels, and complex consonant bursts. Existing static compact models have a constant computational load when processing these signals and cannot adaptively adjust to the real-time complexity of the signal content. For example, when processing a speech segment captured by a microphone that includes pauses between words, the static model still performs a full network forward propagation when processing the silent frames of the pauses, generating unnecessary computational overhead. Conversely, permanent structural simplifications made to achieve preset acceleration goals weaken the model's ability to handle complex acoustic scenarios. When the microphone captures speech at extremely fast speeds or with heavy accents, the static compact model may experience an increased error rate due to insufficient network capacity. This optimization approach forces the system to make a rigid, global trade-off between computational latency and recognition accuracy, failing to dynamically allocate computational resources based on the real-time signal characteristics captured by the microphone. Summary of the Invention

[0005] The main objective of this invention is to provide a sound recognition acceleration method based on AI algorithm optimization, which can effectively solve the problems mentioned above in the background technology.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A method for accelerating voice recognition based on AI algorithm optimization includes the following steps: S1. Instantaneous audio frame composite feature extraction: Obtain audio signal frames, calculate envelope amplitude and rate of change, combine with shallow network phoneme activation intensity, and splice to generate instantaneous audio frame composite feature vector; S2. Dynamic reasoning configuration strategy generation: Based on the composite feature vector of instantaneous audio frames, query the decision matrix to specify the path pattern, establish the masking rules according to the phoneme boundary calculation, and integrate them into a dynamic reasoning configuration strategy. S3. Instantaneous Reconstruction Inference Network Construction: Based on the dynamic inference configuration strategy, the encoder layer pointed to by the path pattern index is marked as inactive, and the masking rules are applied to the computation graph of the decoder layer to establish the instantaneous reconstruction inference network. S4. Accelerate forward inference and decoding grid generation: Invoke the instantaneous reconstruction inference network to propagate the acoustic features forward. The data stream bypasses the encoder layer according to the non-activation marker. In the decoder, the multi-head attention masking branch is set to zero. The end generates an acoustic distribution to update the decoding graph score and generate an accelerated decoding path grid. S5. Optimal Recognition Text Sequence Output: For the accelerated decoding path grid, backtrack from the highest probability end node to the starting point, detect the word units on the path and combine them in order to obtain the optimized recognition text sequence.

[0007] Preferably, the instantaneous audio frame composite feature vector includes envelope amplitude components, envelope rate components, and phoneme category intensity distribution; the dynamic inference configuration strategy includes computation path pattern identifiers, attention head sparsity masks, and hierarchical skip instructions; the instantaneous reconstruction inference network includes an activated subset of encoder layers, dynamically masked decoder layers, and adjusted inter-layer connections; the accelerated decoding path grid includes a candidate word node set, acoustic transfer scores between nodes, and an optimal path backtracking pointer; and the optimized recognition text sequence includes the final output word sequence, the timestamp of each word, and the global confidence score of the entire sequence.

[0008] Preferably, step S1 includes: The system acquires the input audio signal frame, calculates the arithmetic mean of the absolute values ​​of all signal sample points within the frame to obtain the temporal envelope amplitude, and performs a difference operation between the temporal envelope amplitude of the previous frame and the temporal envelope amplitude of the current frame to obtain the envelope change rate. The system combines the temporal envelope amplitude and the envelope change rate to generate instantaneous acoustic temporal feature values. Based on the input audio signal frame, the amplitude component of the temporal envelope in the instantaneous acoustic temporal feature value is compared with the preset silence energy threshold. If the amplitude component is not lower than the threshold, the Mel frequency cepstral coefficient of the current frame is extracted. The coefficient is used as input to the preset shallow feedforward network to perform forward propagation, obtain the activation value of its output layer, and establish a broad overtone activation intensity vector. The instantaneous acoustic temporal feature value and the broad phoneme activation intensity vector are called, and the two vectors are concatenated end to end according to the preset dimensional order. The maximum and minimum value normalization processing is performed on each component of the concatenated vector to generate the instantaneous audio frame composite feature vector.

[0009] Preferably, step S2 includes: Based on the composite feature vector of instantaneous audio frames, its temporal envelope amplitude component is compared with a preset silence energy threshold. If the comparison result does not represent silence, its envelope change rate component is compared with a steady state threshold. Combined with the maximum value index in the phoneme category activation intensity vector, a preset decision matrix is ​​queried to obtain the macroscopic calculation path index. The acoustic feature vector sequence is monitored, the Euclidean distance between consecutive vectors is calculated and compared with the mutation threshold to determine the phoneme boundary, the average number of frames between recent phoneme boundaries is counted, and based on the comparison result of the average number of frames with the preset duration benchmark, a corresponding binary masking vector is selected from a fixed functional profile index library to establish an attention head sparsity mask. The macro-level computation path index and the attention head sparsity mask are invoked. The macro-level computation path index is used as the pattern identifier of the computation path, and the attention head sparsity mask is used as the structural adjustment parameter of the self-attention computation unit under the path. The two are integrated into a single data structure to generate a dynamic inference configuration strategy.

[0010] Preferably, step S3 includes: Based on the dynamic inference configuration strategy, the computation path pattern index is extracted, and the level identifiers of all encoder layers in the original network structure are traversed. The level identifiers that match the index are included in a set, and the execution status of all encoder layers in the set is marked as disabled so that their internal computation can be skipped in subsequent forward propagation, thus establishing an encoder layer pruning configuration scheme. Based on the dynamic inference configuration strategy, the self-attention head masking rules are extracted and the encoder layer pruning configuration scheme is invoked. All encoder layers that are not marked as disabled are retained in the original network structure. At the same time, the masking rules are applied to the computation graph of each decoder layer. The weight matrix multiplication of the attention heads listed in the rules is skipped through conditional judgment logic. Finally, the retained encoder layers and the modified decoder layers are reconnected to establish an instantaneous reconstruction inference network.

[0011] Preferably, step S4 includes: The instantaneous reconstruction inference network is invoked, and the acoustic features of the original audio signal frame are used as the network input to perform forward propagation. The data flow makes conditional judgments based on the deactivation flags of each encoder layer in the network. When the flag is deactivated, the data flow directly bypasses all the calculation operations such as weight matrix multiplication and activation function contained in the layer, and passes the input tensor of the layer to the next layer to obtain the output tensor of the encoder layer. Based on the output tensor of the encoder layer, it is fed into the decoder part of the instantaneous reconstruction inference network. When multi-head self-attention calculation is performed inside each decoder layer, the built-in masking rules of the network are called. The attention head at the index specified by the rule is calculated as a branch, and all components of its output vector are forced to be set to zero. Finally, the outputs of all branches are concatenated and passed through the classification layer at the end of the network to generate frame-level acoustic posterior probabilities. Based on frame-level acoustic posterior probabilities and by calling the language model to query state transition probabilities, all end nodes in the current decoding graph are traversed. The cumulative score of each node is jointly calculated with the state transition probability and the frame-level acoustic posterior probability to update the original path score and expand to generate new hypothetical nodes and connecting edges. The updated set of all nodes and connecting edges is then integrated to obtain the accelerated decoding path grid.

[0012] Preferably, step S5 includes: For the accelerated decoding path grid, all nodes in the graph data structure are traversed, and all candidate nodes representing the end of speech are selected based on the state identifier built into each node. Then, the cumulative probability scores of each candidate node are extracted one by one and the numerical values ​​are compared. The candidate node with the maximum score is selected to establish the optimal path backtracking starting point. Call the optimal path backtracking starting point, and traverse backwards along the forward backtracking pointer stored inside the node to accelerate the decoding of the path grid until the starting node of the graph is reached. During the traversal, the vocabulary units corresponding to each node on the path are detected in turn, and all detected vocabulary units are stored in a sequence in the order of backward traversal to obtain the reversed vocabulary unit list. Based on the reversed list of word units, the order of all word units stored in the sequence is reversed to restore their temporal relationship in the original speech. Then, a preset text delimiter is called to connect all word units in the reversed sequence in the new order and combine them into a single string to obtain the optimized recognition text sequence.

[0013] Compared with the prior art, the present invention has the following beneficial effects: By parallel computing the temporal envelope and phoneme category of each audio frame, a composite feature vector is generated that characterizes the complexity of the current microphone input signal. This vector is used for real-time decision-making, dynamically assigning a macroscopic computational path to different types of signals captured by the microphone (such as silence, vowels, or plosives). Simultaneously, the system measures the real-time speech rate to establish masking rules for the self-attention computation unit, forming a hierarchical control mechanism. During inference, the entire network structure can be instantaneously reconstructed according to this control mechanism. When the microphone input is a simple vowel or silence, a shallow path skipping multiple encoder layers is used; when the input is a complex consonant, a deeper path is invoked. This strategy, which dually binds computational resources to the instantaneous complexity and speech rate characteristics of the microphone's real-time signal acquisition, avoids unnecessary full network computation when processing low-information-density signal segments. Therefore, without sacrificing complex signal processing capabilities, it significantly reduces the average computational load and latency of the terminal device when processing real-world speech streams. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the overall process of the present invention. Detailed Implementation

[0015] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0016] Example 1, as Figure 1 As shown, a voice recognition acceleration method based on AI algorithm optimization includes the following steps: S1. Instantaneous audio frame composite feature extraction: Obtain the input audio signal frame, calculate the arithmetic mean of the absolute values ​​of all signal sample points in the frame to obtain the temporal envelope amplitude, and calculate the difference between the temporal envelope amplitude of the current frame and the previous frame to obtain the envelope change rate. Extract the Mel frequency cepstral coefficients of the current frame, input the coefficients into a preset shallow feedforward network to obtain the activation intensity of each broad overtone category, and concatenate the temporal envelope amplitude, envelope change rate and activation intensity into a vector to generate an instantaneous audio frame composite feature vector. S2. Dynamic inference configuration strategy generation: Based on the composite feature vector of instantaneous audio frames, its temporal envelope amplitude, envelope change rate and phoneme category activation intensity component are called together to query the preset decision matrix to specify the calculation path mode. At the same time, the inter-frame Euclidean distance of the acoustic feature vector sequence is monitored. When it crosses the mutation threshold, it is determined as a phoneme boundary. The average number of frames between recent boundaries is calculated. Based on this, the masking rules of the self-attention head are established. The calculation path mode and the masking rules are integrated to obtain the dynamic inference configuration strategy. S3. Instantaneous Reconstruction Inference Network Construction: Based on the dynamic inference configuration strategy, the computation path pattern index is extracted. In the complete speech recognition network structure, the specific set of encoder layers pointed to by the index is marked as inactive. The attention head masking rule is extracted and applied to the computation graph of each decoder layer. The weight matrix multiplication operation of the attention heads listed in the rule is skipped through the conditional judgment logic to establish the instantaneous reconstruction inference network. S4. Accelerated Forward Inference and Decoding Grid Generation: The instantaneous reconstruction inference network is invoked, and the acoustic features of the original audio signal frame are used as input to perform forward propagation. The data stream bypasses the specified encoder layer according to the non-activation marker. When performing multi-head self-attention calculation inside the decoder layer, the output of the calculation branch specified by the masking rule is set to zero. The acoustic probability distribution is generated by the classification layer at the end of the network. The cumulative score of each hypothetical path in the decoding graph is updated according to the distribution and the transition probability of the language model state, and the accelerated decoding path grid is generated. S5. Optimal Recognition Text Sequence Output: For the accelerated decoding path grid, select the node with the highest cumulative probability score from the nodes representing the end state of speech in the graph as the backtracking starting point, traverse backward along its forward pointer to the starting node, detect the word units corresponding to each node on the path in turn, arrange and combine all the detected word units in the traversal order to obtain the optimized recognition text sequence.

[0017] The instantaneous audio frame composite feature vector includes envelope amplitude components, envelope rate components, and phoneme category intensity distribution. The dynamic inference configuration strategy includes computation path pattern identifiers, attention head sparsity masks, and hierarchical skip instructions. The instantaneous reconstruction inference network includes an activated subset of encoder layers, dynamically masked decoder layers, and adjusted inter-layer connections. The accelerated decoding path lattice includes a candidate word node set, acoustic transfer scores between nodes, and optimal path backtracking pointers. The optimized recognition text sequence includes the final output word sequence, the timestamp of each word, and the global confidence score of the entire sequence.

[0018] Step S1 includes: The system acquires the input audio signal frame, calculates the arithmetic mean of the absolute values ​​of all signal sample points within the frame to obtain the temporal envelope amplitude, and performs a difference operation between the temporal envelope amplitude of the previous frame and the temporal envelope amplitude of the current frame to obtain the envelope change rate. The system combines the temporal envelope amplitude and the envelope change rate to generate instantaneous acoustic temporal feature values. Furthermore, the input audio signal frames are acquired, with each frame's duration set to 25 milliseconds. At a 16kHz sampling rate, this corresponds to 400 signal sample points. First, the arithmetic mean of the absolute values ​​of all 400 signal sample points within the frame is calculated to obtain the time-domain envelope amplitude. This calculation involves taking the absolute value of the amplitude of each sample point, then summing these 400 absolute values ​​and dividing by 400. For example, if the sample point value sequence of the current frame is S1, S2, ..., S... 400 Then the temporal envelope amplitude E of the current frame t， pass: The result obtained from the above formula is the temporal envelope amplitude E of the current t-th frame. t The value is 0.15, and then the temporal envelope amplitude E of the previous frame, i.e., the (t-1)th frame, is called. t−1 Its value is 0.09, and the envelope change rate is obtained by subtracting the two amplitudes: The positive value here indicates that the signal energy shows an increasing trend between frames. Then, the temporal envelope amplitude of the current frame, 0.15, and the calculated envelope change rate, 0.06, are combined into a vector. The combination method is to arrange the two scalar values ​​in a fixed order to form a two-dimensional vector, namely [0.15, 0.06], to generate the instantaneous acoustic temporal feature value.

[0019] Based on the input audio signal frame, the amplitude component of the temporal envelope in the instantaneous acoustic temporal feature value is compared with the preset silence energy threshold. If the amplitude component is not lower than the threshold, the Mel frequency cepstral coefficient of the current frame is extracted. The coefficient is used as input to the preset shallow feedforward network to perform forward propagation, obtain the activation value of its output layer, and establish a broad overtone activation intensity vector. Furthermore, based on the input audio signal frame, the temporal envelope amplitude component 0.15 from the instantaneous acoustic temporal feature value [0.15, 0.06] generated in the previous process is retrieved and compared with a preset silence energy threshold. This threshold is used for numerical comparison. The setting is based on statistical analysis of a large amount of pure background noise audio data that does not contain speech. Taking 1.5 times the average energy value as a reference, it is set to 0.02. Since the current amplitude component is 0.15, which is greater than... If the amplitude value is 0.02, the current audio frame is determined to be a non-silent frame, so the subsequent operations continue. If the amplitude component is lower than the threshold, the processing flow of the current frame terminates. Under this condition, the Mel frequency cepstral coefficients of the current audio signal frame are extracted. That is, the 400 sample points of the signal of this frame are windowed, fast Fourier transform, Mel filter bank filtering and discrete cosine transform to obtain a 13-dimensional coefficient vector. Then, the coefficient vector is used as input and fed into a preset shallow feedforward network. The network contains a hidden layer with 128 neurons and an output layer with 5 neurons. Each neuron in the output layer corresponds to a broad phoneme category, namely vowel, plosive, fricative, nasal and silent. In the forward propagation, the 13-dimensional coefficient vector is multiplied by the network weight matrix and processed by the activation function. Finally, a 5-dimensional activation value vector is obtained in the output layer to establish a broad phoneme activation intensity vector.

[0020] The instantaneous acoustic temporal feature value and the broad phoneme activation intensity vector are called, and the two vectors are concatenated end to end according to the preset dimensional order. The maximum and minimum value normalization processing is performed on each component of the concatenated vector to generate the instantaneous audio frame composite feature vector.

[0021] Furthermore, the two-dimensional instantaneous acoustic temporal feature values ​​[0.15, 0.06] and the five-dimensional wide overtone activation intensity vector [0.85, 0.08, 0.04, 0.02, 0.01] generated in the previous steps are called and concatenated according to a preset dimensional order. This order is fixed as arranging the temporal features first, then the phoneme category features. The two vectors are concatenated end to end to form a seven-dimensional concatenated vector [0.15, 0.06, 0.85, 0.08, 0.04, 0.02, 0.01]. Since the numerical ranges of each component in the concatenated vector are different, such as the different scales of the envelope amplitude and the probability activation value, the maximum and minimum value normalization processing needs to be performed on all seven components of the vector. The maximum and minimum value sets used in this processing are fixed parameters obtained by statistically analyzing the feature values ​​of the same dimension of all frames in a large-scale speech database. For example, the maximum value vector preset for the seven dimensions. The vector is [0.6,0.3,1.0,1.0,1.0,1.0,1.0,1.0][0.6,0.3,1.0,1.0,1.0,1.0,1.0,1.0], with a minimum value of [0.0,−0.3,0.0,0.0,0.0,0.0,0.0][0.0,−0.3,0.0,0.0,0.0,0.0,0.0]. Normalizing the first component 0.15 of the concatenated vector yields (0...). 0.15−0.0) / (0.6−0.0)=0.25, and the calculation result for the second component 0.06 is (0.06−(−0.3)) / (0.3−(−0.3))=0.6(0.06−(−0.3)) / (0.3−(−0.3))=0.6. This process is repeated to complete the calculation of all components, generating the instantaneous audio frame composite feature vector.

[0022] Step S2 includes: Based on the composite feature vector of instantaneous audio frames, its temporal envelope amplitude component is compared with a preset silence energy threshold. If the comparison result does not represent silence, its envelope change rate component is compared with a steady state threshold. Combined with the maximum value index in the phoneme category activation intensity vector, a preset decision matrix is ​​queried to obtain the macroscopic calculation path index. Furthermore, based on the composite feature vector of the instantaneous audio frame, its first component, the normalized temporal envelope amplitude of 0.25, is compared with a preset silence energy threshold of 0.05. This threshold is set based on the 95th percentile of the statistical energy distribution of background noise without speech signals. Since 0.25 is greater than 0.05, it is determined to be non-silent. Then, the second component, the normalized envelope change rate of 0.6, is compared with the steady-state threshold range [-0.2, 0.2], which defines the speech signal... The stable fluctuation range of the energy was determined to be unstable because 0.6 exceeded this range. At the same time, the index of the largest component in the phoneme category activation intensity vector [0.85, 0.08, 0.04, 0.02, 0.01] was extracted, i.e., index 0, corresponding to the vowel class. Finally, the non-silence determination, the unstable state determination, and the vowel class index were used as query keys to query a three-dimensional preset decision matrix and retrieve entries that matched the combination key [non-silence, unstable, vowel class] to obtain the macroscopic calculation path index.

[0023] The acoustic feature vector sequence is monitored, the Euclidean distance between consecutive vectors is calculated and compared with the mutation threshold to determine the phoneme boundary, the average number of frames between recent phoneme boundaries is counted, and based on the comparison result of the average number of frames with the preset duration benchmark, a corresponding binary masking vector is selected from a fixed functional profile index library to establish an attention head sparsity mask. Furthermore, the acoustic feature vector sequence is monitored, and the current frame vector is calculated. Compared with the vector of the previous frame The Euclidean distance between them is: Where n is the vector dimension, the calculated distance value 0.48 is compared with a preset mutation threshold of 0.3. This threshold is set by statistically analyzing the inter-frame distance distribution at different phoneme connections, taking twice the standard deviation of the distribution mean. Since 0.48 is greater than 0.3, the current frame is determined to be a phoneme boundary. After recording the positions of the three most recently detected phoneme boundaries, the average number of frames between them is calculated, for example, 15 frames. This average number of frames is then compared with a preset duration reference range, for example, less than 12 frames for fast speech, 12 to 20 frames for medium speech, and more than 20 frames for slow speech. Since 15 frames fall into the medium speech range, an eight-bit binary masking vector [1,1,1,0,1,1,1,0] corresponding to medium speech is selected from a fixed functional profile index library to establish an attention head sparsity mask.

[0024] The macro-level computation path index and the attention head sparsity mask are invoked. The macro-level computation path index is used as the pattern identifier of the computation path, and the attention head sparsity mask is used as the structural adjustment parameter of the self-attention computation unit under the path. The two are integrated into a single data structure to generate a dynamic inference configuration strategy.

[0025] Furthermore, the macroscopic computation path index generated in the previous steps is invoked, with a value of 2, and the attention head sparsity mask is obtained, with a vector value of [1,1,1,0,1,1,1,0]. The value of the macroscopic computation path index is directly used as an integer variable named path_id to identify the computation path pattern adopted during the subsequent forward propagation of the network. The attention head sparsity mask is used as a boolean array named attention_mask, where 1 represents activation and 0 represents masking, to adjust the structure of the self-attention computation unit inside the decoder under this path. Then, the two different types of variables, path_id and attention_mask, are encapsulated into a unified data structure container, such as a dictionary or a custom object, to generate a dynamic inference configuration strategy.

[0026] Step S3 includes: Based on the dynamic inference configuration strategy, the computation path pattern index is extracted, and the level identifiers of all encoder layers in the original network structure are traversed. The level identifiers that match the index are included in a set, and the execution status of all encoder layers in the set is marked as disabled so that their internal computation can be skipped in subsequent forward propagation, thus establishing an encoder layer pruning configuration scheme. Furthermore, based on the dynamic inference configuration strategy, the stored computation path pattern index is extracted, with a value of 2. A pre-configured hierarchical pruning dictionary is obtained, which maps the path index to specific encoder layer identifiers, for example, {path 1: [10,11,12], path 2: [7,8,9,10,11,12]}. Then, the hierarchical identifiers of all twelve encoder layers in the original network structure are traversed from 1 to 12. The matching degree of each identifier is judged with the pruning list [7,8,9,10,11,12] corresponding to path index 2. When the traversal reaches identifier 7, since it exists in the pruning list, the identifier is added to a set to be disabled. The same judgment and inclusion operation is performed on the subsequent layers 8, 9, 10, 11, and 12. After the traversal is completed, the internal execution status parameters of all encoder layers corresponding to [7,8,9,10,11,12] in the set are uniformly marked as disabled, and the encoder layer pruning configuration scheme is established.

[0027] Based on the dynamic inference configuration strategy, the self-attention head masking rules are extracted and the encoder layer pruning configuration scheme is invoked. All encoder layers that are not marked as disabled are retained in the original network structure. At the same time, the masking rules are applied to the computation graph of each decoder layer. The weight matrix multiplication of the attention heads listed in the rules is skipped through conditional judgment logic. Finally, the retained encoder layers and the modified decoder layers are reconnected to establish an instantaneous reconstruction inference network.

[0028] Furthermore, based on the dynamic inference configuration strategy, the self-attention head masking rule stored therein is extracted, which is an eight-bit binary vector [1,1,1,0,1,1,1,0]. The encoder layer pruning configuration scheme is called to retain all encoder layers that are not marked as disabled in the original network structure, namely layers 1 to 6. At the same time, the masking rule is applied to the computation graph of each decoder layer. When performing multi-head self-attention calculation, the eight attention heads are traversed, and condition judgment is performed according to the value of the corresponding position in the masking rule. For example, when calculating the fourth attention head, since the fourth bit in the rule is 0, the weight matrix multiplication of that head is skipped directly, and its output is set to a zero tensor. Finally, the output of the sixth layer encoder that has been pruned and retained is connected to the input of the first layer decoder that has been modified by the masking rule to reconnect the data stream and establish an instantaneous reconstruction inference network.

[0029] Step S4 includes: The instantaneous reconstruction inference network is invoked, and the acoustic features of the original audio signal frame are used as the network input to perform forward propagation. The data flow makes conditional judgments based on the deactivation flags of each encoder layer in the network. When the flag is deactivated, the data flow directly bypasses all the calculation operations such as weight matrix multiplication and activation function contained in the layer, and passes the input tensor of the layer to the next layer to obtain the output tensor of the encoder layer. Furthermore, the instantaneous reconstruction inference network is invoked, and the acoustic features of the original audio signal frame are used as a 512-dimensional input tensor. Forward propagation is performed, and the data stream is conditionally judged at the entrance of each encoder layer in the network based on the deactivation flag. When the data stream reaches the 7th encoder layer, its deactivation flag is read as true. At this time, the data stream directly bypasses all calculation operations such as weight matrix multiplication and activation function inside this layer, and directly passes the 512-dimensional tensor, which is the input of the 7th layer, to the 8th layer. This process is applicable to all layers 7 to 12 marked as deactivation, while layers 1 to 6 perform calculations normally. Finally, the result tensor calculated by the 6th layer is used as the final output to obtain the encoder layer output tensor.

[0030] Based on the output tensor of the encoder layer, it is fed into the decoder part of the instantaneous reconstruction inference network. When multi-head self-attention calculation is performed inside each decoder layer, the built-in masking rules of the network are called. The attention head at the index specified by the rule is calculated as a branch, and all components of its output vector are forced to be set to zero. Finally, the outputs of all branches are concatenated and passed through the classification layer at the end of the network to generate frame-level acoustic posterior probabilities. Furthermore, based on the output tensor of the encoder layer, it is fed into the decoder part of the instantaneous reconstruction inference network. When performing eight-head self-attention calculation within each decoder layer, the built-in masking rule vector [1,1,1,0,1,1,1,0] is invoked. For the fourth calculation branch with index 0 in the rule, all components of its output vector are forced to be set to zero, while the remaining branches with index 1 are calculated normally. Finally, the outputs of all eight branches are concatenated and passed through the classification layer at the end of the network. This classification layer uses a Softmax function to convert the concatenated high-dimensional vector into a probability distribution vector covering all predefined phoneme units, generating frame-level acoustic posterior probabilities.

[0031] Based on frame-level acoustic posterior probabilities and by calling the language model to query state transition probabilities, all end nodes in the current decoding graph are traversed. The cumulative score of each node is jointly calculated with the state transition probability and the frame-level acoustic posterior probability to update the original path score and expand to generate new hypothetical nodes and connecting edges. The updated set of all nodes and connecting edges is then integrated to obtain the accelerated decoding path grid.

[0032] Furthermore, based on the frame-level acoustic posterior probability and by calling the language model to query the state transition probability, all end nodes in the current decoding graph are traversed. For example, a node "A" with a score of -1.8 has a probability of 0.7 for the phoneme "B" in its posterior probability vector. The language model gives a transition probability of 0.5 from "A" to "B". Then, the new path score is calculated by logarithmic addition, i.e., -1.8 + log(0.7) + log(0.5), to update the original path score and expand to generate a new hypothetical node "B" and connecting edges. The updated set of all nodes and connecting edges is integrated into a data structure to obtain the accelerated decoding path grid.

[0033] The S5 steps include: For the accelerated decoding path grid, all nodes in the graph data structure are traversed, and all candidate nodes representing the end of speech are selected based on the state identifier built into each node. Then, the cumulative probability scores of each candidate node are extracted one by one and the numerical values ​​are compared. The candidate node with the maximum score is selected to establish the optimal path backtracking starting point. Furthermore, for the accelerated decoding path grid, all nodes in the graph are traversed, and a judgment is performed based on the is_final state identifier built into each node. All candidate nodes with this identifier being true are selected, such as nodes A, B, and C. Then, the cumulative probability scores of each candidate node are extracted one by one, with values ​​of -1.2, -0.9, and -1.5 respectively. The full value of these three scores is compared. Since -0.9 is the maximum value, node B corresponding to the score of -0.9 is selected to establish the optimal path backtracking starting point.

[0034] Call the optimal path backtracking starting point, and traverse backwards along the forward backtracking pointer stored inside the node to accelerate the decoding of the path grid until the starting node of the graph is reached. During the traversal, the vocabulary units corresponding to each node on the path are detected in turn, and all detected vocabulary units are stored in a sequence in the order of backward traversal to obtain the reversed vocabulary unit list. Furthermore, the optimal path backtracking starting point is invoked, and the forward backtracking pointer stored inside that node is used to traverse backwards to accelerate the decoding of the path grid until the starting node of the graph is reached. During the traversal, the vocabulary units corresponding to each node on the path are detected in turn. For example, backtracking from the starting point "world" to "hello" and then back to the starting node, all detected vocabulary units "world" and "hello" are stored in a queue data structure in the order of reverse traversal to obtain the reversed vocabulary unit list.

[0035] Based on the reversed list of word units, the order of all word units stored in the sequence is reversed to restore their temporal relationship in the original speech. Then, a preset text delimiter is called to connect all word units in the reversed sequence in the new order and combine them into a single string to obtain the optimized recognition text sequence.

[0036] Furthermore, based on the reversed word unit list ["world", "hello"], the order of all word units stored in the sequence is reversed to obtain a new sequence ["hello", "world"]. Then, the preset text separator, i.e. a single space character, is called to connect all word units in the reversed sequence in the new order to form a single string "helloworld", thus obtaining the optimized recognition text sequence.

[0037] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A method for accelerating voice recognition based on AI algorithm optimization, characterized in that, Includes the following steps: S1. Instantaneous audio frame composite feature extraction: Obtain audio signal frames, calculate envelope amplitude and rate of change, combine with shallow network phoneme activation intensity, and splice to generate instantaneous audio frame composite feature vector; S2. Dynamic reasoning configuration strategy generation: Based on the composite feature vector of instantaneous audio frames, query the decision matrix to specify the path pattern, establish the masking rules according to the phoneme boundary calculation, and integrate them into a dynamic reasoning configuration strategy. S3. Instantaneous Reconstruction Inference Network Construction: Based on the dynamic inference configuration strategy, the encoder layer pointed to by the path pattern index is marked as inactive, and the masking rules are applied to the computation graph of the decoder layer to establish the instantaneous reconstruction inference network. S4. Accelerate forward inference and decoding grid generation: Invoke the instantaneous reconstruction inference network to propagate the acoustic features forward. The data stream bypasses the encoder layer according to the non-activation marker. In the decoder, the multi-head attention masking branch is set to zero. The end generates an acoustic distribution to update the decoding graph score and generate an accelerated decoding path grid. S5. Optimal Recognition Text Sequence Output: For the accelerated decoding path grid, backtrack from the highest probability end node to the starting point, detect the word units on the path and combine them in order to obtain the optimized recognition text sequence.

2. The AI-based algorithm-optimized voice recognition acceleration method according to claim 1, characterized in that: The instantaneous audio frame composite feature vector includes envelope amplitude components, envelope rate components, and phoneme category intensity distribution. The dynamic inference configuration strategy includes computation path pattern identifiers, attention head sparsity masks, and hierarchical skip instructions. The instantaneous reconstruction inference network includes an activated subset of encoder layers, dynamically masked decoder layers, and adjusted inter-layer connections. The accelerated decoding path grid includes a candidate word node set, acoustic transfer scores between nodes, and an optimal path backtracking pointer. The optimized recognition text sequence includes the final output word sequence, the timestamp of each word, and the global confidence score of the entire sequence.

3. The AI-based algorithm-optimized voice recognition acceleration method according to claim 1, characterized in that: Step S1 includes: The system acquires the input audio signal frame, calculates the arithmetic mean of the absolute values ​​of all signal sample points within the frame to obtain the temporal envelope amplitude, and performs a difference operation between the temporal envelope amplitude of the previous frame and the temporal envelope amplitude of the current frame to obtain the envelope change rate. The system combines the temporal envelope amplitude and the envelope change rate to generate instantaneous acoustic temporal feature values. Based on the input audio signal frame, the amplitude component of the temporal envelope in the instantaneous acoustic temporal feature value is compared with the preset silence energy threshold. If the amplitude component is not lower than the threshold, the Mel frequency cepstral coefficient of the current frame is extracted. The coefficient is used as input to the preset shallow feedforward network to perform forward propagation, obtain the activation value of its output layer, and establish a broad overtone activation intensity vector. The instantaneous acoustic temporal feature value and the broad phoneme activation intensity vector are called, and the two vectors are concatenated end to end according to the preset dimensional order. The maximum and minimum value normalization processing is performed on each component of the concatenated vector to generate the instantaneous audio frame composite feature vector.

4. The AI-based algorithm-optimized voice recognition acceleration method according to claim 1, characterized in that: Step S2 includes: Based on the composite feature vector of instantaneous audio frames, its temporal envelope amplitude component is compared with a preset silence energy threshold. If the comparison result does not represent silence, its envelope change rate component is compared with a steady state threshold. Combined with the maximum value index in the phoneme category activation intensity vector, a preset decision matrix is ​​queried to obtain the macroscopic calculation path index. The acoustic feature vector sequence is monitored, the Euclidean distance between consecutive vectors is calculated and compared with the mutation threshold to determine the phoneme boundary, the average number of frames between recent phoneme boundaries is counted, and based on the comparison result of the average number of frames with the preset duration benchmark, a corresponding binary masking vector is selected from a fixed functional profile index library to establish an attention head sparsity mask. The macro-level computation path index and the attention head sparsity mask are invoked. The macro-level computation path index is used as the pattern identifier of the computation path, and the attention head sparsity mask is used as the structural adjustment parameter of the self-attention computation unit under the path. The two are integrated into a single data structure to generate a dynamic inference configuration strategy.

5. The AI-based algorithm-optimized voice recognition acceleration method according to claim 1, characterized in that: Step S3 includes: Based on the dynamic inference configuration strategy, the computation path pattern index is extracted, and the level identifiers of all encoder layers in the original network structure are traversed. The level identifiers that match the index are included in a set, and the execution status of all encoder layers in the set is marked as disabled so that their internal computation can be skipped in subsequent forward propagation, thus establishing an encoder layer pruning configuration scheme. Based on the dynamic inference configuration strategy, the self-attention head masking rules are extracted and the encoder layer pruning configuration scheme is invoked. All encoder layers that are not marked as disabled are retained in the original network structure. At the same time, the masking rules are applied to the computation graph of each decoder layer. The weight matrix multiplication of the attention heads listed in the rules is skipped through conditional judgment logic. Finally, the retained encoder layers and the modified decoder layers are reconnected to establish an instantaneous reconstruction inference network.

6. The AI-based algorithm-optimized voice recognition acceleration method according to claim 1, characterized in that: The S4 step includes: The instantaneous reconstruction inference network is invoked, and the acoustic features of the original audio signal frame are used as the network input to perform forward propagation. The data flow makes conditional judgments based on the deactivation flags of each encoder layer in the network. When the flag is deactivated, the data flow directly bypasses all the calculation operations such as weight matrix multiplication and activation function contained in the layer, and passes the input tensor of the layer to the next layer to obtain the output tensor of the encoder layer. Based on the output tensor of the encoder layer, it is fed into the decoder part of the instantaneous reconstruction inference network. When multi-head self-attention calculation is performed inside each decoder layer, the built-in masking rules of the network are called. The attention head at the index specified by the rule is calculated as a branch, and all components of its output vector are forced to be set to zero. Finally, the outputs of all branches are concatenated and passed through the classification layer at the end of the network to generate frame-level acoustic posterior probabilities. Based on frame-level acoustic posterior probabilities and by calling the language model to query state transition probabilities, all end nodes in the current decoding graph are traversed. The cumulative score of each node is jointly calculated with the state transition probability and the frame-level acoustic posterior probability to update the original path score and expand to generate new hypothetical nodes and connecting edges. The updated set of all nodes and connecting edges is then integrated to obtain the accelerated decoding path grid.

7. The AI-based algorithm-optimized voice recognition acceleration method according to claim 1, characterized in that: Step S5 includes: For the accelerated decoding path grid, all nodes in the graph data structure are traversed, and all candidate nodes representing the end of speech are selected based on the state identifier built into each node. Then, the cumulative probability scores of each candidate node are extracted one by one and the numerical values ​​are compared. The candidate node with the maximum score is selected to establish the optimal path backtracking starting point. Call the optimal path backtracking starting point, and traverse backwards along the forward backtracking pointer stored inside the node to accelerate the decoding of the path grid until the starting node of the graph is reached. During the traversal, the vocabulary units corresponding to each node on the path are detected in turn, and all detected vocabulary units are stored in a sequence in the order of backward traversal to obtain the reversed vocabulary unit list. Based on the reversed list of word units, the order of all word units stored in the sequence is reversed to restore their temporal relationship in the original speech. Then, a preset text delimiter is called to connect all word units in the reversed sequence in the new order and combine them into a single string to obtain the optimized recognition text sequence.

Images