Speech recognition method based on large kernel convolution and linear self-attention

Abstract

The invention relates to the technical field of speech recognition, in particular to a speech recognition method based on large kernel convolution and linear self-attention, which comprises the following steps: preprocessing a speech signal acquired in advance to obtain a corresponding acoustic signal; inputting the acoustic signal into a trained LKCovLFormer model to obtain a voice text; the trained LKCovLFormer model is obtained by adopting a training data set to train the LKCovLFormer model, and the LKCovLFormer model is obtained by adopting the training data set to train the LKCovLFormer model; the training data set comprises multiple segments of first acoustic signals and voice texts in one-to-one correspondence with the segments of first acoustic signals; the LKCovLFormer model comprises N layers of circulation stacking modules; each layer of cyclic stacking module comprises a first feed-forward module, a first residual module, a linear self-attention module, a second residual module, a large kernel convolution module, a third residual module, a second feed-forward module, a fourth residual module and a layer standardization module which are arranged in sequence.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition technology, and in particular to a speech recognition method based on large kernel convolution and linear self-attention. Background Art

[0002] As an important branch of natural language processing, speech recognition technology has been widely studied and applied in recent years. The main task of the speech recognition system is to convert the input speech signal into the corresponding text content. Application scenarios include intelligent voice assistants, voice input methods, automatic speech translation, etc. Traditional speech recognition systems usually adopt a hybrid model architecture based on hidden Markov models and deep neural networks, but these methods have certain limitations in processing complex speech signals and capturing long-distance dependency features. Summary of the invention

[0003] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a speech recognition method based on large kernel convolution and linear self-attention.

[0004] In order to achieve the above object, the main technical solutions adopted by the present invention include:

[0005] The embodiment of the present invention provides a speech recognition method based on large kernel convolution and linear self-attention, comprising:

[0006] S1, preprocessing the pre-acquired speech signal to obtain a corresponding acoustic signal;

[0007] S2, inputting the acoustic signal into the trained LKCovLFormer model to obtain a speech text corresponding to the speech signal;

[0008] Among them, the trained LKCovLFormer model is obtained by training the pre-created LKCovLFormer model using the training data set;

[0009] The training data set includes: a plurality of first acoustic signals and pre-annotated speech texts corresponding to each first acoustic signal; the plurality of first acoustic signals are obtained by pre-processing the plurality of speech signals obtained in advance for training;

[0010] The LKCovLFormer model includes an N-layer loop stacking module;

[0011] Each layer of the cyclic stacking module includes a first feedforward module, a first residual module, a linear self-attention module, a second residual module, a large kernel convolution module, a third residual module, a second feedforward module, a fourth residual module, and a layer normalization module, which are arranged in sequence.

[0012] Preferably, before S1, it also includes:

[0013] S0. Use the training data set to train the pre-created LKCovLFormer model to obtain a trained LKCovLFormer model.

[0014] Preferably, the acoustic signal is an 80-dimensional Fbank feature signal.

[0015] Preferably,

[0016] The first feedforward module in each layer of the cyclic stacking module is used to perform preliminary feature extraction processing on the input data input into the cyclic stacking module of this layer to obtain preliminary feature data;

[0017] The first residual module in each layer of the cyclic stacking module is used to perform residual processing on the input data and preliminary feature data input into the cyclic stacking module of this layer to obtain a first residual result;

[0018] The linear self-attention module in each layer of the loop stacking module is used to process the first residual result using a linear complex self-attention mechanism to obtain a linear self-attention processing result;

[0019] The second residual module in each layer of the cyclic stacking module is used to perform residual processing on the first residual result and the linear self-attention processing result to obtain a second residual result;

[0020] The large kernel convolution module in each layer of the cyclic stacking module is used to perform large kernel convolution processing on the second residual result to obtain a large kernel convolution processing result;

[0021] The third residual module in each layer of the cyclic stacking module is used to perform residual processing on the second residual result and the large kernel convolution processing result to obtain a third residual result;

[0022] The second feedforward module in each layer of the cyclic stacking module is used to perform enhanced feature representation processing on the third residual result to obtain a processing result of the second feedforward module;

[0023] The fourth residual module in each layer of the cyclic stacking module is used to perform residual processing on the third residual result and the processing result of the second feedforward module to obtain a fourth residual result;

[0024] The layer normalization module in each layer of the loop stacking module is used to normalize the fourth residual result to obtain input data for the next layer of the loop stacking module.

[0025] Preferably,

[0026] The large kernel convolution module includes:

[0027] A depth convolution unit, used to perform a convolution operation on each channel of the input second residual result respectively, so as to extract local spatial features in the second residual result;

[0028] A depth and extension convolution unit, connected to the depth convolution unit, for applying an extension convolution operation based on the extracted local spatial features, by increasing the sampling interval of the convolution kernel to expand the receptive field, thereby capturing the long-distance dependency features in the second residual result;

[0029] A point convolution unit connected to the depth and extension convolution units, and configured to perform channel fusion on the output results of the depth and extension convolution units through a 1×1 convolution operation, so as to integrate feature information between different channels and obtain channel fusion features;

[0030] Among them, the large kernel convolution processing result is a fused feature map obtained after being processed in sequence by the depth convolution unit, the depth and extension convolution unit and the point convolution unit. The fused feature map also includes the local spatial features, long-distance dependency features and channel fusion features of the second residual result.

[0031] Preferably, in the process where the linear self-attention module uses a linear complex self-attention mechanism to process the first residual result, the self-attention formula used is:

[0032]

[0033] Y is the result of linear self-attention processing;

[0034] Q, K, and V are query, key, and value matrices, respectively;

[0035] is the kernel function mapping;

[0036] is a normalized diagonal matrix; where 1 is a vector of all ones.

[0037] Preferably,

[0038] Among them, the size and expansion rate of the convolution kernel of the extended convolution are adjusted in the depth and extended convolution units according to formula (1) and formula (2) respectively;

[0039] The formula (1) is:

[0040] k t =k min +σ(W k ·X)×(k max -k min );

[0041] Among them, k minis the minimum size of the convolution kernel in the depth and dilation convolution unit;

[0042] k max is the maximum size of the convolution kernel in the depth and dilation convolution unit;

[0043] σ() is the Sigmoid activation function;

[0044] W k is the preset first trainable parameter matrix;

[0045] X is the data of input depth and dilated convolution unit;

[0046] The formula (2) is:

[0047] d t =d min +tanh(W d ·X)×(d max -d min );

[0048] Among them, d max is the maximum value of the preset expansion rate;

[0049] d min is the minimum value of the preset expansion rate;

[0050] tanh() is the Tanh activation function;

[0051] W d is the second trainable parameter matrix.

[0052] Preferably, the S0 specifically includes:

[0053] Using each first acoustic signal in the training data set and the corresponding pre-annotated speech text, the LKCovLFormer model is trained until a preset regression loss function corresponding to the speech recognition model converges, thereby obtaining a trained LKCovLFormer model;

[0054] The AdamW optimization algorithm is used during the training of the LKCovLFormer model to minimize the preset regression loss function and update all weight parameters in the LKCovLFormer model.

[0055] Preferably,

[0056] During the training of the LKCovLFormer model, the learning rate of the tth iteration is calculated by formula (3);

[0057] The formula (3) is:

[0058]

[0059] Among them, y t is the learning rate of the tth iteration;

[0060] y0 is the initial learning rate;

[0061] g i is the gradient value of the tth iteration;

[0062] ∈ is a preset constant.

[0063] Preferably, 6≤N≤12.

[0064] The beneficial effects of the present invention are:

[0065] The present invention discloses a speech recognition method based on large kernel convolution and linear self-attention, which adopts a new hybrid architecture model LKCovLFormer, combining the long-distance dependent feature extraction capability of large kernel convolution and the efficient global modeling capability of linear self-attention mechanism. Specifically, the present invention realizes multi-level feature extraction and comprehensive representation of speech signals by introducing a cyclic stacking module, including submodules such as a feedforward network, a residual connection, a large kernel convolution module, and a linear self-attention module, thereby solving the problems of limited receptive field, high computational complexity, and low training efficiency in the prior art.

[0066] The present invention discloses a speech recognition method based on large kernel convolution and linear self-attention. By using a large kernel convolution module, the model can capture the long-distance dependency features in the speech signal, effectively expand the receptive field, reduce the dependence on the stacking of convolution layers, and thus reduce the computational complexity.

[0067] The present invention discloses a speech recognition method based on large kernel convolution and linear self-attention. The present invention adopts a linear self-attention mechanism, reduces the complexity of standard self-attention, and significantly improves the computational efficiency when long sequence input is used.

[0068] The speech recognition method based on large kernel convolution and linear self-attention of the present invention includes an alternating design of a feedforward module, a residual module, a self-attention module and a convolution module in each layer of a cyclic stacking module, which effectively integrates local spatial features, long-distance dependency features and channel fusion features, and improves the feature expression ability of the LKCovLFormer model. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 This is a flow chart of a speech recognition method based on large kernel convolution and linear self-attention in the present invention;

[0070] Figure 2 It is a structural schematic diagram of the circular stacking module in the present invention. DETAILED DESCRIPTION

[0071] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation modes in conjunction with the accompanying drawings.

[0072] In order to better understand the above technical solution, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a clearer and more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0073] See also Figure 1 , this embodiment provides a speech recognition method based on large kernel convolution and linear self-attention, including:

[0074] S1, preprocessing the pre-acquired speech signal to obtain a corresponding acoustic signal;

[0075] The acoustic signal is an 80-dimensional Fbank feature signal.

[0076] S2, inputting the acoustic signal into the trained LKCovLFormer model to obtain a speech text corresponding to the speech signal;

[0077] Among them, the trained LKCovLFormer model is obtained by training the pre-created LKCovLFormer model using the training data set;

[0078] The training data set includes: a plurality of first acoustic signals and pre-annotated speech texts corresponding to each first acoustic signal; the plurality of first acoustic signals are obtained by pre-processing the plurality of speech signals obtained in advance for training;

[0079] The LKCovLFormer model includes N layers of loop stacking modules; wherein 6≤N≤12.

[0080] Each layer of the cyclic stacking module includes a first feedforward module, a first residual module, a linear self-attention module, a second residual module, a large kernel convolution module, a third residual module, a second feedforward module, a fourth residual module, and a layer normalization module, which are arranged in sequence.

[0081] In this embodiment, before S1, the following steps are also included:

[0082] S0. Use the training data set to train the pre-created LKCovLFormer model to obtain a trained LKCovLFormer model.

[0083] See also Figure 2 The first feedforward module in each layer of the cyclic stacking module is used to perform preliminary feature extraction processing on the input data of the cyclic stacking module of this layer to obtain preliminary feature data;

[0084] In this embodiment, the first feed-forward module performs preliminary feature extraction processing on the input data of the loop stacking module of this layer, usually using a fully connected layer (Feed-Forward Layer) and an activation function (such as ReLU) to improve the nonlinear expression ability of the features and extract the basic spectral information and time-varying features in the speech signal.

[0085] The first residual module in each layer of the cyclic stacking module is used to perform residual processing on the input data and preliminary feature data input into the cyclic stacking module of this layer to obtain a first residual result;

[0086] In this embodiment, residual processing is used to alleviate the gradient vanishing problem in the deep network, retain the original information in the input features, and enhance the robustness of the LKCovLFormer model.

[0087] The linear self-attention module in each layer of the loop stacking module is used to process the first residual result using a linear complex self-attention mechanism to obtain a linear self-attention processing result;

[0088] In this embodiment, the long-distance temporal dependency is effectively modeled, and the contextual information in the speech signal is captured. The self-attention mechanism with linear complexity is used, which significantly reduces the computational cost and is suitable for processing long sequence speech data.

[0089] The second residual module in each layer of the cyclic stacking module is used to perform residual processing on the first residual result and the linear self-attention processing result to obtain a second residual result;

[0090] Through the processing of the second residual module, the original residual information and the context information extracted by self-attention are retained, effectively integrating local and global features, and improving the generalization ability and stability of the model.

[0091] The large kernel convolution module in each layer of the cyclic stacking module is used to perform large kernel convolution processing on the second residual result to obtain a large kernel convolution processing result;

[0092] In this embodiment, the receptive field is expanded by large-kernel convolution, long-distance dependency features are captured, the global pattern in the speech signal is effectively extracted, and the feature expression capability is improved.

[0093] The third residual module in each layer of the cyclic stacking module is used to perform residual processing on the second residual result and the large kernel convolution processing result to obtain a third residual result;

[0094] The second feedforward module in each layer of the cyclic stacking module is used to perform enhanced feature representation processing on the third residual result to obtain the processing result of the second feedforward module; the nonlinear expression ability of the feature is improved, and deeper feature patterns are extracted.

[0095] The fourth residual module in each layer of the cyclic stacking module is used to perform residual processing on the third residual result and the processing result of the second feedforward module to obtain a fourth residual result;

[0096] In this embodiment, through the processing of the fourth residual module, the enhanced features in the second feedforward module are retained, while the original residual information is maintained, thereby improving the stability and robustness of the model.

[0097] The layer normalization module in each layer of the loop stacking module is used to normalize the fourth residual result to obtain the input data of the next layer of the loop stacking module. Normalizing the input features reduces the internal covariate shift and speeds up the training speed and convergence effect of the LKCovLFormer model.

[0098] In each layer of recurrent stacking modules, the feedforward module, self-attention module and large-kernel convolution module extract local features, global context features and long-distance spatial features respectively. Through residual connection and normalization processing, the LKCovLFormer model can effectively capture the complex patterns in speech signals, improve the accuracy and efficiency of speech recognition, and this design structure is repeated in multiple layers of recurrent stacking, further enhancing the feature expression ability and robustness of the model.

[0099] In this embodiment, the large kernel convolution module includes:

[0100] A depth convolution unit, used to perform a convolution operation on each channel of the input second residual result respectively, so as to extract local spatial features in the second residual result;

[0101] A depth and extension convolution unit, connected to the depth convolution unit, for applying an extension convolution operation based on the extracted local spatial features, by increasing the sampling interval of the convolution kernel to expand the receptive field, thereby capturing the long-distance dependency features in the second residual result;

[0102] A point convolution unit connected to the depth and extension convolution units, and configured to perform channel fusion on the output results of the depth and extension convolution units through a 1×1 convolution operation, so as to integrate feature information between different channels and obtain channel fusion features;

[0103] Among them, the large kernel convolution processing result is a fused feature map obtained after being processed in sequence by the depth convolution unit, the depth and extension convolution unit and the point convolution unit. The fused feature map also includes the local spatial features, long-distance dependency features and channel fusion features of the second residual result.

[0104] For example, assuming that the input is the second residual result after preprocessing, its feature map shape is (T, H, W, C), where: T represents the number of time frames; H and W are the height and width of the feature map; C is the number of channels of the input feature map.

[0105] The deep convolution unit performs convolution operations on each channel of the second residual result of the input to extract local spatial features. Deep convolution only performs convolution operations on spatial dimensions (height and width) and does not involve fusion between channels. Deep convolution significantly reduces the amount of calculation and parameters because each channel performs convolution independently without interaction between channels. It can effectively extract fine-grained local features in each channel, enhancing the model's ability to capture short-term changing features.

[0106] For example, assuming that the input feature map is (100, 20, 20, 64), after 3×3 depth convolution processing, the output feature map shape remains unchanged (100, 20, 20, 64), which contains the extracted local spatial features.

[0107] The deep and extended convolution unit applies extended convolution operation based on the extracted local spatial features, and expands the receptive field by increasing the sampling interval of the convolution kernel, thereby extracting long-distance dependent features in the input data. Extended convolution can effectively expand the receptive field without increasing the size of the convolution kernel by inserting the sampling interval.

[0108] The point convolution unit performs channel fusion on the output results of the depth and extension convolution units through 1×1 convolution operations to integrate the feature information between different channels and obtain channel fusion features. Point convolution can achieve feature fusion between channels without increasing computational complexity, thereby improving the expressiveness of the model. Using a 1×1 convolution kernel, only a small number of parameters are required to effectively fuse channel features, reducing computational overhead.

[0109] The shape of the input feature map is (100, 20, 20, 64). After 1×1 point convolution, the shape of the output feature map is (100, 20, 20, 128). This feature map contains the integrated channel fusion features, which can comprehensively express the various feature information of the input data.

[0110] In this embodiment, the large kernel convolution module realizes the comprehensive extraction and fusion of local features, long-distance dependency features and channel fusion features through the combination of three convolution operations: deep convolution, dilated convolution and point convolution. It not only effectively improves the feature extraction capability, but also significantly reduces the computational complexity, and can provide higher accuracy and faster reasoning speed in speech recognition tasks.

[0111] Among them, the linear self-attention module uses the linear complex self-attention mechanism to process the first residual result, and the self-attention formula used is:

[0112]

[0113] Y is the result of linear self-attention processing;

[0114] Q, K, and V are query, key, and value matrices, respectively;

[0115] is the kernel function mapping;

[0116] is a normalized diagonal matrix; where 1 is a vector of all ones.

[0117] In this embodiment, a self-attention mechanism with linear complexity is adopted, and the complexity is reduced through kernel function mapping, which significantly reduces the computational overhead. Through kernel function mapping, the linear self-attention mechanism can efficiently capture the long-distance dependent features in the input, and is suitable for processing long-time series inputs, such as speech recognition and natural language processing tasks. Using kernel function mapping (such as ReLU activation) and diagonal matrix normalization can effectively avoid numerical instability and maintain the high accuracy of the model during feature extraction.

[0118] Among them, the size and expansion rate of the convolution kernel of the extended convolution are adjusted in the depth and extended convolution units according to formula (1) and formula (2) respectively;

[0119] The formula (1) is:

[0120] k t =k min +σ(W k ·X)×(k max -k min );

[0121] Among them, k min is the minimum size of the convolution kernel in the depth and dilation convolution unit;

[0122] k max is the maximum size of the convolution kernel in the depth and dilation convolution unit;

[0123] σ( ) is the Sigmoid activation function;

[0124] W k is the preset first trainable parameter matrix;

[0125] X is the data of input depth and dilated convolution unit;

[0126] The formula (2) is:

[0127] d t =d min +tanh(W d ·X)×(d max -d min );

[0128] Among them, d max is the maximum value of the preset expansion rate;

[0129] d min is the minimum value of the preset expansion rate;

[0130] tanh() is the Tanh activation function;

[0131] W d is the second trainable parameter matrix.

[0132] In the depth and expansion convolution units, the size of the convolution kernel directly determines the receptive field during the convolution operation, that is, the area that the convolution kernel can cover on the input feature map. In order to more flexibly adapt to different input features and capture spatial information of different scales, in this embodiment, the convolution kernel size is allowed to be in the range of [k min , k max ] range. In speech recognition tasks, input data often contains features of different scales. Using a smaller convolution kernel (such as k min =3) can better extract detail features, while larger convolution kernels (such as k max = 15 or above) can capture global context information. Therefore, dynamically adjusting the convolution kernel size enables the model to better adapt to changes in input features in different scenarios, thereby extracting richer and more diverse information. The larger the convolution kernel, the wider the receptive field, which can cover a larger range of areas on the input feature map. Especially in dilated convolution, the size of the convolution kernel and the dilation rate jointly determine the size of the receptive field.

[0133] Specifically, the S0 specifically includes:

[0134] Using each first acoustic signal in the training data set and the corresponding pre-annotated speech text, the LKCovLFormer model is trained until a preset regression loss function corresponding to the speech recognition model converges, thereby obtaining a trained LKCovLFormer model;

[0135] The AdamW optimization algorithm is used during the training of the LKCovLFormer model to minimize the preset regression loss function and update all weight parameters in the LKCovLFormer model.

[0136] In the practical application of this embodiment, during the training of the LKCovLFormer model, the learning rate of the tth iteration is calculated by formula (3);

[0137] The formula (3) is:

[0138]

[0139] Among them, y t is the learning rate of the tth iteration;

[0140] y0 is the initial learning rate, that is, the learning rate value set at the beginning of model training;

[0141] g i is the gradient value of the tth iteration, which is used to measure the gradient change of this round;

[0142] ∈ is a preset constant. It is used to avoid the denominator being zero and ensure calculation stability.

[0143] In this embodiment, the learning rate is dynamically adjusted by accumulating the square sum of the gradients in formula (3). When the gradient is large, the accumulated square sum of the gradients will also be large, resulting in a learning rate y t is reduced; on the contrary, when the gradient is small, the learning rate y t Will increase, this adaptive adjustment mechanism can help the model converge quickly in the early stage of training, and reduce the learning rate to avoid oscillation and overfitting problems when fine-tuning in the later stage of training. In addition, since the sum of squares of the gradient is used as the denominator, the explosion phenomenon caused by excessive gradients can be effectively suppressed. At the same time, the existence of ∈ can prevent the denominator from being zero, thereby preventing the gradient from disappearing. This allows the model to maintain a stable learning rate adjustment even when training for a long time or when the gradient changes greatly, thereby improving the stability and convergence of the training. Dynamically adjusting the learning rate helps the model adopt different update strategies at different training stages. In the initial training, a larger learning rate can help the model quickly explore the parameter space; in the later fine-tuning, a smaller learning rate is conducive to the model converging to a better local optimal solution. This adjustment strategy can effectively improve the generalization ability of the model and reduce the risk of overfitting, especially when training on complex models and large-scale data sets.

[0144] In the denominator of formula (3), In fact, it is a normalization process, which can dynamically balance the learning rate according to the historical gradient information, making it less likely to experience large fluctuations when the model is updated.

[0145] A speech recognition method based on large kernel convolution and linear self-attention in this embodiment adopts a new hybrid architecture model LKCovLFormer, which combines the long-distance dependent feature extraction capability of large kernel convolution and the efficient global modeling capability of linear self-attention mechanism. Specifically, the present invention realizes multi-level feature extraction and comprehensive representation of speech signals by introducing a cyclic stacking module, including sub-modules such as a feedforward network, a residual connection, a large kernel convolution module, and a linear self-attention module, thereby solving the problems of limited receptive field, high computational complexity, and low training efficiency in the prior art.

[0146] In this embodiment, a speech recognition method based on large kernel convolution and linear self-attention is provided. By using a large kernel convolution module, the model can capture the long-distance dependency features in the speech signal, effectively expand the receptive field, and reduce the dependence on the stacking of convolution layers, thereby reducing the computational complexity.

[0147] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0148] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0149] In the present invention, unless otherwise clearly specified and limited, when a first feature is “on” or “below” a second feature, it may be that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediate medium. Moreover, when a first feature is “above”, “above” or “above” a second feature, it may be that the first feature is directly above or obliquely above the second feature, or it may simply mean that the first feature is higher in level than the second feature. When a first feature is “below”, “below” or “below” a second feature, it may be that the first feature is directly below or obliquely below the second feature, or it may simply mean that the first feature is lower in level than the second feature.

[0150] In the description of this specification, the description of the terms "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0151] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may alter, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A speech recognition method based on large kernel convolution and linear self-attention, characterized in that: include: S1, preprocessing the pre-acquired speech signal to obtain a corresponding acoustic signal; S2, inputting the acoustic signal into the trained LKCovLFormer model to obtain a speech text corresponding to the speech signal; Among them, the trained LKCovLFormer model is obtained by training the pre-created LKCovLFormer model using the training data set; The training data set includes: a plurality of first acoustic signals and pre-annotated speech texts corresponding to each first acoustic signal; the plurality of first acoustic signals are obtained by pre-processing the plurality of speech signals obtained in advance for training; The LKCovLFormer model includes an N-layer loop stacking module; Each layer of the cyclic stacking module includes a first feedforward module, a first residual module, a linear self-attention module, a second residual module, a large kernel convolution module, a third residual module, a second feedforward module, a fourth residual module, and a layer normalization module, which are arranged in sequence.

2. The speech recognition method based on large kernel convolution and linear self-attention according to claim 1, characterized in that: Prior to S1, it also included: S0. Use the training data set to train the pre-created LKCovLFormer model to obtain a trained LKCovLFormer model.

3. The speech recognition method based on large kernel convolution and linear self-attention according to claim 2, characterized in that: in, The acoustic signal is an 80-dimensional Fbank feature signal.

4. The speech recognition method based on large kernel convolution and linear self-attention according to claim 2, characterized in that: The first feedforward module in each layer of the cyclic stacking module is used to perform preliminary feature extraction processing on the input data input into the cyclic stacking module of this layer to obtain preliminary feature data; The first residual module in each layer of the cyclic stacking module is used to perform residual processing on the input data and preliminary feature data input into the cyclic stacking module of this layer to obtain a first residual result; The linear self-attention module in each layer of the loop stacking module is used to process the first residual result using a linear complex self-attention mechanism to obtain a linear self-attention processing result; The second residual module in each layer of the cyclic stacking module is used to perform residual processing on the first residual result and the linear self-attention processing result to obtain a second residual result; The large kernel convolution module in each layer of the cyclic stacking module is used to perform large kernel convolution processing on the second residual result to obtain a large kernel convolution processing result; The third residual module in each layer of the cyclic stacking module is used to perform residual processing on the second residual result and the large kernel convolution processing result to obtain a third residual result; The second feedforward module in each layer of the cyclic stacking module is used to perform enhanced feature representation processing on the third residual result to obtain a processing result of the second feedforward module; The fourth residual module in each layer of the cyclic stacking module is used to perform residual processing on the third residual result and the processing result of the second feedforward module to obtain a fourth residual result; The layer normalization module in each layer of the loop stacking module is used to normalize the fourth residual result to obtain input data for the next layer of the loop stacking module.

5. The speech recognition method based on large kernel convolution and linear self-attention according to claim 4, characterized in that: The large kernel convolution module includes: A depth convolution unit, used to perform a convolution operation on each channel of the input second residual result respectively, so as to extract local spatial features in the second residual result; A depth and extension convolution unit, connected to the depth convolution unit, for applying an extension convolution operation based on the extracted local spatial features, by increasing the sampling interval of the convolution kernel to expand the receptive field, thereby capturing the long-distance dependency features in the second residual result; A point convolution unit connected to the depth and extension convolution units, and configured to perform channel fusion on the output results of the depth and extension convolution units through a 1×1 convolution operation, so as to integrate feature information between different channels and obtain channel fusion features; Among them, the large kernel convolution processing result is a fused feature map obtained after being processed in sequence by the depth convolution unit, the depth and extension convolution unit and the point convolution unit. The fused feature map also includes the local spatial features, long-distance dependency features and channel fusion features of the second residual result.

6. The speech recognition method based on large kernel convolution and linear self-attention according to claim 5, characterized in that: in, The linear self-attention module uses a linear complex self-attention mechanism to process the first residual result. The self-attention formula used is: Y is the result of linear self-attention processing; Q, K, and V are query, key, and value matrices, respectively; is the kernel function mapping; is a normalized diagonal matrix; where 1 is a vector of all ones.

7. The speech recognition method based on large kernel convolution and linear self-attention according to claim 6, It is characterized in that in, In the depth and dilation convolution units, the size and dilation rate of the dilation convolution kernel are adjusted according to formula (1) and formula (2) respectively; The formula (1) is: k t =k min +σ(W k ·X)×(k max -k min ); Among them, k min is the minimum size of the convolution kernel in the depth and dilation convolution unit; k max is the maximum size of the convolution kernel in the depth and dilation convolution unit; σ() is the Sigmoid activation function; W k is the preset first trainable parameter matrix; X is the data of input depth and dilated convolution unit; The formula (2) is: d t =d min +tanh(W d ·X)×(d max -d min ); Among them, d max is the maximum value of the preset expansion rate; d min is the minimum value of the preset expansion rate; tanh() is the Tanh activation function; W d is the second trainable parameter matrix.

8. The speech recognition method based on large kernel convolution and linear self-attention according to claim 7, characterized in that: The S0 specifically includes: Using each first acoustic signal in the training data set and the corresponding pre-annotated speech text, the LKCovLFormer model is trained until a preset regression loss function corresponding to the speech recognition model converges, thereby obtaining a trained LKCovLFormer model; The AdamW optimization algorithm is used during the training of the LKCovLFormer model to minimize the preset regression loss function and update all weight parameters in the LKCovLFormer model.

9. The speech recognition method based on large kernel convolution and linear self-attention according to claim 7, characterized in that: During the training of the LKCovLFormer model, the learning rate of the tth iteration is calculated by formula (3); The formula (3) is: Among them, y t is the learning rate of the tth iteration; y0 is the initial learning rate; g i is the gradient value of the tth iteration; ∈ is a preset constant.

10. The speech recognition method based on large kernel convolution and linear self-attention according to claim 8, characterized in that: in, 6≤N≤12。

Images