Speech recognition method and device, computer equipment and storage medium
By combining speech recognition methods with timing and spatial characteristics, the problem of insufficient speech recognition accuracy in the prior art is solved, and the user experience of the financial service platform is improved.
Patent Information
- Application Number
- CN202410916837.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-09
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-07-09
AI Technical Summary
In the prior art, speech recognition mainly relies on speech features in the timing dimension, and lacks speech information in the spatial dimension, resulting in insufficient recognition accuracy and affecting users' experience on the financial service platform.
A speech recognition method is proposed, through the target time feature extraction network and the target spatial feature extraction network extracts timing features and spatial features from the target speech data, and combines them for speech recognition, and enhances features using self-attention network and compression excitation network Extraction and fusion.
It improves the accuracy of speech recognition, enhances the perception and expression ability in the speech recognition process, and improves the user experience of the financial service platform.
Smart Images

Figure CN118942451A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of financial technology, and in particular to a speech recognition method and apparatus, computer equipment and storage medium. Background Art
[0002] With the combination of computer technology and the financial field, more and more intelligent platforms provide financial services to users to improve the user experience when handling financial business. For example, in insurance business services, in order to save the configuration of insurance business personnel, routine insurance consultation questions are usually answered by financial intelligent robots. However, in order to improve the accuracy of voice answers and improve the user experience in the question-and-answer process, it is necessary to recognize the user's voice and determine the meaning of the user's question based on the voice recognition results. In related technologies, speech recognition mainly recognizes speech features of different time series and completes speech recognition based on the speech features. However, only realizing speech recognition from the speech features of the time dimension cannot accurately identify the true meaning of the speech, which affects the accuracy of speech recognition. Therefore, how to improve the accuracy of speech recognition has become a technical problem that needs to be solved urgently. Summary of the invention
[0003] The main purpose of the embodiments of the present application is to propose a speech recognition method and apparatus, a computer device and a storage medium, aiming to improve the accuracy of speech recognition.
[0004] To achieve the above object, a first aspect of an embodiment of the present application provides a speech recognition method, the method comprising:
[0005] Obtain target voice data;
[0006] Inputting the target speech data into a preset target speech recognition model; wherein the target speech recognition model includes: a target time feature extraction network, a target space feature extraction network and a target speech recognition network;
[0007] Extracting features from the target speech data through the target time feature extraction network to obtain target speech time series features;
[0008] Extracting features from the target speech data through the target space feature extraction network to obtain target speech space features;
[0009] Speech recognition is performed on the target speech temporal features and the target speech spatial features through the target speech recognition network.
[0010] In some embodiments, the target space feature extraction network is a compression excitation network, and the compression excitation network includes: a linear layer and a convolutional layer; the target space feature extraction network is used to extract features from the target speech data to obtain the target speech space features, including:
[0011] Performing feature extraction on the target speech data through the linear layer to obtain at least two initial speech space features; wherein each of the initial speech space features represents a speech feature on a spatial channel;
[0012] Concatenate at least two of the initial speech space features through the linear layer to obtain candidate speech space features;
[0013] The candidate speech spatial features are weightedly calculated through the convolution layer to obtain the target speech spatial features.
[0014] In some embodiments, the step of concatenating at least two of the initial speech space features through the linear layer to obtain a candidate speech space feature includes:
[0015] Performing dimension expansion on each of the initial speech space features through the linear layer to obtain preliminary speech space features;
[0016] At least two of the preliminary speech space features are concatenated through the linear layer to obtain the candidate speech space feature.
[0017] In some embodiments, performing weighted calculation on the candidate speech spatial features through the convolution layer to obtain the target speech spatial features includes:
[0018] Performing pooling processing on the candidate speech spatial features through the convolution layer to obtain speech pooling features; performing normalization processing on the speech pooling features through the convolution layer to obtain a preset weight for each spatial channel;
[0019] The candidate speech spatial features and the preset weights are weightedly summed through the convolution layer to obtain the target speech spatial features.
[0020] In some embodiments, the temporal feature extraction network is a self-attention network, and the self-attention network includes: a feature extraction layer and a self-attention layer; the target speech data is subjected to feature extraction by the target temporal feature extraction network to obtain the target speech temporal feature, including:
[0021] Extracting features from the target speech data through the feature extraction layer to obtain preliminary speech time sequence features;
[0022] Performing self-attention calculation on the preliminary speech time sequence features through the self-attention layer to obtain a self-attention weight;
[0023] The preliminary speech timing features are weighted and calculated according to the self-attention weights to obtain the target speech timing features.
[0024] In some embodiments, performing speech recognition on the target speech temporal features and the target speech spatial features through a target speech recognition network includes:
[0025] Performing feature fusion on the target speech temporal features and the target speech spatial features through the target speech recognition network to obtain target speech fusion features;
[0026] The target speech recognition network performs speech recognition on the target speech fusion features.
[0027] In some embodiments, before inputting the target speech data into a preset target speech recognition model, the method further includes:
[0028] Constructing the target speech recognition model specifically includes:
[0029] Get training speech data;
[0030] The training speech data is input into a preset original speech recognition model; wherein the original speech recognition model includes: an original time feature extraction network, an original space feature extraction network and an original speech recognition network:
[0031] Performing feature extraction on the training speech data through the original time feature extraction network to obtain training speech time series features;
[0032] Performing feature extraction on the training speech data through the original spatial feature extraction network to obtain training speech spatial features;
[0033] Performing speech recognition on the training speech temporal features and the training speech spatial features through the original speech recognition network to obtain a speech recognition result;
[0034] The parameters of the original speech recognition model are adjusted according to the preset speech verification result and the speech recognition result to obtain the target speech recognition model.
[0035] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a speech recognition device, the device comprising:
[0036] A data acquisition module, used to acquire target voice data;
[0037] A speech data processing module, used for inputting the target speech data into a preset target speech recognition model; wherein the target speech recognition model comprises: a target time feature extraction network, a target space feature extraction network and a target speech recognition network;
[0038] A time series feature extraction module, used to extract features from the target speech data through the target time feature extraction network to obtain target speech time series features;
[0039] A spatial feature extraction module, used for extracting features from the target speech data through the target spatial feature extraction network to obtain target speech spatial features;
[0040] The speech recognition module is used to perform speech recognition on the target speech temporal features and the target speech spatial features through a target speech recognition network.
[0041] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes a computer device, which includes a memory and a processor, the memory stores a computer program, and the processor implements the method described in the first aspect when executing the computer program.
[0042] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.
[0043] The speech recognition method and apparatus, computer equipment and storage medium proposed in the present application input the target speech data into a target speech recognition model including a target time feature extraction network, a target space feature extraction network and a target speech recognition network, extract the target speech time sequence features from the target speech data through the target time feature extraction network, and then extract the target speech space features from the target speech data through the target space feature extraction network, and then perform speech recognition on the target speech time sequence features and the target speech space features through the target speech recognition network. Therefore, speech recognition is performed by combining speech time sequence features and speech space features, enhancing different types of speech features in speech recognition, better learning the features in speech data, and improving the accuracy of speech recognition. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Figure 1 is a flow chart of a speech recognition method provided by an embodiment of the present application;
[0045] Figure 2 is a flow chart of a speech recognition method provided by another embodiment of the present application;
[0046] Figure 3is a schematic diagram of the structure of a target speech recognition model in the speech recognition method provided in an embodiment of the present application;
[0047] Figure 4 yes Figure 1 Flow chart of step S103 in FIG.
[0048] Figure 5 It is a structural diagram of a compression excitation network in a speech recognition method provided in an embodiment of the present application;
[0049] Figure 6 yes Figure 1 Flow chart of step S104 in FIG.
[0050] Figure 7 yes Figure 6 Flowchart of step S602 in FIG.
[0051] Figure 8 It is a schematic diagram of the process of the linear layer processing the target speech data in the speech recognition method provided in the embodiment of the present application;
[0052] Fig. 9 yes Figure 6 Flow chart of step S603 in FIG.
[0053] Fig.10 yes Figure 1 Flow chart of step S105 in FIG.
[0054] Fig.11 is an overall flow chart of the speech recognition method provided by the embodiment of the present application;
[0055] Fig.12 is a structural diagram of a speech recognition device provided in an embodiment of the present application;
[0056] Fig.13 It is a schematic diagram of the hardware structure of the computer device provided in the embodiment of the present application. DETAILED DESCRIPTION
[0057] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0058] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.
[0059] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.
[0060] First, some nouns involved in this application are analyzed:
[0061] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.
[0062] Speech Recognition: Also known as Automatic Speech Recognition (ASR), it refers to the process by which a computer or device converts human-speaking speech signals into corresponding text or instructions. The speech recognition system extracts the basic units of speech (such as phonemes and words) by analyzing the frequency, amplitude and time domain characteristics of speech signals, and then combines these units into understandable text or commands through matching and modeling.
[0063] Self-Attention Mechanism: Also known as the internal attention mechanism, it is a special form of attention mechanism that is used to process sequence data (such as text) so that each element in the sequence can interact with other elements in the sequence to capture global dependencies. The self-attention mechanism has been widely used in the Transformer model and is one of its core components.
[0064] Squeeze-and-Excitation Network (SENet): is a technology that improves the architecture of convolutional neural networks (CNNs) to enhance the network's ability to dynamically adjust the importance of features. The core idea of SENet is to improve feature representation capabilities by explicitly modeling the dependencies between channels. SENet introduces a "Squeeze-and-Excitation" module, referred to as the SE module, which can be inserted into existing convolutional neural networks and improve the performance of the model by re-weighting channel features.
[0065] Feedforward Neural Network (FNN): It is the most basic and simplest artificial neural network structure. Information in a feedforward network only propagates in one direction, that is, from the input layer through one or more hidden layers to the output layer, and there is no loop or feedback connection.
[0066] In order to adapt to the development of intelligence, different types of financial service platforms have been set up in the financial field. In order to improve the user experience when using the financial service platform, voice recognition function will be added. Users can consult and handle financial business on the financial service platform through voice. Therefore, the accuracy of voice recognition affects the user experience on the financial service platform, and improving the accuracy of voice recognition has also become a key concern in the financial field.
[0067] In related technologies, speech recognition is mainly achieved through the attention mechanism, which performs weighted aggregation of input feature sequences to achieve learning of prosodic information and weight allocation of speech features of different time sequences. Since the time sequence information in speech signals is very important, speech features of different time sequences are mixed before speech recognition during the speech recognition process. However, speech recognition is performed only through speech features in the time sequence dimension, lacking speech information in space, which affects the accuracy of speech recognition.
[0068] Based on this, the embodiments of the present application provide a speech recognition method and apparatus, a computer device and a storage medium, which extracts target speech time series features from target speech data through a target time feature extraction network, extracts target speech space features from target speech data through a target space feature extraction network, and then performs speech recognition by combining the target speech time series features and the target speech space features. In the speech recognition process, speech features between time and space are integrated to enhance the perception and expression capabilities of speech recognition and improve the accuracy of speech recognition.
[0069] The speech recognition method and apparatus, computer equipment and storage medium provided in the embodiments of the present application are specifically described through the following embodiments. First, the speech recognition method in the embodiments of the present application is described.
[0070] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.
[0071] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.
[0072] The speech recognition method provided in the embodiment of the present application relates to the field of artificial intelligence technology. The speech recognition method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the speech recognition method, etc., but is not limited to the above forms.
[0073] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer computer devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.
[0074] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.
[0075] Figure 1 is an optional flow chart of the speech recognition method provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S105.
[0076] Step S101, obtaining target voice data;
[0077] Step S102, inputting the target speech data into a preset target speech recognition model; wherein the target speech recognition model includes: a target time feature extraction network, a target space feature extraction network and a target speech recognition network;
[0078] Step S103, extracting features from the target speech data through a target time feature extraction network to obtain target speech time series features;
[0079] Step S104, extracting features from the target speech data through a target space feature extraction network to obtain target speech space features;
[0080] Step S105, performing speech recognition on the target speech temporal features and the target speech spatial features through a target speech recognition network.
[0081] Steps S101 to S105 shown in the embodiment of the present application are performed by constructing a target speech recognition module composed of a target time feature extraction network, a target space feature extraction network and a target speech recognition network. The target speech time series features are extracted from the target speech data by the target time feature extraction network, and the target speech space features are extracted from the target speech data by the target space feature extraction network. Then, speech recognition is performed by combining the target speech time series features and the target speech space features. This can enhance the perception and expression capabilities during the speech recognition process to improve the accuracy of speech recognition.
[0082] In step S101 of some embodiments, the speech recognition method is applied to an insurance service platform in the financial field. The target voice data is collected through an audio recording device in the insurance service platform, or the target voice data is uploaded to the insurance service platform through a user terminal. This embodiment does not impose any specific restrictions on the method of obtaining the target voice data.
[0083] Before step S101 of some embodiments, the speech recognition method further includes constructing a target speech recognition model, which is used to recognize the speech content of the target speech data. In some embodiments, the target speech recognition model includes: a target time feature extraction network, a target space feature extraction network and a target speech recognition network, the target time feature extraction network is a self-attention network, the self-attention network is composed of multiple self-attention layers, each self-attention layer contains a self-attention mechanism, and the context information of the target speech data in the time series is captured through the self-attention mechanism to improve the accuracy of speech recognition. The target space feature extraction network is a compression excitation network, referred to as SENet, and the compression excitation network includes a feature perception module, and the feature perception module is an SE module, which can enhance the convolution layer's perception ability of feature importance and output speech features that characterize spatial relationships. The target speech recognition network is a feedforward network, which performs speech recognition based on the input speech features and outputs speech recognition results.
[0084] In some embodiments, the target speech recognition model is obtained by training the original speech recognition model, and the original speech recognition model includes: an original time feature extraction network, an original space feature extraction network and an original speech recognition network. After the original time feature extraction network is trained, the target time feature extraction network is obtained, and the structure of the original time feature extraction network is the same as the target time feature extraction network; after the original space feature extraction network is trained, the target space feature extraction network is obtained, and the structure of the original space feature extraction network is the same as the target space feature extraction network; after the original speech recognition network is trained, the target speech recognition network is obtained, and the structure of the original speech recognition network is the same as the target speech recognition network.
[0085] See also Figure 2 In some embodiments, building a target speech recognition model may include but is not limited to steps S201 to S206:
[0086] Step S201, obtaining training speech data;
[0087] Step S202, input the training speech data into a preset original speech recognition model; wherein the original speech recognition model includes: an original time feature extraction network, an original space feature extraction network and an original speech recognition network:
[0088] Step S203, extracting features from the training speech data through the original time feature extraction network to obtain the training speech time series features;
[0089] Step S204, extracting features from the training speech data using the original spatial feature extraction network to obtain training speech spatial features;
[0090] Step S205, performing speech recognition on the training speech temporal features and the training speech spatial features through the original speech recognition network to obtain a speech recognition result;
[0091] Step S206, adjusting the parameters of the original speech recognition model according to the preset speech verification result and speech recognition result to obtain a target speech recognition model.
[0092] In step S201 of some embodiments, the method for acquiring training voice data is the same as the method for acquiring target voice data, and both are acquired by the insurance service platform. However, the training voice data is not acquired in real time through the insurance service platform, but is acquired before the voice recognition function in the insurance service platform is started, and is also acquired regularly during the use of the voice recognition function of the insurance service platform. It should be noted that the training voice data is acquired before the insurance service platform starts the voice recognition function, and is used to construct a target voice recognition model. When the insurance service platform starts the voice recognition function, the target voice data collected according to the preset period is stored in a pre-set database, and the target voice data is taken out from the database according to the preset period as training voice data, so the training voice data after the voice recognition function is started is used to update the target voice recognition model and continuously improve the recognition performance of the target voice recognition model.
[0093] In step S202 of some embodiments, an original speech recognition model consisting of an original time feature extraction network, an original space feature extraction network and an original speech recognition network is constructed. Then, the original speech recognition model can obtain speech features in time and space, and perform speech recognition after fusing the speech features in time and space. Therefore, speech recognition can understand the relationship between different parts of the speech, and can better understand the complete speech data to achieve accurate speech recognition.
[0094] It should be noted that the original spatial feature extraction network is a compression excitation network, which is a network specifically used for image classification. It optimizes the attention allocation of spatial channels in speech recognition by automatically learning the correlation between channels, and can extract key features from a large amount of speech data and output speech features that more accurately represent spatial relationships. The original temporal feature extraction network uses a self-attention mechanism to obtain speech features from the temporal sequence of speech data, and the self-attention mechanism can pay attention to the correlation of speech data at different temporal sequences and output speech features that more accurately represent temporal relationships.
[0095] In step S203 of some embodiments, the original time feature extraction network is a self-attention network, and the self-attention network includes multiple self-attention layers, and the self-attention mechanism of the self-attention layer can capture the speech features of the training speech data in time sequence, and output the training speech time sequence features that accurately represent the time sequence relationship. It should be noted that the training speech time sequence features are represented by sequences, and the numerical values in the sequences represent the speech information at different time points. The context information of the training speech data on the time axis can be determined by the training speech time sequence features.
[0096] In step S204 of some embodiments, the original spatial feature extraction network is a compression excitation network, and the compression excitation network enhances the perception of speech features by the convolution layer by introducing the SE module. Therefore, speech features that more accurately represent spatial relationships are extracted from the training speech data through the original spatial feature extraction network. It should be noted that the training speech spatial features can represent the spatial relationship of speech, so as to judge the relationship between different parts of speech in space, which is conducive to understanding the information in the speech data.
[0097] In step S205 of some embodiments, the training speech temporal features and the training speech spatial features are simultaneously input into the original speech recognition network, and the original speech recognition network combines the temporal and spatial speech features for speech recognition, which can more accurately recognize the information in the speech features to output accurate speech recognition results. It should be noted that the speech recognition result can be the phonemes, speech content or speech commands of the training speech data, and the speech recognition result is not specifically limited here.
[0098] In step S206 of some embodiments, the speech verification result is a recognition result pre-configured by the training speech data, and is used to determine the accuracy of speech recognition by the original speech recognition model. Specifically, recognition loss data is obtained by performing loss calculation on the language verification result and the language recognition result, and the recognition loss data represents the recognition error of the original speech recognition model. The parameters of the original speech recognition model are adjusted according to the recognition loss data, and the original speech recognition model with adjusted parameters is used as the target speech recognition model after the recognition loss data reaches the minimum value.
[0099] For example, if the original speech recognition model is used to recognize the content of the training speech data, the speech verification result is set in advance, and the speech verification result is a preset insurance question, such as "What types of insurance can a person who is 50 years old and suffers from coronary heart disease buy?". The corresponding speech data is manually recorded as training speech data according to the insurance question, and then the training speech data is input into the original speech model to obtain the speech recognition result. If the speech recognition result is "What types of insurance can a person who is 50 years old and suffers from coronary heart disease buy?", the recognition loss data between the speech verification result and the language recognition result is calculated, and the recognition loss data represents the error of speech content recognition. The original speech recognition model is adjusted by the recognition loss data until the speech recognition result output by the original speech recognition model is "What types of insurance can a person who is 50 years old and suffers from coronary heart disease buy?", so as to complete the training of the original speech recognition model, and the trained original speech recognition model is used as the target speech recognition model.
[0100] As disclosed above, after the target speech recognition model is constructed, when the speech recognition function is started on the insurance service platform, if target speech data is received, the target speech data is directly input into the target speech recognition model, and speech recognition is performed on the target speech data through the target speech recognition model.
[0101] In step S102 of some embodiments, the structure of the target speech recognition model is as follows: Figure 3 As shown, through Figure 3 It can be seen that the target speech recognition model includes: a target time feature extraction network, a target space feature extraction network and a target speech recognition network. The target time feature extraction network is a self-attention network, the target space feature extraction network is a compression excitation network, and the target speech recognition network is a feedforward network. The target speech data is input into the self-attention network and the compression excitation network respectively, and the features output by the self-attention network and the compression excitation network are fused and input into the feedforward network. The feedforward network combines the spatial and temporal speech features for speech recognition, thereby improving the accuracy of speech recognition.
[0102] In some embodiments, the self-attention network includes: a feature extraction layer and a self-attention layer, and at least two self-attention layers are provided, and each self-attention layer includes: a self-attention mechanism component and a normalization layer, the self-attention mechanism component is used to identify the importance of different time points in the speech features, and the normalization layer is used to normalize the importance of different time points.
[0103] See also Figure 4 In some embodiments, step S103 may include but is not limited to steps S401 to S403:
[0104] Step S401, extracting features from the target speech data through a feature extraction layer to obtain preliminary speech time series features;
[0105] Step S402, performing self-attention calculation on the preliminary speech time sequence features through the self-attention layer to obtain a self-attention weight;
[0106] Step S403, weighted calculation is performed on the preliminary speech timing features according to the self-attention weights to obtain the target speech timing features.
[0107] In step S401 of some embodiments, the feature extraction layer is also called a linear transformation layer, and the target speech data is mapped to the space corresponding to the query, key and value through the linear transformation layer to obtain preliminary speech time series features. It should be noted that the preliminary speech time series features are a sequence including a query vector, a key vector and a value vector, and the importance of speech features at different time points can be calculated through the query vector, the key vector and the value vector.
[0108] In step S402 of some embodiments, the self-attention layer performs self-attention calculation on the preliminary speech time series features, that is, calculates the similarity between the query vector and the key vector as the self-attention weight. It should be noted that the self-attention weight represents the importance of speech features at different time points, and the speech features at different time points are paid attention to through the self-attention weight.
[0109] In step S403 of some embodiments, the self-attention weight and the preliminary speech timing feature are weightedly calculated, that is, the feature importance in the preliminary speech timing feature is adjusted to obtain the target speech timing feature, and the target speech timing feature can accurately represent the speech information at different time points.
[0110] In steps S401 to S403 shown in the present embodiment, the feature extraction layer first performs feature extraction on the target speech data to obtain preliminary speech timing features, and then the self-attention layer performs self-attention calculation on the preliminary speech timing features to calculate the self-attention weights representing the importance of the speech features at different time points, and finally the preliminary speech timing features are weighted according to the self-attention weights to output target speech timing features that more accurately represent the time relationship.
[0111] See also Figure 5 In some embodiments, the compression excitation network includes: a linear layer and a convolutional layer. It should be noted that at least two linear layers are provided, and the outputs of at least two linear layers are connected to the convolutional layer, an SE module is added to the convolutional layer, and the SE module can enhance the perception ability of the convolutional layer and improve the performance of the compression excitation network.
[0112] See also Figure 6In some embodiments, step S104 may include but is not limited to steps S601 to S603:
[0113] Step S601, extracting features from the target speech data through a linear layer to obtain at least two initial speech space features; wherein each initial speech space feature represents a speech feature on a spatial channel;
[0114] Step S602, concatenating at least two initial speech space features through a linear layer to obtain candidate speech space features;
[0115] Step S603, weighted calculation is performed on the candidate speech spatial features through a convolution layer to obtain the target speech spatial features.
[0116] In step S601 of some embodiments, the linear layer extracts features from the target speech data, that is, the target speech data is mapped to each spatial channel through the linear layer to obtain initial speech spatial features, and at least two initial speech spatial features are features under different spatial channel mappings, representing speech features under different spatial channels. It should be noted that at least two linear layers are provided, each linear layer maps the target speech data to a space, and different linear layers are configured with different spaces.
[0117] In step S602 of some embodiments, at least two initial speech spatial features are concatenated through a linear layer. It should be noted that concatenation is to combine initial speech spatial features under different spatial channel mappings, so the candidate speech spatial features represent speech features of different spatial relationships.
[0118] In step S603 of some embodiments, an SE module is added to the convolution layer. The SE module can calculate the importance of each spatial channel in the convolution layer, and then weight the candidate speech spatial features of different spatial channels according to the importance of the channel to obtain speech features that more accurately represent the spatial relationship.
[0119] In steps S601 to S603 shown in this embodiment, a compression excitation network including a linear layer and a convolutional layer is set up to extract at least two initial speech space features from the target speech data through the linear layer, and then the at least two initial speech space features are spliced into candidate speech space features through the linear layer. Finally, the convolutional layer weights the candidate speech space features to obtain target speech space features that more accurately represent spatial relationships.
[0120] See also Figure 7 In some embodiments, step S602 may include but is not limited to steps S701 to S702:
[0121] Step S701, dimensionally expanding each initial speech space feature through a linear layer to obtain a preliminary speech space feature;
[0122] Step S702, concatenating at least two preliminary speech space features through a linear layer to obtain candidate speech space features.
[0123] In step S701 of some embodiments, the dimension of the initial speech space feature is the same as the dimension of the target speech data, and the candidate speech space feature is a speech feature containing different spatial features, so the dimension of the candidate speech space feature is greater than the initial speech space feature, so the initial speech space feature cannot be directly spliced into the candidate speech space feature. Therefore, each initial speech space feature is dimensionally expanded through a linear layer, and in this embodiment, the initial speech space feature is expanded by one channel dimension to obtain a preliminary speech space feature.
[0124] For example, if the target speech data is defined as X, and the shape of the target speech data is (B, T, D), and the shape of the initial speech space feature is also (B, T, D), the initial speech space feature is expanded by one channel dimension to obtain the preliminary speech space feature, and the shape of the preliminary speech space feature is (B, 1, T, D).
[0125] In step S702 of some embodiments, at least two preliminary speech spatial features are spliced into a candidate speech spatial feature, that is, speech spatial features of different spatial channels are spliced into a candidate speech spatial feature containing multiple spatial channels.
[0126] For example, Figure 8 As shown, the shape of the preliminary speech space feature is (B, 1, T, D). In this embodiment, C linear layers are set, that is, there are C preliminary speech space features. By concatenating the C preliminary speech space features, a target speech space feature with a shape of (B, C, T, D) is obtained.
[0127] Specifically, for the target speech data X, different spatial channels are used to represent it as shown in equations (1) to (3):
[0128] X′ i =X*W1 i +b1 i (1)
[0129] X′ i =X′ i .unsqueeze (2)
[0130] X″=concat(X′1,…,X′ i ) (3)
[0131] In the formula, X′ iis the i-th initial speech space feature, W1 i is the weight of the spatial channel of the linear layer, b1 i is the bias of the spatial channel of the linear layer, unsqueeze represents the dimension expansion of the initial speech space feature, X″ is the candidate speech space feature, and concat represents the concatenation of the preliminary speech space features.
[0132] In steps S701 to S702 shown in this embodiment, the initial speech space features are dimensionally expanded to obtain preliminary speech space features, and then at least two preliminary speech space features are spliced into candidate speech space features to construct candidate speech space features under the fusion of different spatial channels.
[0133] See also Fig. 9 In some embodiments, step S603 includes but is not limited to steps S901 to S903:
[0134] Step S901, performing pooling processing on the candidate speech spatial features through a convolutional layer to obtain speech pooling features;
[0135] Step S902, normalizing the speech pooling features through a convolutional layer to obtain a preset weight for each spatial channel;
[0136] Step S903, performing weighted summation on the candidate speech spatial features and preset weights through a convolution layer to obtain the target speech spatial features.
[0137] It should be noted that the convolutional layer of this embodiment adds an SE module, and the SE module includes two stages, namely compression and excitation. The compression stage performs pooling processing on the speech features in each spatial channel, and the excitation stage is used to generate the weight of each spatial channel.
[0138] In step S901 of some embodiments, the convolution layer performs pooling processing on the candidate speech spatial features, and the pooling processing is the compression stage of the SE module, that is, the speech features of each spatial channel are determined by the candidate speech spatial features, and the speech features of each spatial channel are compressed into a global description vector. It should be noted that the speech pooling feature is a global description vector, and the global information of each spatial channel is determined by the global description vector.
[0139] Specifically, the candidate speech spatial features are pooled through the convolution layer to obtain the speech pooling features as shown in formula (4):
[0140]
[0141] Where T and D are the height and width of the candidate speech spatial feature, respectively, z is the speech pooling feature, and X″(i, j) is the value of the input candidate speech spatial feature at position (i, j).
[0142] In step S902 of some embodiments, normalization processing is also the excitation stage of the SE module, and the speech pooling feature is a global description vector of each spatial channel. By normalizing the global description vector, a preset weight for each spatial channel is generated, and the preset weight represents the importance of each spatial channel.
[0143] Specifically, the speech pooling features are normalized through the convolution layer to obtain the preset weights as shown in formula (5):
[0144] S=F ex (z)=softmax(z) (5)
[0145] In the formula, softmax is the normalization function.
[0146] In step S903 of some embodiments, at least two selected speech spatial features are obtained by weighted summing the candidate speech spatial features and preset weights, that is, weighting different spatial channels of the candidate speech spatial features according to the preset weights, and then summing the at least two selected speech spatial features to obtain the target speech spatial features.
[0147] For example, the preset weight S is obtained by the above formulas (4) and (5), and the preset weight is assigned to the candidate speech space features to obtain at least two selected speech space features as shown in formula (6):
[0148] X″′ c =S×X″ (6)
[0149] After obtaining the selected speech space feature, at least two selected speech space features are summed to obtain the target speech space feature as shown in formula (7):
[0150]
[0151] In the formula, X″′ c is the selected speech spatial feature corresponding to the cth spatial channel.
[0152] In steps S901 to S903 shown in this embodiment, the candidate speech spatial features are pooled through a convolutional layer and then normalized to implement the two stages of compression and activation of the SE module, and preset weights for characterizing different spatial channels are obtained. The candidate speech spatial features are weighted and summed through the preset weights to obtain target speech spatial features that more accurately characterize spatial relationships.
[0153] See also Fig.10In some embodiments, step S105 may include but is not limited to steps S1001 to S1002:
[0154] Step S1001, performing feature fusion on the target speech temporal features and the target speech spatial features through a target speech recognition network to obtain a target speech fusion feature;
[0155] Step S1002, performing speech recognition on the target speech fusion features through a target speech recognition network.
[0156] It should be noted that the target speech recognition network is a feedforward network, and the target speech recognition network is trained through the original speech recognition network. The training speech temporal features and the training speech spatial features are learned during the training process of the original speech recognition network. Therefore, the trained target speech recognition network has learned rich speech features and can make accurate speech recognition of the target speech temporal features and the target speech spatial features.
[0157] In step S1001 of some embodiments, the target speech temporal features and the target speech spatial features are fused into target speech fusion features, so that the target speech recognition network can perform speech recognition on the target speech fusion features.
[0158] In step S1002 of some embodiments, the target speech recognition network is a feedforward network, and the feedforward network can extract effective information from the speech features, so the feedforward network of this embodiment can extract more effective speech information from the target speech fusion features that fuse the target speech spatial features and the target speech temporal features, thereby achieving accurate speech recognition.
[0159] In steps S1001 to S1002 shown in this embodiment, the target speech temporal features and the target speech spatial features are first fused into target speech fusion features, and then speech recognition is performed on the target speech fusion features, making the speech recognition operation simpler and more accurate.
[0160] In summary, if Fig.11 As shown, Fig.11 The overall flow chart of the speech recognition method according to the embodiment of the present application is shown.
[0161] Step S1101, training a preset original speech recognition model according to training speech data and preset speech verification results to obtain a target speech recognition model.
[0162] It should be noted that this embodiment is applied to the field of insurance services. By setting up an insurance service platform, voice data is recorded in advance according to a pre-configured voice question and answer text as training voice data, and the voice question and answer sentences in the voice question and answer text corresponding to each training voice data are used as voice verification results. Among them, the original voice recognition model includes: an original time feature extraction network, an original space feature extraction network and an original voice recognition network. The training voice temporal features are extracted from the training voice data through the original time feature extraction network, and the training voice spatial features are extracted from the training voice data through the original space feature extraction network. Then, the training voice temporal features and the training voice spatial features are voice recognized by the original voice recognition network to obtain the voice recognition results. It should be noted that the voice recognition results and the voice verification results are calculated for loss to obtain recognition loss data, and the recognition loss data represents the recognition error of the original voice recognition model, and the original voice recognition model is adjusted for parameters through the recognition loss data. Specifically, in the process of adjusting the original voice recognition model, the parameters of the original time feature extraction network, the original space feature extraction network and the original voice recognition network are adjusted based on the recognition loss data. It should be noted that the original speech recognition network is first parameterized so that the original speech recognition network learns speech temporal features and speech spatial features. Then, the original speech recognition network learns richer speech features, and a target speech recognition network with more accurate speech recognition is obtained.
[0163] Step S1102, obtaining target speech data, and inputting the target speech data into a target speech recognition model; wherein the target speech recognition model includes a target time feature extraction network, a target space feature extraction network and a target speech recognition network.
[0164] As disclosed above, after the training of the original speech recognition model is completed, the target speech recognition model is obtained, and the target speech recognition model is applied to the startup process of the speech recognition function of the insurance service platform. When the insurance service platform receives the speech recognition startup instruction, it will collect speech data in real time through the audio recording device as the target speech data.
[0165] In some embodiments, the target speech recognition model includes a target time feature extraction network, a target space feature extraction network and a target speech recognition network, wherein the target time feature extraction network is a self-attention network, and the self-attention network can automatically learn speech features at different time points and output accurate speech timing features. The target space feature extraction network is a compression excitation network, and the compression excitation network can enhance the attention of key speech features in speech data and output accurate speech space features. The target speech recognition network is a feedforward network, and the feedforward network can extract effective speech information based on speech features to achieve accurate speech recognition.
[0166] Step S1103, extracting features from the target speech data through a target time feature extraction network to obtain target speech time series features.
[0167] It should be noted that the target time feature extraction network is a self-attention network, and the self-attention network includes a feature extraction layer and a self-attention layer. The feature extraction layer is also called a linear transformation layer. The target speech data is mapped to the space corresponding to the query, key, and value through the linear transformation layer to obtain the preliminary speech time sequence features. Then, the preliminary speech time sequence features are self-attentionally calculated through the self-attention layer to obtain the self-attention weights that characterize the importance of each feature in the preliminary speech time sequence features. Finally, the preliminary speech time sequence features are weighted based on the self-attention weights to obtain the target speech time sequence features that more accurately characterize the time sequence relationship.
[0168] Step S1104, extracting features from the target speech data through a target space feature extraction network to obtain target speech space features.
[0169] It should be noted that the target space feature extraction network is an excitation compression network, and the excitation compression network includes a linear layer and a convolutional layer. The linear layer first maps the target speech data to different spatial channels to obtain at least two initial speech space features, and then amplifies each initial speech space feature by one dimension through the linear layer to obtain a preliminary speech space feature. Therefore, the at least two preliminary speech space features after the expanded dimension are spliced into candidate speech space features, and the candidate speech space features are speech features after the dimension has changed. Furthermore, the candidate speech space features are pooled through the convolutional layer to obtain a speech pooling feature that represents the global description vector, and the global description vector represents the global information of each spatial channel, so the global description vector is normalized to obtain a preset weight that represents the importance of each spatial channel. Finally, the preset weight and the candidate speech space feature are weighted and summed to obtain a target speech space feature that represents the spatial relationship more accurately.
[0170] Step S1105, performing speech recognition on the target speech temporal features and the target speech spatial features through a target speech recognition network.
[0171] As disclosed above, after the extraction of the target speech temporal features and the target speech spatial features is completed, speech recognition is performed in combination with the target speech temporal features and the target speech spatial features, so that the speech recognition process can better understand the speech characteristics of the speech data and improve the accuracy of speech recognition.
[0172] It should be noted that after completing speech recognition, feedback data from the target speech recognition network for speech recognition will be received, and the target speech recognition network will be updated based on the feedback data to improve the accuracy of speech recognition by the target speech recognition network.
[0173] See also Fig.12 The present application also provides a speech recognition device that can implement the above-mentioned speech recognition method. The device includes:
[0174] The data acquisition module 1201 is used to acquire target voice data;
[0175] The speech data processing module 1202 is used to input the target speech data into a preset target speech recognition model; wherein the target speech recognition model includes: a target time feature extraction network, a target space feature extraction network and a target speech recognition network;
[0176] The time series feature extraction module 1203 is used to extract the target speech data through the target time feature extraction network to obtain the target speech time series feature;
[0177] The spatial feature extraction module 1204 is used to extract the target speech data through the target spatial feature extraction network to obtain the target speech spatial features;
[0178] The speech recognition module 1205 is used to perform speech recognition on the target speech temporal features and the target speech spatial features through a target speech recognition network.
[0179] The specific implementation of the speech recognition device is substantially the same as the specific implementation of the speech recognition method described above, and will not be described in detail herein.
[0180] The embodiment of the present application also provides a computer device, the computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned speech recognition method when executing the computer program. The computer device can be any intelligent terminal including a tablet computer, a car computer, etc.
[0181] See also Fig.13 , Fig.13 The hardware structure of a computer device according to another embodiment is shown, and the computer device includes:
[0182] The processor 1301 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;
[0183] The memory 1302 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1302 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1302, and the processor 1301 calls and executes the speech recognition method of the embodiment of this application;
[0184] Input / output interface 1303, used to implement information input and output;
[0185] The communication interface 1304 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);
[0186] A bus 1305 that transmits information between the various components of the device (e.g., the processor 1301, the memory 1302, the input / output interface 1303, and the communication interface 1304);
[0187] The processor 1301 , the memory 1302 , the input / output interface 1303 and the communication interface 1304 are connected to each other in communication within the device via a bus 1305 .
[0188] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned speech recognition method is implemented.
[0189] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0190] The speech recognition method and apparatus, computer equipment and storage medium provided in the embodiments of the present application respectively extract target speech temporal features and target speech spatial features from target speech data, and perform speech recognition by combining the target speech temporal features and the target speech spatial features, so that speech recognition can learn more speech features of speech data, and can make more accurate speech recognition based on more speech features.
[0191] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.
[0192] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.
[0193] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0194] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.
[0195] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0196] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.
[0197] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0198] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0199] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0200] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.
[0201] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.
Claims
1. A speech recognition method, characterized in that: The method comprises: Obtain target voice data; Inputting the target speech data into a preset target speech recognition model; wherein the target speech recognition model includes: a target time feature extraction network, a target space feature extraction network and a target speech recognition network; Extracting features from the target speech data through the target time feature extraction network to obtain target speech time series features; Extracting features from the target speech data through the target space feature extraction network to obtain target speech space features; Speech recognition is performed on the target speech temporal features and the target speech spatial features through the target speech recognition network.
2. The method according to claim 1, characterized in that The target space feature extraction network is a compression excitation network, and the compression excitation network includes: a linear layer and a convolutional layer; the target space feature extraction network is used to extract the target speech data to obtain the target speech space feature, including: Performing feature extraction on the target speech data through the linear layer to obtain at least two initial speech space features; wherein each of the initial speech space features represents a speech feature on a spatial channel; Concatenate at least two of the initial speech space features through the linear layer to obtain candidate speech space features; The candidate speech spatial features are weightedly calculated through the convolution layer to obtain the target speech spatial features.
3. The method according to claim 2, characterized in that The step of concatenating at least two of the initial speech space features through the linear layer to obtain candidate speech space features includes: Performing dimension expansion on each of the initial speech space features through the linear layer to obtain preliminary speech space features; At least two of the preliminary speech space features are concatenated through the linear layer to obtain the candidate speech space feature.
4. The method according to claim 2, characterized in that The step of performing weighted calculation on the candidate speech spatial features through the convolution layer to obtain the target speech spatial features includes: Performing pooling processing on the candidate speech spatial features through the convolution layer to obtain speech pooling features; Normalizing the speech pooling features through the convolution layer to obtain a preset weight for each spatial channel; The candidate speech spatial features and the preset weights are weightedly summed through the convolution layer to obtain the target speech spatial features.
5. The method according to any one of claims 1 to 4, characterized in that: The temporal feature extraction network is a self-attention network, and the self-attention network includes: a feature extraction layer and a self-attention layer; the target speech data is subjected to feature extraction by the target temporal feature extraction network to obtain the target speech temporal feature, including: Extracting features from the target speech data through the feature extraction layer to obtain preliminary speech time sequence features; Performing self-attention calculation on the preliminary speech time sequence features through the self-attention layer to obtain a self-attention weight; The preliminary speech timing features are weighted and calculated according to the self-attention weights to obtain the target speech timing features.
6. The method according to any one of claims 1 to 4, characterized in that: The performing speech recognition on the target speech temporal features and the target speech spatial features through a target speech recognition network includes: Performing feature fusion on the target speech temporal features and the target speech spatial features through the target speech recognition network to obtain target speech fusion features; The target speech recognition network performs speech recognition on the target speech fusion features.
7. The method according to any one of claims 1 to 4, characterized in that: Before inputting the target speech data into a preset target speech recognition model, the method further includes: Constructing the target speech recognition model specifically includes: Get training speech data; The training speech data is input into a preset original speech recognition model; wherein the original speech recognition model includes: an original time feature extraction network, an original space feature extraction network and an original speech recognition network: Performing feature extraction on the training speech data through the original time feature extraction network to obtain training speech time series features; Performing feature extraction on the training speech data through the original spatial feature extraction network to obtain training speech spatial features; Performing speech recognition on the training speech temporal features and the training speech spatial features through the original speech recognition network to obtain a speech recognition result; The parameters of the original speech recognition model are adjusted according to the preset speech verification result and the speech recognition result to obtain the target speech recognition model.
8. A speech recognition device, characterized in that: The device comprises: A data acquisition module, used to acquire target voice data; A speech data processing module, used for inputting the target speech data into a preset target speech recognition model; wherein the target speech recognition model comprises: a target time feature extraction network, a target space feature extraction network and a target speech recognition network; A time series feature extraction module, used to extract features from the target speech data through the target time feature extraction network to obtain target speech time series features; A spatial feature extraction module, used for extracting features from the target speech data through the target spatial feature extraction network to obtain target speech spatial features; The speech recognition module is used to perform speech recognition on the target speech temporal features and the target speech spatial features through a target speech recognition network.
9. A computer device, characterized in that: The computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements the speech recognition method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the speech recognition method according to any one of claims 1 to 7 is implemented.
Citation Information
Patent Citations
Online speech recognition engine and method
CN108091339A
Inter-channel feature extraction method, audio separation method and device, and computing equipment
CN110400575A
Voice recognition method and device, electronic equipment and storage medium
CN111341307A
Language recognition model training method, language recognition method and related equipment
CN112489623A
Speech emotion recognition method and system based on time-frequency characteristics and global attention
CN115019833A
