Speech recognition method and apparatus, storage medium, and electronic device

Abstract

A speech recognition method, device, storage medium and electronic equipment. The method comprises: obtaining speech information to be recognized, the speech information to be recognized comprising speech information of at least two channels; performing speech activity segment detection on the speech information to be recognized to obtain a plurality of speech activity segments corresponding to each channel; determining noise information of each channel based on cross information of the speech activity segments corresponding to different channels in the time dimension; and performing speech recognition on the speech information of the corresponding channel according to the noise information of each channel to obtain a speech recognition result corresponding to the speech information to be recognized. The method can greatly improve the accuracy of speech recognition.

Description

Technical Field

[0001] This application relates to the field of speech recognition technology, specifically to a speech recognition method, apparatus, storage medium, and electronic device. Background Technology

[0002] With the continuous development of artificial intelligence (AI) technology, various AI-based products are constantly appearing in people's lives and work, bringing great convenience to their daily lives. Among them, speech recognition technology, also known as Automatic Speech Recognition (ASR), is an important branch of AI technology. Its goal is to convert the lexical content of human speech into computer-readable input, such as keystrokes, binary codes, or character sequences.

[0003] Currently, in some scenarios, such as multi-channel speech recognition, there is a problem of inaccurate speech recognition results. Summary of the Invention

[0004] This application provides a speech recognition method, apparatus, storage medium, and electronic device that can improve the accuracy of speech recognition.

[0005] The speech recognition method provided in this application includes:

[0006] Acquire the speech information to be recognized, wherein the speech information to be recognized includes speech information from at least two channels;

[0007] The speech information to be identified is subjected to speech activity segment detection to obtain multiple speech activity segments corresponding to each channel;

[0008] Based on the cross-information of speech activity segments corresponding to different channels in the time dimension, the noise information of each channel is determined.

[0009] Based on the noise information of each channel, speech recognition is performed on the speech information of the corresponding channel to obtain the speech recognition result corresponding to the speech information to be recognized.

[0010] The speech recognition device provided in this application includes:

[0011] An acquisition module is used to acquire speech information to be recognized, wherein the speech information to be recognized includes speech information from at least two channels;

[0012] The detection module is used to detect speech activity segments in the speech information to be identified, and obtain multiple speech activity segments corresponding to each channel.

[0013] The determination module is used to determine the noise information of each channel based on the cross information of speech activity segments corresponding to different channels in the time dimension;

[0014] The recognition module is used to perform speech recognition on the speech information of the corresponding channel based on the noise information of each channel, and obtain the speech recognition result corresponding to the speech information to be recognized.

[0015] The storage medium provided in this application stores a computer program that, when loaded by a processor, executes the steps in the speech recognition method provided in this application.

[0016] The electronic device provided in this application includes a processor and a memory, wherein the memory stores a computer program, and the processor loads the computer program to execute the steps in the speech recognition method provided in this application.

[0017] The computer program product provided in this application includes a computer program / instruction, which, when executed by a processor, implements the steps in the speech recognition method provided in this application.

[0018] In this application, speech information to be recognized is acquired, including speech information from at least two channels. Speech activity segment detection is performed on the speech information to be recognized, obtaining multiple speech activity segments corresponding to each channel. Based on the cross-information of the speech activity segments corresponding to different channels in the time dimension, the noise information of each channel is determined. Speech recognition is then performed on the speech information of the corresponding channel based on the noise information of each channel, resulting in a speech recognition result corresponding to the speech information to be recognized. Compared with related technologies, this application predicts the speech dialogue process by performing speech activity segment detection on the multi-channel speech information to be recognized, and then identifies the noise information of each channel based on this prediction. Subsequently, the noise information of each channel is suppressed accordingly, thereby obtaining a more accurate speech recognition result. This method can greatly improve the noise resistance of the speech recognition system, thus improving the accuracy of speech recognition. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of a scenario for the speech recognition system provided in an embodiment of this application.

[0021] Figure 2This is a flowchart illustrating the speech recognition method provided in the embodiments of this application.

[0022] Figure 3 This is a schematic diagram illustrating the intersection of speech activity segments in the time dimension within dual-channel speech information.

[0023] Figure 4 This is a schematic diagram of the speech recognition process of the speech recognition model provided in the embodiments of this application.

[0024] Figure 5 This is a structural block diagram of the speech recognition device provided in the embodiments of this application.

[0025] Figure 6 This is a structural block diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0026] It should be noted that the principles of this application are illustrated by example in a suitable computing environment. The following description is based on the specific embodiments of this application exemplified, and should not be considered as limiting other specific embodiments not detailed herein. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0027] The relational terms such as "first" and "second" used in the following embodiments of this application are only used to distinguish one object or operation from another, and are not intended to limit the actual order of these objects or operations. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0028] Artificial intelligence (AI) is the theory, methods, technology, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0029] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include machine learning (ML), with deep learning (DL) being a relatively new research direction within ML. It has been introduced into machine learning to bring it closer to its original goal: artificial intelligence. Currently, deep learning is mainly applied in fields such as computer vision and natural language processing.

[0030] Natural Language Processing (NLP) is an important field within computer science and artificial intelligence. It studies the theories and methods for enabling effective communication between humans and computers using natural language. NLP is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language people use in daily life—and thus it has a close relationship with linguistic research. NLP techniques typically include text processing, semantic understanding, machine translation, question answering, and knowledge graphs.

[0031] To improve the accuracy of speech recognition, this application provides a speech recognition method apparatus, a storage medium, and an electronic device. The speech recognition method can be executed by an electronic device.

[0032] Please refer to Figure 1 This application also provides a speech recognition system, such as Figure 1 The diagram shown illustrates a usage scenario of the speech recognition system provided in this application. The speech recognition system includes an electronic device 100. For example, the electronic device can acquire speech information to be recognized, which includes speech information from at least two channels; perform speech activity segment detection on the speech information to be recognized to obtain multiple speech activity segments corresponding to each channel; determine the noise information of each channel based on the cross-information of the speech activity segments corresponding to different channels in the time dimension; and correct the speech recognition result of the corresponding channel according to the noise information of each channel to obtain the speech recognition result corresponding to the speech information to be recognized.

[0033] Electronic device 100 can be any device equipped with a processor and having voice processing capabilities, such as smartphones, tablets, PDAs, laptops, smart speakers and other mobile electronic devices with processors, or desktop computers, televisions, servers and other fixed electronic devices with processors.

[0034] In addition, such as Figure 1 As shown, the speech recognition system may also include a storage device 200 for storing data, including but not limited to raw data, intermediate data and result data obtained during the speech recognition process. For example, the electronic device 100 can store the acquired speech information to be recognized, the recognized speech activity segments, the noise information of each channel and the speech recognition result corresponding to the speech information to be recognized into the storage device 200.

[0035] It should be noted that, Figure 1 The schematic diagram of the speech recognition system shown is merely an example. The speech recognition system and scenarios described in this application are intended to more clearly illustrate the technical solutions of this application and do not constitute a limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of speech recognition systems and the emergence of new business scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

[0036] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating the speech recognition method provided in an embodiment of this application. Figure 2 As shown, the flow of the speech recognition method provided in this application embodiment can be as follows:

[0037] In S301, the speech information to be recognized is acquired.

[0038] Among related technologies, speech recognition technology has been widely applied in intelligent customer service scenarios, including but not limited to outbound calls, navigation, work schedules, and analysis, greatly improving customer service efficiency. In some scenarios, such as automatically checking the service quality of customer service agents, the speech recognition system can automatically recognize the speech of the conversation between the agent and the customer, and then automatically check the service quality of the agent based on the recognition results. This method can greatly improve the efficiency of checking the service quality of customer service agents. However, in some scenarios, such as when the user is in a noisy environment (e.g., in a cafeteria or on a bus), the speech recognition system is affected by background noise, resulting in inaccurate recognition results. To address this, and to improve the robustness of the speech recognition system in such scenarios, i.e., to improve the accuracy of speech recognition in these scenarios, this application provides a speech recognition method to improve the accuracy of multi-channel speech information recognition under noise interference. The speech recognition method provided in this application will be described in detail below.

[0039] First, the voice information to be recognized is acquired. This voice information can be acquired in real-time during a call between a customer service representative and a customer, or the call can be pre-recorded and stored, and then the voice information to be recognized can be obtained from the stored recording. In this embodiment, the voice information to be recognized can be either two-channel or multi-channel. Specifically, this application will use two-channel voice information as an example to describe the voice recognition method provided in detail. Here, two channels specifically refer to the voice channel corresponding to the customer service representative and the voice channel corresponding to the customer.

[0040] In S302, speech activity segment detection is performed on the speech information to be recognized to obtain multiple speech activity segments corresponding to each channel.

[0041] In this embodiment, after acquiring the voice information to be recognized, voice activity segment detection can be performed on the voice information to determine the voice activity segments contained in each channel. In a customer service scenario, voice communication is generally two-channel, meaning one person speaks while the other listens. Specifically, when a customer speaks, the customer service representative usually listens; when the representative answers or follows up on a customer's question, the customer usually listens. That is, generally, there should only be one voice activity segment in two-channel voice information at any given time. If voice activity segments are detected in both channels at the same time, noise can be identified in one channel. Based on the above considerations, this embodiment, after acquiring the voice information to be recognized, can first perform voice activity segment detection on at least two channels of the voice information to determine multiple voice activity segments contained in each channel, so as to further determine noise information based on the voice activity segments corresponding to each channel.

[0042] In some embodiments, the speech information to be identified is subjected to speech activity segment detection to obtain multiple speech activity segments corresponding to each channel, including:

[0043] 1. Detect speech activity endpoints on the speech information to be recognized to obtain multiple sets of speech activity endpoints corresponding to each channel;

[0044] 2. Determine multiple speech activity segments corresponding to each channel based on multiple sets of speech activity endpoints.

[0045] In this embodiment, voice activity endpoint detection is performed on the voice information to be identified, which can be implemented using the Voice Activity Detection (VAD) method. That is, multiple sets of voice activity endpoints are detected in the voice information corresponding to each channel. Each set of voice activity endpoints includes a voice activity start point and a voice activity end point corresponding to that start point. The voice segment between the voice activity start point and the corresponding voice activity end point within a set of voice activity endpoints is the voice activity segment corresponding to that set of voice activity endpoints.

[0046] Generally, the specific process of speech activity endpoint detection (VAD) involves dividing the speech information to be recognized in a certain channel into multiple audio frames, then performing short-time energy detection on each audio frame to obtain its short-time energy value. This short-time energy value is then compared with a preset energy threshold. If, within a first number of audio frames starting from a given audio frame, a second number (less than the first number) of audio frames have short-time energy values ​​greater than the aforementioned energy threshold, then that audio frame can be determined as a speech activity start point, or VAD start point. Energy detection and comparison with the preset energy threshold are then continuously performed on subsequent audio frames. If, within a third consecutive number of audio frames starting from a given audio frame, the short-time energy values ​​are all below the aforementioned energy threshold, then that audio frame can be determined as a speech activity end point, or VAD end point. Consecutive VAD start points and VAD end points constitute a VAD endpoint combination, and one VAD endpoint combination corresponds to one speech activity segment.

[0047] In some embodiments, before performing voice activity endpoint detection on the voice information to be identified to obtain multiple sets of voice activity endpoints corresponding to each channel, the method further includes:

[0048] A. Obtain the first training sample data, which includes multiple segments of first sample audio data and sample speech activity endpoint data contained in each segment of first sample audio data;

[0049] B. Train a speech activity endpoint detection model by taking multiple segments of first sample audio data as input and the corresponding sample speech activity endpoint data of each segment of first sample audio data as output.

[0050] Speech activity endpoint detection is performed on the speech information to be recognized, resulting in multiple sets of speech activity endpoints corresponding to each channel, including:

[0051] C. Input the speech information of each channel into the speech activity endpoint detection model to perform endpoint detection, and obtain multiple sets of speech activity endpoints corresponding to the speech information of each channel.

[0052] In this embodiment, a speech activity endpoint detection model can also be used for speech activity endpoint detection. Specifically, the speech information of each channel can be input into a speech activity endpoint detection model for endpoint detection, thereby obtaining multiple sets of speech activity endpoints corresponding to the speech information of each channel.

[0053] Before using the speech activity endpoint detection model to detect speech activity endpoints in each channel, the model can be trained first. Specifically, training sample data can be acquired first. To distinguish this training sample data from the training sample data of the subsequent speech recognition model, this training sample data can be called the first training sample data. The first training sample data can include multiple first sample audio data segments and the sample speech activity endpoint data contained in each audio segment. That is, a large number of dual-channel audio samples from customer service scenarios can be acquired to construct an audio database, and then each audio segment in the audio database can be labeled with speech activity endpoints. After acquiring the training sample data, the sample audio data can be used as the input to the speech activity endpoint detection model, and the speech activity endpoint labels corresponding to each sample audio segment can be used as the output of the model to train the speech activity endpoint detection model. The speech activity endpoint detection model can be a neural network model, and the training method for this model can use the existing gradient descent algorithm, which will not be elaborated here.

[0054] In this embodiment, the use of dual-channel speech information is described as an example. When the speech information to be identified is multi-channel speech information, the above method can be used to sample the speech information of each channel to detect speech activity segments and obtain the speech activity segments corresponding to the speech information of other channels. This will not be elaborated here.

[0055] In S303, noise information for each channel is determined based on the cross-information of speech activity segments corresponding to different channels in the time dimension.

[0056] In this embodiment of the application, after obtaining the speech activity segment corresponding to each channel, the cross-information of the speech activity segments of different channels in the time dimension can be determined. For example... Figure 3The diagram illustrates the temporal intersection of speech activity segments in dual-channel speech information. As shown, two speech activity segments are detected in the speech information corresponding to the first channel 10. Three speech activity segments are detected in the speech information corresponding to the second channel 20. Based on the temporal intersection of the speech activity segments corresponding to the two channels, seven timelines, from timeline a to timeline g, can be defined. Between timeline a and timeline b, only the second channel has speech activity segments; between timeline b and timeline c, neither channel has speech activity segments; between timeline c and timeline d, only the first channel has speech activity segments; between timeline d and timeline e, both channels have speech activity segments; between timeline e and timeline f, only the second channel has speech activity segments; and between timeline f and timeline g, only the first channel has speech activity segments. Figure 3 The example shown is just one example for understanding the temporal crossover information between detected speech activity segments in two-channel speech information.

[0057] Based on the temporal crossover information between speech activity segments, it can be determined which time periods have speech activity segments in only one channel, which time periods have speech activity segments in both channels, and which time periods have no speech activity segments detected in either channel. As mentioned earlier, speech activity segment detection can be based on short-time energy thresholds or on speech activity endpoint detection models. This means that the determination of a speech activity segment is based on certain conditions; the presence of speech information does not necessarily mean the presence of a speech activity segment. Therefore, the speech information corresponding to the time periods without speech activity segments is silence. In general, speech information still exists in the time periods where no speech activity segments are detected within each channel; however, this portion of speech information may not be recognized as a speech activity segment due to its low volume. This low-volume speech information is highly likely to be environmental noise.

[0058] As mentioned earlier, in a customer service call scenario, generally only one channel should have a voice activity segment at any given time. Therefore, if the time-division cross-reference of voice activity segments from different channels determines that both channels have voice activity segments at the same time, then one channel's voice activity segment might be noise. Thus, in this embodiment, after determining the voice activity segment corresponding to each channel, the noise information for each channel can be determined based on the time-division cross-reference of the voice activity segments from different channels. Then, noise suppression can be further applied to the recognition results based on the noise information of each channel, resulting in a more accurate recognition outcome.

[0059] In some embodiments, taking the example that the speech information to be identified includes two-channel speech information, the noise information of each channel is determined based on the cross-information of speech activity segments corresponding to different channels in the time dimension, including:

[0060] 1. When only one channel detects a speech activity segment in the target time period, the speech information contained in the channel where no speech activity segment was detected in the target time period is determined to be noise information.

[0061] 2. When speech activity segments are detected in both channels during the target time period, noise information is determined based on the voiceprint detection results of the two speech activity segments detected during the target time period.

[0062] In this embodiment, the method for determining noise information based on the temporal crossover information of speech activity segments is described in detail, taking the example of speech information to be identified including dual-channel speech information. Please continue reading. Figure 3 For example, if only one channel detects a speech activity segment within a certain time period—for instance, if only the second channel detects a speech activity segment between timeline a and timeline b—then the speech information in the first channel during that time period can be determined to be background noise. This background noise might be too quiet to be recognized as a speech activity segment. The speech activity segment identified in the second channel during this time period could be valid call speech or loud background noise; to avoid misjudgment, it is not identified as noise. Similarly, if only the first channel detects a speech activity segment between timeline f and timeline g, then the speech information in the second channel during that time period is determined to be background noise. In this case, the speech activity segment identified in the first channel could be valid call speech or loud background noise; again, to avoid misjudgment, it is not identified as noise.

[0063] If speech activity segments are detected in both channels within a certain time period, for example Figure 3 Speech activity segments were detected between timelines d and e. However, since only one channel should have a speech activity segment at a given time point, it is possible that the speech information in one channel is background noise with a large volume. In this case, voiceprint detection can be used to determine which channel contains noise information.

[0064] In some embodiments, noise information is determined based on the speakerprint detection results of two detected speech activity segments within a target time period, including:

[0065] 2.1 Perform voiceprint detection on the speech activity segments detected in the two channels during the target time period;

[0066] 2.2 When the voiceprint of any target speech activity segment is not a preset voiceprint, the speech information corresponding to the target speech activity segment is determined to be noise information.

[0067] In this embodiment, when voice activity segments are detected in both channels within a certain time period, the voiceprint information of the current call's voice in both channels can be obtained first, and this voiceprint information can be used as a preset voiceprint. The method for obtaining the voiceprint information of the call voice in each channel can be achieved by extracting the voiceprints of the detected voice activity segments in that channel, then extracting the voiceprint information with the highest frequency, and setting it as the preset voiceprint.

[0068] Therefore, when a speech activity segment exists in both channels within a certain time period, voiceprint detection can be performed on these two speech activity segments to determine whether the detected voiceprint is one of the aforementioned preset voiceprints. If it is, the speech segment can be determined to be valid call speech; otherwise, the speech segment can be determined to be background noise with a large volume.

[0069] In cases where only one channel has a speech activity segment during the aforementioned time period, voiceprint detection can be used to determine whether the speech information in the channel with the speech activity segment is noise information.

[0070] In this embodiment, only two-channel speech information is used as an example. When the speech information to be identified is three-channel or multi-channel speech information, the above method can also be used to determine the noise information in each channel.

[0071] In S304, speech recognition is performed on the speech information of the corresponding channel based on the noise information of each channel to obtain the speech recognition result corresponding to the speech information to be recognized.

[0072] After determining the noise information in each channel using the above method, noise suppression can be further performed on the speech recognition process based on the noise information of that channel to obtain the speech recognition result for each channel, and thus obtain the speech recognition result corresponding to the speech information to be recognized.

[0073] In some embodiments, speech recognition is performed on the speech information of the corresponding channel based on the noise information of each channel to obtain the speech recognition result corresponding to the speech information to be recognized, including:

[0074] 1. Obtain the preset speech recognition model;

[0075] 2. Input the speech information and corresponding noise information of each channel into the preset speech recognition model for speech recognition to obtain the speech recognition result of each channel;

[0076] 3. Determine the speech recognition result corresponding to the speech information to be recognized based on the speech recognition result of each channel.

[0077] In this embodiment, a speech recognition model that explicitly models noise is also provided, which can be referred to here as a preset speech recognition model. That is, during the speech recognition process, this speech recognition model can not only identify and convert the spoken speech into text, but also identify and display noise information in the spoken speech. The speech recognition model also includes a noise pool to temporarily store speech information identified as noise. When using this speech recognition model to perform speech recognition on the speech information to be recognized, the noise information detected in the speech information to be recognized can first be stored in the noise pool to update the noise pool. Then, the speech information to be recognized is input into the speech recognition model for speech recognition. During the recognition process, the speech recognition model fully considers the characteristics of the noise information present in the noise pool, thus accurately identifying the spoken speech and noise information in the speech to be recognized, thereby making the recognition of spoken speech more accurate and greatly improving the accuracy of speech recognition.

[0078] In some embodiments, the speech recognition method provided in this application further includes:

[0079] A. Obtain the noise pool corresponding to each channel;

[0080] B. Update the corresponding noise pool based on the noise information of each channel;

[0081] The speech information and corresponding noise information of each channel are input into a preset speech recognition model for speech recognition, resulting in the speech recognition result for each channel, including:

[0082] C. Input the speech information of each channel and the corresponding updated noise information in the noise pool into the preset speech recognition model to perform speech recognition and obtain the speech recognition result of each channel.

[0083] In this embodiment, the speech recognition process for multi-channel speech information can be performed by recognizing the speech information of each channel separately, and then determining the final speech recognition result based on the speech recognition results of each channel. In this embodiment, a corresponding noise pool can be set for each channel. For example, if a channel is the customer service channel, during model training, noise data present in the customer service voice information can be determined based on training sample data, and a noise pool for the customer service channel can be maintained accordingly. Thus, during the speech recognition of dual-channel speech information, the noise information recognized in the customer service channel can be added to the noise pool corresponding to the customer service channel, resulting in an updated noise pool for the customer service channel. Then, noise suppression is performed on the speech recognition process of the customer service channel based on the noise pool of the customer service channel, yielding the speech recognition result for the customer service channel. Similarly, a corresponding noise pool can be constructed for the customer service channel's speech information.

[0084] In this embodiment, since a more targeted noise pool is constructed for different vocal channels, the large amount of noise information stored in the noise pool can be avoided from affecting the training efficiency of the speech recognition model. At the same time, since the noise information in the noise pool is more targeted, noise suppression can be performed more accurately on the speech recognition process of each vocal channel, thereby further improving the accuracy of speech recognition.

[0085] In some embodiments, a storage limit can be set for each noise pool to ensure the decoding efficiency of the main model. When the noise information stored in the noise pool exceeds the aforementioned storage limit, low-frequency noise information can be deleted, or the noise information whose storage time is furthest from the current time can be deleted based on the storage time information.

[0086] Among them, such as Figure 4 The figure shows a schematic diagram of the speech recognition process using the speech recognition model provided in this application. As shown, when recognizing the speech information 11 to be recognized, the speech recognition model first encodes the speech information 11 using the first encoder 12 of the main model to obtain speech features. At the same time, the noise information 13 in the noise pool is encoded using the second encoder 14 to obtain noise features. Then, the noise features and speech features are input together into the attention layer 15 for processing. Finally, the output of the attention layer 15 is input into the decoder 16 for decoding and outputting the recognition result.

[0087] The specific processing algorithm formula for the attention layer is expressed as follows:

[0088] a i =Attention(s) i ,a i-1,h,h n ).

[0089] The core part of the attention layer calculation formula, in addition to the output of the encoder segment and the encoder's hidden layer variable h in the usual sense, also includes the noise information vector h. n This allows the model to directly consider the noise information contained in the audio history during the recognition and reasoning process, which helps to improve the noise resistance of the speech recognition model and enhance the overall recognition effect and robustness.

[0090] In some embodiments, before obtaining the preset speech recognition model, the method further includes:

[0091] a. Obtain the second training sample data, which includes multiple segments of second sample audio data and the corresponding text and noise labels for each segment of second sample audio data.

[0092] b. Train a pre-defined speech recognition model by taking multiple segments of second-sample audio data as input and the corresponding text labels and noise labels for each segment of second-sample audio data as output.

[0093] In this embodiment, before performing speech recognition on the speech information to be recognized using the speech recognition model provided in this application, the speech recognition model needs to be trained. Specifically, training sample data for training the speech recognition model can be obtained first, referred to here as second training sample data. The second training sample data includes multiple segments of second sample audio data and text labels corresponding to each segment of second sample audio data. In addition, in this embodiment, the second training sample data also includes noise labels corresponding to each segment of sample audio data, so as to train the speech recognition model's ability to recognize noise. Then, the preset speech recognition model can be trained using multiple segments of second sample audio data as input and the text labels and noise labels corresponding to each segment of second sample audio data as output.

[0094] As described above, the speech recognition method provided in this application acquires speech information to be recognized, which includes speech information from at least two channels; performs speech activity segment detection on the speech information to be recognized to obtain multiple speech activity segments corresponding to each channel; determines the noise information of each channel based on the cross-information of the speech activity segments corresponding to different channels in the time dimension; and performs speech recognition on the speech information of the corresponding channel based on the noise information of each channel to obtain the speech recognition result corresponding to the speech information to be recognized. Compared with related technologies, this application predicts the speech dialogue process by performing speech activity segment detection on the multi-channel speech information to be recognized, and then identifies the noise information of each channel based on this, and then suppresses the noise information of each channel accordingly, thereby obtaining a more accurate speech recognition result. This method can greatly improve the noise resistance of the speech recognition system, thereby improving the accuracy of speech recognition.

[0095] Please refer to Figure 5 To better implement the speech recognition method provided in this application, this application further provides a speech recognition device 400, such as... Figure 5 As shown, the voice recognition device 400 includes:

[0096] The acquisition module 410 is used to acquire the speech information to be recognized, wherein the speech information to be recognized includes speech information from at least two channels;

[0097] The detection module 420 is used to detect speech activity segments in the speech information to be identified, and obtain multiple speech activity segments corresponding to each channel.

[0098] The determination module 430 is used to determine the noise information of each channel based on the cross information of speech activity segments corresponding to different channels in the time dimension.

[0099] The recognition module 440 is used to perform speech recognition on the speech information of the corresponding channel based on the noise information of each channel, and obtain the speech recognition result corresponding to the speech information to be recognized.

[0100] Optionally, in some embodiments, the detection module includes:

[0101] The detection submodule is used to detect the voice activity endpoints of the voice information to be identified, and obtain multiple sets of voice activity endpoints corresponding to each channel.

[0102] The first determining submodule is used to determine multiple speech activity segments corresponding to each channel based on the multiple sets of speech activity endpoints.

[0103] Optionally, in some embodiments, the voice recognition device provided in this application further includes:

[0104] The first acquisition submodule is used to acquire the first training sample data, which includes multiple segments of first sample audio data and sample speech activity endpoint data contained in each segment of first sample audio data.

[0105] The first training submodule is used to train a speech activity endpoint detection model by taking the multiple segments of first sample audio data as input and the sample speech activity endpoint data corresponding to each segment of first sample audio data as output.

[0106] The detection submodule is also used for:

[0107] The speech information of each channel is input into the speech activity endpoint detection model for endpoint detection, thereby obtaining multiple sets of speech activity endpoints corresponding to the speech information of each channel.

[0108] Optionally, in some embodiments, the speech information to be identified includes dual-channel speech information, and the determining module includes:

[0109] The second determining submodule is used to determine that the speech information contained in the channel in which no speech activity segment was detected in the target time period is noise information when only one channel detects a speech activity segment in the target time period.

[0110] The third determination submodule is used to determine noise information based on the voiceprint detection results of the two voice activity segments detected in the target time period when speech activity segments are detected in both channels during the target time period.

[0111] Optionally, in some embodiments, the third determining submodule includes:

[0112] The detection unit is used to perform voiceprint detection on the speech activity segments detected in the two channels during the target time period.

[0113] The determining unit is used to determine the speech information corresponding to the target speech activity segment as noise information when the voiceprint of any target speech activity segment is detected to not belong to a preset voiceprint.

[0114] Optionally, in some embodiments, the identification module includes:

[0115] The second acquisition submodule is used to acquire a preset speech recognition model;

[0116] The recognition submodule is used to input the speech information and corresponding noise information of each channel into the preset speech recognition model for speech recognition, and obtain the speech recognition result of each channel.

[0117] The fourth determining submodule is used to determine the speech recognition result corresponding to the speech information to be recognized based on the speech recognition result of each channel.

[0118] Optionally, in some embodiments, the voice recognition device provided in this application further includes:

[0119] The third acquisition submodule is used to acquire the noise pool corresponding to each channel;

[0120] The update submodule is used to update the corresponding noise pool based on the noise information of each channel;

[0121] The identification submodule is also used for:

[0122] The speech information of each channel and the corresponding updated noise information in the noise pool are input into the preset speech recognition model for speech recognition, and the speech recognition result of each channel is obtained.

[0123] Optionally, in some embodiments, the voice recognition device provided in this application further includes:

[0124] The fourth acquisition submodule is used to acquire the second training sample data, which includes multiple segments of second sample audio data and text labels and noise labels corresponding to each segment of second sample audio data.

[0125] The second training submodule is used to train a preset speech recognition model by taking the multiple segments of second sample audio data as input and the text labels and noise labels corresponding to each segment of second sample audio data as output.

[0126] It should be noted that the speech recognition device 400 provided in this application embodiment belongs to the same concept as the speech recognition method in the above embodiment. Its specific implementation process can be found in the above related embodiments, and will not be repeated here.

[0127] As described above, the speech recognition device provided in this application acquires speech information to be recognized through an acquisition module 410, which includes speech information from at least two channels. A detection module 420 detects speech activity segments in the speech information to be recognized, obtaining multiple speech activity segments corresponding to each channel. A determination module 430 determines the noise information of each channel based on the cross-sectional information of the speech activity segments corresponding to different channels in the time dimension. A recognition module 440 performs speech recognition on the speech information of the corresponding channel based on the noise information of each channel, obtaining the speech recognition result corresponding to the speech information to be recognized. Compared with related technologies, this application predicts the speech dialogue process by detecting speech activity segments in the multi-channel speech information to be recognized, and then identifies the noise information of each channel based on this prediction. Furthermore, it suppresses the noise information of each channel accordingly, thereby obtaining a more accurate speech recognition result. This method can greatly improve the noise resistance of the speech recognition system, thereby improving the accuracy of speech recognition.

[0128] This application also provides an electronic device, including a memory and a processor, wherein the processor executes the steps in the speech recognition method provided in this embodiment by calling a computer program stored in the memory.

[0129] Please refer to Figure 6 , Figure 6 This is a schematic diagram of the structure of the electronic device 100 provided in the embodiments of this application.

[0130] The electronic device 100 may include components such as a network interface 110, a memory 120, a processor 130, and a screen assembly. Those skilled in the art will understand that... Figure 6 The structure of the electronic device 100 shown does not constitute a limitation on the electronic device 100, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0131] Network interface 110 can be used for network connections between devices.

[0132] The memory 120 can be used to store computer programs and data. The computer program stored in the memory 120 contains executable code. The computer program can be divided into various functional modules. The processor 130 executes various functional applications and data processing by running the computer program stored in the memory 120.

[0133] The processor 130 is the control center of the electronic device 100. It connects various parts of the electronic device 100 through various interfaces and lines. By running or executing computer programs stored in the memory 120 and calling data stored in the memory 120, it performs various functions of the electronic device 100 and processes data, thereby controlling the electronic device 100 as a whole.

[0134] In this embodiment, the processor 130 in the electronic device 100 loads executable code corresponding to one or more computer programs into the memory 120 according to the following instructions, and the processor 130 executes the steps in the speech recognition method provided in this application, such as:

[0135] The process involves acquiring speech information to be recognized, which includes speech information from at least two channels; detecting speech activity segments in the speech information to be recognized to obtain multiple speech activity segments corresponding to each channel; determining the noise information of each channel based on the cross-information of the speech activity segments corresponding to different channels in the time dimension; and performing speech recognition on the speech information of the corresponding channel based on the noise information of each channel to obtain the speech recognition result corresponding to the speech information to be recognized.

[0136] It should be noted that the electronic device 100 provided in this application embodiment belongs to the same concept as the speech recognition method in the above embodiment. Its specific implementation process can be found in the above related embodiments, and will not be repeated here.

[0137] This application also provides a computer-readable storage medium storing a computer program thereon. When the computer program stored thereon is executed on the processor of the electronic device provided in the embodiments of this application, the processor of the electronic device performs any of the steps in the above-described speech recognition method suitable for electronic devices. The storage medium may be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0138] The above provides a detailed description of a speech recognition method, apparatus, storage medium, and electronic device provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, those skilled in the art will recognize that there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A speech recognition method, characterized in that, The method includes: Acquire the speech information to be recognized, wherein the speech information to be recognized includes speech information from at least two channels; The speech information to be identified is subjected to speech activity segment detection to obtain multiple speech activity segments corresponding to each channel; Based on the cross-information of speech activity segments corresponding to different channels in the time dimension, the noise information of each channel is determined; wherein, when the speech information to be identified is dual-channel speech information, the cross-information of the speech activity segments in the time dimension is used to indicate the time period when only one channel has the speech activity segment, the time period when both channels have the speech activity segment, and the time period when neither channel has the speech activity segment. Based on the noise information of each channel, speech recognition is performed on the speech information of the corresponding channel to obtain the speech recognition result corresponding to the speech information to be recognized.

2. The method according to claim 1, characterized in that, The process of detecting speech activity segments in the speech information to be identified, resulting in multiple speech activity segments corresponding to each channel, includes: The speech information to be identified is subjected to speech activity endpoint detection to obtain multiple sets of speech activity endpoints corresponding to each channel; Based on the multiple sets of speech activity endpoints, multiple speech activity segments corresponding to each channel are determined.

3. The method according to claim 1, characterized in that, The speech information to be identified includes dual-channel speech information. The step of determining the noise information for each channel based on the temporal cross-information of speech activity segments corresponding to different channels includes: When only one channel detects a speech activity segment during the target time period, the speech information contained in the channel where no speech activity segment was detected during the target time period is determined to be noise information. When speech activity segments are detected in both channels during the target time period, noise information is determined based on the voiceprint detection results of the two speech activity segments detected during the target time period.

4. The method according to claim 3, characterized in that, The step of determining noise information based on the speakerprint detection results of two detected speech activity segments within the target time period includes: Voiceprint detection is performed on the speech activity segments detected in the two vocal channels during the target time period; When the voiceprint of any target speech activity segment is not a preset voiceprint, the speech information corresponding to the target speech activity segment is determined to be noise information.

5. The method according to any one of claims 1 to 4, characterized in that, Speech recognition is performed on the speech information of the corresponding channel based on the noise information of each channel to obtain the speech recognition result corresponding to the speech information to be recognized, including: Obtain the preset speech recognition model; The speech information and corresponding noise information of each channel are input into the preset speech recognition model for speech recognition, and the speech recognition result of each channel is obtained. The speech recognition result corresponding to the speech information to be recognized is determined based on the speech recognition result of each channel.

6. The method according to claim 5, characterized in that, The method further includes: Obtain the noise pool corresponding to each channel; The corresponding noise pool is updated based on the noise information of each channel; The step of inputting the speech information and corresponding noise information of each channel into the preset speech recognition model for speech recognition to obtain the speech recognition result of each channel includes: The speech information of each channel and the corresponding updated noise information in the noise pool are input into the preset speech recognition model for speech recognition, and the speech recognition result of each channel is obtained.

7. The method according to claim 5, characterized in that, Before obtaining the preset speech recognition model, the process also includes: Acquire second training sample data, which includes multiple segments of second sample audio data and text labels and noise labels corresponding to each segment of second sample audio data; Using the multiple segments of second sample audio data as input, and the text labels and noise labels corresponding to each segment of second sample audio data as output, a preset speech recognition model is trained.

8. A voice recognition device, characterized in that, include: An acquisition module is used to acquire speech information to be recognized, wherein the speech information to be recognized includes speech information from at least two channels; The detection module is used to detect speech activity segments in the speech information to be identified, and obtain multiple speech activity segments corresponding to each channel. The determination module is used to determine the noise information of each channel based on the cross-information of speech activity segments corresponding to different channels in the time dimension; wherein, when the speech information to be identified is dual-channel speech information, the cross-information of the speech activity segments in the time dimension is used to indicate the time period when only one channel has the speech activity segment, the time period when both channels have the speech activity segment, and the time period when neither channel has the speech activity segment. The recognition module is used to perform speech recognition on the speech information of the corresponding channel based on the noise information of each channel, and obtain the speech recognition result corresponding to the speech information to be recognized.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is loaded by the processor, it performs the steps of the speech recognition method as described in any one of claims 1-7.

10. An electronic device comprising a processor and a memory, the memory storing a computer program, characterized in that, The processor loads the computer program to perform the steps in the speech recognition method as described in any one of claims 1 to 7.

11. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the speech recognition method according to any one of claims 1 to 7.

Images