Speech recognition method, apparatus, processor, and electronic device

Abstract

The application discloses a speech recognition method and device, a processor and an electronic device. It relates to the field of artificial intelligence, and the method comprises: obtaining speech information of a target scene, and determining the scene type of the target scene; determining a target far-field speech recognition model matched with the scene type, and inputting the speech information into the target far-field speech recognition model to obtain dialogue information of the speech information, wherein the target far-field speech recognition model is obtained by training a preset far-field speech recognition model with a sample data set, the sample data set comprises first sample data and second sample data, each first sample data is composed of original speech information under a scene type and dialogue information of the original speech information, and each second sample data is composed of simulated speech information under a scene type and dialogue information of the simulated speech information. Through the application, the problem that it is difficult to reduce the training sample cost of a speech recognition model in the related art is solved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and more specifically, to a speech recognition method, apparatus, processor, and electronic device. Background Technology

[0002] With the development of artificial intelligence, more and more smart home appliances are equipped with far-field voice recognition technology. This technology recognizes the commands issued by the user and can then perform corresponding operations without the user manually operating the appliance.

[0003] Far-field speech recognition technology is a user-enhancing feature that integrates AI voice search. By training far-field speech recognition models in different scenarios, speech recognition experiences in various scenarios can be achieved. Training methods for far-field speech recognition models typically include geometric processing methods and image source model methods. Geometric processing has the advantage of low cost, but to facilitate data simulation, it requires geometricalizing the spatial data. Due to varying degrees of spatial complexity, this method may result in relatively poor accuracy and low efficiency. Models trained using image source models have higher accuracy, but this method requires consideration of factors such as spatial complexity and energy decay, leading to relatively higher costs and longer simulation times.

[0004] There is currently no effective solution to the problem of reducing the training sample cost of speech recognition models in related technologies. Summary of the Invention

[0005] The main objective of this application is to provide a speech recognition method, device, processor, and electronic device to solve the problem of difficulty in reducing the training sample cost of speech recognition models in related technologies.

[0006] To achieve the above objectives, according to one aspect of this application, a speech recognition method is provided. The method includes: acquiring speech information of a target scene and determining the scene type of the target scene; determining a target far-field speech recognition model matching the scene type, and inputting the speech information into the target far-field speech recognition model to process the resulting speech information into dialogue information. The target far-field speech recognition model is obtained by training a preset far-field speech recognition model using a sample dataset. The sample dataset includes first sample data and second sample data. Each first sample data consists of original speech information under the scene type and dialogue information of the original speech information. Each second sample data consists of simulated speech information under the scene type and dialogue information of the simulated speech information. The simulated speech information is generated from noise information and original speech information under the scene type.

[0007] Optionally, the target far-field speech recognition model is trained as follows: Z original speech information of scene type are obtained from the speech database, and the dialogue information of each original speech information is recognized, where Z is a positive integer; for each original speech information, the original speech information and the dialogue information of the original speech information constitute the first sample data, resulting in Z first sample data; for each original speech information, the original speech information and the noise information in the noise database are mixed to obtain simulated speech information, and the simulated speech information and the dialogue information of the original speech information constitute the second sample data, resulting in Z second sample data; the Z first sample data and the Z second sample data constitute the sample dataset; the preset far-field speech recognition model is trained using the sample dataset to obtain the target far-field speech recognition model.

[0008] Optionally, training a preset far-field speech recognition model from a sample dataset to obtain a target far-field speech recognition model includes: inputting sample speech information from the sample dataset into the preset far-field speech recognition model, processing it to obtain predicted dialogue information and the trained far-field speech recognition model, wherein the sample speech information includes the original speech information in the first sample data and the simulated speech information in the second sample data; calculating model evaluation parameters based on the predicted dialogue information and the dialogue information associated with the sample speech information; if the model evaluation parameters do not meet the preset recognition requirements, adjusting the trained far-field speech recognition model until the model evaluation parameters of the adjusted far-field speech recognition model meet the preset recognition requirements, and determining the adjusted far-field speech recognition model as the target far-field speech recognition model, wherein the model evaluation parameters include accuracy and speech error rate.

[0009] Optionally, before mixing the original speech information with noise information in the noise database for each original speech information to obtain simulated speech information, the method further includes: obtaining an annotation table associated with the original speech information, wherein the annotation table is used to annotate the dialogue time period information in the original speech information, the dialogue time period information including the dialogue time interval and the dialogue text; obtaining the dialogue time period information from the annotation table to obtain M dialogue time period information, and sorting the M dialogue time period information according to the time information to obtain M sorted dialogue time period information, wherein M is a positive integer; segmenting the original speech information according to the M sorted dialogue time period information to obtain M dialogue segments, and adding the non-dialogue segments other than the M dialogue segments in the original speech information to a preset noise database to obtain a noise database.

[0010] Optionally, for each original speech information, the original speech information and the noise information in the noise database are mixed to obtain simulated speech information, including: obtaining N non-dialogue segments from the noise database, arranging and combining the N non-dialogue segments to obtain X non-dialogue combination segments, where N and X are positive integers; selecting any one of the X non-dialogue combination segments, and adding the selected non-dialogue combination segment to the original speech information to obtain simulated speech information.

[0011] Optionally, before mixing the original speech information and noise information from the noise database for each original speech information to obtain simulated speech information, the method further includes: obtaining a first phoneme error rate of the original speech information and a second phoneme error rate of the simulated speech information; determining whether the second phoneme error rate is less than the first phoneme error rate; if the second phoneme error rate is less than the first phoneme error rate, constructing a second sample data from the dialogue information of the simulated speech information and the original speech information; and discarding the simulated speech information if the second phoneme error rate is greater than or equal to the first phoneme error rate.

[0012] Optionally, before mixing the original speech information with noise information in the noise database for each original speech information to obtain analog speech information, the method further includes: acquiring a first simulated signal waveform of the original speech information and a second simulated signal waveform of the analog speech information; determining whether the audio amplitude of the second simulated signal waveform is less than the audio amplitude of the first simulated signal waveform; if the audio amplitude of the second simulated signal waveform is less than the audio amplitude of the first simulated signal waveform, constructing second sample data from the dialogue information of the analog speech information and the original speech information; and discarding the analog speech information if the audio amplitude of the second simulated signal waveform is greater than or equal to the audio amplitude of the first simulated signal waveform.

[0013] To achieve the above objectives, according to another aspect of this application, a speech recognition device is provided. The device includes: a first acquisition unit, configured to acquire speech information of a target scene and determine the scene type of the target scene; and a determination unit, configured to determine a target far-field speech recognition model matching the scene type, input the speech information into the target far-field speech recognition model, and process the speech information to obtain dialogue information. The target far-field speech recognition model is obtained by training a preset far-field speech recognition model using a sample dataset. The sample dataset includes first sample data and second sample data. Each first sample data consists of original speech information under the scene type and dialogue information of the original speech information. Each second sample data consists of simulated speech information under the scene type and dialogue information of the simulated speech information. The simulated speech information is generated from noise information and original speech information under the scene type.

[0014] According to another aspect of the present invention, a processor is also provided, which is used to run a program, wherein the program controls a device containing a non-volatile storage medium to perform a speech recognition method during runtime.

[0015] According to another aspect of the present invention, an electronic device is also provided, comprising one or more processors and a memory; the memory stores computer-readable instructions, and the processor is configured to execute the computer-readable instructions, wherein the computer-readable instructions execute a speech recognition method when they are executed.

[0016] This application employs the following steps: acquiring speech information of a target scene and determining the scene type of the target scene; determining a target far-field speech recognition model matching the scene type, and inputting the speech information into the target far-field speech recognition model to process the speech information into dialogue information. The target far-field speech recognition model is obtained by training a preset far-field speech recognition model using a sample dataset. The sample dataset includes first sample data and second sample data. Each first sample data consists of original speech information under the scene type and dialogue information of the original speech information. Each second sample data consists of simulated speech information under the scene type and dialogue information of the simulated speech information. The simulated speech information is generated from noise information and original speech information under the scene type. This solves the problem of reducing the training sample cost of speech recognition models in related technologies. By superimposing different noise information onto the original speech data, multiple sample data based on the original speech data are established, thereby reducing the cost of acquiring sample data for the model. Attached Figure Description

[0017] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 This is a flowchart of a speech recognition method provided according to an embodiment of this application;

[0019] Figure 2 This is a schematic diagram simulating data in the speech recognition method provided according to the embodiments of this application;

[0020] Figure 3 This is a schematic diagram of the audio amplitude of the simulated signal waveform provided in the embodiments of this application. Figure 1 ;

[0021] Figure 4 This is a schematic diagram of the audio amplitude of the simulated signal waveform provided in the embodiments of this application. Figure 2 ;

[0022] Figure 5This is a schematic diagram of an optional data simulation method provided according to an embodiment of this application;

[0023] Figure 6 This is a schematic diagram of a speech recognition device provided according to an embodiment of this application;

[0024] Figure 7 This is a schematic diagram of an electronic device provided according to an embodiment of this application. Detailed Implementation

[0025] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0026] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] It should be noted that all information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this disclosure are information and data authorized by the user or fully authorized by all parties.

[0029] The present invention will now be described in conjunction with preferred implementation steps. Figure 1 This is a flowchart of a speech recognition method provided according to an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps:

[0030] Step S101: Obtain the voice information of the target scene and determine the scene type of the target scene.

[0031] Specifically, the target scene refers to the environment in which speech recognition needs to be performed. The scene type can include indoor and outdoor environments. For example, the target scene type can be a living room, kitchen, indoor venue, etc. The speech information refers to the recording information collected in the target scene, including dialogue and background sounds when no dialogue occurs. For example, if the target scene type is a living room, the collected speech information includes the dialogue between two people and the background sound from the television in the living room.

[0032] Step S102: Determine the target far-field speech recognition model that matches the scene type, and input the speech information into the target far-field speech recognition model to process the speech information into dialogue information. The target far-field speech recognition model is obtained by training a preset far-field speech recognition model with a sample dataset. The sample dataset includes first sample data and second sample data. Each first sample data consists of the original speech information under the scene type and the dialogue information of the original speech information. Each second sample data consists of the simulated speech information under the scene type and the dialogue information of the simulated speech information. The simulated speech information is generated from noise information and the original speech information under the scene type.

[0033] Specifically, a far-field speech recognition model refers to a model that uses far-field speech recognition technology to recognize user dialogue. Each target scene type can be matched with a type of far-field speech recognition model. By using the far-field speech recognition model to obtain speech information, the dialogue information contained in the speech information can be recognized and output. The first sample data can be the original speech information containing dialogue information obtained in the living room scene, and the second sample data can be the simulated speech information with various noise information superimposed on the original speech information.

[0034] It should be noted that the target far-field speech recognition model is obtained by training a preset far-field speech recognition model using multiple sample data. Since it is difficult to collect sample data of different target scenarios in the existing environment, it is necessary to simulate the sample data input into the model, which can greatly reduce the difficulty of training the model.

[0035] For example, if the target scenario is a living room, the far-field speech recognition model to be trained is the living room recognition model. The original speech information obtained is the user's dialogue information in the living room and the accompanying background sounds. By using the trained model to recognize the speech information, the user can output speech information such as controlling smart devices to perform a certain function. The speech recognition method provided in this application embodiment acquires speech information of a target scene and determines the scene type of the target scene; determines a target far-field speech recognition model matching the scene type, and inputs the speech information into the target far-field speech recognition model to process and obtain the dialogue information of the speech information. The target far-field speech recognition model is obtained by training a preset far-field speech recognition model using a sample dataset. The sample dataset includes first sample data and second sample data. Each first sample data consists of original speech information under the scene type and dialogue information of the original speech information. Each second sample data consists of simulated speech information under the scene type and dialogue information of the simulated speech information. The simulated speech information is generated from noise information and original speech information under the scene type. This method solves the problem of reducing the training sample cost of speech recognition models in related technologies. By superimposing different noise information on the original speech data, multiple sample data based on the original speech data are established, thereby achieving the effect of reducing the acquisition cost of sample data for the model.

[0036] The target far-field speech recognition model needs to be trained using multiple sample data. Optionally, in the speech recognition method provided in this application embodiment, the target far-field speech recognition model is trained in the following way: Z original speech information under scene type are obtained from the speech database, and the dialogue information of each original speech information is identified, where Z is a positive integer; for each original speech information, the original speech information and the dialogue information of the original speech information constitute first sample data, resulting in Z first sample data; for each original speech information, the original speech information and the noise information in the noise database are mixed to obtain simulated speech information, and the simulated speech information and the dialogue information of the original speech information constitute second sample data, resulting in Z second sample data; the Z first sample data and the Z second sample data constitute a sample dataset; and a preset far-field speech recognition model is trained using the sample dataset to obtain the target far-field speech recognition model.

[0037] Specifically, the speech database can be data from CHiME-5. Raw speech information refers to speech information that has not undergone processing. When training the target far-field speech recognition model, it is first necessary to determine the scene type of the target scene and obtain multiple raw speech information corresponding to the scene type from the speech database. For example, if the target scene is a living room, the raw speech information obtained from the speech database is the raw speech information of the living room scene. After obtaining multiple raw speech information, the multiple raw speech information is segmented to obtain noise information, and the noise information is mixed with the raw speech information to obtain multiple simulated speech information. The simulated speech information and the raw speech information are used together as a sample dataset and input into the preset far-field speech recognition model. The model is adjusted by calculating the model evaluation parameters through the dialogue information output by the model until it meets the preset requirements, and then the training of the far-field speech recognition model is completed.

[0038] It should be noted that simulated speech information is used to increase the sample dataset for training the model. Simulated speech information is obtained by adding noise information of different quantities and loudnesses from a noise database to the original speech information. This can be achieved using the librosa library in the software. The model's sample dataset is generated from the original speech information and simulated speech information, while the simulated speech information is based on the original speech information. This expands the sample data while reducing the workload of obtaining sample data.

[0039] Optionally, in the speech recognition method provided in this application embodiment, training a preset far-field speech recognition model from a sample dataset to obtain a target far-field speech recognition model includes: inputting sample speech information from the sample dataset into the preset far-field speech recognition model, processing it to obtain predicted dialogue information and the trained far-field speech recognition model, wherein the sample speech information includes the original speech information in the first sample data and the simulated speech information in the second sample data; calculating model evaluation parameters based on the predicted dialogue information and the dialogue information associated with the sample speech information; if the model evaluation parameters do not meet the preset recognition requirements, adjusting the trained far-field speech recognition model until the model evaluation parameters of the adjusted far-field speech recognition model meet the preset recognition requirements, and determining the adjusted far-field speech recognition model as the target far-field speech recognition model, wherein the model evaluation parameters include accuracy and speech error rate.

[0040] When training a far-field speech recognition model, it is necessary to evaluate the model using model evaluation parameters. Specifically, the predicted dialogue information output by the trained far-field speech recognition model is used to calculate the dialogue accuracy and speech error rate by associating the predicted dialogue information with the sample speech information. The calculated dialogue accuracy and speech error rate are then compared with the preset dialogue accuracy and speech error rate in the preset recognition requirements. If the dialogue accuracy is lower than the preset dialogue accuracy or the speech error rate is higher than the preset speech error rate, it indicates that the trained far-field speech recognition model does not meet the requirements and the parameters in the model need to be adjusted until the adjusted far-field speech recognition model achieves a dialogue accuracy greater than the preset dialogue accuracy while satisfying a speech error rate lower than the preset speech error rate. At this point, the model is used as the target far-field recognition model. By calculating the model evaluation parameters, the effect of model training can be better judged by data quantification.

[0041] Noise information is obtained by segmenting the original speech information. Optionally, in the speech recognition method provided in this application embodiment, before mixing the original speech information and noise information in the noise database for each original speech information to obtain simulated speech information, the method further includes: obtaining a labeling table associated with the original speech information, wherein the labeling table is used to label the dialogue time period information in the original speech information, the dialogue time period information includes the dialogue time interval and the dialogue text; obtaining the dialogue time period information from the labeling table to obtain M dialogue time period information, and sorting the M dialogue time period information according to the time information to obtain M sorted dialogue time period information, wherein M is a positive integer; segmenting the original speech information according to the M sorted dialogue time period information to obtain M dialogue segments, and adding the non-dialogue segments other than the M dialogue segments in the original speech information to a preset noise database to obtain a noise database.

[0042] To obtain noise information, the original speech information needs to be segmented. Specifically, non-dialogue segments, i.e. noise information, are obtained by acquiring a label table containing dialogue time period information associated with the original speech information, identifying the time intervals of multiple dialogues and dialogue text contained in the label table, and segmenting the original speech information by using the chronological order of the time period information of different dialogues, thereby obtaining multiple dialogue segments and multiple non-dialogue segments.

[0043] Furthermore, non-dialogue segments are added to a preset noise database. For example, when the target scene is a living room, after obtaining a 3-minute original audio message of the scene, two segments of dialogue time information are identified, each 30 seconds long, namely 30 seconds to 1 minute and 2 minutes to 2 minutes and 30 seconds. The identified information is then segmented to obtain noise information from 0 to 30 seconds, 1 to 2 minutes, and 2 minutes and 30 seconds to 3 minutes, and added to the noise database.

[0044] To obtain simulated speech information by mixing noise with the original speech information, optionally, in the speech recognition method provided in this application embodiment, for each piece of original speech information, mixing the original speech information with noise information in a noise database to obtain simulated speech information includes: obtaining N non-dialogue segments from the noise database, arranging and combining the N non-dialogue segments to obtain X non-dialogue combination segments, where N and X are positive integers; selecting any one of the X non-dialogue combination segments, and adding the selected non-dialogue combination segment to the original speech information to obtain simulated speech information.

[0045] Specifically, Figure 2 This is a schematic diagram simulating data in the speech recognition method provided in the embodiments of this application, such as... Figure 2 As shown, non-dialogue segments refer to information in the original speech information other than dialogue information. After adding the segmented non-dialogue segments to the noise database, in order to better train the far-field speech recognition model, different combinations of noise information can be added to the simulated speech information. For example, if four non-dialogue segments A, B, C, and D are obtained from the noise database, these non-dialogue segments can be arranged and combined. The result of the arrangement and combination can be four individual noise segments as a non-dialogue combination segment, a combination of two non-dialogue segments as a non-dialogue combination segment (e.g., AB, AC, BA, CA, etc.), a combination of three non-dialogue segments as a non-dialogue combination segment (e.g., ABC, BAC, ABD, etc.), etc. Any one of the arrangement and combination results can be selected and added to the original speech information for mixing to obtain simulated speech information. Through multiple combinations, multiple simulated data can be generated, which greatly increases the number of samples and improves the richness of the samples.

[0046] Phone error rate (PER) is a metric used to evaluate the error rate between predicted text and standard text. Therefore, a lower PER is better. The PER can be used to analyze whether simulated speech information can be used as sample data for training a far-field speech recognition model. Optionally, in the speech recognition method provided in this application embodiment, before mixing the original speech information with noise information from a noise database to obtain simulated speech information for each original speech information, the method further includes: obtaining a first PER of the original speech information and a second PER of the simulated speech information; determining whether the second PER is less than the first PER; if the second PER is less than the first PER, constructing second sample data from the dialogue information of the simulated speech information and the original speech information; and discarding the simulated speech information if the second PER is greater than or equal to the first PER.

[0047] After obtaining the mixed simulated speech information, to determine whether it can be used as sample data for training a far-field speech recognition model, the phoneme error rate of the simulated speech information can be analyzed. Specifically, after obtaining the phoneme error rates of the original speech information and the mixed simulated speech information using a speech recognition tool (e.g., Kaldi), the two phoneme error rates are compared. If the phoneme error rate of the original speech information is greater than that of the simulated speech information, it indicates that the simulated speech information can be used for training the far-field speech recognition model; otherwise, the simulated speech information is deleted, and new simulated speech information is obtained by mixing it with different noise information from a noise database. By using the phoneme error rate as the criterion for judging sample data, more simulated speech information can be obtained, and then this simulated speech information can be used to train a preset far-field speech recognition model to obtain a far-field recognition speech model that meets the requirements.

[0048] Whether simulated speech information can be used as sample data for training a far-field speech recognition model can be determined by analyzing simulated waveforms. Optionally, in the speech recognition method provided in this application embodiment, before mixing the original speech information with noise information in a noise database for each original speech information to obtain simulated speech information, the method further includes: acquiring a first simulated signal waveform of the original speech information and a second simulated signal waveform of the simulated speech information; determining whether the audio amplitude of the second simulated signal waveform is less than the audio amplitude of the first simulated signal waveform; if the audio amplitude of the second simulated signal waveform is less than the audio amplitude of the first simulated signal waveform, the dialogue information of the simulated speech information and the original speech information constitutes the second sample data; if the audio amplitude of the second simulated signal waveform is greater than or equal to the audio amplitude of the first simulated signal waveform, the simulated speech information is discarded.

[0049] After obtaining the mixed simulated speech information, in order to determine whether the simulated speech information can be used as sample data for training the far-field speech recognition model, the simulated waveform of the simulated speech information can be analyzed. Specifically, the original speech information and the mixed simulated speech information are input into the simulation software, and the simulation software is used to display the sound signal waveforms of the two speech information. It is determined whether the waveform amplitude of the original speech information is greater than that of the simulated speech information. When the waveform amplitude of the original speech information is greater than that of the simulated speech information, the dialogue information of the simulated speech information and the original speech information is added to the sample dataset as a second sample data. Otherwise, the simulated speech information is deleted and the simulated speech information is remixed.

[0050] Specifically, Figure 3 This is a schematic diagram of the audio amplitude of the simulated signal waveform provided in the embodiments of this application. Figure 1 ,like Figure 3 As shown in the figure, this is the audio amplitude waveform of the original speech information. The horizontal axis represents the time of the original speech information, and the vertical axis represents the audio amplitude. Figure 4 This is a schematic diagram of the audio amplitude of the simulated signal waveform provided in the embodiments of this application. Figure 2 ,like Figure 4 As shown in the figure, the waveform represents the audio amplitude of the simulated speech information. The horizontal axis represents the time of the simulated speech information, and the vertical axis represents the audio amplitude. The amplitude of the first waveform is greater than that of the second waveform, which means that the intensity of the original speech information is stronger than that of the simulated speech information. This indicates that the background noise in the simulated speech information is greater than the noise in the original speech information. This information has high complexity and can be used to train far-field speech recognition models well.

[0051] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0052] This application also provides an optional speech recognition method. Figure 5 This is a schematic diagram of an optional data simulation method provided according to an embodiment of this application, such as... Figure 5 As shown, the method includes:

[0053] Before acquiring the training data for the model, the operating environment parameters are configured using a speech recognition tool. Furthermore, the speech recognition tool is used to acquire the original speech information of the target scene from the CHiME-5 website, and the morpheme error rate of the original speech information is obtained in order to recognize and judge the mixed information.

[0054] Furthermore, using the dialogue time period information in the annotation table associated with the original speech information and programming software, the original speech information is segmented into speech segments and non-speech segments to obtain multiple speech segments and multiple non-speech segments, and the non-speech segments are stored as noise information in the noise database.

[0055] Then, the processing tools in the software are used to randomly obtain different noise information from the noise database and generate different noise files. The noise files are then randomly mixed with the original speech information to obtain mixed speech information, which is then added to the training data.

[0056] It should be noted that, in order to determine the state of the mixed speech information, a speech recognition tool is used to obtain the morpheme error rate of the mixed speech information, and the morpheme error rate is compared with the morpheme error rate of the original speech information. When the morpheme error rate is less than that of the original speech information, it indicates that the mixed speech information meets the requirements.

[0057] This embodiment achieves the effect of reducing the cost of analog data by using the original speech information to obtain multiple noise information. At the same time, since the scene type of the analog speech information is the same as that of the original speech information, the model recognition accuracy can be improved when using the analog speech information for model training.

[0058] This application also provides a speech recognition device. It should be noted that the speech recognition device of this application can be used to execute the speech recognition method provided in this application. The speech recognition device provided in this application will be described below.

[0059] Figure 6 This is a schematic diagram of a speech recognition device provided according to an embodiment of this application, such as... Figure 6 As shown, the device includes: a first acquisition unit 60 and a determination unit 61.

[0060] The first acquisition unit 60 is used to acquire the voice information of the target scene and determine the scene type of the target scene.

[0061] The determining unit 61 is used to determine the target far-field speech recognition model that matches the scene type, and input the speech information into the target far-field speech recognition model to process the speech information into dialogue information. The target far-field speech recognition model is obtained by training a preset far-field speech recognition model with a sample dataset. The sample dataset includes first sample data and second sample data. Each first sample data consists of the original speech information under the scene type and the dialogue information of the original speech information. Each second sample data consists of the simulated speech information under the scene type and the dialogue information of the simulated speech information. The simulated speech information is generated from noise information and the original speech information under the scene type.

[0062] Optionally, in the speech recognition device provided in this application embodiment, the target far-field speech recognition model is trained in the following manner: a first acquisition module is used to acquire Z original speech information under scene type from a speech database and recognize the dialogue information of each original speech information, where Z is a positive integer; a first composition module is used to construct first sample data from the original speech information and the dialogue information of the original speech information for each original speech information, thereby obtaining Z first sample data; a mixing module is used to mix the original speech information and noise information in a noise database for each original speech information to obtain simulated speech information, and construct second sample data from the simulated speech information and the dialogue information of the original speech information, thereby obtaining Z second sample data; a second composition module is used to construct a sample dataset from the Z first sample data and the Z second sample data; and a training module is used to train a preset far-field speech recognition model from the sample dataset to obtain the target far-field speech recognition model.

[0063] Optionally, in the speech recognition device provided in this application embodiment, the determining unit 61 includes: an input module, used to input sample speech information from the sample dataset into a preset far-field speech recognition model, process it to obtain predicted dialogue information and a trained far-field speech recognition model, wherein the sample speech information includes the original speech information in the first sample data and the simulated speech information in the second sample data; and a calculation module, used to calculate model evaluation parameters based on the predicted dialogue information and the dialogue information associated with the sample speech information, and, if the model evaluation parameters do not meet the preset recognition requirements, adjust the trained far-field speech recognition model until the model evaluation parameters of the adjusted far-field speech recognition model meet the preset recognition requirements, and determine the adjusted far-field speech recognition model as the target far-field speech recognition model, wherein the model evaluation parameters include accuracy and speech error rate.

[0064] Optionally, in the speech recognition device provided in this application embodiment, the device further includes: a second acquisition unit, configured to acquire an annotation table associated with the original speech information before mixing the original speech information and noise information in the noise database for each original speech information to obtain simulated speech information, wherein the annotation table is used to annotate the dialogue time period information in the original speech information, the dialogue time period information including the dialogue time interval and the dialogue text; a third acquisition unit, configured to acquire the dialogue time period information in the annotation table to obtain M dialogue time period information, and sort the M dialogue time period information according to time information to obtain M sorted dialogue time period information, wherein M is a positive integer; and a segmentation unit, configured to segment the original speech information according to the M sorted dialogue time period information to obtain M dialogue segments, and add non-dialogue segments other than the M dialogue segments in the original speech information to a preset noise database to obtain a noise database.

[0065] Optionally, in the speech recognition device provided in this application embodiment, the determining unit 61 includes: a second acquisition module, used to acquire N non-dialogue segments from a noise database, and to arrange and combine the N non-dialogue segments to obtain X non-dialogue combination segments, where N and X are positive integers; and a selection module, used to select any one of the X non-dialogue combination segments, and to add the selected non-dialogue combination segment to the original speech information to obtain simulated speech information.

[0066] Optionally, in the speech recognition device provided in this application embodiment, the method further includes: a fourth acquisition unit, configured to acquire a first phoneme error rate of the original speech information and a second phoneme error rate of the simulated speech information before mixing the original speech information and noise information in the noise database for each original speech information to obtain simulated speech information; a first judgment unit, configured to judge whether the second phoneme error rate is less than the first phoneme error rate; a first composition unit, configured to construct second sample data from the dialogue information of the simulated speech information and the original speech information when the second phoneme error rate is less than the first phoneme error rate; and a first discard unit, configured to discard the simulated speech information when the second phoneme error rate is greater than or equal to the first phoneme error rate.

[0067] Optionally, in the speech recognition device provided in this application embodiment, the device further includes: a fifth acquisition unit, configured to acquire a first simulated signal waveform of the original speech information and a second simulated signal waveform of the simulated speech information before mixing the original speech information and noise information in a noise database for each original speech information to obtain simulated speech information; a second judgment unit, configured to judge whether the audio amplitude of the second simulated signal waveform is less than the audio amplitude of the first simulated signal waveform; a second composition unit, configured to construct second sample data from the dialogue information of the simulated speech information and the original speech information when the audio amplitude of the second simulated signal waveform is less than the audio amplitude of the first simulated signal waveform; and a second discard unit, configured to discard the simulated speech information when the audio amplitude of the second simulated signal waveform is greater than or equal to the audio amplitude of the first simulated signal waveform.

[0068] The speech recognition device provided in this application embodiment acquires speech information of a target scene and determines the scene type of the target scene through a first acquisition unit 60; and determines a target far-field speech recognition model that matches the scene type, and inputs the speech information into the target far-field speech recognition model to process and obtain the dialogue information of the speech information. The target far-field speech recognition model is obtained by training a preset far-field speech recognition model with a sample dataset. The sample dataset includes first sample data and second sample data. Each first sample data consists of the original speech information under the scene type and the dialogue information of the original speech information. Each second sample data consists of simulated speech information under the scene type and the dialogue information of the simulated speech information. The simulated speech information is generated by noise information and the original speech information under the scene type. This solves the problem of reducing the training sample cost of speech recognition models in related technologies. By superimposing different noise information on the original speech data, multiple sample data based on the original speech data are established, thereby achieving the effect of reducing the acquisition cost of sample data for the model.

[0069] The voice recognition device includes a processor and a memory. The first acquisition unit 60, the determination unit 61, etc. are all stored in the memory as program units. The processor executes the program units stored in the memory to realize the corresponding functions.

[0070] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and adjusting kernel parameters can address the challenge of reducing the training sample cost for speech recognition models in related technologies.

[0071] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0072] This invention provides a computer-readable storage medium storing a program that, when executed by a processor, implements the speech recognition method.

[0073] This invention provides a processor for running a program, wherein the program executes the speech recognition method during runtime.

[0074] Figure 7 This is a schematic diagram of an electronic device provided according to an embodiment of this application, such as... Figure 7 As shown, this embodiment of the invention provides an electronic device 70, which includes a processor, a memory, and a program stored in the memory and executable on the processor. The processor is used to execute computer-readable instructions, wherein the computer-readable instructions execute a speech recognition method. The device described herein may be a server, PC, PAD, mobile phone, etc.

[0075] This application also provides a computer program product that, when executed on a data processing device, is suitable for performing a speech recognition method.

[0076] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0077] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0078] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0079] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0080] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0081] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0082] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0083] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0084] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0085] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A speech recognition method, characterized by, The method comprises: acquiring voice information of a target scene and determining a scene type of the target scene; determining a target far-field voice recognition model matched with the scene type, and inputting the voice information into the target far-field voice recognition model to process dialogue information of the voice information, wherein the target far-field voice recognition model is obtained by training a preset far-field voice recognition model based on a sample data set, the sample data set comprises first sample data and second sample data, each first sample data is composed of original voice information under the scene type and dialogue information of the original voice information, and each second sample data is composed of simulated voice information under the scene type and dialogue information of the simulated voice information, the simulated voice information is generated based on noise information and the original voice information under the scene type; wherein, before the simulated voice information is generated based on the noise information and the original voice information under the scene type, the method comprises: acquiring a first simulation signal waveform of the original voice information and a second simulation signal waveform of the simulated voice information; determining whether the audio amplitude of the second simulation signal waveform is less than the audio amplitude of the first simulation signal waveform; when the audio amplitude of the second simulation signal waveform is less than the audio amplitude of the first simulation signal waveform, the second sample data is composed of the simulated voice information and the dialogue information of the original voice information; and when the audio amplitude of the second simulation signal waveform is greater than or equal to the audio amplitude of the first simulation signal waveform, the simulated voice information is discarded.

2. The method of claim 1, wherein, The target far-field voice recognition model is obtained by training in the following manner: acquiring Z original voice information under the scene type from a voice database and recognizing dialogue information of each original voice information, wherein Z is a positive integer; for each original voice information, the first sample data is composed of the original voice information and the dialogue information of the original voice information, and Z first sample data is obtained; for each original voice information, the simulated voice information is obtained by mixing the original voice information and noise information in a noise database, the second sample data is composed of the simulated voice information and the dialogue information of the original voice information, and Z second sample data is obtained; the sample data set is composed of the Z first sample data and the Z second sample data; the preset far-field voice recognition model is trained based on the sample data set to obtain the target far-field voice recognition model.

3. The method of claim 2, wherein, The target far-field voice recognition model is obtained by training the preset far-field voice recognition model based on the sample data set, which comprises: inputting sample voice information in the sample data set into the preset far-field voice recognition model to process predicted dialogue information and a trained far-field voice recognition model, wherein the sample voice information comprises original voice information in the first sample data and simulated voice information in the second sample data; According to the predicted dialogue information and the dialogue information associated with the sample voice information, a model evaluation parameter is calculated, and in a case where the model evaluation parameter does not meet a preset recognition requirement, the trained remote voice recognition model is adjusted until the model evaluation parameter of the adjusted remote voice recognition model meets the preset recognition requirement, and the adjusted remote voice recognition model is determined as the target remote voice recognition model, wherein the model evaluation parameter includes an accuracy rate and a speech error rate.

4. The method of claim 2, wherein, Before mixing the original voice information and noise information in the noise database to obtain simulated voice information for each original voice information, the method further comprises: obtaining a label table associated with the original voice information, wherein the label table is used to label dialogue time period information of a dialogue in the original voice information, and the dialogue time period information includes a time interval of the dialogue and dialogue text of the dialogue; obtaining dialogue time period information of a dialogue from the label table, obtaining M dialogue time period information, and sorting the M dialogue time period information according to time information to obtain M sorted dialogue time period information, wherein M is a positive integer; cutting the original voice information according to the M sorted dialogue time period information to obtain M dialogue segments, and adding non-dialogue segments other than the M dialogue segments in the original voice information to a preset noise database to obtain the noise database.

5. The method of claim 2, wherein, Mixing the original voice information and noise information in the noise database to obtain simulated voice information for each original voice information comprises: obtaining N non-dialogue segments from the noise database, performing permutation and combination on the N non-dialogue segments to obtain X non-dialogue combined segments, wherein N and X are positive integers; selecting any one of the X non-dialogue combined segments, and adding the selected non-dialogue combined segment to the original voice information to obtain the simulated voice information.

6. The method according to any one of claims 2 to 5, characterized in that, Before mixing the original voice information and noise information in the noise database to obtain simulated voice information for each original voice information, the method further comprises: obtaining a first phoneme error rate of the original voice information and a second phoneme error rate of the simulated voice information; determining whether the second phoneme error rate is less than the first phoneme error rate; in a case where the second phoneme error rate is less than the first phoneme error rate, constructing the second sample data from the simulated voice information and dialogue information of the original voice information; in a case where the second phoneme error rate is greater than or equal to the first phoneme error rate, discarding the simulated voice information.

7. A speech recognition apparatus characterized by comprising: comprises: a first obtaining unit, configured to obtain voice information of a target scene and determine a scene type of the target scene; The determining unit is configured to determine a target far-field speech recognition model matched with the scene type, input the speech information into the target far-field speech recognition model, and obtain dialogue information of the speech information through processing. The fifth obtaining unit is configured to obtain a first simulation signal waveform of the original speech information and a second simulation signal waveform of the simulated speech information before the simulated speech information is generated from the noise information and the original speech information in the scene type.

8. A processor, comprising: The second judging unit is configured to judge whether an audio amplitude of the second simulation signal waveform is less than an audio amplitude of the first simulation signal waveform.

9. An electronic device, comprising: The second constituting unit is configured to constitute the second sample data from the simulated speech information and the dialogue information of the original speech information when the audio amplitude of the second simulation signal waveform is less than the audio amplitude of the first simulation signal waveform. The processor is configured to run a program, and the program performs the speech recognition method in any one of claims 1 to 6 when the program is run. The device comprises one or more processors and a memory, and the memory is configured to store one or more programs, and the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the speech recognition method in any one of claims 1 to 6.

Images