Speech recognition method, apparatus, device, and storage medium

Abstract

This disclosure provides a speech recognition method, apparatus, device, and storage medium. The method involves acquiring initial text and an initial speech unit sequence corresponding to the speech to be processed in a target scene; acquiring a specified number of target data groups from a dataset, wherein the target data groups include target text and target speech unit sequences, the initial speech unit sequence and the target speech unit sequence satisfy similarity conditions, and the data information included in the dataset is related to the target scene; and inputting the initial text corresponding to the speech to be processed, the initial speech unit sequence, and the specified number of target data groups into a target recognition model.

Description

Technical Field

[0001] This disclosure relates to the field of speech processing technology, and in particular to a speech recognition method, apparatus, device, and storage medium. Background Technology

[0002] Voice-based intelligent customer service has a wide range of applications. The process involves transcribing the user's speech into text using Automatic Speech Recognition (ASR), and then passing the text to Natural Language Understanding (NLU) for intent recognition.

[0003] One of the challenges facing intelligent voice customer service is the transmission of erroneous text resulting from errors in the transcription of user speech by Automatic Speech Recognition (ASR). Errors in ASR transcription can easily lead to the loss of important information and the inclusion of significant noise, hindering proper understanding by subsequent Natural Language Understanding (NLU). Summary of the Invention

[0004] This disclosure provides a speech recognition method, apparatus, device, and storage medium.

[0005] According to a first aspect of this disclosure, a speech recognition method is provided, comprising:

[0006] Obtain the initial text and initial speech unit sequence corresponding to the speech to be processed in the target scene;

[0007] Obtain a specified number of target data sets from the dataset, wherein the target data sets include target text and target speech unit sequences, the initial speech unit sequence and the target speech unit sequence satisfy similarity conditions, and the data information included in the dataset is related to the target scene;

[0008] The initial text corresponding to the speech to be processed, the initial speech unit sequence, and the specified number of target data groups are input into the target recognition model to recognize the speech to be processed.

[0009] In one possible implementation, obtaining the initial text and initial speech unit sequence corresponding to the speech to be processed in the target scene includes:

[0010] The speech to be processed is input into the speech recognition model to obtain the initial text of the speech to be processed;

[0011] The initial text is matched with information in the syllable database to obtain the initial speech unit sequence corresponding to the initial text.

[0012] In one possible implementation, matching the initial text with information in a syllable database to obtain an initial speech unit sequence corresponding to the initial text includes at least one of the following:

[0013] The characters in the initial text are matched with the characters in the syllable database to obtain the syllable information corresponding to each character in the initial text. The syllable information corresponding to each character in the initial text is arranged in the order of the characters in the initial text to obtain the initial speech unit sequence corresponding to the initial text. The syllable database includes characters and their corresponding syllable information, and the syllable information is composed of multiple phoneme information.

[0014] The initial text is segmented into words to obtain initial word segments. The word segments in the initial word segments are matched with the text in the syllable database to obtain the syllable segment information corresponding to each word segment in the initial word segments. The syllable segment information corresponding to each word segment in the initial word segments is arranged according to the word segmentation order in the initial word segments to obtain the initial speech unit sequence corresponding to the initial text.

[0015] In one possible implementation, obtaining a specified number of target data groups from the dataset includes:

[0016] Obtain the speech unit sequence of each text information in the dataset;

[0017] The similarity values ​​of each text information are obtained by performing similarity matching between the speech unit sequence of the text information in the dataset and the initial speech unit sequence.

[0018] The text information corresponding to the similarity value that meets the similarity condition is taken as the target text, and the speech unit sequence corresponding to the target text is taken as the target speech unit sequence. Among them, multiple target texts that meet the similarity condition and their corresponding target speech unit sequences constitute a specified number of target data groups.

[0019] In one possible implementation, the step of performing similarity matching between the speech unit sequences of text information in the dataset and the initial speech unit sequence to obtain similarity values ​​for each piece of text information includes at least one of the following:

[0020] According to the order of the same position in the sequence, the phoneme information in the initial speech unit sequence is matched with the phoneme information of each speech unit sequence in the dataset to obtain the similarity value of the text information corresponding to each speech unit sequence.

[0021] The syllable segment information within the initial speech unit sequence is matched with the syllable segment information of each speech unit sequence in the dataset to obtain the similarity value of the text information corresponding to each speech unit sequence.

[0022] In one possible implementation, inputting the initial text corresponding to the speech to be processed, the initial speech unit sequence, and the specified number of target data sets into the target recognition model includes:

[0023] The initial text corresponding to the speech to be processed, the initial speech unit sequence, and the specified number of target data groups are concatenated or fused to obtain the data group to be recognized.

[0024] The data set to be identified is input into the target recognition model.

[0025] In one possible implementation, the step of inputting the initial text corresponding to the speech to be processed, the speech unit sequence, and the specified number of target data sets into the target recognition model to recognize the speech to be processed includes:

[0026] The initial text corresponding to the speech to be processed, the speech unit sequence, and the specified number of target data groups are input into the target recognition model to obtain the text information corresponding to the speech to be processed.

[0027] In one possible implementation, the step of inputting the initial text corresponding to the speech to be processed, the speech unit sequence, and the specified number of target data sets into the target recognition model to recognize the speech to be processed includes:

[0028] The initial text corresponding to the speech to be processed, the speech unit sequence, and the specified number of target data groups are input into the target recognition model to obtain the intent information corresponding to the speech to be processed, so as to respond based on the intent information.

[0029] In one embodiment, the target recognition model is a model trained based on text samples and their corresponding speech unit sequence samples, and the training process includes:

[0030] Obtain text samples and their corresponding speech unit sequence samples;

[0031] In the dataset, obtain a specified number of target data group samples corresponding to the speech unit sequence samples;

[0032] The text samples, the speech unit sequence samples, and the specified number of target data group samples are combined into training pairs, and text labels are assigned to them.

[0033] The training pair and its text labels are input into the target recognition model for training to obtain the training results. The target recognition model is then adjusted based on the training results and the text labels to obtain a mature target recognition model.

[0034] According to a second aspect of this disclosure, a speech recognition device is provided, the device comprising:

[0035] The first acquisition module is used to acquire the initial text and initial speech unit sequence corresponding to the speech to be processed in the target scene;

[0036] The second acquisition module is used to acquire a specified number of target data groups from the dataset, wherein the target data groups include target text and target speech unit sequences, the initial speech unit sequence and the target speech unit sequence satisfy similarity conditions, and the data information included in the dataset is related to the target scene;

[0037] The recognition module is used to input the initial text corresponding to the speech to be processed, the initial speech unit sequence, and the specified number of target data groups into the target recognition model to recognize the speech to be processed.

[0038] In one possible implementation, the first acquisition module is specifically used for:

[0039] The speech to be processed is input into the speech recognition model to obtain the initial text of the speech to be processed;

[0040] The initial text is matched with information in the syllable database to obtain the initial speech unit sequence corresponding to the initial text.

[0041] In one possible implementation, the first acquisition module is further configured to be used for at least one of the following:

[0042] The characters in the initial text are matched with the characters in the syllable database to obtain the syllable information corresponding to each character in the initial text. The syllable information corresponding to each character in the initial text is arranged in the order of the characters in the initial text to obtain the initial speech unit sequence corresponding to the initial text. The syllable database includes characters and their corresponding syllable information, and the syllable information is composed of multiple phoneme information.

[0043] The initial text is segmented into words to obtain initial word segments. The word segments in the initial word segments are matched with the text in the syllable database to obtain the syllable segment information corresponding to each word segment in the initial word segments. The syllable segment information corresponding to each word segment in the initial word segments is arranged according to the word segmentation order in the initial word segments to obtain the initial speech unit sequence corresponding to the initial text.

[0044] In one possible implementation, the second acquisition module is specifically used for:

[0045] Obtain the speech unit sequence of each text information in the dataset;

[0046] The similarity values ​​of each text information are obtained by performing similarity matching between the speech unit sequence of the text information in the dataset and the initial speech unit sequence.

[0047] The text information corresponding to the similarity values ​​that meet the similarity criteria is taken as the target text, and the speech unit sequence corresponding to the target text is taken as the target speech unit sequence. Multiple target texts that meet the similarity criteria and their corresponding target speech unit sequences constitute a specified number of target data groups.

[0048] In one possible implementation, the second acquisition module is further configured to include at least one of the following:

[0049] According to the order of the same position in the sequence, the phoneme information in the initial speech unit sequence is matched with the phoneme information of each speech unit sequence in the dataset to obtain the similarity value of the text information corresponding to each speech unit sequence.

[0050] The syllable segment information within the initial speech unit sequence is matched with the syllable segment information of each speech unit sequence in the dataset to obtain the similarity value of the text information corresponding to each speech unit sequence.

[0051] In one possible implementation, the identification module is specifically used for:

[0052] The initial text corresponding to the speech to be processed, the initial speech unit sequence, and the specified number of target data groups are concatenated or fused to obtain the data group to be recognized.

[0053] The data set to be identified is input into the target recognition model.

[0054] In one possible implementation, the recognition module is specifically used to: input the initial text corresponding to the speech to be processed, the speech unit sequence, and the specified number of target data groups into the target recognition model to obtain the text information corresponding to the speech to be processed.

[0055] In one possible implementation, the recognition module is specifically used to: input the initial text corresponding to the speech to be processed, the speech unit sequence, and the specified number of target data groups into the target recognition model to obtain the intent information corresponding to the speech to be processed, so as to respond based on the intent information.

[0056] In one possible implementation, it further includes:

[0057] The training module is specifically used for: the target recognition model being a model trained based on text samples and their corresponding speech unit sequence samples.

[0058] Obtain text samples and their corresponding speech unit sequence samples;

[0059] In the dataset, obtain a specified number of target data group samples corresponding to the speech unit sequence samples;

[0060] The text samples, the speech unit sequence samples, and the specified number of target data group samples are combined into training pairs, and text labels are assigned to them.

[0061] The training pair and its text labels are input into the target recognition model for training to obtain the training results. The target recognition model is then adjusted based on the training results and the text labels to obtain a mature target recognition model.

[0062] According to a third aspect of this disclosure, an electronic device is provided, comprising:

[0063] At least one processor; and

[0064] A memory communicatively connected to the at least one processor; wherein,

[0065] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the methods described in this disclosure.

[0066] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the methods described in this disclosure.

[0067] The speech recognition method, apparatus, device, and storage medium disclosed herein acquire initial text and initial speech unit sequence corresponding to the speech to be processed in a target scene; acquire a specified number of target data groups in a dataset, wherein the target data groups include target text and target speech unit sequences, the initial speech unit sequence and the target speech unit sequence satisfy similarity conditions, and the data information included in the dataset is related to the target scene; input the initial text corresponding to the speech to be processed, the initial speech unit sequence, and the specified number of target data groups into a target recognition model to recognize the speech to be processed, thereby improving the recognition accuracy of the model.

[0068] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0069] The above and other objects, features, and advantages of this disclosure will become readily apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. Several embodiments of this disclosure are illustrated in the drawings by way of example and not limitation, in which:

[0070] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0071] Figure 1 A schematic diagram illustrating the implementation flow of a speech recognition method provided in an embodiment of this disclosure is shown;

[0072] Figure 2 A schematic diagram illustrating the implementation flow of step S110 in a speech recognition method provided in an embodiment of this disclosure is shown.

[0073] Figure 3 This illustration shows a schematic diagram of the implementation flow of step S120 in a speech recognition method provided in an embodiment of the present disclosure;

[0074] Figure 4 This illustration shows a current speech recognition method provided by an embodiment of the present disclosure;

[0075] Figure 5 A schematic diagram illustrating the implementation flow of a speech recognition method provided in an embodiment of this disclosure is shown;

[0076] Figure 6 A schematic diagram illustrating the implementation flow of a speech recognition training method provided in an embodiment of this disclosure is shown.

[0077] Figure 7 A schematic diagram of the structure of a speech recognition device provided in Embodiment 3 of this disclosure is shown;

[0078] Figure 8 A schematic diagram of the composition structure of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0079] To make the objectives, features, and advantages of this disclosure more apparent and understandable, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0080] Figure 1 This is a flowchart illustrating a speech recognition method provided in Embodiment 1 of this disclosure. This method can be executed by the speech recognition device provided in this embodiment, which can be implemented using software and / or hardware. Specifically, the method includes:

[0081] S110. Obtain the initial text and the initial speech unit sequence corresponding to the speech to be processed in the target scenario.

[0082] Among them, the target scenario can be a relevant scenario used by a voice intelligent customer service. For example, if the intelligent voice customer service is a customer service for selling item A, the target scenario can be a scenario related to item A and sales; if the intelligent voice customer service is a tour guide customer service for a certain park, the target scenario can be a scenario related to that park and tourism.

[0083] Among them, the speech to be processed is the speech with recognition requirements. The initial text is the text content obtained by converting the speech to be processed, and the initial speech unit sequence is the speech feature corresponding to the speech to be processed, which is a sequence composed of syllable segment information, a sequence composed of syllable information, or a sequence called phoneme composition. For example, the initial speech unit sequence can be composed of multiple syllable information, and each syllable information is composed of different phoneme information, which can carry word segmentation information or not. And its word segmentation information can be segmented by characters or by words, and this embodiment does not limit it. Exemplarily, if the initial text of the speech to be processed is "Go boating in the park?", its corresponding initial speech unit sequence can be "qu gongyuan youhu me", or "qugongyuanyouhume", or "qu gongyuan you hu me".

[0084] In the process of ASR speech-to-text conversion, although there will be phoneme information in the intermediate quantity, its phoneme information is related to the conversion rule and is different from the result obtained by the method of obtaining phonemes from the text. Therefore, in this embodiment, the corresponding initial speech unit sequence is obtained through the initial text. This embodiment can obtain the speech to be processed in the target scenario, and obtain the corresponding initial text and initial speech unit sequence through the speech to be processed; this embodiment can also directly obtain the initial text and initial speech unit sequence corresponding to the speech to be processed, for example, it can be the content transferred by other electronic devices.

[0085] S120. Obtain a specified number of target data groups from the data set.

[0086] Among them, the target data group includes the target text and the target speech unit sequence, the initial speech unit sequence and the target speech unit sequence meet the similarity condition, and the data information included in the data set is related to the target scenario.

[0087] The dataset contains data information related to the target scenario. For example, if the target scenario is the sale of item A, the dataset contains data information related to the sale of item A; if the target scenario is a tour guide and tourists visiting a park, the dataset contains data information related to the park and tourism. The data information can be text information or the corresponding speech unit sequences.

[0088] In this embodiment, the target data set consists of more information derived from the speech to be processed, the target text consists of text information derived from the speech to be processed, and the target speech unit sequence consists of speech-related information derived from the speech to be processed. For example, if the initial text of the speech to be processed is "Want to go boating in the park?", its target data set could be two sets of target text and their corresponding target speech unit sequences. Exemplary examples could be "Boating in the park; gongyuan youhu" and "Is the park open for boating?; gongyuan youhu kaimen me". It should be noted that the target data set in this embodiment is merely an example and its specific content is not limited.

[0089] Specifically, in order to obtain more comprehensive information about the speech to be processed, this embodiment can obtain a target data group related to the speech in the dataset. This target data group may contain target text and target speech unit sequences. For example, this embodiment can use an initial speech unit sequence to find speech unit sequences in the dataset that meet similarity criteria as target speech unit sequences, and use the text corresponding to the target speech unit sequences as target text. The target text and the target speech unit sequences are then combined to form the target data group. It should be noted that the specified quantity in this embodiment can be any value set according to requirements, such as the top 3 texts with the highest similarity, without limiting its specific quantity.

[0090] S130. Input the initial text corresponding to the speech to be processed, the initial speech unit sequence, and a specified number of target data groups into the target recognition model to recognize the speech to be processed.

[0091] Among them, the target recognition model refers to the model used to identify textual information and intent information in the speech to be processed.

[0092] Specifically, in this embodiment, after obtaining relatively comprehensive information about the speech to be processed, all of it is input into the target recognition model. Specifically, the initial text corresponding to the speech to be processed, the initial speech unit sequence, and a specified number of target data sets are input into the target recognition model, thereby obtaining a relatively accurate recognition result.

[0093] In one embodiment, the initial text corresponding to the speech to be processed, the initial speech unit sequence, and a specified number of target data groups are input into the target recognition model, including: concatenating or fusing the initial text corresponding to the speech to be processed, the initial speech unit sequence, and the specified number of target data groups to obtain the data group to be recognized; and inputting the data group to be recognized into the target recognition model.

[0094] The data set to be recognized is the data obtained by splicing or fusing the initial text, initial speech unit sequence and a specified number of target data sets corresponding to the speech to be processed. This allows the initial text, initial speech unit sequence and the specified number of target data sets to be connected in an orderly manner, and then the data set to be recognized is directly input into the target recognition model.

[0095] In this embodiment, the initial text corresponding to the speech to be processed, the initial speech unit sequence, and a specified number of target data groups are spliced ​​or fused to obtain the data group to be recognized. The input data is then organized to make the data input to the target recognition model more neat and orderly.

[0096] In one possible implementation, the initial text corresponding to the speech to be processed, the speech unit sequence, and a specified number of target data sets are input into a target recognition model to recognize the speech to be processed. This includes: inputting the initial text corresponding to the speech to be processed, the speech unit sequence, and a specified number of target data sets into the target recognition model to obtain the text information corresponding to the speech to be processed.

[0097] Specifically, in this embodiment, after inputting the initial text corresponding to the speech to be processed, the speech unit sequence, and a specified number of target data groups into the target recognition model, the text information corresponding to the speech to be processed can be obtained. This text information can completely express the information conveyed by the speech to be processed.

[0098] In one possible implementation, the initial text corresponding to the speech to be processed, the speech unit sequence, and a specified number of target data sets are input into the target recognition model to obtain the intent information corresponding to the speech to be processed, so as to respond based on the intent information.

[0099] Specifically, in this embodiment, after inputting the initial text, speech unit sequence, and a specified number of target data groups corresponding to the speech to be processed into the target recognition model, the intent information corresponding to the speech to be processed can be obtained. This intent information can be a more concise and accurate content extracted from the above text information, that is, intent information, which can accurately and concisely express the intent information of the speech to be processed, and respond based on the intent information.

[0100] In this embodiment, the initial text and the initial speech unit sequence corresponding to the speech to be processed in the target scenario are obtained; a specified number of target data groups are obtained from the dataset; the initial text, the initial speech unit sequence corresponding to the speech to be processed, and the specified number of target data groups are input into the target recognition model to recognize the speech to be processed. In this embodiment, without changing any parameters in the target recognition model, the recognition accuracy of the model can be improved, and a higher NLU recognition rate can be achieved.

[0101] In an optional embodiment, in step S110, the initial text and the initial speech unit sequence corresponding to the speech to be processed in the target scenario are obtained, as Figure 2 shown, which specifically includes:

[0102] Step S110a, input the speech to be processed into the speech recognition model to obtain the initial text of the speech to be processed.

[0103] Among them, the speech recognition model can be a model that converts the speech to be processed into the initial text. The initial text is the text after the speech to be processed is transformed. Specifically, in this embodiment, the speech to be processed can be input into the speech recognition model to obtain the initial text corresponding to the speech to be processed.

[0104] Step S110b, match the initial text with the information in the syllable database to obtain the initial speech unit sequence corresponding to the initial text.

[0105] Among them, the syllable database includes characters and their corresponding syllable information, and the syllable information is composed of multiple phoneme information. For example, the syllable information corresponding to the character "公" is "gong", which is composed of four phoneme information "g, o, n, g".

[0106] Since all characters and their syllable information are included in the syllable database, in this embodiment, the syllable information corresponding to each character in the initial text can be determined through the syllable database, and then the initial speech unit sequence can be obtained by combining the syllable information corresponding to each character.

[0107] In an optional embodiment, the initial text is matched with the information in the syllable database to obtain an initial speech unit sequence corresponding to the initial text, including at least one of the following: the characters in the initial text are matched with the characters in the syllable database to obtain the syllable information corresponding to each character in the initial text, and the syllable information corresponding to each character in the initial text is arranged in the order of the characters in the initial text to obtain the initial speech unit sequence corresponding to the initial text, where the syllable database includes characters and their corresponding syllable information, and the syllable information is composed of multiple phoneme information; the initial text is segmented to obtain initial segments, the segments in the initial segments are matched with the characters in the syllable database to obtain the syllable segment information corresponding to each segment in the initial segments, and the syllable segment information corresponding to each segment in the initial segments is arranged in the order of the segments in the initial segments to obtain the initial speech unit sequence corresponding to the initial text.

[0108] Among them, the initial segment is text information with segmentation information. The syllable segment information can be composed of multiple syllable information. For example, after segmenting "going to the park to row a boat on the lake", we get "park" and "row a boat on the lake", and there are also two syllable segment information according to the segmentation. Exemplarily, the syllable segment information corresponding to "park" is "gongyuan", and the syllable segment information corresponding to "row a boat on the lake" is "youhu".

[0109] Specifically, the method of matching the initial text with the information in the syllable database in this embodiment can include two types. One is to match according to each character in the initial text in the syllable database, and the other is to first perform segmentation processing on the initial text and then match according to each segment in the syllable database.

[0110] Specifically, in this embodiment, in order, each character in the initial text can be matched with the characters in the syllable database to obtain the syllable information corresponding to each character in the initial text, and the syllable information corresponding to each character in the initial text is arranged in the order of the characters in the initial text to obtain the initial speech unit sequence corresponding to the initial text. For example, if the initial text is "Are you going to the park to row a boat on the lake?", then each of its characters is matched with the characters in the syllable database, and the initial speech unit sequence corresponding to the initial text is "qugongyuanyouhume" or "qu gong yuan you hume".

[0111] Specifically, in this embodiment, the initial text can also be segmented by a mature word segmentation software to obtain initial word segmentation. For example, if the initial text is "Go to the park to row a boat?", the initial word segmentation is "go / park / row a boat / ?" In this embodiment, the words in the initial word segmentation are then matched with the words in the syllable database to obtain the syllable segment information corresponding to each word arranged in the order of the words in the initial word segmentation. The four words "go / park / row a boat / ?" are respectively matched to obtain the sorted syllable segment information as "qu gongyuan youhu me", and the initial speech unit sequence corresponding to the initial text can also be obtained.

[0112] This embodiment provides two methods for obtaining the initial speech unit sequence, which can quickly and efficiently obtain the initial speech unit sequence corresponding to the initial text and meet the needs of users with different requirements.

[0113] In an optional embodiment, in step S120, a specified number of target data groups are obtained from the data set, as Figure 3 shown, including:

[0114] S120a, obtaining the speech unit sequence of each text information in the data set.

[0115] Specifically, in this embodiment, all the text information in the data set is also matched through the syllable database to obtain the speech unit sequence of each text information. The specific matching method is as above and will not be elaborated here.

[0116] S120b, respectively performing similarity matching between the speech unit sequence of the text information in the data set and the initial speech unit sequence to obtain the similarity value of each text information.

[0117] In order to prevent errors in speech transcription and resulting in incorrect recognition of the intention of the initial text in this embodiment, more information is obtained through the data set. Specifically, in this embodiment, the initial speech unit sequence can be used to match the speech unit sequence of the text information in the data set to obtain the similarity value of each text information.

[0118] In an optional embodiment, performing similarity matching between the speech unit sequence of the text information in the data set and the initial speech unit sequence to obtain the similarity value of each text information includes at least one of the following: in the order of the same positions in the sequence, matching each phoneme information in the initial speech unit sequence with the phoneme information of each speech unit sequence in the data set to obtain the similarity value of the text information corresponding to each speech unit sequence; matching each syllable segment information in the initial speech unit sequence with the syllable segment information of each speech unit sequence in the data set to obtain the similarity value of the text information corresponding to each speech unit sequence.

[0119] Specifically, in this embodiment, the phoneme information within the initial speech unit sequence is matched with the phonemes of each speech unit sequence in the dataset according to the order of their corresponding positions. If a match is successful, it is set as the first number; if a match fails, it is set as the second number. After matching, a set of numbers corresponding to each speech unit sequence is obtained. Finally, the cosine similarity method is used to calculate the similarity value of each set of numbers, thus obtaining the similarity value of the text information corresponding to each speech unit sequence.

[0120] Specifically, this embodiment can also be based on the one-hot coding algorithm to match the syllable segment information of each syllable segment in the initial speech unit sequence with the syllable segment information of each speech unit sequence in the dataset to obtain a set of numbers; then, the cosine similarity method is used to calculate the similarity value of the text information corresponding to each speech unit sequence.

[0121] S120c, the text information corresponding to the similarity value that meets the similarity condition is taken as the target text, and the speech unit sequence corresponding to the target text is taken as the target speech unit sequence.

[0122] Among them, multiple target texts that meet similar conditions and their corresponding target speech unit sequences constitute a specified number of target data groups.

[0123] Specifically, this embodiment can set a similarity threshold, and use the text information corresponding to all similarity values ​​that reach the threshold as the target text. Alternatively, it can sort all similarity values, set a specified number, and use the text information corresponding to the highest similarity values ​​(a specified number) as the target text. Furthermore, the speech unit sequence corresponding to the target text is used as the target speech unit sequence. Several sets of target texts and target speech unit sequences that meet the conditions are combined to obtain a specified number of target data sets.

[0124] Figure 4 This illustration of a current speech recognition method provided in this disclosure includes speech to be processed 410 and initial text 420. For example... Figure 4 As shown, the current error tolerance is very low. When an error occurs when the speech to be processed 410 is converted into the initial text 420, it affects the subsequent intent recognition.

[0125] Figure 5This is a schematic diagram of a speech recognition method provided in an embodiment of the present disclosure, including: speech to be processed 510, initial text 520, initial speech unit sequence 530, acquisition of a specified number of similarity values ​​540, dataset 550, and target data group 560. Specifically, in this embodiment, the speech to be processed 510 is first converted into initial text 520 using ASR speech recognition, and the initial speech unit sequence 530 of the initial text 520 is obtained. Then, similarity values ​​corresponding to each text information are obtained from the dataset 550, a specified number of similarity values ​​540 are obtained, and finally, the text information and speech unit sequence corresponding to the specified number of similarity values ​​540 in the dataset are used as the target data group 560. Figure 5 As shown, the initial text 520 in this embodiment is "Is the Summer Palace open today?", and its initial speech unit sequence 530 is "yiheyuan jinri youhu kaimen ma". Three target data sets are extracted, including "Is the Summer Palace open today for lake tours?"; "Is the Summer Palace open for lake tours today?"; and "Is the Summer Palace open for lake tours today?". Although a text error occurs in the initial text conversion, due to the acquisition of more relevant information from the dataset, the target data sets, initial text, and initial speech unit sequence are finally fed into NLU recognition (i.e.,...). Figure 5 The Tramsformer in the system can provide more accurate intent information such as "lake tour - opening time".

[0126] It should be noted that this embodiment Figure 5 The complete information for target data group 560 consists of three groups. Figure 5 The arrows are used for illustration only; the complete information is not shown.

[0127] This embodiment retains erroneous text, reducing error propagation caused by error correction. It uses non-parametric external corpus to enhance the model, achieving a higher NLU recognition rate. Furthermore, it requires no additional corpus, saving time and effort while maintaining high recognition efficiency.

[0128] In one embodiment, the target recognition model is a model trained based on text samples and their corresponding speech unit sequence samples. The training process includes: acquiring text samples and their corresponding speech unit sequence samples; acquiring a specified number of target data group samples corresponding to the speech unit sequence samples in the dataset; combining the text samples, speech unit sequence samples, and the specified number of target data group samples into training pairs and labeling them with text tags; inputting the training pairs and their text tags into the target recognition model for training to obtain training results, and adjusting the target recognition model according to the training results and text tags to obtain a mature target recognition model.

[0129] Here, text samples refer to the text information used during training, while speech unit sequence samples are sequences of syllables, phonemes, or other information obtained by matching text samples within a syllable database. Target data set samples can be data obtained from the dataset that matches the speech unit sequence samples. Text labels can be accurate, labeled intent information.

[0130] Since this embodiment describes the training process of an object recognition model, it combines text samples, speech unit sequence samples, and a specified number of target data sets into training pairs, labels them with text, and inputs them into the object recognition model for learning. This embodiment continuously compares the training results with the labeled text tags to adjust the parameters within the object recognition model, resulting in a more mature object recognition model. Figure 6 As shown, Figure 6 This is a schematic diagram of a speech recognition training method provided in an embodiment of the present disclosure, as shown below. Figure 6 As shown, the target data group samples are schematically indicated as "Is the Summer Palace open for lake tours today?" and "Is the Summer Palace open for lake tours today?" etc.

[0131] This embodiment obtains a specified number of target data group samples by using centroid-assisted decision-making based on similar content. The target data group samples form training pairs and text labels that mark the training pairs. The correct intent information is labeled to allow the target recognition model to learn. This approach addresses the ASR error problem from a new angle by incorporating speech unit sequence samples into the recognition process. This allows for the correct intent recognition even when erroneous content is retained. The target recognition model has good generalization ability and improves the accuracy of intent recognition.

[0132] Figure 7 This is a schematic diagram of the structure of a speech recognition device provided in an embodiment of this disclosure. The device specifically includes:

[0133] The first acquisition module 710 is used to acquire the initial text and initial speech unit sequence corresponding to the speech to be processed in the target scene;

[0134] The second acquisition module 720 is used to acquire a specified number of target data groups from the dataset, wherein the target data groups include target text and target speech unit sequences, the initial speech unit sequence and the target speech unit sequence satisfy similarity conditions, and the data information included in the dataset is related to the target scene;

[0135] The recognition module 730 is used to input the initial text corresponding to the speech to be processed, the initial speech unit sequence, and the specified number of target data groups into the target recognition model to recognize the speech to be processed.

[0136] In one embodiment, the first acquisition module 710 is specifically used to: input the speech to be processed into a speech recognition model to obtain the initial text of the speech to be processed; and match the initial text with information in a syllable database to obtain the initial speech unit sequence corresponding to the initial text.

[0137] In one embodiment, the first acquisition module 710 is further configured to: match the text in the initial text with the text in the syllable database to obtain the syllable information corresponding to each text in the initial text, and arrange the syllable information corresponding to each text in the initial text according to the text order to obtain the initial speech unit sequence corresponding to the initial text, wherein the syllable database includes text and its corresponding syllable information, and the syllable information is composed of multiple phoneme information; segment the initial text to obtain initial words, match the words in the initial words with the text in the syllable database to obtain the syllable segment information corresponding to each word in the initial words, and arrange the syllable segment information corresponding to each word in the initial words according to the word order to obtain the initial speech unit sequence corresponding to the initial text.

[0138] In one embodiment, the second acquisition module 720 is specifically configured to: acquire the speech unit sequences of each text information within the dataset; perform similarity matching between the speech unit sequences of the text information within the dataset and the initial speech unit sequence to obtain similarity values ​​for each text information; take the text information corresponding to the similarity values ​​that satisfy the similarity conditions as target text, and take the speech unit sequences corresponding to the target text as target speech unit sequences, wherein multiple target texts that satisfy the similarity conditions and their corresponding target speech unit sequences constitute a specified number of target data groups.

[0139] In one embodiment, the second acquisition module 720 is further configured to include at least one of the following: matching the phoneme information of each phoneme in the initial speech unit sequence with the phoneme information of each speech unit sequence in the dataset according to the order of the same position in the sequence, to obtain a similarity value of the text information corresponding to each speech unit sequence; and matching the syllable segment information of each syllable segment information in the initial speech unit sequence with the syllable segment information of each speech unit sequence in the dataset, to obtain a similarity value of the text information corresponding to each speech unit sequence.

[0140] In one embodiment, the recognition module 730 is specifically used to: concatenate or fuse the initial text corresponding to the speech to be processed, the initial speech unit sequence, and the specified number of target data groups to obtain a data group to be recognized; and input the data group to be recognized into the target recognition model.

[0141] In one embodiment, the recognition module 730 is specifically used to: input the initial text corresponding to the speech to be processed, the speech unit sequence, and the specified number of target data groups into the target recognition model to obtain the text information corresponding to the speech to be processed.

[0142] In one embodiment, the recognition module 730 is specifically used to: input the initial text corresponding to the speech to be processed, the speech unit sequence, and the specified number of target data groups into the target recognition model to obtain the intent information corresponding to the speech to be processed, so as to respond based on the intent information.

[0143] In one possible implementation, it further includes:

[0144] The training module is specifically used for: obtaining text samples and their corresponding speech unit sequence samples, since the target recognition model is a model trained based on text samples and their corresponding speech unit sequence samples; obtaining a specified number of target data group samples corresponding to the speech unit sequence samples in the dataset; combining the text samples, the speech unit sequence samples, and the specified number of target data group samples into training pairs and labeling them with text tags; inputting the training pairs and their text tags into the target recognition model for training to obtain training results; and adjusting the target recognition model according to the training results and the text tags to obtain a mature target recognition model.

[0145] According to embodiments of this disclosure, this disclosure also provides an electronic device and a readable storage medium.

[0146] Figure 8A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0147] like Figure 8 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0148] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0149] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as speech recognition methods. For example, in some embodiments, the speech recognition method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the speech recognition method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform speech recognition methods by any other suitable means (e.g., by means of firmware).

[0150] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0151] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0152] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0153] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0154] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0155] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0156] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0157] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.

[0158] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A speech recognition method, the method comprising: Obtain the initial text and initial speech unit sequence corresponding to the speech to be processed in the target scene; Obtain a specified number of target data sets from the dataset, wherein the target data sets include target text and target speech unit sequences, the initial speech unit sequence and the target speech unit sequence satisfy similarity conditions, and the data information included in the dataset is related to the target scene; The initial text corresponding to the speech to be processed, the initial speech unit sequence, and the specified number of target data groups are input into the target recognition model to recognize the speech to be processed. The step of inputting the initial text corresponding to the speech to be processed, the initial speech unit sequence, and the specified number of target data groups into the target recognition model includes: The initial text corresponding to the speech to be processed, the initial speech unit sequence, and the specified number of target data groups are concatenated or fused to obtain the data group to be recognized. The data set to be identified is input into the target recognition model.

2. The method according to claim 1, wherein obtaining the initial text and initial speech unit sequence corresponding to the speech to be processed in the target scene includes: The speech to be processed is input into the speech recognition model to obtain the initial text of the speech to be processed; The initial text is matched with information in the syllable database to obtain the initial speech unit sequence corresponding to the initial text.

3. The method according to claim 2, wherein matching the initial text with information in the syllable database to obtain the initial speech unit sequence corresponding to the initial text comprises at least one of the following: The characters in the initial text are matched with the characters in the syllable database to obtain the syllable information corresponding to each character in the initial text. The syllable information corresponding to each character in the initial text is then arranged according to the order of the characters in the initial text to obtain the initial speech unit sequence corresponding to the initial text. The syllable database includes text and its corresponding syllable information, and the syllable information is composed of multiple phoneme information; The initial text is segmented into words to obtain initial word segments. The word segments in the initial word segments are matched with the text in the syllable database to obtain the syllable segment information corresponding to each word segment in the initial word segments. The syllable segment information corresponding to each word segment in the initial word segments is arranged according to the word segmentation order in the initial word segments to obtain the initial speech unit sequence corresponding to the initial text.

4. The method according to claim 1, wherein obtaining a specified number of target data groups from the dataset comprises: Obtain the speech unit sequence of each text information in the dataset; The similarity values ​​of each text information are obtained by performing similarity matching between the speech unit sequence of the text information in the dataset and the initial speech unit sequence. The text information corresponding to the similarity value that meets the similarity condition is taken as the target text, and the speech unit sequence corresponding to the target text is taken as the target speech unit sequence. Among them, multiple target texts that meet the similarity condition and their corresponding target speech unit sequences constitute a specified number of target data groups.

5. The method according to claim 4, wherein performing similarity matching between the speech unit sequence of text information in the dataset and the initial speech unit sequence to obtain a similarity value for each piece of text information includes at least one of the following: According to the order of the same position in the sequence, the phoneme information in the initial speech unit sequence is matched with the phoneme information of each speech unit sequence in the dataset to obtain the similarity value of the text information corresponding to each speech unit sequence. The syllable segment information within the initial speech unit sequence is matched with the syllable segment information of each speech unit sequence in the dataset to obtain the similarity value of the text information corresponding to each speech unit sequence.

6. The method according to claim 1, wherein inputting the initial text corresponding to the speech to be processed, the initial speech unit sequence, and the specified number of target data groups into the target recognition model to recognize the speech to be processed comprises: The initial text corresponding to the speech to be processed, the initial speech unit sequence, and the specified number of target data groups are input into the target recognition model to obtain the text information corresponding to the speech to be processed.

7. The method according to claim 1, wherein inputting the initial text corresponding to the speech to be processed, the initial speech unit sequence, and the specified number of target data groups into the target recognition model to recognize the speech to be processed comprises: The initial text corresponding to the speech to be processed, the initial speech unit sequence, and the specified number of target data groups are input into the target recognition model to obtain the intent information corresponding to the speech to be processed, so as to respond based on the intent information.

8. The method according to claim 1, wherein the target recognition model is a model trained based on text samples and their corresponding speech unit sequence samples, and the training process includes: Obtain text samples and their corresponding speech unit sequence samples; In the dataset, obtain a specified number of target data group samples corresponding to the speech unit sequence samples; The text samples, the speech unit sequence samples, and the specified number of target data group samples are combined into training pairs, and text labels are assigned to them. The training pair and its text labels are input into the target recognition model for training to obtain the training results. The target recognition model is then adjusted based on the training results and the text labels to obtain a mature target recognition model.

9. A voice processing apparatus, the apparatus comprising: The first acquisition module is used to acquire the initial text and initial speech unit sequence corresponding to the speech to be processed in the target scene; The second acquisition module is used to acquire a specified number of target data groups from the dataset, wherein the target data groups include target text and target speech unit sequences, the initial speech unit sequence and the target speech unit sequence satisfy similarity conditions, and the data information included in the dataset is related to the target scene; The recognition module is used to input the initial text corresponding to the speech to be processed, the initial speech unit sequence, and the specified number of target data groups into the target recognition model to recognize the speech to be processed. The recognition module is specifically used to: concatenate or fuse the initial text corresponding to the speech to be processed, the initial speech unit sequence, and the specified number of target data groups to obtain a data group to be recognized; and input the data group to be recognized into the target recognition model.

Images