Model processing method and device, speech recognition method and device

Abstract

The embodiments of this specification provide a model processing method and device, and a speech recognition method and device. A specific implementation of the model processing method includes: obtaining at least one training sample, the training sample includes speech information of words of the target category, and annotated text, the annotated text is used to represent the semantics of the speech information, and is added with a slot mark corresponding to the target category, and the slot mark is added to the original appearance position of the word of the target category in the annotated text; according to the at least one training sample, the end-to-end model to be trained is trained to obtain a target end-to-end model for speech recognition and with a slot prediction function.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of computer technology, and specifically, to a model processing method and device, a speech recognition method and device, and an interactive device. Background Art

[0002] Existing speech recognition technology solutions are usually based on the theory of deep learning and adopt end-to-end modeling solutions. Although end-to-end speech recognition works well in general scenarios, it is not very effective in scenarios such as names of people and places.

[0003] Therefore, a reasonable and reliable solution is needed to improve the recognition effect when talking about similar scenarios such as names of people and places. Summary of the invention

[0004] The embodiments of this specification provide a model processing method and device, a speech recognition method and device, and an interactive device.

[0005] In a first aspect, an embodiment of the present specification provides a model processing method, comprising: obtaining at least one training sample, the training sample comprising speech information of words of a target category, and annotated text, the annotated text being used to characterize the semantics of the speech information and being added with slot marks corresponding to the target category, the slot marks being added to the original occurrence positions of the words of the target category in the annotated text; training an end-to-end model to be trained according to the at least one training sample to obtain a target end-to-end model for speech recognition and with a slot prediction function.

[0006] In some embodiments, the target category includes at least one of the following categories: person name, place name, organization name.

[0007] In some embodiments, when the target category includes personal names, the personal name slot mark added in the annotation text is used to indicate that a personal name should appear at the position it occupies; when the target category includes place names, the place name slot mark added in the annotation text is used to indicate that a place name should appear at the position it occupies; when the target category includes institution names, the institution name slot mark added in the annotation text is used to indicate that an institution name should appear at the position it occupies.

[0008] In some embodiments, the dictionary used by the end-to-end model to be trained is supplemented with slot labels corresponding to the target category.

[0009] In some embodiments, the end-to-end model to be trained includes a natural language processing model based on a self-attention mechanism and adopting an encoder-decoder architecture.

[0010] In some embodiments, the annotated text is text in the form of a word sequence; and the training of the end-to-end model to be trained based on the at least one training sample includes: taking the speech information respectively included in the at least one training sample as input, and taking the annotated text corresponding to the speech information as a label, and training the end-to-end model to be trained.

[0011] In some embodiments, the annotated text is text that has not been segmented; and the training of the end-to-end model to be trained based on the at least one training sample includes: for each training sample in the at least one training sample, segmenting the annotated text included in the training sample, and forming a word sequence of words obtained by the segmentation process; taking the speech information respectively included in the at least one training sample as input, and taking the word sequence corresponding to the speech information as a label, to train the end-to-end model to be trained.

[0012] In a second aspect, an embodiment of the present specification provides a speech recognition method applied to an optimization processor in a speech recognition system, wherein the speech recognition system also includes a target end-to-end model for speech recognition and with a slot prediction function, the method including: obtaining a prediction result output by the target end-to-end model, the prediction result including multiple text messages; in response to reading a slot mark corresponding to a target category at the same position of the multiple text messages, extracting from the multiple text messages respectively words that are adjacent to the slot mark position and appear after the slot mark; determining a first score corresponding to the extracted word based on the slot mark; and determining a target word from the extracted word based on the determined first score, the target word serving as the recognition result corresponding to the position occupied by the slot mark.

[0013] In some embodiments, the target end-to-end model includes a target end-to-end model for speech recognition and with a slot prediction function that is trained using the implementation method of the first aspect.

[0014] In some embodiments, determining the first score corresponding to the extracted word according to the slot mark includes: using a scoring model corresponding to the slot mark to determine the first score corresponding to the extracted word.

[0015] In some embodiments, the scoring model includes a pre-established data mapping table for characterizing the correspondence between words under the target category and the first score; and the scoring model corresponding to the slot mark is used to determine the first score corresponding to the extracted word, including: searching the scoring model for a record including the extracted word, and determining the first score in the found record as the first score corresponding to the extracted word.

[0016] In some embodiments, the scoring model includes a pre-trained prediction model for predicting a first score corresponding to a word under a target category; and the scoring model corresponding to the slot mark is used to determine the first score corresponding to the extracted word, including: inputting the extracted word into the scoring model to obtain the first score output by the scoring model.

[0017] In some embodiments, the prediction result also includes second scores corresponding to the multiple text information respectively; and determining the target word from the extracted words based on the determined first score includes: for each extracted word, determining the screening score corresponding to the word based on the first score corresponding to the word and the second score corresponding to the text information where the word is located; determining the target word from the extracted words based on the determined screening score.

[0018] In some embodiments, the scoring model corresponds to a preset score adjustment coefficient; and determining the screening score corresponding to the word based on the first score corresponding to the word and the second score corresponding to the text information where the word is located includes: determining the product of the first score corresponding to the word and the score adjustment coefficient; determining the sum of the product and the second score corresponding to the text information where the word is located as the screening score corresponding to the word.

[0019] On the third aspect, an embodiment of the present specification provides a model processing method, comprising: obtaining at least one training sample, the training sample comprising speech information including a person's name, and annotated text, the annotated text being used to characterize the semantics of the speech information and adding a name slot mark, the name slot mark being added to the original appearance position of the word belonging to the person's name in the annotated text; according to the at least one training sample, training an end-to-end model to be trained to obtain a target end-to-end model for speech recognition and with a name slot prediction function.

[0020] In a fourth aspect, an embodiment of the present specification provides a speech recognition method, which is applied to an optimization processor in a speech recognition system, wherein the speech recognition system also includes a target end-to-end model for speech recognition and with a name slot prediction function, the method including: obtaining a prediction result output by the target end-to-end model, the prediction result including multiple text messages; in response to reading a name slot mark at the same position of the multiple text messages, extracting from the multiple text messages respectively words that are adjacent to the name slot mark and appear after the name slot mark; determining a first score corresponding to the extracted word based on the name slot mark; determining a target word from the extracted word based on the determined first score, the target word serving as the recognition result corresponding to the position occupied by the name slot mark.

[0021] In some embodiments, the target end-to-end model includes a target end-to-end model for speech recognition and with a name slot prediction function obtained by training using the implementation method in the third aspect.

[0022] In some embodiments, determining the first score corresponding to the extracted word based on the name slot mark includes: using a scoring model corresponding to the name slot mark to determine the first score corresponding to the extracted word.

[0023] In a fifth aspect, an embodiment of the present specification provides a model processing method, comprising: obtaining at least one training sample, the training sample comprising text information of words of a target category, and slot mark information, the slot mark information showing the occurrence position of the words of the target category and the slot mark corresponding to the target category; training an end-to-end model to be trained according to the at least one training sample to obtain a target end-to-end model for slot prediction.

[0024] In a sixth aspect, an embodiment of the present specification provides a speech recognition method, which is applied to an optimization processor in a speech recognition system, wherein the speech recognition system also includes a speech recognition model and a target end-to-end model for slot prediction, wherein the method includes: obtaining a first prediction result output by the speech recognition model, wherein the first prediction result includes multiple text messages; obtaining a second prediction result output by the target end-to-end model, wherein the second prediction result is obtained by performing slot prediction on the first prediction result; in response to the second prediction result, showing the appearance position of a word of a target category and a slot mark corresponding to the target category, and reading a word located at the appearance position in the multiple text messages, determining a first score corresponding to the read word located at the appearance position according to the slot mark; determining a target word from the read word located at the appearance position according to the determined first score, wherein the target word serves as the recognition result corresponding to the appearance position.

[0025] In some embodiments, the target end-to-end model includes a target end-to-end model for slot prediction trained using the implementation method in the fifth aspect.

[0026] In some embodiments, determining the first score corresponding to the word read at the occurrence position based on the slot mark includes: using a scoring model corresponding to the slot mark to determine the first score corresponding to the word read at the occurrence position.

[0027] In the seventh aspect, an embodiment of the present specification provides a model processing device, comprising: an acquisition unit, configured to acquire at least one training sample, the training sample comprising speech information of a word of a target category, and an annotated text, the annotated text being used to characterize the semantics of the speech information and being added with a slot mark corresponding to the target category, the slot mark being added to the original appearance position of the word of the target category in the annotated text; a model training unit, configured to train an end-to-end model to be trained according to the at least one training sample, to obtain a target end-to-end model for speech recognition with a slot prediction function.

[0028] In an eighth aspect, an embodiment of the present specification provides a speech recognition device, which is applied to an optimization processor in a speech recognition system, wherein the speech recognition system also includes a target end-to-end model for speech recognition and with a slot prediction function, and the device includes: an acquisition unit, configured to acquire a prediction result output by the target end-to-end model, wherein the prediction result includes multiple text messages; an extraction unit, configured to extract, in response to reading a slot mark corresponding to a target category at the same position of the multiple text messages, words that are adjacent to the slot mark position and appear after the slot mark from the multiple text messages; a score determination unit, configured to determine a first score corresponding to the extracted word based on the slot mark; and a recognition result determination unit, configured to determine a target word from the extracted word based on the determined first score, wherein the target word serves as the recognition result corresponding to the position occupied by the slot mark.

[0029] In some embodiments, the target end-to-end model includes a target end-to-end model for speech recognition and with a slot prediction function that is trained using the implementation method of the first aspect.

[0030] In some embodiments, the score determination unit is further configured to: determine a first score corresponding to the extracted word using a scoring model corresponding to the slot mark.

[0031] In the ninth aspect, an embodiment of the present specification provides a model processing device, comprising: an acquisition unit, configured to acquire at least one training sample, the training sample comprising speech information including a person's name, and annotated text, the annotated text being used to characterize the semantics of the speech information and adding a name slot mark, the name slot mark being added to the original appearance position of the word belonging to the person's name in the annotated text; a model training unit, configured to train an end-to-end model to be trained according to the at least one training sample, to obtain a target end-to-end model for speech recognition and with a name slot prediction function.

[0032] In the tenth aspect, an embodiment of the present specification provides a speech recognition device, which is applied to an optimization processor in a speech recognition system, wherein the speech recognition system also includes a target end-to-end model for speech recognition and with a name slot prediction function, and the device includes: an acquisition unit, configured to acquire a prediction result output by the target end-to-end model, wherein the prediction result includes multiple text messages; an extraction unit, configured to extract, in response to reading a name slot mark at the same position of the multiple text messages, words that are adjacent to the name slot mark and appear after the name slot mark from the multiple text messages; a score determination unit, configured to determine a first score corresponding to the extracted word according to the name slot mark; and a recognition result determination unit, configured to determine a target word from the extracted word according to the determined first score, wherein the target word serves as a recognition result corresponding to the position occupied by the name slot mark.

[0033] In some embodiments, the target end-to-end model includes a target end-to-end model for speech recognition and with a name slot prediction function obtained by training using the implementation method in the third aspect.

[0034] In some embodiments, the score determination unit is further configured to: determine a first score corresponding to the extracted word using a scoring model corresponding to the name slot mark.

[0035] On the eleventh aspect, an embodiment of the present specification provides a model processing device, comprising: an acquisition unit, configured to acquire at least one training sample, the training sample comprising text information of words of a target category, and slot mark information, the slot mark information showing the occurrence position of the words of the target category and the slot mark corresponding to the target category; a model training unit, configured to train an end-to-end model to be trained according to the at least one training sample to obtain a target end-to-end model for slot prediction.

[0036] In the twelfth aspect, an embodiment of the present specification provides a speech recognition device, which is applied to an optimization processor in a speech recognition system, wherein the speech recognition system also includes a speech recognition model and a target end-to-end model for slot prediction, and the device includes: a first acquisition unit, configured to acquire a first prediction result output by the speech recognition model, wherein the first prediction result includes multiple text messages; a second acquisition unit, configured to acquire a second prediction result output by the target end-to-end model, wherein the second prediction result is obtained by performing slot prediction on the first prediction result; a score determination unit, configured to display the appearance position of a word of a target category and a slot mark corresponding to the target category in response to the second prediction result, and to read a word located at the appearance position in the multiple text messages, and determine a first score corresponding to the read word located at the appearance position according to the slot mark; a recognition result determination unit, configured to determine a target word from the read word located at the appearance position according to the determined first score, wherein the target word serves as the recognition result corresponding to the appearance position.

[0037] In some embodiments, the target end-to-end model includes a target end-to-end model for slot prediction trained using the implementation method in the fifth aspect.

[0038] In some embodiments, the score determination unit is further configured to: determine a first score corresponding to the read word located at the occurrence position using a scoring model corresponding to the slot mark.

[0039] In the thirteenth aspect, an embodiment of the present specification provides an interactive device, including an optimization processor; the optimization processor is configured to: obtain a prediction result output by a target end-to-end model, the target end-to-end model is used for speech recognition and has a slot prediction function, and the prediction result includes multiple text messages; in response to reading a slot mark corresponding to a target category at the same position of the multiple text messages, extracting from the multiple text messages respectively words that are adjacent to the slot mark position and appear after the slot mark; according to the slot mark, determining a first score corresponding to the extracted word; according to the determined first score, determining a target word from the extracted words, the target word serving as the recognition result corresponding to the position occupied by the slot mark.

[0040] In a fourteenth aspect, an embodiment of the present specification provides an interactive device, comprising an optimization processor; the optimization processor is configured to: obtain a first prediction result output by a speech recognition model, the first prediction result comprising multiple text messages; obtain a second prediction result output by a target end-to-end model, the target end-to-end model being used for slot prediction, the second prediction result being obtained by performing slot prediction on the first prediction result; in response to the second prediction result, the appearance position of a word of a target category and a slot mark corresponding to the target category are shown, and a word located at the appearance position is read from the multiple text messages, and according to the slot mark, a first score corresponding to the word read at the appearance position is determined; according to the determined first score, a target word is determined from the word read at the appearance position, the target word serving as the recognition result corresponding to the appearance position.

[0041] In a fifteenth aspect, an embodiment of the present specification provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed in a computer, the computer is caused to execute a method as described in any one of the implementations in the first to sixth aspects.

[0042] In the sixteenth aspect, an embodiment of the present specification provides a computing device, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the method described in any implementation method of the first to sixth aspects.

[0043] The model processing method and device provided by the above-mentioned embodiments of this specification obtain at least one training sample including the following contents: speech information including words of the target category, annotated text for characterizing the semantics of the speech information and added with a slot mark corresponding to the target category, and the slot mark is added to the original appearance position of the words of the target category in the annotated text, and then the end-to-end model to be trained is trained according to the at least one training sample, so as to train and obtain a target end-to-end model for speech recognition and with a slot prediction function. In the speech recognition scenario, when the prediction result output by the target end-to-end model is added with a slot mark corresponding to the target category, the optimization processor can know the words of the target category that should be optimized based on the slot mark, so that the optimization processor can accurately identify the words of the target category. Therefore, the recognition accuracy of the words of the target category can be improved, and the false alarm rate can be reduced. Therefore, the speech recognition effect for the speech information including the words of the target category can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the multiple embodiments disclosed in this specification, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only the multiple embodiments disclosed in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 is an exemplary system architecture diagram to which some embodiments of the present specification may be applied;

[0046] Figure 2 is a flow chart of an embodiment of a model processing method according to the present specification;

[0047] Figure 3a It is a schematic diagram for adding slot marks to the text information to be annotated;

[0048] Figure 3b It is another schematic diagram for adding slot marks for the text information to be annotated;

[0049] Figure 4 is a flow chart of another embodiment of the model processing method according to the present specification;

[0050] Figure 5 is a flow chart of an embodiment of a speech recognition method according to the present specification;

[0051] Figure 6 is a flow chart of an embodiment of a method for determining a recognition result corresponding to a position occupied by a slot marker corresponding to a target category;

[0052] Figure 7 is a flow chart of another embodiment of the speech recognition method according to the present specification;

[0053] Figure 8 is another exemplary system architecture diagram to which some embodiments of the present specification may be applied;

[0054] Fig. 9 is a flow chart of another embodiment of the model processing method according to the present specification;

[0055] Fig.10 is a flow chart of another embodiment of the speech recognition method according to the present specification;

[0056] Fig.11 is a structural schematic diagram of a model processing device according to the present specification;

[0057] Fig.12 is a structural schematic diagram of a speech recognition device according to the present specification;

[0058] Fig.13 is another structural schematic diagram of a model processing device according to the present specification;

[0059] Fig.14 is another structural schematic diagram of a speech recognition device according to the present specification;

[0060] Fig.15 is another structural schematic diagram of the model processing device according to the present specification;

[0061] Fig.16 is another structural schematic diagram of the speech recognition device according to the present specification;

[0062] Fig.17 is a schematic diagram of a scenario of an interactive device according to this specification;

[0063] Fig.18 It is another scene schematic diagram of the interactive device according to this specification. DETAILED DESCRIPTION

[0064] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. The described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of this application.

[0065] It should be noted that, for the convenience of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments in this specification can be combined with each other.

[0066] As mentioned earlier, although end-to-end speech recognition works well in general scenarios, it performs poorly when it comes to scenarios such as names of people and places.

[0067] The inventor has found through research that when you want to optimize the recognition effect of a certain type of words, you can integrate the optimization of the recognition effect of this type of words into the training of the end-to-end model, so that you can optimize the recognition effect with the help of the powerful modeling ability of the end-to-end model. Because the model end can directly output information to tell the optimization processor when to optimize this type of words, it can effectively improve the recognition efficiency.

[0068] Specifically, when you want to improve the recognition effect of a certain type of words, you can pre-set the corresponding slot mark for the words in this type. For example, when you want to improve the recognition effect of a person's name, you can pre-set the person's name slot mark. When you want to improve the recognition effect of a place name, you can pre-set the place name slot mark. Here, the certain type of words as described above can be called a target category. In the training data preparation stage, the slot mark corresponding to the target category can be associated with the occurrence position of the words of the target category in the text information serving as the training data. In this way, in the training stage of the end-to-end model, by combining the slot mark corresponding to the target category, the trained end-to-end model can have a slot prediction function, that is, it can predict the occurrence position of the words of the target category. Subsequently, the optimization processor can accurately identify the words of the target category based on the slot mark and the position corresponding to the slot mark shown in the prediction result output by the end-to-end model.

[0069] In addition, the inventors found that through the following two model processing schemes, a target end-to-end model with slot prediction function can be trained, and the prediction results output by the target end-to-end model can enable the optimization processor to accurately identify words of the target category.

[0070] Model processing solution 1: Integrate slot mark prediction into the training phase of the end-to-end model for speech recognition, so that when the trained target end-to-end model receives speech information including words of the target category, it outputs text information that is used to represent the semantics of the speech information and is added with slot marks corresponding to the target category. The slot mark is added at the predicted occurrence position of the word of the target category. In addition, in the speech recognition scenario, the prediction result output by the target end-to-end model is used as the input of the optimization processor.

[0071] Model processing solution 2: Train an end-to-end model specifically for slot prediction, so that the trained target end-to-end model receives the prediction results output by the speech recognition model, performs slot prediction based on the prediction results, and outputs the corresponding slot prediction results. In the speech recognition scenario, the prediction results output by the speech recognition model and the target end-to-end model are used as inputs to the optimization processor.

[0072] Below, we will first introduce in detail the content related to model processing solution 1.

[0073] Some embodiments of this specification disclose a model processing method and a speech recognition method related to the model processing solution 1. Specifically, Figure 1 An exemplary system architecture diagram suitable for use with these embodiments is shown.

[0074] like Figure 1As shown, the system architecture includes a model training system and a speech recognition system. The model training system is used to train the end-to-end model to be trained based on the training samples corresponding to the speech information including the words of the target category, so as to train and obtain the target end-to-end model for speech recognition and with the slot prediction function. It should be pointed out that on the basis of using the training samples corresponding to the speech information including the words of the target category, the training samples corresponding to the speech information not including the words of the target category can also be used.

[0075] The target category may include a category related to a name. Further, the target category may include at least one of the following categories: a person's name, a place name, an organization name, an animal name, an audio name, a video name, etc. The audio name may include, but is not limited to, a song name. The video name may include, but is not limited to, the name of at least one of the following videos: a live video, a TV series, a movie, a variety show, etc.

[0076] In practice, the target end-to-end model is a component of the speech recognition system. In addition, the speech recognition system also includes an optimization processor. In the speech recognition scenario, the speech information to be recognized can be input into the target end-to-end model. The target end-to-end model can perform speech recognition on the speech information and output the prediction result obtained by the speech recognition to the optimization processor. The optimization processor can optimize the prediction result and output the speech recognition result obtained after the optimization process.

[0077] It should be noted that when the speech information to be recognized is not speech information including words of the target category, the prediction result output by the target end-to-end model is similar to the prediction result output by the existing end-to-end model for speech recognition, and the optimization processor can use conventional processing methods to optimize the prediction result.

[0078] When the speech information to be recognized includes words of the target category, the prediction result output by the target end-to-end model is different from the prediction result output by the existing end-to-end model for speech recognition. The text information in the prediction result output by the target end-to-end model is added with slot tags that can help the optimization processor accurately recognize the words of the target category.

[0079] The specific implementation steps of the above method are described below in conjunction with specific embodiments.

[0080] See also Figure 2 , which shows a process 200 of an embodiment of a model processing method. The execution subject of the method can be Figure 1 The model training system shown. The method comprises the following steps:

[0081] Step 201, obtaining at least one training sample, the training sample including speech information of a word of a target category and an annotated text, the annotated text is used to represent the semantics of the speech information and is added with a slot mark corresponding to the target category, the slot mark is added to the original occurrence position of the word of the target category in the annotated text;

[0082] Step 202: Train the end-to-end model to be trained based on at least one training sample to obtain a target end-to-end model for speech recognition with a slot prediction function.

[0083] Next, steps 201-202 are described in detail.

[0084] In step 201, the speech information respectively included in the at least one training sample may be spectrum information, and the spectrum information may be extracted using a feature extraction method such as Mel-Frequency Cepstral Coefficients (MFCC) or Linear Predictive Cepstral Coefficients (LPCC).

[0085] The at least one training sample may be pre-generated, and the at least one training sample may be stored in a specific storage location, such as a location locally on the execution subject or on another server that is communicatively connected to the execution subject. The execution subject may obtain the at least one training sample from the storage location.

[0086] In practice, the slot marks in the annotated text respectively included in the at least one training sample may be manually annotated or automatically annotated using a specific tool (e.g., a tool for named entity recognition). For example, when you want to identify a certain named entity (e.g., a person's name, a place name, or an institution name, etc.), you can set the category of the named entity and the slot mark corresponding to the category in the tool. The tool can then identify the words belonging to the category from the text to be annotated based on the category, and add the slot mark corresponding to the category in front of the word, so that the slot mark occupies the original position of the word in the text.

[0087] For example, the target category is a person's name, and the name slot is marked as "CLASS_PERSON". Figure 3a As shown in FIG. 1 , it shows a schematic diagram of adding slot marks to the text information to be marked. Figure 3a In the figure, the text information to be annotated is "My name is Xiao Ming". Among them, "Xiao Ming" is a person's name. You can use manual annotation or automatic annotation to add a name slot mark "CLASS_PERSON" in front of "Xiao Ming" (such as on the left). Figure 3a As shown, the annotation text after adding the name slot mark can be "My name is CLASS_PERSON Xiao Ming". Among them, "CLASS_PERSON" occupies the original appearance position of the name "Xiao Ming". "CLASS_PERSON" can be used to indicate that a name should appear at the position it occupies.

[0088] Let's take the target category as a place name and the place name slot as "CLASS_PLACE" as an example. Figure 3b As shown, it shows another schematic diagram of adding slot marks to the text information to be marked. Figure 3b In the figure, the text information to be annotated is "I live in Beijing". Among them, "Beijing" is a place name. You can use manual annotation or automatic annotation to add a place name slot mark "CLASS_PLACE" in front of "Beijing" (such as on the left). Figure 3b As shown in the figure, after adding the place name slot marker, the annotation text can be "I live in CLASS_PLACE Beijing". Among them, "CLASS_PLACE" occupies the original position of the place name "Beijing". "CLASS_PLACE" can be used to indicate that the place name should appear at the position it occupies.

[0089] It should be understood that when the target category includes a person's name, the name slot mark added in the annotation text can be used to indicate that a person's name should appear at the position it occupies. When the target category includes a place name, the place name slot mark added in the annotation text can be used to indicate that a place name should appear at the position it occupies. When the target category includes an institution name, the institution name slot mark added in the annotation text can be used to indicate that an institution name should appear at the position it occupies. When the target category includes other categories, the content represented by the slot mark corresponding to the other category added in the annotation text can be obtained by analogy based on the content described above, and will not be described in detail here.

[0090] Optionally, the at least one training sample mentioned above may be a part of samples in a pre-collected training sample set. The training sample set may also include training samples corresponding to speech information that does not include words of the target category. The training sample includes speech information that does not include words of the target category, and text information used to characterize the semantics of the speech information. It should be understood that the training sample set can be used for training an end-to-end model to be trained. On the basis of training the end-to-end model to be trained according to the training samples corresponding to the speech information that includes words of the target category, the end-to-end model to be trained can also be trained according to the training samples corresponding to the speech information that does not include words of the target category.

[0091] In step 202, the execution entity may train the end-to-end model to be trained according to the at least one training sample to obtain a target end-to-end model for speech recognition with a slot prediction function.

[0092] The end-to-end model to be trained may be an untrained or uncompleted model. In addition, the end-to-end model to be trained may be any type of model suitable for speech recognition and using an end-to-end architecture. Furthermore, the end-to-end model to be trained may include but is not limited to a Transformer-based model, etc. A Transformer-based model may be referred to as a Transformer model. The Transformer model is a natural language processing model based on a self-attention mechanism and using an encoder-decoder architecture. The Transformer model typically processes all words or symbols in a sequence in parallel, while using a self-attention mechanism to combine context with more distant words. By processing all words in parallel and allowing each word to notice other words in the sentence in multiple processing steps. In addition, the training speed of the Transformer model is usually faster, and its processing effect is also better.

[0093] It should be noted that the dictionary used by the end-to-end model to be trained has slot labels corresponding to the target category added to it, so that the end-to-end model to be trained can learn slot prediction.

[0094] In practice, the annotated text included in the at least one training sample may be a text in the form of a word sequence or a text without word segmentation.

[0095] When the annotated text respectively included in the at least one training sample is a text in the form of a word sequence, the voice information respectively included in the at least one training sample can be used as input, and the annotated text corresponding to the voice information can be used as a label to train the end-to-end model to be trained.

[0096] When the annotated text included in the at least one training sample is a text that has not been segmented, for each training sample in the at least one training sample, the annotated text included in the training sample can be segmented, and the words obtained after the segmentation can be combined into a word sequence. Then, the voice information included in the at least one training sample can be used as input, and the word sequence corresponding to the voice information can be used as a label to train the end-to-end model to be trained. In practice, the existing text segmentation technology can be used to segment the annotated text, and the specific segmentation method will not be described in detail here.

[0097] The model processing method provided in this embodiment obtains at least one training sample as described above, and trains the end-to-end model to be trained according to the at least one training sample, so as to obtain a target end-to-end model for speech recognition and with a slot prediction function. In this way, the prediction results output by the target end-to-end model can help the optimization processor to accurately identify the words of the target category, improve the recognition accuracy of the words of the target category, and reduce the false alarm rate. Therefore, the speech recognition effect for speech information including words of the target category can be improved.

[0098] Next, we introduce Figure 2 The solution provided by the corresponding embodiment is applied in the scenario of mentioning a person's name.

[0099] In the scenario of speaking a person's name, the target category may be referred to as a person's name. Before model training, a name slot mark may be set. In addition, at least one voice information of speaking a person's name (such as spectral information obtained through preprocessing) may be collected in advance, and corresponding text information may be generated for the at least one voice information collected. The text information is used to characterize the semantics of the corresponding voice information. Then, a manual or automatic annotation method may be used to add a name slot mark in front of the words belonging to the person's name in the generated text information. It should be understood that the name slot mark is located at the original position of the word in the text information. Here, the text information after adding the name slot mark may be referred to as annotated text.

[0100] Afterwards, for each of the at least one voice message collected, the voice message and the annotated text may be combined into a training sample. In addition, the combined training samples may be stored in a specific storage location for use in model training.

[0101] like Figure 4 As shown, it shows a process 400 of another embodiment of the model processing method. The execution subject of the training method can be as follows Figure 1 The model training system shown. The training method includes the following steps:

[0102] Step 401, obtaining at least one training sample, the training sample including speech information including a person's name, and annotated text, the annotated text is used to represent the semantics of the speech information, and a name slot mark is added, and the name slot mark is added to the original occurrence position of the word belonging to the person's name in the annotated text;

[0103] Step 402: Train the end-to-end model to be trained based on at least one training sample to obtain a target end-to-end model for speech recognition with a name slot prediction function.

[0104] In the scenario of speaking a person's name, the target end-to-end model trained by steps 401-402 can be used for speech recognition and has a name slot prediction function. The prediction results output by the target end-to-end model can help the optimization processor to accurately recognize the name, improve the recognition accuracy of the name, and reduce the false alarm rate. Therefore, the speech recognition effect for voice information including the name can be improved.

[0105] It should be noted that, based on the application of the model processing method in the scenario of talking about names, those skilled in the art can infer the application scheme of the model processing method in the scenario of talking about other categories of words by analogy, and examples are not given one by one here.

[0106] In adopting Figure 2 After the solution provided in the corresponding embodiment is trained to obtain the target end-to-end model for speech recognition with slot prediction function, the model can be applied to the speech recognition system so that the prediction results output by the model can be used as the input of the optimization processor in the speech recognition system.

[0107] In practice, the prediction results output by the target end-to-end model usually include multiple text messages, and the multiple text messages have the same length. The optimization processor needs to read the words located at the same position from the multiple text messages in a specified reading order (for example, from left to right, or from right to left, etc.), and determine the words that serve as the recognition results from the read words. It should be noted that for the words in the prediction results that are not related to the slot mark, conventional processing methods can be used for optimization processing. For words related to the slot mark, the following can be used Figure 5 The process shown is optimized.

[0108] Figure 5 FIG. 5 shows a flow chart 500 of an embodiment of a speech recognition method. The execution subject of the speech recognition method may be as follows: Figure 1 The optimization processor in the speech recognition system shown in the figure. The speech recognition method specifically shows the optimization processing process of words related to slot marks, and the speech recognition method includes the following steps:

[0109] Step 501, obtaining the prediction result output by the target end-to-end model, where the prediction result includes multiple text information;

[0110] Step 502, in response to reading a slot mark corresponding to the target category at the same position of the plurality of text messages, extracting words that are adjacent to the slot mark and appear after the slot mark from the plurality of text messages respectively;

[0111] Step 503, determining a first score corresponding to the extracted word according to the slot mark;

[0112] Step 504: determine a target word from the extracted words according to the determined first score, the target word serving as the recognition result corresponding to the position occupied by the slot mark.

[0113] In this embodiment, in order to distinguish the score corresponding to the word from the score corresponding to the text information mentioned below, the score corresponding to the word is referred to as the first score, and the score corresponding to the text information is referred to as the second score. It should be noted that the first score and the second score can both be values ​​in the interval [0, 1].

[0114] In practice, each target category may be provided with a corresponding scoring model. The slot mark corresponding to the target category may include an identifier of the scoring model. The scoring model may be used to characterize the correspondence between the words under the corresponding target category and the first score. Specifically, the scoring model may include a pre-established data mapping table for characterizing the correspondence between the words under the target category and the first score. Alternatively, the scoring model may include a pre-trained prediction model for predicting the first score corresponding to the words under the target category.

[0115] Next, steps 501-504 are described in detail.

[0116] In step 501, the target end-to-end model may include using Figure 2 The target end-to-end model for speech recognition and with slot prediction function obtained by training according to the method described in the corresponding embodiment. If the prediction result is obtained by performing speech recognition on speech information including words of the target category, the multiple text messages in the prediction result can be respectively added with slot marks corresponding to the target category, and the position occupied by the slot mark is the predicted appearance position of the word of the target category. If the prediction result is obtained by performing speech recognition on speech information not including words of the target category, the multiple text messages in the prediction result are not added with slot marks corresponding to the target category.

[0117] In step 502, in the process of optimizing the prediction results, the execution subject can extract the words that are adjacent to the slot mark position and appear after the slot mark from the multiple text messages in response to reading the slot mark corresponding to the target category at the same position of the multiple text messages in the prediction results. When the reading order is from left to right, the words that appear after the slot mark refer to the words located on the right side of the slot mark. When the reading order is from right to left, the words that appear after the slot mark refer to the words located on the left side of the slot mark.

[0118] As an example, suppose the prediction result shows the following two pieces of text information: "My name is CLASS_PERSON Xiao Ming", "My name is CLASS_PERSON nickname", and the above execution subject adopts a reading order from left to right. After "My name is" is identified from the two pieces of text information, the name slot mark "CLASS_PERSON" will be read. After reading the name slot mark, it can be known that the word belonging to the name should appear at the position occupied by the name slot mark, and it can be determined that the words adjacent to the position of the name slot mark and located on the right side of the name slot mark, namely "Xiao Ming" and "Xiao Ming", are the candidate names corresponding to the position occupied by the name slot mark. Therefore, "Xiao Ming" and "Xiao Ming" can be extracted from the two pieces of information respectively.

[0119] In step 503, the scoring model corresponding to the slot mark can be used to determine the first scores corresponding to the extracted words, so as to provide score incentives for the extracted words. In this way, the extracted words can be optimized according to the scores.

[0120] As an implementation, when the scoring model is the data mapping table as described above, for each extracted word, a record including the word can be searched in the scoring model, and the first score in the found record is determined as the first score corresponding to the word. Alternatively, if no record including the word is found, the number 0 can be determined as the first score corresponding to the word.

[0121] As another implementation manner, when the scoring model is the prediction model as described above, each extracted word can be input into the scoring model to obtain a first score output by the scoring model.

[0122] In step 504, the execution entity may determine, from the extracted words, a target word that serves as a recognition result corresponding to the position occupied by the slot mark according to the determined first score.

[0123] As an implementation method, the word with the highest first score among the extracted words can be determined as the target word. Continuing with the above two text messages as an example, assuming that the first score corresponding to "Xiao Ming" is 0.9 and the first score corresponding to "nickname" is 0.2, "Xiao Ming" can be determined as the recognition result corresponding to the position occupied by the name slot label "CLASS_PERSON". At this time, the final speech recognition results corresponding to the above two text messages can include "My name is Xiao Ming".

[0124] As another implementation, the prediction result in step 501 also includes second scores corresponding to the multiple text messages. The second score corresponding to each text message can be calculated by the target end-to-end model based on the probability distribution of each character in the text message. For each extracted word, the screening score corresponding to the word can be determined based on the first score corresponding to the word and the second score corresponding to the text message where the word is located. For example, the sum of the first score corresponding to the word and the second score corresponding to the text message where the word is located can be determined as the screening score corresponding to the word. Then, based on the determined screening scores, the target word can be determined from the extracted words, for example, the word corresponding to the highest screening score can be determined as the target word.

[0125] Optionally, the scoring model may be pre-set with a score adjustment coefficient. The score adjustment coefficient is used to adjust the first score corresponding to the extracted word when the extracted word is score-incentivized. The score adjustment coefficient may be a value in the interval [0, 1]. In practice, a word set corresponding to the target category is collected in advance, and the target category is associated with speech recognition statistical information corresponding to the word set. The statistical information may include, for example, but is not limited to, word correctness, recognition accuracy for words in the target category, and the like. The score adjustment coefficient may be determined based on the statistical information.

[0126] In the case where the scoring model is pre-set with a score adjustment coefficient, step 504 can be performed by Figure 6 The process shown is implemented. Figure 6 The flow chart of an embodiment of a method for determining a recognition result corresponding to a position occupied by a slot mark corresponding to a target category is shown. The determination method comprises the following steps:

[0127] Step 5041, for each extracted word, determine the product of the first score corresponding to the word and the adjustment coefficient of the scoring model corresponding to the slot mark, and determine the sum of the product and the second score corresponding to the text information where the word is located as the screening score corresponding to the word;

[0128] Step 5042, based on the determined screening score, determine the target word from the extracted words, and the target word serves as the recognition result corresponding to the position occupied by the slot mark corresponding to the target category.

[0129] In step 5042, the word corresponding to the highest screening score may be determined as the target word.

[0130] Figure 5The speech recognition method provided by the corresponding embodiment, in the process of optimizing the prediction result, in response to reading the slot mark corresponding to the target category at the same position of multiple text information in the prediction result, respectively extract the words that are adjacent to the slot mark position and appear after the slot mark from the multiple text information, and then determine the first score corresponding to the extracted words according to the slot mark (for example, using the scoring model corresponding to the slot mark), so as to determine the target word that acts as the recognition result corresponding to the position occupied by the slot mark from the extracted words according to the determined first score. In this way, the word corresponding to the position occupied by the slot mark can be accurately identified based on the slot mark corresponding to the target category and the scoring model corresponding to the slot mark. Therefore, the recognition accuracy of the words for the target category can be improved, the false alarm rate can be reduced, and therefore, the speech recognition effect for the speech information including the words of the target category can be improved.

[0131] Next, we introduce Figure 5 The solution provided by the corresponding embodiment is applied in the scenario of mentioning a person's name.

[0132] In the scene where people's names are mentioned, Figure 4 After the solution provided in the corresponding embodiment is trained to obtain the target end-to-end model for speech recognition with the name slot prediction function, the model can be applied to the speech recognition system so that the prediction results output by the model can be used as the input of the optimization processor in the speech recognition system.

[0133] In a speech recognition scenario, after a piece of speech information to be recognized that includes a person's name is input into the target end-to-end model, the target end-to-end model can output a corresponding prediction result after analyzing and processing the speech information. The prediction result may include multiple text information that is used to characterize the semantics of the speech information and has a name slot mark added. After receiving the prediction result, the optimization processor can optimize the prediction result and output the speech recognition result obtained after the optimization process. For words in the prediction result that are not related to the name slot mark, conventional processing methods can be used for optimization processing. For words that are related to the name slot mark, the following methods can be used: Figure 7 The process shown is optimized.

[0134] Figure 7 FIG. 7 shows a flow chart 700 of an embodiment of a speech recognition method. The execution subject of the speech recognition method may be as follows: Figure 1 The optimization processor in the speech recognition system shown. The speech recognition method specifically shows the optimization processing process of words related to the name slot mark, and the speech recognition method includes the following steps:

[0135] Step 701, obtaining the prediction result output by the target end-to-end model, where the prediction result includes multiple text information;

[0136] Step 702, in response to reading a name slot mark at the same position of the plurality of text messages, extracting words that are adjacent to the name slot mark and appear after the name slot mark from the plurality of text messages;

[0137] Step 703, determining a first score corresponding to the extracted word according to the name slot mark;

[0138] Step 704: determine a target word from the extracted words according to the determined first score, the target word serving as a recognition result corresponding to the position occupied by the name slot mark.

[0139] In step 701, the target end-to-end model may include: Figure 4 The target end-to-end model for speech recognition and with name slot tag prediction function obtained by training the method described in the corresponding embodiment.

[0140] In step 703, the scoring model corresponding to the name slot mark can be used to determine the first score corresponding to the extracted word.

[0141] In the scenario of speaking a person's name, the optimization processor can accurately recognize the person's name by adopting steps 701-704, thereby improving the recognition accuracy of the person's name and reducing the false alarm rate, and thus improving the speech recognition effect for the speech information including the person's name.

[0142] It should be noted that, based on the application of the speech recognition method in the scenario of speaking people's names, those skilled in the art can infer the application scheme of the speech recognition method in the scenario of speaking other categories of words by analogy, and examples are not given one by one here.

[0143] The above describes the content related to the model processing scheme 1 as described above. The following describes the content related to the model processing scheme 2 as described above. It should be noted that in order to facilitate the distinction between the prediction results output by the target end-to-end model and the speech recognition model, hereinafter, the prediction result output by the speech recognition model is referred to as the first prediction result, and the prediction result output by the target end-to-end model is referred to as the second prediction result.

[0144] Some embodiments of this specification disclose a model processing method and a speech recognition method related to the model processing solution 2. Specifically, Figure 8 An exemplary system architecture diagram suitable for use with these embodiments is shown.

[0145] like Figure 8As shown, the system architecture includes a model training system and a speech recognition system. The model training system is used to train the end-to-end model to be trained based on training samples including text information and slot marking information, so as to train a target end-to-end model for slot prediction. The text information includes words of the target category. The slot marking information shows the occurrence position of the word and the slot marking corresponding to the target category. Here, the target category is similar to the target category mentioned in the previous text, and no specific explanation is given here.

[0146] In practice, the target end-to-end model is an integral part of the speech recognition system. In addition, the speech recognition system also includes a speech recognition model and an optimization processor. The speech recognition model can be a model that adopts an end-to-end architecture. In a speech recognition scenario, when performing speech recognition on the speech information to be recognized, the speech information can be input into the speech recognition model, so that the speech recognition model outputs a first prediction result for the speech information to the target end-to-end model and the optimization processor, respectively. The target end-to-end model can perform slot prediction on the first prediction result, and output the second prediction result obtained by the slot prediction to the optimization processor. The optimization processor can optimize the first prediction result based on the second prediction result, and output the speech recognition result obtained after the optimization process.

[0147] The specific implementation steps of the above method are described below in conjunction with specific embodiments.

[0148] See also Fig. 9 , which shows a process 900 of an embodiment of a model processing method. The execution subject of the method can be as follows Figure 8 The model training system shown. The method comprises the following steps:

[0149] Step 901, obtaining at least one training sample, the training sample including text information of a word of a target category and slot mark information, the slot mark information showing the occurrence position of the word of the target category and the slot mark corresponding to the target category;

[0150] Step 902: Train the end-to-end model to be trained according to at least one training sample to obtain a target end-to-end model for slot prediction.

[0151] Next, steps 901-902 are described in detail.

[0152] In step 901, the at least one training sample may be pre-generated, and the at least one training sample may be stored in a specific storage location, such as a location locally on the execution subject or on another server that is communicatively connected to the execution subject. The execution subject may obtain the at least one training sample from the storage location.

[0153] In practice, the slot marking information in the at least one training sample may be manually labeled or automatically labeled, which is not specifically limited here.

[0154] In step 902, the execution entity may train the end-to-end model to be trained according to the at least one training sample to obtain a target end-to-end model for slot prediction.

[0155] The end-to-end model to be trained may be an untrained or incompletely trained model. In addition, the end-to-end model to be trained may be any type of model that is suitable for text processing and adopts an end-to-end architecture. Furthermore, the end-to-end model to be trained may include but is not limited to a Transformer-based model.

[0156] In practice, the text information respectively included in the at least one training sample may be text in the form of a word sequence, or may be text without word segmentation.

[0157] When the text information respectively included in the at least one training sample is text in the form of a word sequence, the text information respectively included in the at least one training sample can be used as input, and the slot mark information corresponding to the text information can be used as a label to train the end-to-end model to be trained.

[0158] When the text information respectively included in the at least one training sample is a text that has not been segmented, for each of the at least one training sample, the text information included in the training sample can be segmented, and the words obtained by the segmentation process can be combined into a word sequence. Then, the word sequence corresponding to the text information respectively included in the at least one training sample can be used as input, and the slot mark information corresponding to the text information can be used as a label to train the end-to-end model to be trained.

[0159] The model processing method provided in this embodiment obtains at least one training sample, and then trains the end-to-end model to be trained according to the at least one training sample, so as to train and obtain the target end-to-end model for slot prediction. In this way, the second prediction result output by the target end-to-end model can help the optimization processor to accurately identify the words of the target category from the first prediction result output by the speech recognition model, improve the recognition accuracy of the words of the target category, and reduce the false alarm rate. Therefore, the speech recognition effect for the speech information including the words of the target category can be improved.

[0160] In adopting Fig. 9 After the solution provided in the corresponding embodiment trains the target end-to-end model for slot prediction, the model can be applied to the speech recognition system so that the second prediction result output by the model serves as the input of the optimization processor in the speech recognition system.

[0161] In practice, the first prediction result output by the speech recognition model usually includes multiple text messages, and the multiple text messages have the same length. The optimization processor needs to read the words located at the same position from the multiple text messages in a specified reading order (for example, from left to right, or from right to left, etc.), and determine the word corresponding to the position that serves as the recognition result from the read words. It should be noted that for the words in the first prediction result that are not related to the slot mark, conventional processing methods can be used for optimization processing. For words related to the slot mark, the following can be used Fig.10 The process shown is optimized.

[0162] Fig.10 FIG. 1 shows a flow chart 1000 of an embodiment of a speech recognition method. The execution subject of the speech recognition method may be as follows: Figure 8 The optimization processor in the speech recognition system shown. The speech recognition method shows the optimization process of the words related to the slot mark when the second prediction result shows the occurrence position of the word of the target category and the slot mark corresponding to the target category, and the speech recognition method includes the following steps:

[0163] Step 1001, obtaining a first prediction result output by a speech recognition model, where the first prediction result includes a plurality of text information;

[0164] Step 1002, obtaining a second prediction result output by the target end-to-end model, where the second prediction result is obtained by performing slot prediction on the first prediction result;

[0165] Step 1003, in response to the second prediction result showing the appearance position of the word of the target category and the slot mark corresponding to the target category, and reading the word located at the appearance position in the multiple text messages, determining a first score corresponding to the read word located at the appearance position according to the slot mark shown in the second prediction result;

[0166] Step 1004: Determine a target word from the words read at the occurrence position according to the determined first score, and the target word serves as the recognition result corresponding to the occurrence position.

[0167] In step 1002, the target end-to-end model may include: Fig. 9 The target end-to-end model for slot prediction is trained by the method described in the corresponding embodiment.

[0168] In step 1003, in response to the appearance position of the word of the target category and the slot mark corresponding to the target category shown in the second prediction result, and the word located at the appearance position is read in multiple text messages, the first score corresponding to the read word located at the appearance position can be determined using the scoring model corresponding to the slot mark shown in the second prediction result.

[0169] In this embodiment, the scoring model corresponding to the slot mark is used to determine the first score corresponding to the word read at the occurrence position, and the method of determining the target word from the word read at the occurrence position according to the determined first score can be referred to Figure 5 Related description in the corresponding embodiment.

[0170] The speech recognition method provided in the present embodiment, in the case where the second prediction result shows the occurrence position of the word of the target category and the slot mark corresponding to the target category, in the process of optimizing the first prediction result, in response to reading the word located at the occurrence position in multiple text information in the first prediction result, according to the slot mark (for example, using the scoring model corresponding to the slot mark), determine the first score corresponding to the word located at the occurrence position read, so as to determine the target word from the word located at the occurrence position read according to the determined first score, and the target word acts as the recognition result corresponding to the occurrence position. In this way, the word corresponding to the occurrence position shown in the second prediction result can be accurately identified based on the second prediction result and the scoring model corresponding to the slot mark shown in the second prediction result. Therefore, the recognition accuracy of the word for the target category can be improved, and the false alarm rate can be reduced. Therefore, the speech recognition effect for the speech information including the word of the target category can be improved.

[0171] Further references Fig.11 As an implementation of the methods shown in some of the above figures, this specification provides an embodiment of a model processing device, which is similar to Figure 2 Corresponding to the method embodiment shown, the device can be applied to Figure 1 The model training system shown.

[0172] like Fig.11As shown, the model processing device 1100 of this embodiment includes: an acquisition unit 1101 and a model training unit 1102. The acquisition unit 1101 is configured to acquire at least one training sample, the training sample includes speech information of words of the target category, and annotated text, the annotated text is used to represent the semantics of the speech information, and is added with slot marks corresponding to the target category, and the slot marks are added to the original occurrence positions of the words of the target category in the annotated text; the model training unit 1102 is configured to train the end-to-end model to be trained according to the at least one training sample, and obtain a target end-to-end model for speech recognition and with a slot prediction function.

[0173] In this embodiment, the specific processing of the acquisition unit 1101 and the model training unit 1102 and the technical effects thereof can be referred to respectively. Figure 2 The relevant descriptions of step 201 and step 202 in the corresponding embodiment are not repeated here.

[0174] Optionally, the target category may include at least one of the following categories: person name, place name, organization name, animal name, audio name, video name, etc.

[0175] Optionally, the dictionary used by the end-to-end model to be trained is added with slot markers corresponding to the target category. The end-to-end model to be trained may include a natural language processing model based on a self-attention mechanism and using an encoder-decoder architecture.

[0176] Optionally, when the target category includes personal names, the personal name slot mark added in the annotation text is used to indicate that a personal name should appear at the position it occupies; when the target category includes place names, the place name slot mark added in the annotation text is used to indicate that a place name should appear at the position it occupies; when the target category includes institution names, the institution name slot mark added in the annotation text is used to indicate that an institution name should appear at the position it occupies.

[0177] Optionally, when the annotated text is in the form of a word sequence, the model training unit 1102 can be further configured to: take the speech information included in the above-mentioned at least one training sample as input, and use the annotated text corresponding to the speech information as a label to train the end-to-end model to be trained.

[0178] Optionally, when the annotated text is an unsegmented text, the model training unit 1102 can be further configured to: for each training sample in the at least one training sample mentioned above, perform segmentation on the annotated text included in the training sample, and form a word sequence with the words obtained after the segmentation; use the speech information respectively included in the at least one training sample mentioned above as input, and use the word sequence corresponding to the speech information as a label to train the end-to-end model to be trained.

[0179] The model processing device provided in this embodiment obtains the above-mentioned at least one training sample through the acquisition unit, and then trains the end-to-end model to be trained according to the above-mentioned at least one training sample through the model training unit to obtain a target end-to-end model for speech recognition and with a slot prediction function. The prediction result output by the target end-to-end model can help the optimization processor to accurately identify the words of the target category, improve the recognition accuracy of the words of the target category, and reduce the false alarm rate. Therefore, the speech recognition effect for speech information including the words of the target category can be improved.

[0180] Further references Fig.12 As an implementation of the methods shown in some of the above figures, this specification provides an embodiment of a speech recognition device, which is similar to Figure 5 Corresponding to the method embodiment shown, the device is applied to Figure 1 The optimization processor in the speech recognition system shown. The speech recognition system also includes a target end-to-end model for speech recognition and with a slot prediction function. The optimization processor is used to optimize the prediction results output by the target end-to-end model.

[0181] like Fig.12 As shown, the speech recognition device 1200 of this embodiment includes: an acquisition unit 1201, an extraction unit 1202, a score determination unit 1203 and a recognition result determination unit 1204. The acquisition unit 1201 is configured to acquire the prediction result output by the target end-to-end model, and the prediction result includes multiple text information; the extraction unit 1202 is configured to extract, in response to reading the slot mark corresponding to the target category at the same position of the multiple text information, the words that are adjacent to the slot mark position and appear after the slot mark from the multiple text information; the score determination unit 1203 is configured to determine the first score corresponding to the extracted word according to the slot mark; the recognition result determination unit 1204 is configured to determine the target word from the extracted word according to the determined first score, and the target word serves as the recognition result corresponding to the position occupied by the slot mark.

[0182] In this embodiment, the specific processing of the acquisition unit 1201, the extraction unit 1202, the score determination unit 1203 and the recognition result determination unit 1204 and the technical effects thereof can be referred to respectively. Figure 5 The relevant descriptions of step 501, step 502, step 503 and step 504 in the corresponding embodiment are not repeated here.

[0183] Optionally, the target end-to-end model may include, using Figure 2 The target end-to-end model for speech recognition and with slot prediction function obtained by training using the method described in the corresponding embodiment.

[0184] Optionally, the score determination unit 1203 may be further configured to: determine a first score corresponding to the extracted word using a scoring model corresponding to the slot mark.

[0185] Optionally, the scoring model may include a pre-established data mapping table for characterizing the correspondence between words under the target category and the first scores; and the score determination unit 1203 may be further configured to: search for records including the extracted words in the scoring model, and determine the first score in the found records as the first score corresponding to the extracted words.

[0186] Optionally, the scoring model may include a pre-trained prediction model for predicting a first score corresponding to a word under a target category; and the score determination unit 1203 may be further configured to: input the extracted word into the scoring model to obtain a first score output by the scoring model.

[0187] Optionally, the prediction result may also include second scores corresponding to the above-mentioned multiple text information respectively; and the recognition result determination unit 1204 may be further configured to: for each extracted word, determine the screening score corresponding to the word according to the first score corresponding to the word and the second score corresponding to the text information where the word is located; and determine the target word from the extracted words according to the determined screening score.

[0188] Optionally, the scoring model may correspond to a preset score adjustment coefficient; and the recognition result determination unit 1204 may be further configured to: for each extracted word, determine the product between the first score corresponding to the word and the score adjustment coefficient; and determine the sum of the product and the second score corresponding to the text information in which the word is located as the screening score corresponding to the word.

[0189] The speech recognition device provided by the present embodiment obtains the prediction result output by the target end-to-end model through the acquisition unit, and the prediction result includes multiple text messages, and then the extraction unit responds to the slot mark corresponding to the target category at the same position of the multiple text messages, and extracts the words that are adjacent to the slot mark position and appear after the slot mark from the multiple text messages, and then determines the first score corresponding to the extracted words according to the slot mark (for example, using the scoring model corresponding to the slot mark) through the score determination unit, so that the recognition result determination unit determines the target word from the extracted words according to the determined first score, and the target word acts as the recognition result corresponding to the position occupied by the slot mark. In this way, the word corresponding to the position occupied by the slot mark can be accurately identified according to the slot mark and the scoring model corresponding to the slot mark. Therefore, the recognition accuracy of the words for the target category can be improved, and the false alarm rate can be reduced. Therefore, the speech recognition effect of the speech information for the words including the target category can be improved.

[0190] Further references Fig.13 As an implementation of the methods shown in some of the above figures, this specification provides another embodiment of a model processing device, which is similar to Figure 4 Corresponding to the method embodiment shown, the device can be applied to Figure 1 The model training system shown.

[0191] like Fig.13 As shown, the model processing device 1300 of this embodiment includes: an acquisition unit 1301 and a model training unit 1302. The acquisition unit 1301 is configured to acquire at least one training sample, the training sample includes voice information including a name, and an annotated text, the annotated text is used to represent the semantics of the voice information, and a name slot mark is added, and the name slot mark is added to the original appearance position of the word belonging to the name in the annotated text; the model training unit 1302 is configured to train the end-to-end model to be trained according to the at least one training sample, and obtain a target end-to-end model for speech recognition and with a name slot prediction function.

[0192] In this embodiment, the specific processing of the acquisition unit 1301 and the model training unit 1302 and the technical effects thereof can be referred to respectively. Figure 4 The relevant descriptions of step 401 and step 402 in the corresponding embodiment are not repeated here.

[0193] The model processing device provided in this embodiment obtains the at least one training sample through the acquisition unit, and then trains the end-to-end model to be trained according to the at least one training sample through the model training unit, so as to obtain a target end-to-end model for speech recognition with a name slot prediction function. The prediction result output by the target end-to-end model can help the optimization processor to accurately recognize the name, improve the recognition accuracy of the name, and reduce the false alarm rate, and therefore, can improve the speech recognition effect for the speech information including the name.

[0194] Further references Fig.14 As an implementation of the methods shown in some of the above figures, this specification provides an embodiment of a speech recognition device, which is similar to Figure 7 Corresponding to the method embodiment shown, the device is applied to Figure 1 The optimization processor in the speech recognition system shown. The speech recognition system also includes a target end-to-end model for speech recognition and with a name slot prediction function. The optimization processor is used to optimize the prediction results output by the target end-to-end model.

[0195] like Fig.14 As shown, the speech recognition device 1400 of this embodiment includes: an acquisition unit 1401, an extraction unit 1402, a score determination unit 1403 and a recognition result determination unit 1404. The acquisition unit 1401 is configured to acquire the prediction result output by the target end-to-end model, and the prediction result includes multiple text information; the extraction unit 1402 is configured to extract the words that are adjacent to the position of the name slot mark and appear after the name slot mark from the multiple text information in response to reading the name slot mark at the same position of the multiple text information; the score determination unit 1403 is configured to determine the first score corresponding to the extracted word according to the name slot mark; the recognition result determination unit 1404 is configured to determine the target word from the extracted word according to the determined first score, and the target word acts as the recognition result corresponding to the position occupied by the name slot mark.

[0196] In this embodiment, the specific processing of the acquisition unit 1401, the extraction unit 1402, the score determination unit 1403 and the recognition result determination unit 1404 and the technical effects thereof can be referred to respectively. Figure 7 The relevant descriptions of step 701, step 702, step 703 and step 704 in the corresponding embodiment are not repeated here.

[0197] Optionally, the target end-to-end model may include, using Figure 4 The target end-to-end model for speech recognition and with name slot prediction function obtained by training using the method described in the corresponding embodiment.

[0198] Optionally, the score determination unit 1403 may be further configured to: determine a first score corresponding to the extracted word using a scoring model corresponding to the name slot mark.

[0199] The speech recognition device provided by the present embodiment obtains the prediction result output by the target end-to-end model through the acquisition unit, and the prediction result includes multiple text information, and then the extraction unit responds to the same position of the multiple text information to read the name slot mark, and extracts the words that are close to the name slot mark position and appear after the name slot mark from the multiple text information, and then the score determination unit determines the first score corresponding to the extracted word according to the name slot mark (for example, using the scoring model corresponding to the name slot mark), so that the recognition result determination unit determines the target word from the extracted word according to the determined first score, and the target word acts as the recognition result corresponding to the position occupied by the name slot mark. In this way, according to the name slot mark and the scoring model corresponding to the name slot mark, the name can be accurately identified. Therefore, the recognition accuracy for the name can be improved, the false alarm rate can be reduced, and therefore, the speech recognition effect for the voice information including the name can be improved.

[0200] Further references Fig.15 As an implementation of the methods shown in some of the above figures, this specification provides another embodiment of a model processing device, which is similar to Fig. 9 Corresponding to the method embodiment shown, the device can be applied to Figure 8 The model training system shown.

[0201] like Fig.15 As shown, the model processing device 1500 of this embodiment includes: an acquisition unit 1501 and a model training unit 1502. The acquisition unit 1501 is configured to acquire at least one training sample, the training sample includes text information of a word of a target category, and slot mark information, the slot mark information shows the occurrence position of the word of the target category and the slot mark corresponding to the target category; the model training unit 1502 is configured to train the end-to-end model to be trained according to the at least one training sample to obtain a target end-to-end model for slot prediction.

[0202] In this embodiment, the specific processing of the acquisition unit 1501 and the model training unit 1502 and the technical effects thereof can be referred to respectively. Fig. 9 The relevant descriptions of step 901 and step 902 in the corresponding embodiment are not repeated here.

[0203] The model processing device provided in this embodiment obtains the above-mentioned at least one training sample through the acquisition unit, and then trains the end-to-end model to be trained according to the at least one training sample through the model training unit, so as to train and obtain the target end-to-end model for slot prediction. In this way, the second prediction result output by the target end-to-end model can help the optimization processor to accurately identify the words of the target category from the first prediction result output by the speech recognition model, improve the recognition accuracy of the words of the target category, and reduce the false alarm rate. Therefore, the speech recognition effect for the speech information including the words of the target category can be improved.

[0204] Further references Fig.16 As an implementation of the methods shown in some of the above figures, this specification provides an embodiment of a speech recognition device, which is similar to Fig.10 Corresponding to the method embodiment shown, the device is applied to Figure 8 The optimization processor in the speech recognition system shown. The speech recognition system also includes a speech recognition model and a target end-to-end model for slot prediction. The target end-to-end model is used to perform slot prediction on the first prediction result output by the speech recognition model, and output the second prediction result obtained by the slot prediction to the optimization processor. The optimization processor is used to optimize the first prediction result according to the second prediction result.

[0205] like Fig.16 As shown, the speech recognition device 1600 of this embodiment includes: a first acquisition unit 1601, a second acquisition unit 1602, a score determination unit 1603 and a recognition result determination unit 1604. The first acquisition unit 1601 is configured to acquire a first prediction result output by the speech recognition model, the first prediction result includes multiple text information; the second acquisition unit 1602 is configured to acquire a second prediction result output by the target end-to-end model, the second prediction result is obtained by performing slot prediction on the first prediction result; the score determination unit 1603 is configured to display the appearance position of the word of the target category and the slot mark corresponding to the target category in response to the second prediction result, and read the word located at the appearance position in the multiple text information, and determine the first score corresponding to the word located at the appearance position according to the slot mark; the recognition result determination unit 1604 is configured to determine the target word from the word located at the appearance position according to the determined first score, and the target word serves as the recognition result corresponding to the appearance position.

[0206] In this embodiment, the specific processing of the first acquisition unit 1601, the second acquisition unit 1602, the score determination unit 1603 and the recognition result determination unit 1604 and the technical effects thereof can be referred to in Fig.10The relevant descriptions of step 1001, step 1002, step 1003 and step 1004 in the corresponding embodiment are not repeated here.

[0207] Optionally, the target end-to-end model may include, using Fig. 9 The target end-to-end model for slot prediction is trained by the method described in the corresponding embodiment.

[0208] Optionally, the score determination unit 1603 can be further configured to: in response to the second prediction result showing the appearance position of the word of the target category and the slot mark corresponding to the target category, and reading the word located at the appearance position in multiple text information, use the scoring model corresponding to the slot mark to determine the first score corresponding to the read word located at the appearance position.

[0209] The speech recognition device provided in this embodiment obtains the first prediction result output by the speech recognition model through the first acquisition unit, the first prediction result includes multiple text information, and the second prediction result output by the target end-to-end model is obtained through the second acquisition unit, the second prediction result is obtained by performing slot prediction on the first prediction result, and then the score determination unit responds to the second prediction result to show the appearance position of the word of the target category and the slot mark corresponding to the target category, and reads the word located at the appearance position in the multiple text information, and determines the first score corresponding to the word read at the appearance position according to the slot mark (for example, using the scoring model corresponding to the slot mark), so that the recognition result determination unit determines the target word from the word read at the appearance position according to the determined first score, and the target word acts as the recognition result corresponding to the appearance position. In this way, the word corresponding to the appearance position shown by the second prediction result can be accurately identified based on the scoring model corresponding to the slot mark shown by the second prediction result and the second prediction result. Therefore, the recognition accuracy of the word for the target category can be improved, and the false alarm rate can be reduced. Therefore, the overall speech recognition effect of the speech information including the word of the target category can be improved.

[0210] Further references Fig.17 , which shows a scenario diagram of an interactive device according to this specification.

[0211] like Fig.17 As shown, the interactive device may include an optimization processor, wherein the optimization processor may be connected to a target end-to-end model, wherein the target end-to-end model is used for speech recognition and has a slot prediction function.

[0212] It should be noted that the target end-to-end model may be included in the interactive device or in other devices, which is not specifically limited here. Here, the target end-to-end model is included in the interactive device as an example for description.

[0213] Specifically, the interactive device can obtain the user's voice information and input the voice information into the target end-to-end model. The target end-to-end model can perform voice recognition and slot prediction on the voice information, and output the prediction result to the optimization processor. Among them, the prediction result may include multiple text information, and the multiple text information is used to characterize the semantics of the voice information. It should be understood that when the user's voice information includes words of the target category, the multiple text information is added with slot marks corresponding to the target category, and the words of the target category should appear at the position occupied by the slot mark. Afterwards, the optimization processor can respond to the slot mark corresponding to the target category read at the same position of the multiple text information, and extract the words that are adjacent to the slot mark position and appear after the slot mark from the multiple text information. Then, the optimization processor determines the target word from the extracted words, and the target word acts as the recognition result corresponding to the position occupied by the slot mark. Specifically, the optimization processor can determine the first score corresponding to the extracted word according to the slot mark. Then, the optimization processor can determine the target word from the extracted words according to the determined first score.

[0214] Optionally, the target end-to-end model may include, using Figure 2 The target end-to-end model for speech recognition and with slot prediction function obtained by training using the method described in the corresponding embodiment.

[0215] Optionally, the optimization processor may be further configured to: determine a first score corresponding to the extracted word according to a scoring model corresponding to the slot mark.

[0216] It should be noted that for a detailed explanation of the operations performed by the optimization processor and the technical effects brought about, please refer to Figure 5 The relevant descriptions in the corresponding embodiments will not be described in detail here.

[0217] The interactive device provided in this embodiment includes an optimization processor that can enable the interactive device to have a higher recognition accuracy rate for words of the target category in voice information, and to have a better voice recognition effect for voice information including words of the target category.

[0218] Further references Fig.18 , which shows another scenario schematic diagram of the interactive device according to the present specification.

[0219] like Fig.18 As shown, the interactive device may include an optimization processor, wherein the optimization processor may be connected to the speech recognition model and the target end-to-end model, wherein the target end-to-end model is used for slot prediction.

[0220] It should be noted that the speech recognition model and / or the target end-to-end model may be included in the interactive device or in other devices, and no specific limitation is made here. Here, the speech recognition model and the target end-to-end model are both included in the interactive device for illustration.

[0221] Specifically, the interactive device can obtain the user's voice information and input the voice information into the voice recognition model. The voice recognition model can perform voice recognition on the voice information to obtain a first prediction result, and output the first prediction result to the target end-to-end model and the optimization processor respectively. The first prediction result includes multiple text information. The multiple text information is used to represent the semantics of the voice information respectively.

[0222] The target end-to-end model can perform slot prediction on the received first prediction result to obtain a second prediction result, and output the second prediction result to the optimization processor. It should be understood that when the user's voice information includes words of the target category, the second prediction result can show the occurrence position of the words of the target category and the slot mark corresponding to the target category.

[0223] After receiving the first prediction result and the second prediction result, the optimization processor can analyze and process the first prediction result and the second prediction result. Specifically, the optimization processor can respond to the second prediction result by showing the appearance position of the word of the target category and the slot mark corresponding to the target category, and read the word located at the appearance position in the above-mentioned multiple text information, and determine the target word from the words read at the appearance position, and the target word serves as the recognition result corresponding to the appearance position. Specifically, the optimization processor can determine the first score corresponding to the word read at the appearance position according to the slot mark, and determine the target word from the words read at the appearance position according to the determined first score.

[0224] Optionally, the target end-to-end model may include, using Fig. 9 The target end-to-end model for slot prediction is trained by the method described in the corresponding embodiment.

[0225] Optionally, the optimization processor may be further configured to determine a first score corresponding to the read word located at the above-mentioned occurrence position according to a scoring model corresponding to the slot mark.

[0226] It should be noted that for a detailed explanation of the operations performed by the optimization processor and the technical effects brought about, please refer to Fig.10 The relevant descriptions in the corresponding embodiments will not be described in detail here.

[0227] The interactive device provided in this embodiment includes an optimization processor that can enable the interactive device to have a higher recognition accuracy rate for words of the target category in voice information, and to have a better voice recognition effect for voice information including words of the target category.

[0228] It should be noted that Fig.17 , 18 The interactive devices in the corresponding embodiments may be any type of devices with voice recognition and interactive functions, including but not limited to smart speakers, smart robots, etc.

[0229] The embodiments of this specification also provide a computer-readable storage medium on which a computer program is stored. When the computer program is executed in a computer, the computer is caused to execute the methods respectively shown in the above method embodiments.

[0230] The embodiments of the present specification also provide a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the methods shown in the above method embodiments are implemented.

[0231] The embodiments of this specification also provide a computer program product, which, when executed on a data processing device, enables the data processing device to implement the methods respectively shown in the above method embodiments.

[0232] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the multiple embodiments disclosed in this specification can be implemented by hardware, software, firmware, or any combination thereof. When implemented by software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0233] In some cases, the actions or steps described in the claims may be performed in a different order than in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0234] The specific implementation methods described above further illustrate in detail the purposes, technical solutions and beneficial effects of the multiple embodiments disclosed in this specification. It should be understood that the above description is only the specific implementation methods of the multiple embodiments disclosed in this specification, and is not used to limit the protection scope of the multiple embodiments disclosed in this specification. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the multiple embodiments disclosed in this specification should be included in the protection scope of the multiple embodiments disclosed in this specification.

Claims

1. A speech recognition method, applied to an optimization processor in a speech recognition system, wherein the speech recognition system also includes a target end-to-end model for speech recognition and having a slot prediction function, the method comprising: Obtaining a prediction result output by the target end-to-end model, wherein the prediction result includes multiple pieces of text information; In response to reading a slot mark corresponding to the target category at the same position of the plurality of text messages, extracting from the plurality of text messages respectively words that are adjacent to the slot mark and appear after the slot mark; Determining a first score corresponding to the extracted word according to the slot mark; Determining a target word from the extracted words according to the determined first score, the target word serving as a recognition result corresponding to the position occupied by the slot mark; The target end-to-end model is obtained by training with training samples, wherein the training samples include speech information of words of the target category and annotated text, wherein the annotated text is used to represent the semantics of the speech information and is added with a slot mark corresponding to the target category, and the slot mark is added to the original occurrence position of the word of the target category in the annotated text; After the slot marker is added, the word of the target category is located after the slot marker.

2. The method according to claim 1, wherein: The target end-to-end model includes a target end-to-end model for speech recognition and having a slot prediction function, which is trained using the method according to claim 1.

3. The method according to claim 1, wherein: The step of determining a first score corresponding to the extracted word according to the slot mark includes: The first score corresponding to the extracted word is determined by using the scoring model corresponding to the slot mark.

4. The method according to claim 3, wherein: The scoring model includes a pre-established data mapping table for characterizing the correspondence between words under the target category and the first scores; as well as The step of using the scoring model corresponding to the slot mark to determine the first score corresponding to the extracted word includes: searching the scoring model for a record including the extracted word, and determining the first score in the found record as the first score corresponding to the extracted word.

5. The method according to claim 3, wherein: The scoring model includes a pre-trained prediction model for predicting a first score corresponding to a word under a target category; as well as The step of using the scoring model corresponding to the slot mark to determine the first score corresponding to the extracted word includes: inputting the extracted word into the scoring model to obtain the first score output by the scoring model.

6. The method according to claim 1, wherein: The prediction result also includes second scores corresponding to the plurality of pieces of text information respectively; as well as The step of determining a target word from the extracted words according to the determined first score comprises: For each extracted word, determining a screening score corresponding to the word according to a first score corresponding to the word and a second score corresponding to the text information where the word is located; Based on the determined screening score, a target word is determined from the extracted words.

7. The method according to claim 6, wherein: The scoring model corresponds to a preset score adjustment coefficient; and the method of determining the screening score corresponding to the word according to the first score corresponding to the word and the second score corresponding to the text information in which the word is located comprises: Determine the product of the first score corresponding to the word and the adjustment coefficient; The sum of the product and the second score corresponding to the text information where the word is located is determined as the screening score corresponding to the word.

8. A model processing method comprising: Acquire at least one training sample, the training sample including speech information of a word of a target category, and annotated text, the annotated text is used to characterize the semantics of the speech information, and is added with a slot mark corresponding to the target category, the slot mark is added to the original appearance position of the word of the target category in the annotated text, and after the slot mark is added, the word of the target category is located after the slot mark; train the end-to-end model to be trained according to the at least one training sample to obtain a target end-to-end model for speech recognition and with a slot prediction function; in, The target end-to-end model is used for speech recognition and includes: Obtaining a prediction result output by the target end-to-end model, the prediction result comprising a plurality of text messages; in response to reading a slot mark corresponding to a target category at the same position of the plurality of text messages, extracting from the plurality of text messages words that are adjacent to the slot mark and appear after the slot mark; Determining a first score corresponding to the extracted word according to the slot mark; According to the determined first score, a target word is determined from the extracted words, and the target word serves as a recognition result corresponding to the position occupied by the slot mark.

9. The method according to claim 8, wherein: The target category includes at least one of the following categories: person name, place name, and organization name.

10. The method according to claim 9, wherein: When the target category includes a person's name, the name slot mark added to the annotation text is used to indicate that a person's name should appear at the position it occupies; When the target category includes a place name, the place name slot mark added to the annotation text is used to indicate that a place name should appear at the position it occupies; When the target category includes an institution name, the institution name slot mark added to the annotation text is used to indicate that the institution name should appear at the position it occupies.

11. The method according to claim 8, wherein: The dictionary used by the end-to-end model to be trained is supplemented with slot labels corresponding to the target category.

12. The method according to claim 8, wherein: The end-to-end model to be trained includes a natural language processing model based on a self-attention mechanism and adopting an encoder-decoder architecture.

13. The method according to any one of claims 8 to 12, wherein: The annotated text is a text in the form of a word sequence; And the end-to-end model to be trained is trained according to the at least one training sample, including: taking the speech information respectively included in the at least one training sample as input, and taking the annotated text corresponding to the speech information as a label to train the end-to-end model to be trained.

14. The method according to any one of claims 8 to 12, wherein: The annotated text is the text without word segmentation; as well as The step of training the end-to-end model to be trained according to the at least one training sample includes: For each training sample in the at least one training sample, performing word segmentation processing on the annotated text included in the training sample, and forming word groups obtained through the word segmentation processing into word sequences; The speech information respectively included in the at least one training sample is used as input, and the word sequence corresponding to the speech information is used as a label to train the end-to-end model to be trained.

15. A speech recognition method, applied to an optimization processor in a speech recognition system, wherein the speech recognition system also includes a target end-to-end model for speech recognition and having a name slot prediction function, the method comprising: Obtaining a prediction result output by the target end-to-end model, wherein the prediction result includes multiple pieces of text information; In response to reading a name slot mark at the same position of the plurality of text messages, extracting words that are adjacent to the name slot mark and appear after the name slot mark from the plurality of text messages respectively; Determining a first score corresponding to the extracted word according to the name slot mark; Determining a target word from the extracted words according to the determined first score, the target word serving as a recognition result corresponding to the position occupied by the name slot mark; The target end-to-end model is obtained by training with training samples, wherein the training samples include speech information of words of the target category and annotated text, wherein the annotated text is used to represent the semantics of the speech information and is added with a slot mark corresponding to the target category, and the slot mark is added to the original occurrence position of the word of the target category in the annotated text; After the slot mark is added, the word of the target category is located after the slot mark.

16. A model processing method, comprising: Acquire at least one training sample, the training sample comprising speech information including a person's name, and annotated text, the annotated text is used to represent the semantics of the speech information, and a name slot mark is added, the name slot mark is added to the original appearance position of the word belonging to the person's name in the annotated text, and after the name slot mark is added, the word belonging to the person's name is located after the name slot mark; According to the at least one training sample, the end-to-end model to be trained is trained to obtain a target end-to-end model for speech recognition and having a name slot prediction function; in, The target end-to-end model is used for speech recognition and includes: Obtaining a prediction result output by the target end-to-end model, the prediction result comprising a plurality of text messages; in response to reading a slot mark corresponding to a target category at the same position of the plurality of text messages, extracting from the plurality of text messages words that are adjacent to the slot mark and appear after the slot mark; Determining a first score corresponding to the extracted word according to the slot mark; According to the determined first score, a target word is determined from the extracted words, and the target word serves as a recognition result corresponding to the position occupied by the slot mark.

17. A speech recognition method, applied to an optimization processor in a speech recognition system, wherein the speech recognition system further comprises a speech recognition model and a target end-to-end model for slot prediction, the method comprising: Obtaining a first prediction result output by the speech recognition model, wherein the first prediction result includes a plurality of text information; Obtaining a second prediction result output by the target end-to-end model, where the second prediction result is obtained by performing slot prediction on the first prediction result; In response to the second prediction result showing the occurrence position of a word of the target category and the slot mark corresponding to the target category, and reading the word located at the occurrence position in the plurality of text messages, determining a first score corresponding to the read word located at the occurrence position according to the slot mark; Determining a target word from the words read at the occurrence position according to the determined first score, the target word serving as a recognition result corresponding to the occurrence position; wherein the appearance position is the position occupied by the slot mark; the target end-to-end model is obtained by training with training samples, the training samples include speech information of words of a target category, and annotated text, the annotated text is used to characterize the semantics of the speech information, and is added with a slot mark corresponding to the target category, the slot mark is added to the original appearance position of the word of the target category in the annotated text; wherein after the slot mark is added, the word of the target category is located after the slot mark.

18. A model processing method, comprising: Acquire at least one training sample, the training sample including text information of a word of a target category and slot mark information, the slot mark information showing an appearance position of the word of the target category and a slot mark corresponding to the target category, and after the slot mark is added, the word of the target category is located after the slot mark; Training the end-to-end model to be trained according to the at least one training sample to obtain a target end-to-end model for slot prediction; The target end-to-end model is used for speech recognition and includes: Obtain a first prediction result output by a speech recognition model, the first prediction result including a plurality of text information; obtain a second prediction result output by the target end-to-end model, the second prediction result being obtained by performing slot prediction on the first prediction result; In response to the second prediction result showing the occurrence position of a word of the target category and the slot mark corresponding to the target category, and reading the word located at the occurrence position in the plurality of text messages, determining a first score corresponding to the read word located at the occurrence position according to the slot mark; According to the determined first score, a target word is determined from the read words located at the appearance position, and the target word serves as a recognition result corresponding to the appearance position.

19. A model processing device comprising: an acquisition unit configured to acquire at least one training sample, the training sample comprising speech information including a word of a target category, and an annotated text, the annotated text being used to represent the semantics of the speech information and having a slot mark corresponding to the target category added thereto, the slot mark being added to the original occurrence position of the word of the target category in the annotated text; A model training unit is configured to train the end-to-end model to be trained according to the at least one training sample to obtain a target end-to-end model for speech recognition and with a slot prediction function; in, The target end-to-end model is used for speech recognition and includes: Obtaining a prediction result output by the target end-to-end model, wherein the prediction result includes multiple pieces of text information; In response to reading a slot mark corresponding to the target category at the same position of the plurality of text messages, extracting from the plurality of text messages respectively words that are adjacent to the slot mark and appear after the slot mark; Determining a first score corresponding to the extracted word according to the slot mark; According to the determined first score, a target word is determined from the extracted words, and the target word serves as a recognition result corresponding to the position occupied by the slot mark.

20. A speech recognition device, applied to an optimization processor in a speech recognition system, wherein the speech recognition system also includes a target end-to-end model for speech recognition and having a slot prediction function, the device comprising: an acquisition unit, configured to acquire a prediction result output by the target end-to-end model, wherein the prediction result includes a plurality of text information; an extraction unit configured to, in response to reading a slot mark corresponding to a target category at a same position in the plurality of text messages, extract from the plurality of text messages words that are adjacent to the slot mark and appear after the slot mark; a score determination unit, configured to determine a first score corresponding to the extracted word according to the slot mark; a recognition result determination unit configured to determine a target word from the extracted words according to the determined first score, the target word serving as a recognition result corresponding to the position occupied by the slot mark; The target end-to-end model is obtained by training with training samples, wherein the training samples include speech information of words of the target category and annotated text, wherein the annotated text is used to represent the semantics of the speech information and is added with a slot mark corresponding to the target category, and the slot mark is added to the original occurrence position of the word of the target category in the annotated text; After the slot mark is added, the word of the target category is located after the slot mark.

21. A model processing device, comprising: An acquisition unit is configured to acquire at least one training sample, wherein the training sample includes speech information including a person's name, and an annotated text, wherein the annotated text is used to represent the semantics of the speech information and is added with a name slot mark, wherein the name slot mark is added to the original occurrence position of the word belonging to the person's name in the annotated text, and after the name slot mark is added, the word belonging to the person's name is located after the name slot mark; The model training unit is configured to train the end-to-end model to be trained according to the at least one training sample to obtain a target end-to-end model for speech recognition and with a name slot prediction function; wherein the target end-to-end model is used for speech recognition and includes: Obtaining a prediction result output by the target end-to-end model, wherein the prediction result includes multiple pieces of text information; In response to reading a name slot mark at the same position of the plurality of text messages, extracting words that are adjacent to the name slot mark and appear after the name slot mark from the plurality of text messages respectively; Determining a first score corresponding to the extracted word according to the name slot mark; According to the determined first score, a target word is determined from the extracted words, and the target word serves as a recognition result corresponding to the position occupied by the name slot mark.

22. A speech recognition device, applied to an optimization processor in a speech recognition system, wherein the speech recognition system also includes a target end-to-end model for speech recognition and having a name slot prediction function, the device comprising: an acquisition unit, configured to acquire a prediction result output by the target end-to-end model, wherein the prediction result includes a plurality of text information; an extracting unit configured to, in response to reading a name slot mark at the same position of the plurality of text messages, extract from the plurality of text messages respectively words that are adjacent to the name slot mark and appear after the name slot mark; a score determination unit, configured to determine a first score corresponding to the extracted word according to the name slot mark; A recognition result determination unit is configured to determine a target word from the extracted words according to the determined first score, wherein the target word serves as a recognition result corresponding to the position occupied by the name slot mark; wherein the target end-to-end model is obtained by training a training sample, wherein the training sample includes speech information of a word of a target category, and an annotated text, wherein the annotated text is used to represent the semantics of the speech information and is added with a slot mark corresponding to the target category, wherein the slot mark is added to the original occurrence position of the word of the target category in the annotated text; After the slot marker is added, the word of the target category is located after the slot marker.

23. A model processing device, comprising: an acquisition unit, configured to acquire at least one training sample, the training sample comprising text information of a word of a target category and slot mark information, the slot mark information showing an appearance position of the word of the target category and a slot mark corresponding to the target category, and after the slot mark is added, the word of the target category is located after the slot mark; A model training unit is configured to train the end-to-end model to be trained according to the at least one training sample to obtain a target end-to-end model for slot prediction; The target end-to-end model is used for speech recognition and includes: Obtain a first prediction result output by a speech recognition model, the first prediction result including a plurality of text information; obtain a second prediction result output by the target end-to-end model, the second prediction result being obtained by performing slot prediction on the first prediction result; In response to the second prediction result showing the occurrence position of a word of the target category and the slot mark corresponding to the target category, and reading the word located at the occurrence position in the plurality of text messages, determining a first score corresponding to the read word located at the occurrence position according to the slot mark; According to the determined first score, a target word is determined from the read words located at the appearance position, and the target word serves as a recognition result corresponding to the appearance position.

24. A speech recognition device, applied to an optimization processor in a speech recognition system, wherein the speech recognition system further comprises a speech recognition model and a target end-to-end model for slot prediction, the device comprising: A first acquisition unit is configured to acquire a first prediction result output by the speech recognition model, wherein the first prediction result includes a plurality of text information; A second acquisition unit is configured to acquire a second prediction result output by the target end-to-end model, where the second prediction result is obtained by performing slot prediction on the first prediction result; a score determination unit, configured to display the occurrence position of the word of the target category and the slot mark corresponding to the target category in response to the second prediction result, and read the word located at the occurrence position in the plurality of text information, Determining, according to the slot mark, a first score corresponding to the read word located at the occurrence position; The recognition result determination unit is configured to determine a target word from the words read at the appearance position according to the determined first score, and the target word serves as the recognition result corresponding to the appearance position; wherein the appearance position is the position occupied by the slot mark; the target end-to-end model is obtained by training training samples, and the training samples include speech information of words of a target category, and annotated text, wherein the annotated text is used to represent the semantics of the speech information and is added with a slot mark corresponding to the target category, and the slot mark is added to the original appearance position of the word of the target category in the annotated text, wherein after the slot mark is added, the word of the target category is located after the slot mark.

25. An interactive device comprising an optimization processor; The optimization processor is configured to: Obtaining a prediction result output by a target end-to-end model, wherein the target end-to-end model is used for speech recognition and has a slot prediction function, and the prediction result includes multiple text information; In response to reading a slot mark corresponding to the target category at the same position of the plurality of text messages, extracting from the plurality of text messages respectively words that are adjacent to the slot mark and appear after the slot mark; Determining a first score corresponding to the extracted word according to the slot mark; Determining a target word from the extracted words according to the determined first score, the target word serving as a recognition result corresponding to the position occupied by the slot mark; The target end-to-end model is obtained by training with training samples, wherein the training samples include speech information of words of the target category and annotated text, wherein the annotated text is used to represent the semantics of the speech information and is added with a slot mark corresponding to the target category, and the slot mark is added to the original occurrence position of the word of the target category in the annotated text; After the slot mark is added, the word of the target category is located after the slot mark.

26. An interactive device comprising an optimization processor; The optimization processor is configured to: Obtain a first prediction result output by a speech recognition model, the first prediction result including a plurality of text information; obtain a second prediction result output by a target end-to-end model, the target end-to-end model is used for slot prediction, the second prediction result is obtained by performing slot prediction on the first prediction result; In response to the second prediction result showing the occurrence position of a word of the target category and the slot mark corresponding to the target category, and reading the word located at the occurrence position in the plurality of text messages, determining a first score corresponding to the read word located at the occurrence position according to the slot mark; Determining a target word from the words read at the occurrence position according to the determined first score, the target word serving as a recognition result corresponding to the occurrence position; The target end-to-end model is obtained by training with training samples, wherein the training samples include speech information of words of a target category and annotated text, wherein the annotated text is used to characterize the semantics of the speech information and is added with a slot mark corresponding to the target category, wherein the slot mark is added to the original occurrence position of the word of the target category in the annotated text, wherein after the slot mark is added, the word of the target category is located after the slot mark.

27. A computer-readable storage medium having a computer program stored thereon, wherein: When the computer program is executed in a computer, the computer is caused to execute the method according to any one of claims 1 to 18.

28. A computing device comprising a memory and a processor, wherein: The memory stores executable codes, and when the processor executes the executable codes, the method according to any one of claims 1 to 18 is implemented.

Images