Speech recognition method, device and storage medium

Abstract

The embodiments of the present application provide a speech recognition method, device, and storage medium. The method includes: obtaining cloud-side recognition results at an edge end, wherein the cloud-side recognition results include at least one target sequence obtained by the cloud for identifying a target recognition object; and correcting the at least one target sequence based on a local reference text corresponding to the target recognition object to obtain an edge-side recognition result. The present application obtains cloud-side recognition results for a target recognition object at an edge end, and corrects the cloud-side recognition results based on a local reference text corresponding to the target recognition object, thereby optimizing the cloud-side recognition results and improving the accuracy of speech recognition.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a speech recognition method and device. Background Art

[0002] With the development of the internet, speech recognition has played an increasingly important role. Automatic speech recognition (ASR) technology enables machines to convert speech signals into corresponding text or commands through recognition and understanding. Large vocabulary continuous speech recognition (LVCSR) technology has developed rapidly in recent years and has been widely applied in many fields.

[0003] In existing technologies, cloud-based speech recognition engines are typically used, and language models are typically trained using general-purpose corpora. However, due to limited data, comprehensive coverage is not possible. When performing speech recognition tasks in specific fields, such as medicine, architecture, and artificial intelligence, insufficient language model coverage or the presence of many out-of-vocabulary (OOV) words can lead to reduced ASR performance and accuracy. Summary of the Invention

[0004] The embodiments of the present application provide a speech recognition method, device, and storage medium to address the defect of the prior art that speech cannot be accurately recognized and to improve the accuracy of speech recognition.

[0005] In a first aspect, an embodiment of the present application provides a speech recognition method, comprising:

[0006] The edge end obtains the cloud-side recognition result, which includes at least one target sequence obtained by the cloud for the target recognition object; based on the local reference text corresponding to the target recognition object, the at least one target sequence is corrected to obtain the edge-side recognition result.

[0007] Optionally, according to a speech recognition method according to an embodiment of the present application, the step of correcting the at least one target sequence based on the local reference text corresponding to the target recognition object to obtain an edge recognition result includes:

[0008] Performing vocabulary replacement on an alternative sequence in the target sequence based on a language model corresponding to the target sequence and the local reference text; wherein the language model corresponding to the target sequence is obtained by training based on the local reference text;

[0009] and / or,

[0010] Based on named entity recognition (NER), vocabulary replacement is performed on one of the candidate sequences in the target sequence.

[0011] Optionally, according to a speech recognition method of an embodiment of the present application, the vocabulary matching probability of the candidate sequence is the highest among the vocabulary matching probabilities of all the target sequences;

[0012] Wherein, for each target sequence, the vocabulary matching probability is calculated based on the language model corresponding to the target sequence and the target sequence;

[0013] The vocabulary matching probability of the target sequence is used to describe the frequency with which the vocabulary in the target sequence appears in the local reference text.

[0014] Optionally, according to a speech recognition method according to an embodiment of the present application, performing vocabulary replacement on an alternative sequence in the target sequence based on the language model corresponding to the target sequence and the local reference text includes:

[0015] Obtaining, based on the language model corresponding to the target sequence, a continuous matching probability corresponding to a continuous vocabulary combination in the candidate sequence, wherein the continuous matching probability is used to represent a frequency of occurrence of the continuous vocabulary combination corresponding to the continuous matching probability in the local reference text;

[0016] If the continuous matching probability corresponding to any continuous word combination in the candidate sequence is lower than a first preset threshold, replacing the first continuous word combination with the continuous matching probability lower than the first preset threshold with a replacement text in the local reference text;

[0017] The phoneme matching degree between the phoneme sequence corresponding to the replacement text and the phoneme sequence of the first continuous vocabulary combination is greater than a second preset threshold, and the replacement text appears more frequently in the local reference text than the first continuous vocabulary combination.

[0018] Optionally, according to a speech recognition method of an embodiment of the present application, the target sequence includes: a phoneme sequence and / or a word sequence;

[0019] Correspondingly, the language model corresponding to the phoneme sequence includes a phoneme sequence language model; the language model corresponding to the word sequence includes a word sequence language model.

[0020] Optionally, according to a speech recognition method of an embodiment of the present application, if the target sequence includes a phoneme sequence, performing vocabulary replacement on an alternative sequence in the target sequence based on a language model corresponding to the target sequence and the local reference text includes:

[0021] Based on the phoneme sequence language model, performing vocabulary replacement on an alternative sequence in the phoneme sequence to obtain a first phoneme sequence;

[0022] Based on the word sequence language model, vocabulary replacement is performed on an alternative sequence in at least one word sequence corresponding to the first phoneme sequence.

[0023] Optionally, according to a speech recognition method according to an embodiment of the present application, if the target sequence also includes a word sequence, performing vocabulary replacement on an alternative sequence in the target sequence based on the language model corresponding to the target sequence and the local reference text further includes:

[0024] If the fourth matching probability corresponding to the first word sequence is greater than the fourth matching probability corresponding to the second word sequence, determining that the first word sequence is the edge recognition result;

[0025] If the fourth matching probability corresponding to the second word sequence is greater than the fourth matching probability corresponding to the first word sequence, determining that the second word sequence is the edge recognition result;

[0026] The first word sequence is obtained by performing lexical replacement on an alternative sequence in at least one word sequence corresponding to the first phoneme sequence;

[0027] The second word sequence is obtained by performing vocabulary replacement on an alternative sequence in the word sequence in the target sequence based on the word sequence language model;

[0028] The fourth matching probability corresponding to the first word sequence is used to describe the matching degree between the first word sequence and the local reference text;

[0029] The fourth matching probability corresponding to the second word sequence is used to describe the matching degree between the second word sequence and the local reference text.

[0030] Optionally, according to a speech recognition method according to an embodiment of the present application, the step of performing vocabulary replacement on an alternative sequence in the target sequence based on named entity recognition (NER) to obtain the edge recognition result includes:

[0031] Based on the NER vocabulary, replacing a vocabulary in an alternative sequence in the target sequence with a first replacement vocabulary in the NER vocabulary to obtain the edge recognition result;

[0032] The phoneme matching probability between the phoneme sequence corresponding to the replacement vocabulary and the phoneme sequence of the vocabulary in the candidate sequence is greater than a third preset threshold.

[0033] Optionally, according to a speech recognition method according to an embodiment of the present application, the method further includes:

[0034] Recognize the local reference text based on the NER to generate a NER vocabulary;

[0035] Based on a dictionary and / or character-to-phoneme G2P technology, the phonemes corresponding to the NER vocabulary are obtained.

[0036] Optionally, according to a speech recognition method according to an embodiment of the present application, the local reference text includes: professional information, participant information, activity background information, and activity content of the activity to which the local reference text belongs.

[0037] Optionally, according to a speech recognition method according to an embodiment of the present application, after obtaining the edge recognition result, the method further includes:

[0038] Based on the professional information of the activity to which the local reference text belongs, relevant information of the local reference text is saved in a server.

[0039] In a second aspect, an embodiment of the present application provides a speech recognition method, comprising:

[0040] The device side obtains a cloud recognition result or an edge recognition result, wherein the cloud recognition result includes a target sequence obtained by the cloud side from recognizing the target recognition object;

[0041] Based on the local vocabulary on the device side, the target sequence is modified to obtain a device side recognition result.

[0042] Optionally, according to a speech recognition method of an embodiment of the present application, the step of modifying the target sequence based on a local vocabulary on the device to obtain a device-side recognition result includes:

[0043] replacing a word in the target sequence corresponding to the second replacement word based on the second replacement word in the local word list to obtain the device-side recognition result;

[0044] The phoneme matching probability between the phoneme sequence of the vocabulary corresponding to the second replacement vocabulary and the phoneme sequence of the second replacement vocabulary is greater than a fourth preset threshold.

[0045] Optionally, according to a speech recognition method according to an embodiment of the present application, the vocabulary includes: a pre-set error-prone vocabulary and / or a real-time NER vocabulary.

[0046] Optionally, according to a speech recognition method of an embodiment of the present application, the method further includes: acquiring real-time NER vocabulary in the display information corresponding to the target recognition object in real time based on NER technology.

[0047] In a third aspect, an embodiment of the present application provides a speech recognition device, including a memory, a transceiver, and a processor:

[0048] A memory for storing a computer program; a transceiver for transmitting and receiving data under the control of the processor; and a processor for reading the computer program in the memory and performing the following operations:

[0049] The edge end obtains a cloud recognition result, where the cloud recognition result includes at least one target sequence obtained by the cloud for target recognition object recognition;

[0050] Based on the local reference text corresponding to the target recognition object, the at least one target sequence is corrected to obtain an edge recognition result.

[0051] Optionally, in a speech recognition device according to an embodiment of the present application, the step of correcting the at least one target sequence based on the local reference text corresponding to the target recognition object to obtain an edge recognition result includes:

[0052] Performing vocabulary replacement on an alternative sequence in the target sequence based on a language model corresponding to the target sequence and the local reference text; wherein the language model corresponding to the target sequence is obtained by training based on the local reference text;

[0053] and / or,

[0054] Based on named entity recognition (NER), vocabulary replacement is performed on one of the candidate sequences in the target sequence.

[0055] Optionally, in a speech recognition device according to an embodiment of the present application, the vocabulary matching probability of the candidate sequence is the highest among the vocabulary matching probabilities of all the target sequences;

[0056] Wherein, for each target sequence, the vocabulary matching probability is calculated based on the language model corresponding to the target sequence and the target sequence;

[0057] The vocabulary matching probability of the target sequence is used to describe the frequency with which the vocabulary in the target sequence appears in the local reference text.

[0058] Optionally, in a speech recognition device according to an embodiment of the present application, performing vocabulary replacement on an alternative sequence in the target sequence based on the language model corresponding to the target sequence and the local reference text includes:

[0059] Obtaining, based on the language model corresponding to the target sequence, a continuous matching probability corresponding to a continuous vocabulary combination in the candidate sequence, wherein the continuous matching probability is used to represent a frequency of occurrence of the continuous vocabulary combination corresponding to the continuous matching probability in the local reference text;

[0060] If the continuous matching probability corresponding to any continuous word combination in the candidate sequence is lower than a first preset threshold, replacing the first continuous word combination with the continuous matching probability lower than the first preset threshold with a replacement text in the local reference text;

[0061] The phoneme matching degree between the phoneme sequence corresponding to the replacement text and the phoneme sequence of the first continuous vocabulary combination is greater than a second preset threshold, and the replacement text appears more frequently in the local reference text than the first continuous vocabulary combination.

[0062] Optionally, according to a speech recognition device according to one embodiment of the present application, the target sequence includes:

[0063] Phoneme sequence, and / or, word sequence;

[0064] Accordingly, the language model corresponding to the phoneme sequence includes a phoneme sequence language model;

[0065] The language model corresponding to the word sequence includes a word sequence language model.

[0066] Optionally, in a speech recognition device according to an embodiment of the present application, if the target sequence includes a phoneme sequence, performing vocabulary replacement on an alternative sequence in the target sequence based on a language model corresponding to the target sequence and the local reference text includes:

[0067] Based on the phoneme sequence language model, performing vocabulary replacement on an alternative sequence in the phoneme sequence to obtain a first phoneme sequence;

[0068] Based on the word sequence language model, vocabulary replacement is performed on an alternative sequence in at least one word sequence corresponding to the first phoneme sequence.

[0069] Optionally, according to a speech recognition device according to an embodiment of the present application, if the target sequence also includes a word sequence, performing vocabulary replacement on an alternative sequence in the target sequence based on the language model corresponding to the target sequence and the local reference text further includes:

[0070] If the fourth matching probability corresponding to the first word sequence is greater than the fourth matching probability corresponding to the second word sequence, determining that the first word sequence is the edge recognition result;

[0071] If the fourth matching probability corresponding to the second word sequence is greater than the fourth matching probability corresponding to the first word sequence, determining that the second word sequence is the edge recognition result;

[0072] The first word sequence is obtained by performing lexical replacement on an alternative sequence in at least one word sequence corresponding to the first phoneme sequence;

[0073] The second word sequence is obtained by performing vocabulary replacement on an alternative sequence in the word sequence in the target sequence based on the word sequence language model;

[0074] The fourth matching probability corresponding to the first word sequence is used to describe the matching degree between the first word sequence and the local reference text;

[0075] The fourth matching probability corresponding to the second word sequence is used to describe the matching degree between the second word sequence and the local reference text.

[0076] Optionally, according to a speech recognition device according to an embodiment of the present application, the step of performing vocabulary replacement on an alternative sequence in the target sequence based on named entity recognition (NER) to obtain the edge recognition result includes:

[0077] Based on the NER vocabulary, replacing a vocabulary in an alternative sequence in the target sequence with a first replacement vocabulary in the NER vocabulary to obtain the edge recognition result;

[0078] The phoneme matching probability between the phoneme sequence corresponding to the replacement vocabulary and the phoneme sequence of the vocabulary in the candidate sequence is greater than a third preset threshold.

[0079] Optionally, according to a speech recognition device according to one embodiment of the present application, the operation further includes:

[0080] Recognize the local reference text based on the NER to generate a NER vocabulary;

[0081] Based on a dictionary and / or character-to-phoneme G2P technology, the phonemes corresponding to the NER vocabulary are obtained.

[0082] Optionally, in a speech recognition device according to one embodiment of the present application, the local reference text includes:

[0083] The local reference text contains professional information, participant information, background information, and content of the activity.

[0084] Optionally, according to a speech recognition device according to an embodiment of the present application, after obtaining the edge recognition result, the operation further includes:

[0085] Based on the professional information of the activity to which the local reference text belongs, relevant information of the local reference text is saved in a server.

[0086] In a fourth aspect, an embodiment of the present application further provides a device, including a memory, a transceiver, and a processor:

[0087] A memory for storing a computer program; a transceiver for transmitting and receiving data under the control of the processor; and a processor for reading the computer program in the memory and performing the following operations:

[0088] The device side obtains a cloud recognition result or an edge recognition result, wherein the cloud recognition result includes a target sequence obtained by the cloud side from recognizing the target recognition object;

[0089] Based on the local vocabulary on the device side, the target sequence is modified to obtain a device side recognition result.

[0090] Optionally, in a speech recognition device according to an embodiment of the present application, the step of modifying the target sequence based on a local vocabulary on the device to obtain a device-side recognition result includes:

[0091] replacing a word in the target sequence corresponding to the second replacement word based on the second replacement word in the local word list to obtain the device-side recognition result;

[0092] The phoneme matching probability between the phoneme sequence of the vocabulary corresponding to the second replacement vocabulary and the phoneme sequence of the second replacement vocabulary is greater than a fourth preset threshold.

[0093] Optionally, according to a speech recognition device of an embodiment of the present application, the vocabulary includes: a pre-set error-prone vocabulary and / or a real-time NER vocabulary.

[0094] Optionally, according to a speech recognition device according to an embodiment of the present application, the operation further includes: acquiring real-time NER vocabulary in the display information corresponding to the target recognition object in real time based on NER technology.

[0095] In a fifth aspect, an embodiment of the present application further provides a speech recognition device, comprising:

[0096] A first acquisition unit is configured to acquire, at the edge end, a cloud recognition result, wherein the cloud recognition result includes at least one target sequence obtained by the cloud for identifying a target recognition object;

[0097] The first correction unit is configured to correct the at least one target sequence based on a local reference text corresponding to the target recognition object to obtain an edge recognition result.

[0098] In a sixth aspect, an embodiment of the present application further provides a speech recognition device, comprising:

[0099] A second acquisition unit is configured to acquire, on the device side, a cloud-side recognition result or an edge-side recognition result, wherein the cloud-side recognition result includes a target sequence obtained by the cloud side from recognizing the target recognition object;

[0100] The second correction unit is used to correct the target sequence based on the local vocabulary of the device to obtain a device-side recognition result.

[0101] In the seventh aspect, an embodiment of the present application further provides a processor-readable storage medium, wherein the processor-readable storage medium stores a computer program, and the computer program is used to enable the processor to execute the steps of the first aspect method described in the above-mentioned preliminary selection aspect.

[0102] The embodiments of the present application provide a speech recognition method, device, and storage medium, which obtain the cloud-based recognition results of a target recognition object through the edge terminal, and correct the cloud-based recognition results based on the local reference text corresponding to the target recognition object, thereby optimizing the cloud-based recognition results and improving the accuracy of speech recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0103] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0104] Figure 1 It is a flowchart of speech recognition provided by the prior art;

[0105] Figure 2 This is one of the flow charts of speech recognition provided in the embodiment of the present application;

[0106] Figure 3 This is the second flow chart of speech recognition provided in the embodiment of the present application;

[0107] Figure 4 This is the third flowchart of the speech recognition provided in the embodiment of the present application;

[0108] Figure 5 This is one of the structural diagrams of the speech recognition device provided in the embodiment of the present application;

[0109] Figure 6 This is the second structural diagram of the speech recognition device provided in the embodiment of the present application;

[0110] Figure 7 This is the third structural diagram of the speech recognition device provided in the embodiment of the present application;

[0111] Figure 8 This is the fourth structural diagram of the speech recognition device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0112] In the embodiments of this application, the term "and / or" describes the association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0113] In the embodiments of the present application, the term "plurality" refers to two or more than two, and other quantifiers are similar.

[0114] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0115] The embodiments of the present application provide a speech recognition method and apparatus to improve the accuracy of speech recognition.

[0116] Among them, the method and the device are based on the same application concept. Since the principles of solving problems by the method and the device are similar, the implementation of the device and the method can refer to each other, and the repeated parts will not be repeated.

[0117] The technical solution provided in the embodiment of the present application can be applicable to a variety of systems, especially 5G systems. For example, the applicable system can be a global system of mobile communication (GSM) system, a code division multiple access (CDMA) system, a wideband code division multiple access (WCDMA) general packet radio service (GPRS) system, a long term evolution (LTE) system, a LTE frequency division duplex (FDD) system, a LTE time division duplex (TDD) system, an advanced long term evolution (LTE-A) system, a universal mobile telecommunication system (UMTS), a world-wide interoperability for microwave access (WiMAX) system, a 5G new air interface (NR) system, etc. These various systems include terminal equipment and network equipment. The system may also include a core network part, such as an evolved packet system (EPS), a 5G system (5GS), etc.

[0118] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0119] In the prior art, Figure 1 This is a flow chart of speech recognition provided by the prior art, such as Figure 1 As shown, speech recognition is the process of converting speech into text.

[0120] Taking into account the auditory characteristics of the human ear, Mel cepstral coefficients or perceptual linear prediction coefficients have become one of the mainstream methods for extracting speech feature vectors. Combined with their first-order and second-order differences and normalized processing of the feature vectors, good results have been achieved in the problem of large-vocabulary continuous speech recognition.

[0121] The acoustic model is the underlying model of a speech recognition system and its most critical component. Continuous speech signals are composed of basic speech units, such as sentences, phrases, words, syllables, subsyllables, or phonemes. The choice of speech units for the acoustic model is determined by the specific application (e.g., vocabulary size, speech corpus size, required performance, and other objective factors). Generally speaking, the selected modeling units should meet the following requirements: 1) Robustness: Each model has sufficient samples to estimate model parameters; 2) Consistency: The modeling units should be stable, with minimal variation in acoustic features under different conditions. In continuous speech, due to the influence of coarticulation, the implementation of modeling units in different contexts can sometimes vary significantly. To improve model accuracy, it is necessary to consider the impact of context on modeling units. In acoustic model research, context-sensitive modeling units (such as diphones and triphones) are gaining increasing attention and have become the mainstream modeling units in current acoustic models.

[0122] With the continuous advancement of speech recognition technology, the role of language models in speech recognition has become increasingly important. Due to the dynamic, time-varying, instantaneous, and random nature of acoustic signals, accurate speech recognition and understanding cannot be achieved solely through acoustic pattern matching and interpretation. Leveraging higher-level linguistic knowledge can reduce ambiguity in pattern matching at the acoustic recognition level, thereby improving recognition accuracy. Furthermore, a large-vocabulary continuous speech recognition system must detect speech articulation boundaries at every moment, resulting in the recognition of many different words from different speech streams. To eliminate ambiguity between these words, language models are essential. Language models can provide contextual and semantic information between words. Language models are used not only in speech recognition systems but also in research fields such as machine translation and information retrieval. With the development of statistical language processing methods, statistical language models have become the mainstream language processing technology in speech recognition.

[0123] Search is the process of finding the optimal sentence within the sentence space, following a specific optimization criterion. This involves leveraging existing knowledge (acoustic, phonetic, lexicon, language model, and grammatical and semantic knowledge) to find the optimal sequence of states within the state space (referring to the states of phrases, words, modeling units, or HMMs). The acoustic model, pronunciation lexicon, and language model are tightly integrated through a finite state transformer (FST), and search is performed on the FST.

[0124] Training the language model of a cloud-based speech recognition engine requires a large amount of text data. This data is primarily general-purpose, with some data from other fields also included, but it is impossible to cover all domains. When performing speech recognition tasks in specific fields, such as medicine, architecture, and artificial intelligence, the performance of automatic speech recognition (ASR) technology can be significantly degraded due to insufficient coverage of the language model in these areas.

[0125] Therefore, for a specific recognition task, if some text corpus in this field can be added to optimize the recognition results, the recognition accuracy of the current recognition task can be significantly improved. In addition, it has become a trend to link the cloud, edge computing, and device-side computing on the Internet to build a core computing capability of the Internet of Things (IoT). For the Internet of Things, breakthroughs in edge computing technology mean that many controls (calculations) will be implemented by local devices without having to be handed over to the cloud, and the processing process will be completed at the local edge computing layer. This will undoubtedly greatly improve processing efficiency and reduce the load on the cloud. Because it is closer to the user, it can also provide users with faster responses and solve their needs at the edge. Edge computing, as a key technology in the 5G era, will become one of the indispensable infrastructures in the future. Therefore, after obtaining the basic ASR recognition results on the cloud, we can further improve the ASR recognition accuracy on the edge and device sides. This application adopts a cloud-edge-end combination method to improve recognition accuracy.

[0126] Figure 2 This is one of the flow charts of the speech recognition method provided in the embodiment of the present application, such as Figure 2 Said method comprises the following steps:

[0127] Step 201: The edge terminal obtains a cloud recognition result, where the cloud recognition result includes at least one target sequence obtained by the cloud for target recognition object recognition;

[0128] Optionally, the target recognition object includes but is not limited to audio or video files that can be obtained by the cloud or sent to the cloud. The cloud recognizes the target audio or video file to obtain a cloud recognition result containing at least one target sequence, and the edge obtains the cloud recognition result.

[0129] For example, the target recognition object may be a real-time speech of one or more speakers in a professional conference, and the speech of the real-time speech may be recognized as a target recognition object in units of sentences or segments.

[0130] Step 202: Based on the local reference text corresponding to the target recognition object, modify at least one target sequence to obtain an edge recognition result.

[0131] Through the cloud-edge speech recognition method of this embodiment, the cloud recognition results are corrected to improve the accuracy of speech recognition.

[0132] Optionally, the local reference text includes: professional information, participant information, activity background information, and activity content of the activity to which the local reference text belongs.

[0133] Optionally, the application scenario of the recognition object, that is, the activity to which the local reference text belongs, may be a meeting, a speech, a lecture, or the like.

[0134] Optionally, the edge can obtain the cloud recognition results, including at least one target sequence obtained by the cloud for target recognition object recognition, and then can correct the at least one target sequence based on the local reference text to obtain the edge recognition result.

[0135] Optionally, in this application, a combination of cloud and edge is used to improve the accuracy of speech recognition, and local reference text does not need to be uploaded to the cloud. The ASR recognition results returned from the cloud are optimized at the edge using local reference text such as conference notes, screencast PPTs, and other materials, thereby protecting the privacy of conference materials. There is no need to upload meeting-related materials to the cloud, but only to process them at the edge and device end. By implementing a higher level of network security control within the organization, the security of activities can be improved.

[0136] Optionally, the cloud uses a general ASR that does not need to be modified according to the specific application scenario.

[0137] Optionally, at the edge, data related to the current recognition task, i.e., local reference text, is used to optimize the recognition results.

[0138] Optionally, the embodiments of the present application can improve the recognition accuracy of speech recognition in specific scenarios. The specific scenario can be a scenario where there is reference material and speech to be recognized. The embodiments of the present application are not limited to this, that is, speech can be recognized based on reference material.

[0139] A speech recognition method provided in an embodiment of the present application obtains the cloud-side recognition result of a target recognition object through the edge terminal, and corrects the cloud-side recognition result based on the local reference text corresponding to the target recognition object, thereby optimizing the cloud-side recognition result and improving the accuracy of speech recognition.

[0140] Optionally, the correcting the at least one target sequence based on the local reference text corresponding to the target recognition object to obtain the edge recognition result includes:

[0141] Performing vocabulary replacement on an alternative sequence in the target sequence based on a language model corresponding to the target sequence and the local reference text; wherein the language model corresponding to the target sequence is obtained by training based on the local reference text;

[0142] and / or,

[0143] Based on named entity recognition (NER), vocabulary replacement is performed on one of the candidate sequences in the target sequence.

[0144] Optionally, a vocabulary replacement may be performed on an alternative sequence in the target sequence based on a language model corresponding to the target sequence and the local reference text;

[0145] Optionally, the language model corresponding to the target sequence is obtained by training based on the local reference text.

[0146] Alternatively, the language model can obtain a probability distribution over a sequence of words. For a given sequence of length m, it can generate a probability P(w_1, w_2, ..., w_m) for the entire sequence. In fact, the goal is to find a probability distribution that can represent the probability of any sentence or sequence appearing.

[0147] Optionally, the local reference text may be used as a sample to train and obtain a corresponding language model.

[0148] Optionally, for text content, training can be performed directly to obtain a language model for text sequences (ie, word sequences), ie, a word sequence language model.

[0149] Optionally, for speech content, the text content of the reference text may be converted into phonemes and then trained to obtain a language model for phoneme sequences, namely, a phoneme sequence language model.

[0150] Optionally, a word sequence language model can be used to perform probability calculation on a word sequence in the target sequence to determine the degree of matching with the reference text, and replace the words or word combinations with low matching degrees, wherein they can be replaced with words or phrases in the local reference text that have similar pronunciation but higher probability of occurrence.

[0151] Optionally, a phoneme sequence language model can be used to perform probability calculation on a phoneme sequence in the target sequence to determine the degree of matching with the reference text (when the local reference text is known, the phoneme sequence of the local reference text is known and unique), and some phonemes with low matching degrees can be replaced, where they can be replaced with phonemes in the local reference text that have similar pronunciation but higher probability of occurrence.

[0152] Optionally, based on named entity recognition (NER), vocabulary replacement may be performed on one of the candidate sequences in the target sequence.

[0153] Alternatively, named entity recognition (NER), also known as proper name recognition, refers to identifying entities with specific meanings in text, mainly including names of people, places, organizations, proper nouns, etc. It usually includes two parts: (1) entity boundary recognition; (2) determining the entity category (personal name, place name, organization name, or other).

[0154] Optionally, in order to solve the problem of inaccurate recognition of proper nouns, names of institutions, organizations or people, and other similar words in speech recognition, based on named entity recognition (NER), one of the alternative sequences in the target sequence can be replaced with a word, specifically replacing the inaccurately recognized names of people, places, institutions or other proper nouns.

[0155] Optionally, the phoneme can be understood as the pinyin of a Chinese character, or as the pronounced letters in a word, for example, the phoneme corresponding to the word "name" is "nam".

[0156] Optionally, the vocabulary matching probability of the candidate sequence is the highest among the vocabulary matching probabilities of all the target sequences;

[0157] Wherein, for each target sequence, the vocabulary matching probability is calculated based on the language model corresponding to the target sequence and the target sequence;

[0158] The vocabulary matching probability of the target sequence is used to describe the frequency with which the vocabulary in the target sequence appears in the local reference text.

[0159] Optionally, the candidate sequence may have a vocabulary matching probability that is the highest among all the target sequences; that is, the vocabulary included in the candidate sequence has the highest frequency of appearance in the local reference text among all the target sequences.

[0160] Optionally, when determining the alternative sequences in the target sequence, the corresponding language model (LM) can be used to perform probability calculation on the recognition results obtained from the cloud to obtain the recognition result with the highest probability as the alternative sequence, where the probability is the vocabulary matching probability. For a word sequence, it can be the frequency of the words therein appearing in the local reference text. For a phoneme sequence, it can be the frequency of the phonemes therein appearing in the phoneme sequence corresponding to the local reference text, or the frequency of the words corresponding to the phonemes therein appearing in the local reference text.

[0161] Optionally, the vocabulary corresponding to a group of phonemes may be a most likely vocabulary or multiple vocabulary. For example, the vocabulary corresponding to a group of phonemes "shan feng" may be "mountain peak" or "mountain wind".

[0162] Optionally, performing vocabulary replacement on an alternative sequence in the target sequence based on the language model corresponding to the target sequence and the local reference text includes:

[0163] Obtaining, based on the language model corresponding to the target sequence, a continuous matching probability corresponding to a continuous vocabulary combination in the candidate sequence, wherein the continuous matching probability is used to represent a frequency of occurrence of the continuous vocabulary combination corresponding to the continuous matching probability in the local reference text;

[0164] If the continuous matching probability corresponding to any continuous word combination in the candidate sequence is lower than a first preset threshold, replacing the first continuous word combination with the continuous matching probability lower than the first preset threshold with a replacement text in the local reference text;

[0165] The phoneme matching degree between the phoneme sequence corresponding to the replacement text and the phoneme sequence of the first continuous vocabulary combination is greater than a second preset threshold, and the replacement text appears more frequently in the local reference text than the first continuous vocabulary combination.

[0166] Optionally, the continuous matching probabilities corresponding to the continuous word combinations in the candidate sequence may be obtained based on the language model corresponding to the target sequence;

[0167] Optionally, the continuous matching probability may be used to indicate the frequency of occurrence of the continuous vocabulary combination corresponding to the continuous matching probability in the local reference text; that is, a higher the continuous matching probability, the higher the frequency of occurrence of the corresponding continuous vocabulary combination in the local reference text.

[0168] Optionally, the target sequence may include a phoneme sequence and / or a word sequence;

[0169] Taking the target sequence including a phoneme sequence as an example, its corresponding language model can be a phoneme sequence language model. Therefore, the edge device can calculate the continuous matching probability of an alternative sequence in the phoneme sequence by using the phoneme sequence language model.

[0170] Taking the word sequence including a word sequence as an example, its corresponding language model can be a word sequence language model. Therefore, the edge device can calculate the continuous matching probability of an alternative sequence in the word sequence by using the word sequence language model.

[0171] Optionally, the frequency of the vocabulary included in the alternative sequence appearing in the local reference text is the highest among all target sequences.

[0172] Optionally, after calculating the continuous matching probability of all or part of the continuous vocabulary combinations in the alternative sequence, if it is determined that the continuous matching probability corresponding to part of the continuous vocabulary combinations is lower than the first preset threshold, it can be considered that the frequency of occurrence of this part of the continuous vocabulary combinations in the local reference text is very low or even not present, and there are words or continuous words in the local reference text that are similar in pronunciation to it and have a higher frequency of occurrence in the local reference text. Then, it can be considered that the word or phrase with a similar pronunciation and a higher frequency of occurrence is the replacement text, and the part of the continuous vocabulary combinations with a continuous matching probability lower than the first preset threshold is replaced.

[0173] Optionally, the phoneme matching degree between the phoneme sequence corresponding to the replacement text and the phoneme sequence of the first continuous vocabulary combination is greater than the second preset threshold (i.e., similar in pronunciation), and the replacement text has a higher frequency of occurrence in the local reference text than the first continuous vocabulary combination. For example, if the replacement text is "Zhang San" and the first continuous vocabulary combination is "Zhang San", the two are similar in pronunciation, but "Zhang San" has a higher frequency of occurrence in the local reference text, then the first continuous vocabulary combination "Zhang San" in the alternative sequence can be replaced with "Zhang San".

[0174] Optionally, the first preset threshold can be a positive number less than 1 set in advance, and the second preset threshold can be a positive number less than 1. The phoneme matching degree between the phoneme sequence corresponding to the replacement text and the phoneme sequence of the first continuous vocabulary combination being greater than the second preset threshold indicates that in the reference text, the pronunciation of the replacement text is similar to the pronunciation of the first continuous vocabulary combination. Replacing the continuous vocabulary combination in the alternative sequence with a replacement text that is similar in pronunciation and has a higher probability of occurrence in the reference text can improve the accuracy of speech recognition.

[0175] Optionally, the target sequence includes:

[0176] Phoneme sequence, and / or, word sequence;

[0177] Optionally, the phoneme can be understood as the pinyin of a Chinese character, or as the pronounced letters in a word, for example, the phoneme corresponding to the word "name" is "nam".

[0178] Optionally, the vocabulary corresponding to a group of phonemes may be a most likely vocabulary or multiple vocabulary. For example, the vocabulary corresponding to a group of phonemes "shan feng" may be "mountain peak" or "mountain wind".

[0179] Accordingly, the language model corresponding to the phoneme sequence includes a phoneme sequence language model;

[0180] The language model corresponding to the word sequence includes a word sequence language model.

[0181] Optionally, the target sequence may be a phoneme sequence, a word sequence, or a phoneme sequence and a word sequence at the same time.

[0182] Correspondingly, the language model corresponding to the phoneme sequence includes a phoneme sequence language model; the language model corresponding to the word sequence includes a word sequence language model.

[0183] Optionally, the phoneme sequence language model and the word sequence language model are obtained by training based on the local reference text.

[0184] Alternatively, a phoneme sequence language model or word sequence language model can obtain a probability distribution over a phoneme sequence or word sequence. For a given sequence of length m, it can generate a probability P(w_1, w_2, …, w_m) for the entire sequence. This essentially involves finding a probability distribution that represents the probability of any sentence or sequence occurring.

[0185] Specifically, the recognition results obtained from the cloud can be the n-best (top N best) phoneme sequence recognition results, the n-best (top N best) word sequence recognition results, or the n-best (top N best) phoneme sequence recognition results and the n-best (top N best) word sequence recognition results. The phoneme sequence is a sequence of the smallest speech units, the word sequence is a sequence of characters or words, and N can be a natural number greater than 0.

[0186] Optionally, if the target sequence includes a word sequence, performing vocabulary replacement on an alternative sequence in the target sequence based on the language model corresponding to the target sequence and the local reference text includes:

[0187] Based on the word sequence language model, vocabulary replacement is performed on an alternative sequence in the word sequence to obtain a first word sequence.

[0188] Optionally, the word sequence includes N word sequences. For example, the phonetic pronunciation of "shanfeng" contains three word sequences of "mountain peak, mountain wind, and fan wind". The probability of the above three word sequences is calculated through a word sequence language model. If "mountain peak" has the highest probability of appearing in the local reference text, "mountain peak" is obtained as the alternative sequence.

[0189] Optionally, if the target sequence includes a phoneme sequence, performing vocabulary replacement on an alternative sequence in the target sequence based on the language model corresponding to the target sequence and the local reference text includes:

[0190] Based on the phoneme sequence language model, performing vocabulary replacement on an alternative sequence in the phoneme sequence to obtain a first phoneme sequence;

[0191] Based on the word sequence language model, vocabulary replacement is performed on an alternative sequence in at least one word sequence corresponding to the first phoneme sequence.

[0192] Optionally, based on the phoneme sequence language model, vocabulary replacement is performed on an alternative sequence in the phoneme sequence to obtain a first phoneme sequence.

[0193] For example, the pronunciation in the target recognition result is "zhangsan", and the phoneme sequence obtained from the cloud can be "zhangsan, zhangshan, zhangsang". The above phoneme sequence is probability calculated through the phoneme language model, and the phoneme zhang and phoneme san have the highest probability in the local reference text respectively. Then the phoneme sequence "zhangsan" is selected as the alternative sequence. If the "zhuangsan" in the alternative sequence "zhangsan" with a similar pronunciation in the local reference text has a higher probability of appearing in the local reference text than the alternative sequence "zhangsan", then "zhuangsan" is used as the first phoneme sequence.

[0194] For another example, the pronunciation in the target recognition result is "wo de ming zi shi zhangsan", and the phoneme sequences obtained from the cloud can be "wo de ming zi shi zhangsan", "wo de ming zi shi zhangshan", and "wo deming zi shi zhangsang". The above phoneme sequences are probability calculated through the phoneme language model, and the phoneme sequence "wo de ming zi shi zhangsan" has the highest probability in the local reference text. Then the phoneme sequence "wo de ming zi shi zhangsan" is selected as the alternative sequence. If the "zhangsan" in the alternative sequence "wo de ming zi shi zhangsan" has a similar pronunciation to "zhuangsan" in the local reference text and has a higher probability of appearing in the local reference text than the alternative sequence "zhangsan", then "zhuangsan" replaces the phoneme "zhangsan" to obtain the first phoneme sequence "wo deming zi shi zhuangsan".

[0195] Optionally, based on the word sequence language model, lexical replacement is performed on an alternative sequence in the at least one word sequence corresponding to the first phoneme sequence. The first phoneme sequence may correspond to multiple word sequences. For example, the phoneme sequence "wo demingzi shi zhangsan" may correspond to multiple word sequences such as "my name is Zhang San," "my name is Zhang San," and "my name is Zhang San." If "my name is Zhang San" has the highest frequency of occurrence in the local reference text, "my name is Zhang San" is selected as an alternative sequence in the at least one word sequence corresponding to the first phoneme sequence.

[0196] Optionally, if the target sequence includes a phoneme sequence and a word sequence, performing vocabulary replacement on an alternative sequence in the target sequence based on the language model corresponding to the target sequence and the local reference text further includes:

[0197] If the fourth matching probability corresponding to the first word sequence is greater than the fourth matching probability corresponding to the second word sequence, determining that the first word sequence is the edge recognition result;

[0198] If the fourth matching probability corresponding to the second word sequence is greater than the fourth matching probability corresponding to the first word sequence, determining that the second word sequence is the edge recognition result;

[0199] The first word sequence is obtained by performing lexical replacement on an alternative sequence in at least one word sequence corresponding to the first phoneme sequence;

[0200] Optionally, the first phoneme sequence may correspond to multiple word sequences. One of the word sequences is selected as an alternative sequence, and the lexical matching probability of the alternative sequence is the highest among the lexical matching probabilities of all the target sequences. For example, the words corresponding to a group of phonemes "shan feng" can be "山峰" (mountain peak) or "山风" (mountain breeze). If the occurrence probability of "山峰" in the local reference text is the highest, then "山峰" is selected as the alternative sequence for the phonemes "shan feng".

[0201] Optionally, if the speech pronunciation of the target recognition object is "wo de ming zi shi zhang san", and the word sequences include "我的名字是张三" (My name is Zhang San), "我得名字是张叁" (My name is Zhang San (a different character)), and "我的名字是璋三" (My name is Zhang San (a different character)), among the above three word sequences, the occurrence probability of "我的名字是张三" in the local reference text is the highest, and "我的名字是张三" is selected as the alternative sequence. If the occurrence probability of the next word "张三" after "是" (is) in the alternative sequence calculated through the language model is low, then a character or word with a similar pronunciation and a higher occurrence probability in the reference text can be selected for replacement. For example, if "张山" appears multiple times in the reference text, then "张山" is used to replace "张三", resulting in the first word sequence "我的名字是张山".

[0202] The second word sequence is obtained by performing lexical replacement on an alternative sequence in the word sequences of the target sequence based on the word sequence language model;

[0203] The fourth matching probability corresponding to the first word sequence is used to describe the matching degree between the first word sequence and the local reference text;

[0204] The fourth matching probability corresponding to the second word sequence is used to describe the matching degree between the second word sequence and the local reference text.

[0205] Optionally, the obtaining of the edge - end recognition result by performing lexical replacement on an alternative sequence in the target sequence based on named - entity recognition (NER) includes:

[0206] Based on the NER word list, the words in an alternative sequence in the target sequence are replaced with the first replacement words in the NER word list to obtain the edge - end recognition result;

[0207] Wherein, the phoneme matching probability between the phoneme sequence corresponding to the replacement word and the phoneme sequence of the word in the alternative sequence is greater than a third preset threshold.

[0208] Optionally, a NER vocabulary is generated using local reference text, and the edge end replaces a vocabulary in an alternative sequence in the target sequence with a first replacement vocabulary in the NER vocabulary to obtain the edge end recognition result, wherein the first replacement vocabulary is a vocabulary whose corresponding phoneme sequence has a phoneme matching probability with the phoneme sequence of the vocabulary in the alternative sequence greater than a third preset threshold, that is, the first replacement vocabulary has a similar pronunciation to the corresponding vocabulary in the alternative sequence. The third preset threshold is a positive number less than 1, and the phoneme matching probability of the phoneme sequence corresponding to the replacement vocabulary with the phoneme sequence of the vocabulary in the alternative sequence is greater than the third preset threshold, indicating that the replacement vocabulary in the NER vocabulary has a similar pronunciation to the vocabulary in the alternative sequence.

[0209] Optionally, the method further includes:

[0210] Recognize the local reference text based on the NER to generate a NER vocabulary;

[0211] Based on a dictionary and / or grapheme-to-phoneme (G2P) technology, the phonemes corresponding to the NER vocabulary are obtained.

[0212] Optionally, based on the NER vocabulary, the edge recognition result is obtained by replacing the vocabulary in an alternative sequence in the target sequence with the first replacement vocabulary in the NER vocabulary. For example, if the vocabulary in an alternative sequence in the target sequence is "open ai", and the vocabulary "OpenAI" corresponding to the pronunciation of "open ai" in the NER vocabulary has a greater matching probability, then the vocabulary "open ai" in the alternative sequence is replaced with "OpenAI", where "OpenAI" is an artificial intelligence non-profit organization jointly established by many Silicon Valley tycoons.

[0213] Specifically, a dictionary is requested from the cloud and downloaded to the edge.

[0214] Named Entity Recognition (NER) technology is used to identify names of people, places, organizations, proper nouns, etc. in text content and generate a NER vocabulary.

[0215] Optionally, the local reference text includes:

[0216] The local reference text contains professional information, participant information, background information, and content of the activity.

[0217] Optionally, the local reference text may include conference notes, a conference projection PPT, names, positions, background identities of participants, background information of the conference, etc.

[0218] Optionally, in this application, a combination of cloud and edge is used to improve the accuracy of speech recognition, and local reference text does not need to be uploaded to the cloud. The ASR recognition results returned from the cloud are optimized at the edge using local reference text such as conference notes, screencast PPTs, and other materials, thereby protecting the privacy of conference materials. There is no need to upload meeting-related materials to the cloud, but only to process them at the edge and device end. By implementing a higher level of network security control within the organization, the security of activities can be improved.

[0219] Optionally, the names, titles, etc. of the participants are added to the NER vocabulary to improve the accuracy of NER.

[0220] Specifically, if the names, titles, and other information of the attendees may contain uncommon characters, the cloud-based recognition results for these uncommon characters may be inaccurate. This application adds the names, titles, and other names of the attendees to the NER vocabulary, and can replace the cloud-based recognition results for these uncommon characters with the uncommon characters in the NER vocabulary to obtain more accurate recognition results.

[0221] Optionally, if the local reference text includes conference notes, screencast PPT, etc., text recognition technology is used to automatically perform text recognition to identify the text content in the image. If the local reference text is an electronic version, text recognition is not required.

[0222] Optionally, the local reference text also includes meeting-related background information such as the names and job descriptions of the participants.

[0223] At the edge, the voice recognition results returned from the cloud are optimized through conference presentations, screencast PPTs, and other materials.

[0224] Optionally, you can use a downloaded dictionary to get the pronunciation corresponding to the words in the NER vocabulary, or you can use character-to-phoneme G2P technology to get the pronunciation corresponding to the words in the NER vocabulary, or you can use character-to-phoneme G2P technology based on the downloaded dictionary to get the pronunciation corresponding to the words in the NER vocabulary.

[0225] Optionally, when a word in the NER vocabulary has no corresponding pronunciation in the downloaded dictionary, G2P technology can be used to convert the words in the NER vocabulary using the downloaded dictionary as a reference model to obtain the pronunciation corresponding to the words in the NER vocabulary.

[0226] By using the cloud-edge-end collaboration approach in this embodiment, the computing resources of the cloud-edge-end are fully utilized, which reduces the tasks processed by the cloud, reduces the cloud network latency, and improves the response speed of speech recognition.

[0227] Optionally, after obtaining the edge recognition result, the method further includes:

[0228] Based on the professional information of the activity to which the local reference text belongs, relevant information of the local reference text is saved in a server.

[0229] Specifically, local reference texts are categorized by the topic of the activity to which they belong, and their related information is stored on the server. For example, if the theme of a conference is artificial intelligence, AI can be used as a category, and all local reference texts related to subsequent conferences with the same theme can be stored in the same AI category on the server. This allows the organization to continuously accumulate relevant text data and continuously improve recognition accuracy in this field.

[0230] Through the voice recognition method provided by the embodiment of the present application, the edge optimizes the recognition results, and there is no need to transmit meeting-related materials to the cloud. Instead, they are processed at the edge, which can protect the privacy of meeting materials and improve the security of meetings by performing higher-level network security controls within the organization. The application scenarios of this application are not limited to meetings, but can also be speeches, lectures and other activities. The cloud uses general voice recognition technology and does not need to be changed according to specific application scenarios.

[0231] A speech recognition method provided in an embodiment of the present application obtains the cloud-side recognition result of a target recognition object through the edge terminal, and corrects the cloud-side recognition result based on the local reference text corresponding to the target recognition object, thereby optimizing the cloud-side recognition result and improving the accuracy of speech recognition.

[0232] Figure 3 This is the second flow chart of the speech recognition method proposed in the embodiment of the present application. Figure 3 Said method comprises the following steps:

[0233] Step 301: The device obtains a cloud recognition result or an edge recognition result, wherein the cloud recognition result includes a target sequence obtained by the cloud from recognizing the target object;

[0234] Optionally, the edge recognition result is a recognition result obtained after the edge performs vocabulary replacement, i.e., optimization, on the cloud recognition result.

[0235] Step 302: Based on the local vocabulary on the device, the target sequence is modified to obtain a device-side recognition result.

[0236] Optionally, if the device side obtains the edge side recognition result, the device side may continue to optimize the speech recognition result returned by the edge side.

[0237] A speech recognition method provided in an embodiment of the present application obtains the cloud-side recognition result of a target recognition object through the edge terminal, and corrects the cloud-side recognition result based on the local reference text corresponding to the target recognition object, thereby optimizing the cloud-side recognition result and improving the accuracy of speech recognition.

[0238] Optionally, the modifying the target sequence based on a local vocabulary on the device to obtain a device-side recognition result includes:

[0239] replacing a word in the target sequence corresponding to the second replacement word based on the second replacement word in the local word list to obtain the device-side recognition result;

[0240] Optionally, the second replacement vocabulary is a vocabulary in the local vocabulary that has a similar pronunciation to the vocabulary in the target sequence and has a higher probability of occurrence.

[0241] For example, the target sequence is "My name is Zhang San", where the vocabulary "Zhang San" has similar pronunciations to "Zhang San" and "Zhang San" in the local vocabulary. The vocabulary "Zhang San" has a higher probability of appearing than the vocabulary "Zhang San" in the target sequence, so "Zhang San" is the second replacement vocabulary in the local vocabulary.

[0242] The phoneme matching probability between the phoneme sequence of the vocabulary corresponding to the second replacement vocabulary and the phoneme sequence of the second replacement vocabulary is greater than a fourth preset threshold.

[0243] Specifically, the fourth preset threshold is a positive number less than 1, and the phoneme matching probability between the phoneme sequence of the vocabulary corresponding to the second replacement vocabulary and the phoneme sequence of the second replacement vocabulary is greater than the fourth preset threshold, indicating that the pronunciation of the vocabulary in the local vocabulary is similar to the pronunciation of the vocabulary in the target sequence corresponding to the vocabulary in the local vocabulary.

[0244] Optionally, the vocabulary includes: a preset error-prone vocabulary and / or a real-time NER vocabulary.

[0245] The pre-set error-prone word list includes words that are easily misidentified and words that appear frequently but are uncommon in this meeting.

[0246] Optionally, the method further includes: acquiring real-time NER vocabulary in the display information corresponding to the target recognition object based on NER technology in real time.

[0247] Optionally, real-time NER vocabulary in the display information corresponding to the target recognition object is obtained in real time.

[0248] Specifically, the NER vocabulary on the current screen is obtained by capturing the information on the screen in real time based on the NER technology. However, in this application, it is not limited to the information on the screen, and can also be the information of the entity text captured in real time.

[0249] Optionally, the vocabulary includes: a preset error-prone vocabulary and / or a real-time NER vocabulary.

[0250] Optionally, grammatical analysis technology is used to analyze the recognition results obtained by the device on the cloud or edge side, and the subject, object and other vocabulary therein are replaced with words with similar pronunciation in the vocabulary, thereby further improving the recognition accuracy.

[0251] The embodiments of this application provide a voice recognition method that optimizes recognition results on the device side. This eliminates the need to upload meeting-related materials to the cloud, allowing them to be processed on the device side. This protects the privacy of meeting materials and improves meeting security by implementing higher-level network security controls within the organization. In this application, this is not limited to meetings, but can also include activities such as speeches and lectures.

[0252] Optionally, the cloud-based recognition results of the target recognition object are obtained through the edge, and the cloud-based recognition results are corrected based on the local reference text corresponding to the target recognition object, thereby optimizing the cloud-based recognition results and improving the accuracy of speech recognition.

[0253] Figure 4 This is the third flow chart of the speech recognition method provided in the embodiment of the present application, such as Figure 4 The specific process is as follows:

[0254] Step 401: Obtain the n-best recognition results of the phoneme sequence and the n-best recognition results of the word sequence from the cloud;

[0255] Step 402: Request a dictionary from the cloud and download the dictionary to the edge.

[0256] Step 403: Generate a LM on the edge, generate a NER vocabulary, obtain the pronunciations corresponding to the words in the NER vocabulary, use the LM to select the recognition results returned from the cloud, and optimize the recognition results using the LM and the NER vocabulary;

[0257] Step 404: construct a device-side word list, obtain the NER vocabulary on the current screen, and further optimize the recognition result.

[0258] Figure 5 This is one of the structural diagrams of the speech recognition device provided in the embodiment of the present application. Figure 5 The speech recognition device includes a memory, a transceiver, and a processor:

[0259] A memory for storing a computer program; a transceiver for transmitting and receiving data under the control of the processor; and a processor for reading the computer program in the memory and performing the following operations:

[0260] The edge end obtains a cloud recognition result, where the cloud recognition result includes at least one target sequence obtained by the cloud for target recognition object recognition;

[0261] Based on the local reference text corresponding to the target recognition object, the at least one target sequence is corrected to obtain an edge recognition result.

[0262] The transceiver 502 is configured to receive and send data under the control of the processor 503 .

[0263] Among them, Figure 5 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking together various circuits of one or more processors represented by processor 503 and memory represented by memory 501. The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are all well known in the art and therefore will not be described further herein. The bus interface provides an interface. The transceiver 502 may be a plurality of components, i.e., a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium, such as a wireless channel, a wired channel, an optical cable, and the like. For different user devices, the user interface 504 may also be an interface capable of connecting external or internal devices as required, and the connected devices include but are not limited to a keypad, a display, a speaker, a microphone, a joystick, and the like.

[0264] The processor 503 is responsible for managing the bus architecture and general processing, and the memory 501 can store data used by the processor 503 when performing operations.

[0265] Optionally, the processor 503 may be a CPU (central processing unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) or a CPLD (Complex Programmable Logic Device), and the processor may also adopt a multi-core architecture.

[0266] Optionally, in one embodiment, the modifying the at least one target sequence based on the local reference text corresponding to the target recognition object to obtain the edge recognition result includes:

[0267] Based on the language model corresponding to the target sequence and the local reference text, vocabulary replacement is performed on an alternative sequence in the target sequence; wherein the language model corresponding to the target sequence is obtained by training based on the local reference text; and / or, based on named entity recognition (NER), vocabulary replacement is performed on one of the alternative sequences in the target sequence.

[0268] Optionally, in one embodiment, the vocabulary matching probability of the candidate sequence is the highest among the vocabulary matching probabilities of all the target sequences;

[0269] Wherein, for each target sequence, the vocabulary matching probability is calculated based on the language model corresponding to the target sequence and the target sequence;

[0270] The vocabulary matching probability of the target sequence is used to describe the frequency with which the vocabulary in the target sequence appears in the local reference text.

[0271] Optionally, in one embodiment, performing vocabulary replacement on an alternative sequence in the target sequence based on the language model corresponding to the target sequence and the local reference text includes:

[0272] Obtaining, based on the language model corresponding to the target sequence, a continuous matching probability corresponding to a continuous vocabulary combination in the candidate sequence, wherein the continuous matching probability is used to represent a frequency of occurrence of the continuous vocabulary combination corresponding to the continuous matching probability in the local reference text;

[0273] If the continuous matching probability corresponding to any continuous word combination in the candidate sequence is lower than a first preset threshold, replacing the first continuous word combination with the continuous matching probability lower than the first preset threshold with a replacement text in the local reference text;

[0274] The phoneme matching degree between the phoneme sequence corresponding to the replacement text and the phoneme sequence of the first continuous vocabulary combination is greater than a second preset threshold, and the replacement text appears more frequently in the local reference text than the first continuous vocabulary combination.

[0275] Optionally, in one embodiment, the target sequence comprises:

[0276] Phoneme sequence, and / or word sequence; accordingly, the language model corresponding to the phoneme sequence includes a phoneme sequence language model; the language model corresponding to the word sequence includes a word sequence language model.

[0277] Optionally, in one embodiment, if the target sequence includes a phoneme sequence, performing vocabulary replacement on an alternative sequence in the target sequence based on the language model corresponding to the target sequence and the local reference text includes:

[0278] Based on the phoneme sequence language model, performing vocabulary replacement on an alternative sequence in the phoneme sequence to obtain a first phoneme sequence;

[0279] Based on the word sequence language model, vocabulary replacement is performed on an alternative sequence in at least one word sequence corresponding to the first phoneme sequence.

[0280] Optionally, in one embodiment, if the target sequence further includes a word sequence, performing vocabulary replacement on an alternative sequence in the target sequence based on the language model corresponding to the target sequence and the local reference text further includes:

[0281] If the fourth matching probability corresponding to the first word sequence is greater than the fourth matching probability corresponding to the second word sequence, determining that the first word sequence is the edge recognition result;

[0282] If the fourth matching probability corresponding to the second word sequence is greater than the fourth matching probability corresponding to the first word sequence, determining that the second word sequence is the edge recognition result;

[0283] The first word sequence is obtained by performing lexical replacement on an alternative sequence in at least one word sequence corresponding to the first phoneme sequence;

[0284] The second word sequence is obtained by performing vocabulary replacement on an alternative sequence in the word sequence in the target sequence based on the word sequence language model;

[0285] The fourth matching probability corresponding to the first word sequence is used to describe the matching degree between the first word sequence and the local reference text;

[0286] The fourth matching probability corresponding to the second word sequence is used to describe the matching degree between the second word sequence and the local reference text.

[0287] Optionally, in one embodiment, performing vocabulary replacement on an alternative sequence in the target sequence based on named entity recognition (NER) to obtain the edge recognition result includes:

[0288] Based on the NER vocabulary, replacing a vocabulary in an alternative sequence in the target sequence with a first replacement vocabulary in the NER vocabulary to obtain the edge recognition result;

[0289] The phoneme matching probability between the phoneme sequence corresponding to the replacement vocabulary and the phoneme sequence of the vocabulary in the candidate sequence is greater than a third preset threshold.

[0290] Optionally, in one embodiment, the operation further includes:

[0291] Recognize the local reference text based on the NER to generate a NER vocabulary;

[0292] The phonemes corresponding to the NER vocabulary are obtained based on a dictionary and / or character-to-phoneme G2P technology. Optionally, in one embodiment, the local reference text includes: professional information, participant information, activity background information, and activity content of the activity to which the local reference text belongs.

[0293] Optionally, in one embodiment, after obtaining the edge recognition result, the operation further includes: saving relevant information of the local reference text to a server based on professional information of the activity to which the local reference text belongs.

[0294] A speech recognition device provided in an embodiment of the present application obtains the cloud-based recognition results of a target recognition object through an edge terminal, and corrects the cloud-based recognition results based on the local reference text corresponding to the target recognition object, thereby optimizing the cloud-based recognition results and improving the accuracy of speech recognition.

[0295] The processor calls the computer program stored in the memory to execute any of the methods provided in the embodiments of the present application according to the obtained executable instructions. The processor and the memory can also be arranged physically separately.

[0296] It should be noted here that the above-mentioned device provided by the embodiment of the present invention can implement all the method steps implemented by the above-mentioned method embodiment and can achieve the same technical effect. The parts and beneficial effects that are the same as the method embodiment in this embodiment will not be described in detail here.

[0297] Figure 6 This is the second structural diagram of the speech recognition device provided in the embodiment of the present application. Figure 6 The speech recognition device includes a memory, a transceiver, and a processor:

[0298] A memory for storing a computer program; a transceiver for transmitting and receiving data under the control of the processor; and a processor for reading the computer program in the memory and performing the following operations:

[0299] The device side obtains a cloud recognition result or an edge recognition result, wherein the cloud recognition result includes a target sequence obtained by the cloud side from recognizing the target recognition object;

[0300] Based on the local vocabulary on the device side, the target sequence is modified to obtain a device side recognition result.

[0301] The transceiver 602 is configured to receive and send data under the control of the processor 603 .

[0302] Among them, Figure 6 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking together various circuits of one or more processors represented by processor 603 and memory represented by memory 601. The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are all well known in the art and therefore will not be described further herein. The bus interface provides an interface. The transceiver 602 may be a plurality of components, i.e., a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium, such as a wireless channel, a wired channel, an optical cable, and the like. For different user devices, the user interface 604 may also be an interface capable of connecting external or internal devices as required, and the connected devices include but are not limited to a keypad, a display, a speaker, a microphone, a joystick, and the like.

[0303] The processor 603 is responsible for managing the bus architecture and general processing, and the memory 601 can store data used by the processor 603 when performing operations.

[0304] Optionally, the processor 603 may be a CPU (central processing unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) or a CPLD (Complex Programmable Logic Device), and the processor may also adopt a multi-core architecture.

[0305] Optionally, the modifying the target sequence based on a local vocabulary on the device to obtain a device-side recognition result includes:

[0306] replacing a word in the target sequence corresponding to the second replacement word based on the second replacement word in the local word list to obtain the device-side recognition result;

[0307] The phoneme matching probability between the phoneme sequence of the vocabulary corresponding to the second replacement vocabulary and the phoneme sequence of the second replacement vocabulary is greater than a fourth preset threshold.

[0308] Optionally, the vocabulary includes: a preset error-prone vocabulary and / or a real-time NER vocabulary.

[0309] Optionally, the operation further includes: acquiring real-time NER vocabulary in the display information corresponding to the target recognition object based on NER technology in real time.

[0310] A speech recognition device provided in an embodiment of the present application obtains the cloud-based recognition results of a target recognition object through an edge terminal, and corrects the cloud-based recognition results based on the local reference text corresponding to the target recognition object, thereby optimizing the cloud-based recognition results and improving the accuracy of speech recognition.

[0311] The processor calls the computer program stored in the memory to execute any of the methods provided in the embodiments of the present application according to the obtained executable instructions. The processor and the memory can also be arranged physically separately.

[0312] It should be noted here that the above-mentioned device provided by the embodiment of the present invention can implement all the method steps implemented by the above-mentioned method embodiment and can achieve the same technical effect. The parts and beneficial effects that are the same as the method embodiment in this embodiment will not be described in detail here.

[0313] Figure 7 This is the third structural diagram of the speech recognition device provided in the embodiment of the present application. Figure 7 The speech recognition device includes a first acquisition unit 710 and a first correction unit 720, wherein:

[0314] The first acquisition unit 710 is configured to acquire a cloud recognition result at the edge, where the cloud recognition result includes at least one target sequence obtained by the cloud for target recognition object recognition;

[0315] The first correction unit 720 is configured to correct the at least one target sequence based on the local reference text corresponding to the target recognition object to obtain an edge recognition result.

[0316] Optionally, the speech recognition device obtains the cloud recognition result through the first acquisition unit 710, and then can correct the at least one target sequence through the first correction unit 720 based on the local reference text corresponding to the target recognition object to obtain the edge recognition result.

[0317] A speech recognition device provided in an embodiment of the present application obtains the cloud-based recognition results of a target recognition object through an edge terminal, and corrects the cloud-based recognition results based on the local reference text corresponding to the target recognition object, thereby optimizing the cloud-based recognition results and improving the accuracy of speech recognition.

[0318] It should be noted that the division of units in the embodiments of the present application is schematic and is merely a logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0319] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0320] Optionally, the first correction unit 720 is configured to:

[0321] Performing vocabulary replacement on an alternative sequence in the target sequence based on a language model corresponding to the target sequence and the local reference text; wherein the language model corresponding to the target sequence is obtained by training based on the local reference text;

[0322] and / or,

[0323] Based on named entity recognition (NER), vocabulary replacement is performed on one of the candidate sequences in the target sequence.

[0324] Optionally, the vocabulary matching probability of the candidate sequence is the highest among the vocabulary matching probabilities of all the target sequences;

[0325] Wherein, for each target sequence, the vocabulary matching probability is calculated based on the language model corresponding to the target sequence and the target sequence;

[0326] The vocabulary matching probability of the target sequence is used to describe the frequency with which the vocabulary in the target sequence appears in the local reference text.

[0327] Optionally, the first correction unit 720 is configured to: obtain, based on a language model corresponding to the target sequence, a continuous matching probability corresponding to a continuous word combination in the candidate sequence, wherein the continuous matching probability is used to represent a frequency of occurrence of the continuous word combination corresponding to the continuous matching probability in the local reference text;

[0328] If the continuous matching probability corresponding to any continuous word combination in the candidate sequence is lower than a first preset threshold, replacing the first continuous word combination with the continuous matching probability lower than the first preset threshold with a replacement text in the local reference text;

[0329] The phoneme matching degree between the phoneme sequence corresponding to the replacement text and the phoneme sequence of the first continuous vocabulary combination is greater than a second preset threshold, and the replacement text appears more frequently in the local reference text than the first continuous vocabulary combination.

[0330] Optionally, the target sequence includes:

[0331] Phoneme sequence, and / or, word sequence;

[0332] Accordingly, the language model corresponding to the phoneme sequence includes a phoneme sequence language model;

[0333] The language model corresponding to the word sequence includes a word sequence language model.

[0334] Optionally, if the target sequence includes a phoneme sequence, the first correction unit 720 is configured to: perform vocabulary replacement on an alternative sequence in the phoneme sequence based on the phoneme sequence language model to obtain a first phoneme sequence;

[0335] Based on the word sequence language model, vocabulary replacement is performed on an alternative sequence in at least one word sequence corresponding to the first phoneme sequence.

[0336] Optionally, if the target sequence further includes a word sequence, the first correction unit 720 is configured to: if the fourth matching probability corresponding to the first word sequence is greater than the fourth matching probability corresponding to the second word sequence, determine that the first word sequence is the edge recognition result;

[0337] If the fourth matching probability corresponding to the second word sequence is greater than the fourth matching probability corresponding to the first word sequence, determining that the second word sequence is the edge recognition result;

[0338] The first word sequence is obtained by performing lexical replacement on an alternative sequence in at least one word sequence corresponding to the first phoneme sequence;

[0339] The second word sequence is obtained by performing vocabulary replacement on an alternative sequence in the word sequence in the target sequence based on the word sequence language model;

[0340] The fourth matching probability corresponding to the first word sequence is used to describe the matching degree between the first word sequence and the local reference text;

[0341] The fourth matching probability corresponding to the second word sequence is used to describe the matching degree between the second word sequence and the local reference text.

[0342] Optionally, the first correction unit 720 is configured to: based on the NER vocabulary, replace a vocabulary in an alternative sequence in the target sequence with a first replacement vocabulary in the NER vocabulary to obtain the edge recognition result;

[0343] The phoneme matching probability between the phoneme sequence corresponding to the replacement vocabulary and the phoneme sequence of the vocabulary in the candidate sequence is greater than a third preset threshold.

[0344] Optionally, it also includes:

[0345] A first generating unit is configured to recognize the local reference text based on the NER and generate a NER vocabulary;

[0346] The third acquisition unit is configured to obtain the phonemes corresponding to the NER vocabulary based on a dictionary and / or character-to-phoneme G2P technology.

[0347] Optionally, the local reference text includes:

[0348] The local reference text contains professional information, participant information, background information, and content of the activity.

[0349] Optionally, it also includes:

[0350] After the edge recognition result is obtained, the relevant information of the local reference text is saved in the server based on the professional information of the activity to which the local reference text belongs.

[0351] Figure 8 This is the fourth structural diagram of the speech recognition device provided in the embodiment of the present application. Figure 8 The speech recognition device includes a second acquisition unit 810 and a second correction unit 820, wherein:

[0352] A second acquisition unit 810 is configured to acquire a cloud-side recognition result or an edge-side recognition result on the device side, wherein the cloud-side recognition result includes a target sequence obtained by the cloud side from recognizing the target object;

[0353] The second correction unit 820 is configured to correct the target sequence based on a local vocabulary on the device side to obtain a device side recognition result.

[0354] Optionally, the speech recognition device obtains the cloud recognition result or the edge recognition result through the second acquisition unit 810, and then can correct the target sequence through the second correction unit 820 based on the local vocabulary of the device to obtain the device-side recognition result.

[0355] A speech recognition device provided in an embodiment of the present application obtains the cloud-based recognition results of a target recognition object through an edge terminal, and corrects the cloud-based recognition results based on the local reference text corresponding to the target recognition object, thereby optimizing the cloud-based recognition results and improving the accuracy of speech recognition.

[0356] Optionally, the second correction unit 820 is configured to:

[0357] replacing a word in the target sequence corresponding to the second replacement word based on the second replacement word in the local word list to obtain the device-side recognition result;

[0358] The phoneme matching probability between the phoneme sequence of the vocabulary corresponding to the second replacement vocabulary and the phoneme sequence of the second replacement vocabulary is greater than a fourth preset threshold.

[0359] Optionally, the vocabulary includes: a preset error-prone vocabulary and / or a real-time NER vocabulary.

[0360] Optionally, it also includes:

[0361] The fourth acquisition module is used to acquire real-time NER vocabulary in the display information corresponding to the target recognition object in real time based on the NER technology.

[0362] It should be noted that the division of units in the embodiments of the present application is schematic and is merely a logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0363] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0364] It should be noted here that the above-mentioned device provided by the embodiment of the present invention can implement all the method steps implemented by the above-mentioned method embodiment and can achieve the same technical effect. The parts and beneficial effects that are the same as the method embodiment in this embodiment will not be described in detail here.

[0365] On the other hand, an embodiment of the present application further provides a processor-readable storage medium, wherein the processor-readable storage medium stores a computer program, wherein the computer program is configured to cause the processor to execute the methods provided in the above embodiments, including:

[0366] The edge end obtains a cloud recognition result, where the cloud recognition result includes at least one target sequence obtained by the cloud for target recognition object recognition;

[0367] Based on the local reference text corresponding to the target recognition object, the at least one target sequence is corrected to obtain an edge recognition result.

[0368] or

[0369] The device side obtains a cloud recognition result or an edge recognition result, wherein the cloud recognition result includes a target sequence obtained by the cloud side from recognizing the target recognition object;

[0370] Based on the local vocabulary on the device side, the target sequence is modified to obtain a device side recognition result.

[0371] The processor-readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO)), optical storage (such as CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (such as ROMs, EPROMs, EEPROMs, non-volatile memories (NANDFLASH), solid-state drives (SSDs)), etc.

[0372] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) that contain computer-usable program code.

[0373] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer-executable instructions. These computer-executable instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0374] These processor-executable instructions may also be stored in a processor-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the processor-readable memory produce an article of manufacture comprising an instruction device that implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0375] These processor-executable instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0376] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A speech recognition method, characterized in that: include: The edge end obtains a cloud recognition result, where the cloud recognition result includes at least one target sequence obtained by the cloud for target recognition object recognition; Based on the local reference text corresponding to the target recognition object, the at least one target sequence is modified to obtain an edge recognition result; The modifying the at least one target sequence based on the local reference text corresponding to the target recognition object to obtain the edge recognition result includes: Performing vocabulary replacement on an alternative sequence in the target sequence based on a language model corresponding to the target sequence and the local reference text; wherein the language model corresponding to the target sequence is obtained by training based on the local reference text; and / or, Based on named entity recognition (NER), vocabulary replacement is performed on one of the candidate sequences in the target sequence.

2. The speech recognition method according to claim 1, wherein: The vocabulary matching probability of the candidate sequence is the highest among the vocabulary matching probabilities of all the target sequences; Wherein, for each target sequence, the vocabulary matching probability is calculated based on the language model corresponding to the target sequence and the target sequence; The vocabulary matching probability of the target sequence is used to describe the frequency with which the vocabulary in the target sequence appears in the local reference text.

3. The speech recognition method according to claim 1, characterized in that The step of performing vocabulary replacement on an alternative sequence in the target sequence based on the language model corresponding to the target sequence and the local reference text includes: Obtaining, based on the language model corresponding to the target sequence, a continuous matching probability corresponding to a continuous vocabulary combination in the candidate sequence, wherein the continuous matching probability is used to represent a frequency of occurrence of the continuous vocabulary combination corresponding to the continuous matching probability in the local reference text; If the continuous matching probability corresponding to any continuous word combination in the candidate sequence is lower than a first preset threshold, replacing the first continuous word combination with the continuous matching probability lower than the first preset threshold with a replacement text in the local reference text; The phoneme matching degree between the phoneme sequence corresponding to the replacement text and the phoneme sequence of the first continuous vocabulary combination is greater than a second preset threshold, and the replacement text appears more frequently in the local reference text than the first continuous vocabulary combination.

4. The speech recognition method according to any one of claims 1 to 3, characterized in that: The target sequence includes: Phoneme sequence, and / or, word sequence; Accordingly, the language model corresponding to the phoneme sequence includes a phoneme sequence language model; The language model corresponding to the word sequence includes a word sequence language model.

5. The speech recognition method according to claim 4, characterized in that If the target sequence includes a phoneme sequence, performing vocabulary replacement on an alternative sequence in the target sequence based on the language model corresponding to the target sequence and the local reference text includes: Based on the phoneme sequence language model, performing vocabulary replacement on an alternative sequence in the phoneme sequence to obtain a first phoneme sequence; Based on the word sequence language model, vocabulary replacement is performed on an alternative sequence in at least one word sequence corresponding to the first phoneme sequence.

6. The speech recognition method according to claim 5, characterized in that If the target sequence also includes a word sequence, performing vocabulary replacement on an alternative sequence in the target sequence based on the language model corresponding to the target sequence and the local reference text further includes: If the fourth matching probability corresponding to the first word sequence is greater than the fourth matching probability corresponding to the second word sequence, determining that the first word sequence is the edge recognition result; If the fourth matching probability corresponding to the second word sequence is greater than the fourth matching probability corresponding to the first word sequence, determining that the second word sequence is the edge recognition result; The first word sequence is obtained by performing lexical replacement on an alternative sequence in at least one word sequence corresponding to the first phoneme sequence; The second word sequence is obtained by performing vocabulary replacement on an alternative sequence in the word sequence in the target sequence based on the word sequence language model; The fourth matching probability corresponding to the first word sequence is used to describe the matching degree between the first word sequence and the local reference text; The fourth matching probability corresponding to the second word sequence is used to describe the matching degree between the second word sequence and the local reference text.

7. The speech recognition method according to claim 1, wherein: The step of performing vocabulary replacement on an alternative sequence in the target sequence based on named entity recognition (NER) to obtain the edge recognition result includes: Based on the NER vocabulary, replacing a vocabulary in an alternative sequence in the target sequence with a first replacement vocabulary in the NER vocabulary to obtain the edge recognition result; The phoneme matching probability between the phoneme sequence corresponding to the replacement vocabulary and the phoneme sequence of the vocabulary in the candidate sequence is greater than a third preset threshold.

8. The speech recognition method according to claim 7, characterized in that: The method further comprises: Recognize the local reference text based on the NER and generate a NER vocabulary; Based on a dictionary and / or character-to-phoneme G2P technology, the phonemes corresponding to the NER vocabulary are obtained.

9. The speech recognition method according to any one of claims 1 to 3 or any one of claims 5 to 8, characterized in that: The local reference texts include: The local reference text contains professional information, participant information, background information, and content of the activity.

10. The speech recognition method according to any one of claims 1 to 3 or any one of claims 5 to 8, characterized in that: After obtaining the edge recognition result, the method further includes: Based on the professional information of the activity to which the local reference text belongs, relevant information of the local reference text is saved in a server.

11. A speech recognition method, characterized in that: include: The device obtains a cloud recognition result or an edge recognition result, wherein the cloud recognition result includes at least one target sequence obtained by the cloud for identifying the target object; the edge recognition result is a recognition result obtained after the edge corrects the cloud recognition result; Based on a local vocabulary on the device, modify the at least one target sequence to obtain a device-side recognition result; The modifying of the at least one target sequence based on the local vocabulary of the device to obtain a device-side recognition result includes: replacing a word in the target sequence corresponding to the second replacement word based on the second replacement word in the local word list to obtain the device-side recognition result; The phoneme matching probability between the phoneme sequence of the vocabulary corresponding to the second replacement vocabulary and the phoneme sequence of the second replacement vocabulary is greater than a fourth preset threshold.

12. The speech recognition method according to claim 11, characterized in that: The vocabulary includes: a pre-set error-prone vocabulary and / or a real-time NER vocabulary.

13. The speech recognition method according to claim 12, wherein: The method further includes: acquiring real-time NER vocabulary in the display information corresponding to the target recognition object in real time based on NER technology.

14. A speech recognition device comprising a memory, a transceiver, and a processor: memory for storing computer programs; a transceiver, configured to transmit and receive data under the control of the processor; A processor, configured to read the computer program in the memory and execute the steps of the speech recognition method according to any one of claims 1 to 10.

15. A speech recognition device comprising a memory, a transceiver, and a processor: memory for storing computer programs; a transceiver, configured to transmit and receive data under the control of the processor; A processor is configured to read the computer program in the memory and execute the steps of the speech recognition method according to any one of claims 11 to 13.

16. A speech recognition device, characterized in that: include: A first acquisition unit is configured to acquire, at the edge end, a cloud recognition result, wherein the cloud recognition result includes at least one target sequence obtained by the cloud for identifying a target recognition object; A first correction unit is configured to correct the at least one target sequence based on a local reference text corresponding to the target recognition object to obtain an edge recognition result; The modifying the at least one target sequence based on the local reference text corresponding to the target recognition object to obtain the edge recognition result includes: Performing vocabulary replacement on an alternative sequence in the target sequence based on a language model corresponding to the target sequence and the local reference text; wherein the language model corresponding to the target sequence is obtained by training based on the local reference text; and / or, Based on named entity recognition (NER), vocabulary replacement is performed on one of the candidate sequences in the target sequence.

17. A speech recognition device, characterized in that: include: A second acquisition unit is configured to acquire, on the device side, a cloud recognition result or an edge recognition result, wherein the cloud recognition result includes at least one target sequence obtained by the cloud side from recognizing the target object; and the edge recognition result is a recognition result obtained after the edge side corrects the cloud recognition result; A second correction unit, configured to correct the at least one target sequence based on a local vocabulary on the device to obtain a device-side recognition result; The modifying of the at least one target sequence based on the local vocabulary of the device to obtain a device-side recognition result includes: replacing a word in the target sequence corresponding to the second replacement word based on the second replacement word in the local word list to obtain the device-side recognition result; The phoneme matching probability between the phoneme sequence of the vocabulary corresponding to the second replacement vocabulary and the phoneme sequence of the second replacement vocabulary is greater than a fourth preset threshold.

18. A processor-readable storage medium, characterized in that: The processor-readable storage medium stores a computer program, and the computer program is configured to cause the processor to execute the method according to any one of claims 1 to 13.

Images