Speech model-based vocabulary recognition method and device, electronic equipment and medium

Abstract

Embodiments of the present application provide a word recognition method and device based on a speech model, electronic equipment and a medium, belonging to the technical field of financial technology. The method comprises: obtaining to-be-recognized audio and target audio; performing text conversion on the target audio to obtain hot word text corresponding to the target hot word audio; obtaining a prefix sequence of a speech model, and inputting the to-be-recognized audio and the target audio into the speech model to perform feature splicing, and outputting a spliced feature sequence; adding the target hot word audio to the spliced feature sequence to obtain a target audio sequence, and adding the hot word text to the prefix sequence to obtain a target prefix sequence; inputting the target audio sequence and the target prefix sequence into the speech model to perform hot word recognition on the to-be-recognized audio, and outputting recognized hot word audio; and performing text conversion on the to-be-recognized audio and the recognized hot word audio to obtain target audio text. The embodiments of the present application can realize accurate recognition of hot words and improve the accuracy of hot word recognition.

Description

Technical Field

[0001] The present application relates to the field of financial technology, and in particular to a vocabulary recognition method, device, electronic device and medium based on a speech model. Background Art

[0002] Speech recognition is one of the most promising technologies of the 21st century, with widespread applications across multiple fields, including artificial intelligence, natural language processing, and machine translation. With the continuous advancement of artificial intelligence, speech recognition technology has also made significant progress. Traditional speech recognition relies primarily on manual input, a process that is inefficient and costly. However, large speech recognition models, trained on vast amounts of internet speech data, are capable of natural speech recognition in multiple languages, significantly improving the efficiency and cost of speech recognition.

[0003] Hotword technology for large speech recognition models further optimizes specific vocabulary based on existing pre-trained models, improving the model's recognition accuracy for certain proper nouns (such as place names, personal names, and product names). Current large speech recognition models often misidentify other words while optimizing specific vocabulary, resulting in lower hotword recognition accuracy. Summary of the Invention

[0004] The main purpose of the embodiments of the present application is to propose a vocabulary recognition method, device, electronic device and medium based on a speech model, which can achieve accurate recognition of hot words and improve the accuracy of hot word recognition.

[0005] To achieve the above objectives, a first aspect of an embodiment of the present application provides a vocabulary recognition method based on a speech model, the method comprising:

[0006] Acquire the audio to be recognized and the target audio, wherein the target audio includes at least one target hot word audio;

[0007] Performing text conversion on the target audio to obtain a hot word text corresponding to the target hot word audio;

[0008] Obtaining a prefix sequence of the speech model, inputting the audio to be recognized and the target audio into the speech model for feature splicing, and outputting a spliced ​​feature sequence;

[0009] Add the target hot word audio to the concatenated feature sequence to obtain a target audio sequence, and add the hot word text to the prefix sequence to obtain a target prefix sequence;

[0010] Inputting the target audio sequence and the target prefix sequence into the speech model, so that the speech model performs hot word recognition on the audio to be recognized based on the target audio sequence and the target prefix sequence, and outputs the recognized hot word audio;

[0011] The audio to be recognized and the recognized hot word audio are converted into text to obtain a target audio text.

[0012] In some embodiments, the target audio is obtained by the following steps:

[0013] Obtaining artificial audio and target hot word vocabulary, and counting the number of vocabulary of the target hot word vocabulary, wherein the artificial audio is obtained by manual recording;

[0014] Performing an audio quality analysis on the artificial audio to obtain an audio quality score, wherein the audio quality score is used to characterize the sound quality and fluency of the artificial audio;

[0015] When the number of the vocabulary is less than a preset number and the audio quality score is greater than or equal to a first preset score, using the artificial audio as the target audio;

[0016] When the number of words is greater than or equal to a preset number or the audio quality score is less than a first preset score, audio synthesis is performed on the target hot word vocabulary through a preset deep learning model to obtain target audio.

[0017] In some embodiments, performing audio quality analysis on the artificial audio to obtain an audio quality score includes:

[0018] Performing time-frequency transformation on the artificial audio, and extracting features from the artificial audio after the time domain transformation to obtain artificial audio features;

[0019] Performing sound quality detection on the artificial audio based on the artificial audio feature;

[0020] When the artificial audio contains noise, determining a second preset score as the audio quality score, wherein the second preset score is less than the first preset score;

[0021] When there is no noise in the artificial audio, clarity detection is performed on the artificial audio based on the artificial audio features to obtain an audio quality score.

[0022] In some embodiments, synthesizing the target hot word vocabulary through a preset deep learning model to obtain the target audio includes:

[0023] Inputting the target hot word vocabulary into a preset deep learning model, so that the deep learning model performs phoneme analysis on the target hot word vocabulary and outputs a spectrogram;

[0024] Performing audio conversion on the spectrogram using a preset vocoder to obtain an audio waveform;

[0025] Generate target audio according to the audio waveform.

[0026] In some embodiments, converting the target audio into text to obtain a hot word text corresponding to the target hot word audio includes:

[0027] Extracting features of the target audio to obtain target audio features corresponding to the target audio;

[0028] Performing feature mapping on the target audio feature using a preset acoustic model to obtain a target candidate sequence of the target audio feature;

[0029] Inputting the target candidate sequence into a preset language model, so that the language model analyzes the context of the target audio based on the target candidate sequence and outputs a target text sequence;

[0030] The target text sequence is decoded to obtain a hot word text corresponding to the target hot word audio.

[0031] In some embodiments, inputting the to-be-recognized audio and the target audio into a speech model for feature splicing and outputting a spliced ​​feature sequence includes:

[0032] Extracting a first splicing feature sequence of the audio to be recognized and a second splicing feature sequence of the target audio through the speech model;

[0033] Feature splicing is performed on the first splicing feature sequence and the second splicing feature sequence to obtain a splicing feature sequence.

[0034] In some embodiments, inputting the to-be-recognized audio and the target audio into a speech model for feature splicing and outputting a spliced ​​feature sequence further includes:

[0035] Performing audio splicing on the audio to be recognized and the target audio to obtain a spliced ​​audio;

[0036] An audio feature sequence of the spliced ​​audio is extracted using the speech model.

[0037] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a vocabulary recognition device based on a speech model, the device comprising:

[0038] An audio acquisition module, configured to acquire audio to be recognized and target audio, wherein the target audio includes at least one target hot word audio;

[0039] A text conversion module, configured to perform text conversion on the target audio to obtain a hot word text corresponding to the target hot word audio;

[0040] A feature splicing module is used to obtain a prefix sequence of the speech model, input the audio to be recognized and the target audio into the speech model for feature splicing, and output a spliced ​​feature sequence;

[0041] A sequence splicing module, configured to add the target hot word audio to the splicing feature sequence to obtain a target audio sequence, and to add the hot word text to the prefix sequence to obtain a target prefix sequence;

[0042] a hot word recognition module, configured to input the target audio sequence and the target prefix sequence into the speech model, so that the speech model performs hot word recognition on the audio to be recognized based on the target audio sequence and the target prefix sequence, and outputs recognized hot word audio;

[0043] The target conversion module is used to perform text conversion on the audio to be recognized and the recognized hot word audio to obtain a target audio text.

[0044] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the speech model-based vocabulary recognition method as described in the first aspect.

[0045] To achieve the above-mentioned purpose, the fourth aspect of the embodiment of the present application proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the vocabulary recognition method based on the speech model as described in the first aspect.

[0046] The present application proposes a speech model-based vocabulary recognition method, device, electronic device and storage medium. First, the audio to be recognized and the target audio are obtained, and the target audio includes at least one target hot word audio, that is, the audio corresponding to the hot word that needs to be recognized. Then, the target audio is converted into text to obtain the hot word text corresponding to the target hot word audio, and the textual display of the target hot word audio is realized, thereby improving the accuracy of information recognition. After that, the prefix sequence of the speech model is obtained to facilitate the subsequent decoding of the audio by the speech model, and the audio to be recognized and the target audio are input into the speech model for feature splicing, and the spliced ​​feature sequence is output, thereby improving the recognition accuracy of the speech, eliminating irrelevant or redundant features, reducing the complexity of the model, and avoiding The risk of overfitting, the target hot word audio is added to the splicing feature sequence to obtain the target audio sequence, and the hot word text is added to the prefix sequence to obtain the target prefix sequence, so that the target audio sequence and the target prefix sequence can be combined with the hot word and the hot word text corresponding to the hot word, adding the hot word prefix, and then the target audio sequence and the target prefix sequence are input into the speech model, so that the speech model performs hot word recognition on the audio to be recognized based on the target audio sequence and the target prefix sequence, outputs the recognized hot word audio, realizes accurate recognition of the audio corresponding to the hot word, and improves the recognition accuracy of the hot word audio, and finally, the audio to be recognized and the recognized hot word audio are converted into text to obtain the target audio text, thereby improving the hot word recognition accuracy in the target audio text. The embodiment of the present application utilizes the contextual reasoning ability and autoregressive generation mechanism in the speech model, and adds the target hot word audio to the splicing feature sequence during the model processing, and adds the hot word text to the prefix sequence to increase the hot word prefix, thereby improving the hot word recognition accuracy of the speech model by combining the input hot word audio sequence with the text, and realizing accurate recognition of the hot word. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a flowchart of a method for vocabulary recognition based on a speech model provided by one embodiment of the present application;

[0048] Figure 2 This is a flowchart of obtaining target audio provided by an embodiment of the present application;

[0049] Figure 3 yes Figure 2 Flowchart of step S202 in FIG.

[0050] Figure 4 yes Figure 2 Flowchart of step S204 in FIG.

[0051] Figure 5 yes Figure 1 Flowchart of step S102 in FIG.

[0052] Figure 6A flow chart of a method for inputting the audio to be recognized and the target audio into a speech model for feature splicing provided in an embodiment of the present application;

[0053] Figure 7 A flowchart of a method for inputting the audio to be recognized and the target audio into a speech model for feature splicing provided in another embodiment of the present application;

[0054] Figure 8 Schematic diagram of the structure of a speech model-based vocabulary recognition device provided in an embodiment of the present application;

[0055] Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0057] It should be noted that although the device schematics illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the device or the sequence in the flowcharts. The terms "first," "second," and so on, in the specification, claims, and drawings, are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0059] First, let’s analyze some of the terms used in this application:

[0060] Natural Language Processing (NLP): NLP uses computers to process, understand, and apply human languages ​​(such as Chinese and English). NLP is a branch of artificial intelligence and an interdisciplinary subject between computer science and linguistics. It is often referred to as computational linguistics. Natural language processing includes grammatical analysis, semantic analysis, and text understanding. Natural language processing is commonly used in technical fields such as machine translation, handwritten and printed character recognition, speech recognition and text-to-speech conversion, information intent recognition, information extraction and filtering, text classification and clustering, public opinion analysis, and opinion mining. It involves data mining related to language processing, machine learning, knowledge acquisition, knowledge engineering, artificial intelligence research, and linguistic research related to language computing.

[0061] Text-to-Speech (TTS): TTS technology refers to the technology of converting text information into audible speech, widely used in speech synthesis, voice assistants, e-readers, etc.

[0062] Hidden Markov Model (HMM): Hidden Markov Model is a statistical model that describes a Markov process with hidden unknown parameters. It is widely used in speech recognition, natural language processing, etc.

[0063] Fast Fourier Transform (FFT): FFT is an algorithm for efficiently computing the discrete Fourier transform and its inverse, widely used in signal processing, image analysis, audio processing, etc.

[0064] Short Time Fourier Transform (STFT): STFT is a method of decomposing a signal into different frequency components over time, especially suitable for non-stationary signal spectrum analysis, and is an important tool in speech signal processing.

[0065] Based on this, the embodiments of the present application provide a speech model-based word recognition method and device, electronic equipment and storage medium, which can accurately identify hot words and improve the accuracy of hot word recognition.

[0066] The speech model-based word recognition method and device, electronic equipment and storage medium provided by the embodiments of the present application are specifically described by the following embodiments. First, the speech model-based word recognition method in the embodiments of the present application is described.

[0067] The embodiments of the present application can acquire and process related data based on artificial intelligence technology. Artificial intelligence (AI) is the use of digital computers or computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0068] Artificial intelligence basic technologies generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, robot technology, biometric technology, module management technology of online customer service system, natural language processing technology and machine learning / deep learning, etc.

[0069] The speech model-based vocabulary recognition method provided in the embodiment of the present application relates to the field of financial technology. The speech model-based vocabulary recognition method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or as a server cluster or distributed system composed of multiple physical servers, or as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements a speech model-based vocabulary recognition method, etc., but is not limited to the above forms.

[0070] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0071] It should be noted that in each specific embodiment of the present application, when it comes to the need to perform relevant processing based on data related to the user's identity or characteristics, such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of such data will comply with the relevant laws, regulations, and standards of the relevant countries and regions. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0072] Speech recognition is one of the most promising technologies of the 21st century, with widespread applications across multiple fields, including artificial intelligence, natural language processing, and machine translation. With the continuous advancement of artificial intelligence, speech recognition technology has also made significant progress. Traditional speech recognition relies primarily on manual input, a process that is inefficient and costly. However, large speech recognition models, trained on vast amounts of internet speech data, are capable of natural speech recognition in multiple languages, significantly improving the efficiency and cost of speech recognition.

[0073] Hotword technology for large speech recognition models further optimizes specific vocabulary based on existing pre-trained models, improving the model's recognition accuracy for certain proper nouns (such as place names, personal names, and product names). Current large speech recognition models often misidentify other words while optimizing specific vocabulary, resulting in lower hotword recognition accuracy.

[0074] In order to solve the above problems, the present embodiment provides a vocabulary recognition method, device, electronic device and storage medium based on a speech model. First, the audio to be recognized and the target audio are obtained, and the target audio includes at least one target hot word audio, that is, the audio corresponding to the hot word that needs to be recognized. Then, the target audio is converted into text to obtain the hot word text corresponding to the target hot word audio, and the textual display of the target hot word audio is realized, thereby improving the accuracy of information recognition. Afterwards, the prefix sequence of the speech model is obtained to facilitate the subsequent decoding of the audio by the speech model, and the audio to be recognized and the target audio are input into the speech model for feature splicing, and the spliced ​​feature sequence is output, thereby improving the recognition accuracy of the speech, eliminating irrelevant or redundant features, and reducing the complexity of the model. clutter, avoid the risk of overfitting, add the target hot word audio to the splicing feature sequence to obtain the target audio sequence, and add the hot word text to the prefix sequence to obtain the target prefix sequence, so that the target audio sequence and the target prefix sequence can be combined with the hot word and the hot word text corresponding to the hot word, adding the hot word prefix, and then input the target audio sequence and the target prefix sequence into the speech model, so that the speech model performs hot word recognition on the audio to be recognized based on the target audio sequence and the target prefix sequence, outputs the recognized hot word audio, realizes accurate recognition of the audio corresponding to the hot word, and improves the recognition accuracy of the hot word audio, and finally, performs text conversion on the audio to be recognized and the recognized hot word audio to obtain the target audio text, thereby improving the hot word recognition accuracy in the target audio text. The embodiment of the present application utilizes the contextual reasoning ability and autoregressive generation mechanism in the speech model, adds the target hot word audio to the splicing feature sequence during the model processing, and adds the hot word text to the prefix sequence to increase the hot word prefix, thereby improving the hot word recognition accuracy of the speech model by combining the input hot word audio sequence with the text, and realizing accurate recognition of the hot word.

[0075] The specific description is made below with reference to the drawings.

[0076] Figure 1 is an optional flowchart of a word recognition method based on a speech model provided by the embodiment of the present application, Figure 1 The method in the embodiment can include, but is not limited to, steps S101 to S106.

[0077] In step S101, the to-be-recognized audio and the target audio are obtained, wherein the target audio includes at least one target hotword audio.

[0078] In step S101 of some embodiments, the to-be-recognized audio and the target audio are obtained, wherein the target audio includes at least one target hotword audio. Specifically, the target hotword audio can be a place name, a person name, a company name, etc., facilitating subsequent recognition of a hotword corresponding to the target hotword audio in the to-be-recognized audio.

[0079] It can be understood that the to-be-recognized audio in the embodiment of the present application can be obtained by artificial recording or extracted from an open source database, and the present application does not make specific limitations.

[0080] In step S102, the target audio is subjected to text conversion to obtain hotword text corresponding to the target hotword audio.

[0081] In step S102 of some embodiments, the target audio is subjected to text conversion, the context relationship of words in a sentence is predicted, the accuracy of recognition is improved through context information, and hotword text corresponding to the target hotword audio is obtained. Through the automatic speech-to-text operation, the time of manual input can be greatly saved, and the hotword text can be used as a prefix of a prefix sequence in the subsequent process.

[0082] In step S103, the prefix sequence of the speech model is obtained, and the to-be-recognized audio and the target audio are input into the speech model for feature splicing to output a spliced feature sequence.

[0083] In step S103 of some embodiments, the prefix sequence of the speech model is obtained, wherein the prefix sequence refers to prior text information used by the model during decoding, which is used to initialize the decoding process of the model. The to-be-recognized audio and the target audio are input into the speech model for feature splicing to output a spliced feature sequence. By splicing multiple feature sequences, a more comprehensive feature sequence can be constructed, which can capture complex patterns and subtle differences in the audio, thereby improving the performance of the audio processing task.

[0084] It should be noted that the speech model in the embodiment of the present application is a Whisper model.

[0085] It is understandable that the prefix sequence contains special tokens, such as task type (translation or transcription), language tag, and timestamp.

[0086] Step S104: Add the target hot word audio to the concatenated feature sequence to obtain a target audio sequence, and add the hot word text to the prefix sequence to obtain a target prefix sequence.

[0087] In step S104 of some embodiments, the target hot word audio is added to the splicing feature sequence to obtain a target audio sequence, and the hot word text is added to the prefix sequence to obtain a target prefix sequence, so that the target audio sequence and the target prefix sequence can be combined with the hot word and the hot word text corresponding to the hot word, and the hot word prefix is ​​added to facilitate subsequent recognition of the hot word.

[0088] It is worth noting that compared with the method of directly adding hot words in the prefix sequence Prompt, the embodiment of the present application combines the audio information corresponding to the hot words, avoiding the problem of misrecognition of other words caused by introducing hot words in Prompt; in addition, compared with the method of jointly training the hot word model and the large speech model, the embodiment of the present application does not need to train the model, thereby speeding up the efficiency of hot word recognition and reducing the cost of model training.

[0089] Step S105: input the target audio sequence and the target prefix sequence into the speech model, so that the speech model performs hot word recognition on the audio to be recognized based on the target audio sequence and the target prefix sequence, and outputs the recognized hot word audio.

[0090] In step S105 of some embodiments, the target audio sequence and the target prefix sequence are input into the speech model, so that the speech model performs hot word recognition on the audio to be recognized based on the target audio sequence and the target prefix sequence, thereby improving the hot word recognition accuracy of the speech recognition model by combining the input hot word audio sequence with the text, outputting the recognized hot word audio, and improving the accuracy of the hot word audio recognition.

[0091] Step S106: convert the audio to be recognized and the recognized hot word audio into text to obtain the target audio text.

[0092] In step S106 of some embodiments, the audio to be recognized and the recognized hot word audio are converted into text, and the recognized hot word audio in the audio to be recognized is recognized during the text conversion process to obtain the target audio text.

[0093] In some embodiments, the embodiments of the present application only need to replace the target hot word audio and hot word text when updating hot words, without the need to retrain the voice model, thereby achieving rapid iterative updates of hot words and reducing the cost of iterative updates of hot words.

[0094] See also Figure 2 , Figure 2 This is a flowchart of obtaining target audio provided by an embodiment of the present application. Figure 2 The method may include but is not limited to steps S201 to S204.

[0095] Step S201: obtain artificial audio and target hot word vocabulary, and count the vocabulary number of the target hot word vocabulary.

[0096] It should be noted that artificial audio is obtained by manual recording.

[0097] Step S202: Perform audio quality analysis on the artificial audio to obtain an audio quality score.

[0098] It should be noted that the audio quality score is used to characterize the sound quality and fluency of artificial audio.

[0099] Step S203: When the number of words is less than a preset number and the audio quality score is greater than or equal to a first preset score, the artificial audio is used as the target audio.

[0100] In some embodiments, in steps S201 to S203, in the process of obtaining the target audio, the embodiment of the present application will first obtain artificial audio and target hot word vocabulary, wherein the artificial audio is obtained by artificial recording, the target hot word vocabulary is a hot word set or marked in advance, and the vocabulary number of the target hot word vocabulary is counted. Specifically, the embodiment of the present application can first classify the target hot word vocabulary to obtain a hot word classification, and then for each hot word classification, count the number of hot words under each hot word classification to obtain the target hot word vocabulary number, and then perform audio quality analysis on the artificial audio to detect whether the artificial audio has noise, whether it is smooth or clear, and obtain an audio quality score to achieve a quality assessment of the artificial audio, which is convenient for subsequent selection of whether to select the artificial audio as the target audio. Then, the vocabulary number is compared with the preset number, and the audio quality score is compared with the first preset score. When the vocabulary number is less than the preset number and the audio quality score is greater than or equal to the first preset score, it means that the number of target hot word vocabulary is small and the artificial audio is relatively clear, without noise or unclearness. The artificial audio can be directly used as the target audio, thereby ensuring the quality and clarity of the target audio.

[0101] Step S204: When the number of words is greater than or equal to the preset number or the audio quality score is less than the first preset score, audio synthesis is performed on the target hot word vocabulary through a preset deep learning model to obtain the target audio.

[0102] In step S204 of some embodiments, when the number of words is greater than or equal to the preset number or the audio quality score is less than the first preset score, it means that the number of target hot words is large, or there is a lot of noise in the artificial audio, the audio is unclear, not smooth, etc. At this time, it is necessary to use a preset deep learning model to perform audio synthesis on the target hot words to obtain the target audio, so as to ensure the integrity of the target audio, avoid the loss of the target hot words, and improve the quality of the target audio.

[0103] It should be noted that the preset number in the embodiment of the present application can be set according to the needs of the user, for example, the preset number is set to 10, 20 or 15, etc. The first preset score is related to the audio quality and can be set according to the needs of the user, for example, the first preset score is set to 70 points, 80 points, 90 points, etc. The embodiment of the present application does not impose any specific restrictions.

[0104] See also Figure 3 In some embodiments, step S202 may also include but is not limited to steps S301 to S304.

[0105] Step S301: perform time-frequency transformation on the artificial audio, and perform feature extraction on the artificial audio after the time domain transformation to obtain artificial audio features.

[0106] Step S302: Perform sound quality detection on the artificial audio based on the artificial audio features.

[0107] Step S303: When the artificial audio contains noise, the second preset score is determined as the audio quality score.

[0108] It should be noted that the second preset score is smaller than the first preset score.

[0109] In some embodiments, during the audio quality analysis of artificial audio in steps S301 to S303, embodiments of the present application first perform a time-frequency transform on the artificial audio, such as a Fourier transform or a short-time Fourier transform, to convert the audio signal from the time domain to the frequency domain to facilitate analysis of its frequency components. Feature extraction is then performed on the artificial audio after the time domain transform. In the frequency domain, noise is identified by extracting features of the audio signal to obtain artificial audio features. Artificial audio features include, but are not limited to, signal intensity, frequency distribution, cepstrum features, harmonic features, and long-term information. Sound quality detection is then performed on the artificial audio based on the artificial audio features. Specifically, the extracted features are used to determine whether the signal contains noise by setting a threshold, using a statistical model, or using machine learning methods. For example, noise can be determined by comparing the signal's power spectrum with a threshold, reducing manual intervention and improving detection efficiency. If noise is present in the artificial audio, a second preset score is directly determined as the audio quality score, and the second preset score is less than the first preset score, thereby ensuring that the target audio is free of noise.

[0110] In some embodiments, during the process of performing sound quality testing on artificial audio based on artificial audio features, embodiments of the present application analyze the extracted artificial audio features, for example, through statistical analysis, pattern recognition, etc., to identify specific patterns or anomalies in the audio and obtain analysis results. The audio quality is then classified based on the analysis results to determine whether the artificial audio is subject to noise interference.

[0111] Step S304: When there is no noise in the artificial audio, the clarity of the artificial audio is detected based on the artificial audio features to obtain an audio quality score.

[0112] In step S304 of some embodiments, when there is no noise in the artificial audio, it means that the artificial audio at this time has no noise interference and is relatively pure. The clarity of the artificial audio can be directly detected based on the artificial audio features to obtain an audio quality score, which is convenient for subsequent determination of clear and clean target audio and further improves the accuracy of hot word recognition.

[0113] See also Figure 4 In some embodiments, step S204 may also include but is not limited to steps S401 to S403.

[0114] In step S401, the target hot word vocabulary is input into a preset deep learning model, so that the deep learning model performs phoneme analysis on the target hot word vocabulary and outputs a spectrogram.

[0115] Step S402: convert the spectrogram into audio using a preset vocoder to obtain an audio waveform.

[0116] Step S403: Generate target audio according to the audio waveform.

[0117] In some embodiments, in steps S401 to S403, during the process of synthesizing the target hotword vocabulary using a preset deep learning model, the target hotword vocabulary is first input into the preset deep learning model so that the deep learning model performs phoneme analysis on the target hotword vocabulary, decomposes the target hotword vocabulary into phonemes, and determines appropriate intonation patterns and pause positions in the sentence. By marking intonation and pauses, the synthesized speech sounds more natural and closer to the way humans speak, thereby avoiding pronunciation errors. A spectrogram is output, wherein the spectrogram contains frequency and amplitude information of the speech. The spectrogram provides details of the synthesized speech, facilitating the subsequent generation of clear and understandable speech. Subsequently, the spectrogram is converted to audio using a preset vocoder to obtain an audio waveform, thereby ensuring that the generated audio waveform is clear and can quickly generate an audio waveform. Finally, the audio waveform output by the vocoder is synthesized to generate a final audio file, i.e., the target audio. Through feature extraction and vocoder processing, high-quality speech can be generated, improving the intelligibility and naturalness of the speech.

[0118] It should be noted that the deep learning model in the embodiment of the present application can be a text-to-speech synthesis model, Google's Tacotron, DeepMind's WaveNet or open source ChatTTS, etc. The deep learning model in the embodiment of the present application is a text-to-speech synthesis model.

[0119] See also Figure 5 In some embodiments, step S102 may also include but is not limited to steps S501 to S504.

[0120] Step S501: extract features of the target audio to obtain target audio features corresponding to the target audio.

[0121] Step S502 : Feature mapping is performed on the target audio features using a preset acoustic model to obtain a target candidate sequence of the target audio features.

[0122] Step S503: input the target candidate sequence into a preset language model, so that the language model analyzes the context of the target audio based on the target candidate sequence and outputs a target text sequence.

[0123] Step S504: decoding the target text sequence to obtain the hot word text corresponding to the target hot word audio.

[0124] In steps S501 to S504 of some embodiments, during the text conversion of the target audio, feature extraction is performed on the to-be-recognized audio and the target audio. Specifically, the target audio is first preprocessed, including denoising, segmentation, silence removal, etc., to improve the quality of the speech signal, making the speech clearer and improving the recognition accuracy by removing background noise and unnecessary silent parts. Then, features are extracted from the preprocessed target audio, such as mel-frequency cepstral coefficients, to obtain target audio features corresponding to the target audio. Then, the target audio features are mapped through a preset acoustic model. Specifically, the audio is resampled to ensure that its sampling rate is 16 kHz, and the acoustic pattern in the speech signal is recognized, corresponding to a specific phoneme or word. Then, the target candidate sequence of the target audio features is generated according to the phonemes or words. Then, the target candidate sequence is input into a preset language model to enable the language model to analyze the context relationship of the target audio based on the target candidate sequence, predict the context relationship of the words in the sentence, adjust the sequence according to the language rules and statistical information, and output the target text sequence. The context information improves the accuracy of recognition. Then, the first text sequence is decoded to decode the output of the acoustic model and the language model into readable text to obtain the audio text of the to-be-recognized audio. The target text sequence is decoded to decode the output of the acoustic model and the language model into readable text to obtain the hotword text corresponding to the target hotword audio. Through the automatic speech-to-text operation, the time of manual input can be greatly saved, and the hotword text can be used as a prefix of the prefix sequence in the subsequent process.

[0125] It should be noted that the acoustic model can be a hidden Markov model, a deep learning model, a Whisper model, etc., and the embodiments of the present application do not make specific limitations.

[0126] Please refer to Figure 6 , Figure 6 The method flowchart provided by the embodiments of the present application for inputting the to-be-recognized audio and the target audio into the speech model for feature splicing includes but is not limited to steps S601 to S602.

[0127] Step S601: Extracting a first splicing feature sequence of the to-be-recognized audio and a second splicing feature sequence of the target audio through the speech model.

[0128] Step S602: Splicing the first splicing feature sequence and the second splicing feature sequence to obtain a splicing feature sequence.

[0129] In steps S601 to S602 of some embodiments, in the process of inputting the audio to be recognized and the target audio into the speech model for feature splicing, the first splicing feature sequence of the audio to be recognized and the second splicing feature sequence of the target audio are first extracted through the speech model. Specifically, the embodiment of the present application takes the speech model as the Whisper model as an example. First, the audio to be recognized and the target audio are resampled by the Whisper model to ensure that its sampling rate is 16kHz, and then the resampled audio signal is converted into a Mel spectrogram. The Mel spectrogram used by Whisper is calculated by short-time Fourier transform (STFT), and the spectrogram has 80 frequency bands. Afterwards, a first splicing feature sequence and a second splicing feature sequence are obtained according to the Mel spectrogram. After obtaining the first splicing feature sequence and the second splicing feature sequence, the first splicing feature sequence and the second splicing feature sequence are feature spliced ​​to obtain a splicing feature sequence. By splicing multiple feature sequences, a more comprehensive feature sequence can be constructed, which can capture complex patterns and subtle differences in the audio, thereby improving the performance of the audio processing task.

[0130] See also Figure 7 , Figure 7 A flowchart of a method for inputting the audio to be recognized and the target audio into a speech model for feature splicing is provided in another embodiment of the present application. The method includes but is not limited to steps S701 to S702.

[0131] Step S701: splice the audio to be recognized and the target audio to obtain spliced ​​audio.

[0132] Step S702: extracting an audio feature sequence of the spliced ​​audio using a speech model.

[0133] In steps S701 to S702 of some embodiments, in the process of inputting the audio to be recognized and the target audio into the speech model for feature splicing, the embodiments of the present application can also first perform audio splicing on the audio to be recognized and the target audio to obtain spliced ​​audio, thereby reducing the volume of the original data, and then extract the audio feature sequence of the spliced ​​audio through the speech model, thereby eliminating irrelevant or redundant features, reducing the complexity of the model, avoiding the risk of overfitting, and improving the training efficiency and prediction performance of the model.

[0134] See also Figure 8 The present application also provides a speech model-based vocabulary recognition device, comprising:

[0135] The audio acquisition module 801 is used to acquire the audio to be recognized and the target audio, wherein the target audio includes at least one target hot word audio;

[0136] A text conversion module 802 is configured to perform text conversion on the target audio to obtain a hot word text corresponding to the target hot word audio;

[0137] The feature splicing module 803 is used to obtain a prefix sequence of the speech model, input the audio to be recognized and the target audio into the speech model for feature splicing, and output a spliced ​​feature sequence;

[0138] A sequence splicing module 804 is configured to add the target hot word audio to the splicing feature sequence to obtain a target audio sequence, and to add the hot word text to the prefix sequence to obtain a target prefix sequence;

[0139] Hot word recognition module 805 is used to input the target audio sequence and the target prefix sequence into the speech model, so that the speech model performs hot word recognition on the audio to be recognized based on the target audio sequence and the target prefix sequence, and outputs the recognized hot word audio;

[0140] The target conversion module 806 is used to convert the audio to be recognized and the recognized hot word audio into text to obtain the target audio text.

[0141] The specific implementation of the speech model-based vocabulary recognition device is substantially the same as the specific embodiment of the speech model-based vocabulary recognition method described above, and will not be described in detail here.

[0142] In order to explain the above-mentioned speech model-based vocabulary recognition method, device, electronic device and medium more clearly and concisely, specific examples are given below for illustration.

[0143] Example 1:

[0144] Taking the recognition of the target audio "I went to Jialian Payment Building today." as an example, the target hot word in the audio is "Jialian Payment Building". When the hot word technology of the embodiment of the present application is not used and the model is directly input for transcription, it is easy to transcribe it into "I went to Jialian Zhifu Building today" or "I went to Jialian Payment Building today" and other incorrect results.

[0145] The specific steps for recognizing the audio in this embodiment of the application are as follows:

[0146] Step 1: Use manual recording or TTS model to synthesize the target hot word audio corresponding to "Jia Lian Payment Building".

[0147] Step 2: Extract the target hot word audio and the audio feature sequence of the target audio separately and splice them together (or splice the target hot word audio and the target audio first and then merge them to extract the audio feature sequence, both methods are acceptable), input them into the model encoder, and add the target hot word text after the model prefix sequence text to obtain a prefix sequence in the style of "<|startoftranscript|><|zh|><|transcribe|><|notimestamps|>JiaLian Payment Building", and then convert it into a token that can be processed by the model through the tokenizer and input it into the model decoder.

[0148] Step 3: The model performs autoregressive iterative decoding and gradually generates information such as "<|startoftranscript|><|zh|><|transcribe|><|notimestamps|>JiaLian Payment Building I", "<|startoftranscript|><|zh|><|transcribe|><|notimestamps|>JiaLian Payment Building I today", and "<|startoftranscript|><|zh|><|transcribe|><|notimestamps|>JiaLian Payment Building I went to JiaLian Payment Building today." based on the prefix sequence.

[0149] Step 4: After obtaining multiple target sentences in step 3, remove the prefix sequence "<|startoftranscript|><|zh|><|transcribe|><|notimestamps|>JiaLian Payment Building" in the model output to obtain the final result "I went to JiaLian Payment Building today" in which the target hot word was correctly identified.

[0150] The embodiment of the present application adds a speech prefix as a hot word optimization solution based on the Transformer structure of the speech model. This hot word technology makes full use of the contextual reasoning ability and autoregressive generation mechanism in the speech recognition large model, and adds hot word prefixes to the encoder and decoder inputs of the original model respectively. Without increasing additional training costs and the risk of misidentification, the embodiment of the present application improves the hot word recognition accuracy of the speech recognition large model by combining the input hot word audio sequence with text. At the same time, this solution only needs to use manual recording or TTS model synthesis to obtain hot word audio, which can achieve rapid iterative updates of hot words in actual production use.

[0151] An embodiment of the present application further provides an electronic device comprising: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for enabling communication between the processor and the memory. When the program is executed by the processor, the aforementioned speech model-based vocabulary recognition method is implemented. The electronic device may be any intelligent terminal, including a tablet computer and an in-vehicle computer.

[0152] See also Figure 9 , Figure 9 The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0153] The processor 901 may be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0154] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called by the processor 901 to execute the speech model-based vocabulary recognition method of the embodiments of this application.

[0155] Input / output interface 903, used to implement information input and output;

[0156] Communication interface 904, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0157] Bus 905 , which transmits information between various components of the device (e.g., processor 901 , memory 902 , input / output interface 903 , and communication interface 904 );

[0158] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0159] An embodiment of the present application further provides a computer-readable storage medium storing a computer program, which implements the above-mentioned speech model-based vocabulary recognition method when executed by a processor.

[0160] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0161] The embodiments of the present application provide a speech model-based vocabulary recognition method, device, electronic device and storage medium. First, the audio to be recognized and the target audio are obtained, and the target audio includes at least one target hot word audio, that is, the audio corresponding to the hot word that needs to be recognized. Then, the audio to be recognized and the target audio are converted into text to obtain the audio text corresponding to the audio to be recognized and the hot word text corresponding to the target hot word audio, so as to realize the textual display of the audio to be recognized and the target hot word audio, thereby improving the accuracy of information recognition. After that, the prefix sequence of the speech model is obtained to facilitate the subsequent decoding of the audio by the speech model, and the audio to be recognized and the target audio are input into the speech model for feature splicing, and the spliced ​​feature sequence is output, thereby improving the recognition accuracy of the speech and eliminating irrelevant or redundant ones. Features, reduce the complexity of the model, avoid the risk of overfitting, add the target hot word audio to the splicing feature sequence to obtain the target audio sequence, and add the hot word text to the prefix sequence to obtain the target prefix sequence, so that the target audio sequence and the target prefix sequence can be combined with the hot word and the hot word text corresponding to the hot word, adding the hot word prefix, and then input the target audio sequence and the target prefix sequence into the speech model, so that the speech model performs hot word recognition on the audio to be recognized based on the target audio sequence and the target prefix sequence, outputs the recognized hot word audio, realizes accurate recognition of the audio corresponding to the hot word, improves the recognition accuracy of the hot word audio, and finally, converts the audio to be recognized and the recognized hot word audio into text to obtain the target audio text, thereby improving the hot word recognition accuracy in the target audio text. The embodiment of the present application utilizes the contextual reasoning ability and autoregressive generation mechanism in the speech model, adds the target hot word audio to the splicing feature sequence during the model processing, and adds the hot word text to the prefix sequence to increase the hot word prefix, thereby improving the hot word recognition accuracy of the speech model by combining the input hot word audio sequence with the text, and realizing accurate recognition of the hot word.

[0162] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0163] It will be understood by those skilled in the art that Figure 1-7 The technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than those shown in the figures, or a combination of certain steps, or different steps.

[0164] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0165] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0166] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0167] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0168] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0169] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0170] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0171] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0172] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A vocabulary recognition method based on a speech model, characterized in that: The method comprises: Acquire the audio to be recognized and the target audio, wherein the target audio includes at least one target hot word audio; Performing text conversion on the target audio to obtain a hot word text corresponding to the target hot word audio; Obtaining a prefix sequence of the speech model, inputting the audio to be recognized and the target audio into the speech model for feature splicing, and outputting a spliced ​​feature sequence; wherein the prefix sequence is prior text information used by the speech model during decoding, and is used to initialize the decoding process of the speech model; Add the target hot word audio to the concatenated feature sequence to obtain a target audio sequence, and add the hot word text to the prefix sequence to obtain a target prefix sequence; Inputting the target audio sequence and the target prefix sequence into the speech model, so that the speech model performs hot word recognition on the audio to be recognized based on the target audio sequence and the target prefix sequence, and outputs the recognized hot word audio; The audio to be recognized and the recognized hot word audio are converted into text to obtain a target audio text.

2. The vocabulary recognition method based on the speech model according to claim 1, characterized in that The target audio is obtained by the following steps: Obtaining artificial audio and target hot word vocabulary, and counting the number of vocabulary of the target hot word vocabulary, wherein the artificial audio is obtained by manual recording; Performing an audio quality analysis on the artificial audio to obtain an audio quality score, wherein the audio quality score is used to characterize the sound quality and fluency of the artificial audio; When the number of the vocabulary is less than a preset number and the audio quality score is greater than or equal to a first preset score, using the artificial audio as the target audio; When the number of words is greater than or equal to a preset number or the audio quality score is less than a first preset score, audio synthesis is performed on the target hot word vocabulary through a preset deep learning model to obtain the target audio.

3. The vocabulary recognition method based on speech model according to claim 2, characterized in that The performing audio quality analysis on the artificial audio to obtain an audio quality score includes: Performing time-frequency transformation on the artificial audio, and extracting features from the artificial audio after the time domain transformation to obtain artificial audio features; Performing sound quality detection on the artificial audio based on the artificial audio feature; When the artificial audio contains noise, determining a second preset score as the audio quality score, wherein the second preset score is less than the first preset score; When there is no noise in the artificial audio, clarity detection is performed on the artificial audio based on the artificial audio features to obtain an audio quality score.

4. The method for vocabulary recognition based on speech model according to claim 2, characterized in that: The method of synthesizing the target hot word vocabulary through a preset deep learning model to obtain the target audio includes: Inputting the target hot word vocabulary into a preset deep learning model, so that the deep learning model performs phoneme analysis on the target hot word vocabulary and outputs a spectrogram; Performing audio conversion on the spectrogram using a preset vocoder to obtain an audio waveform; Generate target audio according to the audio waveform.

5. The vocabulary recognition method based on speech model according to claim 1, characterized in that The step of converting the target audio into text to obtain a hot word text corresponding to the target hot word audio includes: Extracting features of the target audio to obtain target audio features corresponding to the target audio; Performing feature mapping on the target audio feature using a preset acoustic model to obtain a target candidate sequence of the target audio feature; Inputting the target candidate sequence into a preset language model, so that the language model analyzes the context of the target audio based on the target candidate sequence and outputs a target text sequence; The target text sequence is decoded to obtain a hot word text corresponding to the target hot word audio.

6. The method for vocabulary recognition based on speech model according to claim 1, characterized in that: The step of inputting the audio to be recognized and the target audio into a speech model for feature splicing and outputting a spliced ​​feature sequence includes: Extracting a first splicing feature sequence of the audio to be recognized and a second splicing feature sequence of the target audio through the speech model; Feature splicing is performed on the first splicing feature sequence and the second splicing feature sequence to obtain a splicing feature sequence.

7. The vocabulary recognition method based on speech model according to claim 1, characterized in that The step of inputting the audio to be recognized and the target audio into a speech model for feature splicing and outputting a spliced ​​feature sequence further includes: Performing audio splicing on the audio to be recognized and the target audio to obtain a spliced ​​audio; An audio feature sequence of the spliced ​​audio is extracted using the speech model.

8. A vocabulary recognition device based on a speech model, characterized in that: The device comprises: An audio acquisition module, configured to acquire audio to be recognized and target audio, wherein the target audio includes at least one target hot word audio; A text conversion module, configured to perform text conversion on the target audio to obtain a hot word text corresponding to the target hot word audio; a feature splicing module, configured to obtain a prefix sequence of the speech model, input the audio to be recognized and the target audio into the speech model for feature splicing, and output a spliced ​​feature sequence; wherein the prefix sequence is a priori text information used by the speech model during decoding, and is used to initialize the decoding process of the speech model; A sequence splicing module, configured to add the target hot word audio to the splicing feature sequence to obtain a target audio sequence, and to add the hot word text to the prefix sequence to obtain a target prefix sequence; a hot word recognition module, configured to input the target audio sequence and the target prefix sequence into the speech model, so that the speech model performs hot word recognition on the audio to be recognized based on the target audio sequence and the target prefix sequence, and outputs recognized hot word audio; The target conversion module is used to perform text conversion on the audio to be recognized and the recognized hot word audio to obtain a target audio text.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the speech model-based vocabulary recognition method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the speech model-based vocabulary recognition method according to any one of claims 1 to 7 is implemented.

Images