Speech recognition method and apparatus, storage medium, and electronic device

Abstract

The application discloses a speech recognition method and device, a storage medium and an electronic device. The method comprises the following steps: acquiring scene description information and recognition guide prompt information corresponding to a speech interaction scene; performing signal conversion on an audio signal collected in the speech interaction scene to obtain at least one audio segment feature vector; in the case that i-1 text segments obtained by the i-1 audio segment feature vectors are acquired, performing recognition on the i audio segment feature vector based on the i-1 text segments, the scene description information and the recognition guide prompt information to obtain the i text segment, wherein i is a natural number greater than 1 and less than or equal to N, and N is the total number of the at least one audio segment feature vector; and generating a speech recognition text corresponding to the audio signal collected in the speech interaction scene based on the N text segments. The application solves the technical problem of poor accuracy of the speech recognition result caused by ignoring the complete speech information of the audio.

Description

Technical Field

[0001] This invention relates to the field of computers, and more specifically, to a speech recognition method and apparatus, a storage medium, and an electronic device. Background Technology

[0002] With the continuous development of artificial intelligence and deep learning technologies, speech recognition technology has gradually matured, especially in the recognition of short audio segments, where significant progress has been made. However, in scenarios involving the recognition of extremely long audio signals (audio playback duration exceeding a preset threshold) (such as voice interaction activities like meetings and lectures), bottlenecks still exist in the recognition of these long audio resources.

[0003] Current speech recognition methods for very long audio signals typically segment the signal into multiple independent audio clauses based on semantics, and then perform speech recognition on each clause separately. However, this approach often focuses only on the speech information conveyed by a single audio clause, neglecting the overall complete speech information of the long audio signal. This results in fragmented meaning in the final speech recognition results, deviating from the interactive theme of the speech interaction activity.

[0004] In other words, the speech recognition methods provided by related technologies can only perform fragmented recognition and processing of individual audio clause segments in ultra-long audio signals in voice interaction scenarios, resulting in poor accuracy of speech recognition results.

[0005] There is currently no effective solution to the above problems. Summary of the Invention

[0006] This invention provides a speech recognition method, apparatus, storage medium, and electronic device to at least solve the technical problem of poor speech recognition accuracy caused by ignoring complete audio speech information.

[0007] According to one aspect of the present invention, a speech recognition method is provided, comprising: acquiring scene description information corresponding to a speech interaction scenario and recognition guidance prompt information configured for the speech interaction scenario; performing signal conversion on an audio signal acquired in the speech interaction scenario to obtain at least one audio segment feature vector; when acquiring the first i-1 text segments obtained by speech recognition of the first i-1 audio segment feature vectors, recognizing the i-th audio segment feature vector based on the first i-1 text segments, the scene description information and the recognition guidance prompt information to obtain the i-th text segment, wherein i is a natural number greater than 1 and less than or equal to N, and N is the total number of the at least one audio segment feature vectors; and generating speech recognition text corresponding to the audio signal acquired in the speech interaction scenario based on the acquired N text segments.

[0008] As an optional implementation, the above-described method of identifying the feature vector of the i-th audio segment based on the first i-1 text segments, the scene description information, and the recognition guidance prompt information to obtain the i-th text segment includes: obtaining a scene keyword vector that matches the scene description information and a recognition guidance text vector that matches the recognition guidance prompt information, wherein the scene keywords indicated by the scene keyword vector are used to assist in identifying the speech interaction domain information corresponding to the feature vector of the audio segment, and the recognition guidance text vector is used to guide the recognition of the resulting text; in the speech recognition model, under the guidance constraints indicated by the recognition guidance text vector, the feature vector of the i-th audio segment is identified based on the scene keyword vector and the first i-1 text segments to obtain the i-th text segment, wherein the speech recognition model is a neural network model obtained by step-by-step training of the model network parameters for recognizing the text information corresponding to the feature vector of the audio segment.

[0009] As an optional implementation, in the speech recognition model described above, under the guidance constraints indicated by the recognition guidance text vector, recognizing the feature vector of the i-th audio segment based on the scene keyword vector and the first i-1 text segments to obtain the i-th text segment includes: in the speech recognition model, under the guidance constraints, generating first recognition reference information based on the scene keyword vector and the first i-1 text segments; and decoding the feature vector of the i-th audio segment according to the first recognition reference information to obtain the i-th text segment.

[0010] As an optional implementation, in the speech recognition model described above, under the guidance constraints indicated by the recognition guidance text vector, recognizing the feature vector of the i-th audio segment based on the scene keyword vector and the first i-1 text segments to obtain the i-th text segment includes: in the speech recognition model, under the guidance constraints, generating second recognition reference information based on the scene keyword vector, the first i-1 text segments, and the feature vector of the i-th audio segment; and decoding the feature vector of the i-th audio segment according to the second recognition reference information to obtain the i-th text segment.

[0011] As an optional implementation, when a reference pre-recognition text matching the complete audio signal collected in the above-described voice interaction scenario is obtained, in the speech recognition model, under the guidance constraints indicated by the recognition guidance text vector, based on the scenario keyword vector and the first i-1 text segments, the feature vector of the i-th audio segment is recognized to obtain the i-th text segment. This further includes: in the speech recognition model, under the guidance constraints, based on the scenario keyword vector, the first i-1 text segments, and the reference pre-recognition text, generating third recognition reference information; and decoding the feature vector of the i-th audio segment according to the third recognition reference information to obtain the i-th text segment.

[0012] As an optional implementation, under the guidance constraints indicated by the above-mentioned identification guidance text vector, the identification of the feature vector of the i-th audio segment based on the above-mentioned scene keyword vector and the above-mentioned first i-1 text segments to obtain the i-th text segment includes: identifying the feature vector of the i-th audio segment based on the above-mentioned scene keyword vector and the above-mentioned first i-1 text segments to obtain an intermediate text segment; and adjusting the intermediate text segment to the i-th text segment that satisfies the above-mentioned guidance constraints.

[0013] As an optional implementation, the above-mentioned signal conversion of the audio signal collected in the above-mentioned voice interaction scenario to obtain at least one audio segment feature vector includes: encoding the audio signal to obtain an audio encoding matrix; mapping the audio encoding matrix to a text space according to the above-mentioned recognition guidance prompt information to obtain a candidate text vector group; and using each candidate text vector in the above-mentioned candidate text vector group as a feature vector of the above-mentioned audio segment.

[0014] As an optional implementation, the above-mentioned signal conversion of the audio signal collected in the above-mentioned voice interaction scenario to obtain at least one audio segment feature vector includes: segmenting the audio signal to obtain multiple audio segments; encoding the multiple audio segments respectively to obtain multiple audio segment encoding vectors; and mapping the multiple audio segment encoding vectors to the text space according to the above-mentioned recognition guidance prompt information to obtain the at least one audio segment feature vector.

[0015] As an optional implementation, after identifying the feature vector of the i-th audio segment to obtain the i-th text segment, the method further includes: performing at least one of the following processing tasks on the i-th text segment: adding a separator symbol for dividing the sentence structure to the i-th text segment; removing tone words from the i-th text segment; performing reverse text formatting on the text content that belongs to strings identified in the i-th text segment; and tagging the voice interaction events that appear in the i-th text segment to obtain the i-th text segment carrying event tags.

[0016] As an optional implementation, after performing signal conversion on the audio signal collected in the above-mentioned voice interaction scenario to obtain at least one audio segment feature vector, the method further includes: if the i-th audio segment feature vector is the first audio segment feature vector, under the above-mentioned guiding constraints, decoding the first audio segment feature vector based on the above-mentioned scenario keyword vector to obtain the first text segment.

[0017] As an optional implementation, before obtaining the scene description information corresponding to the voice interaction scenario and the recognition guidance prompt information configured for the voice interaction scenario, the method further includes: obtaining the sample scene description information corresponding to the sample voice interaction scenario and the sample recognition guidance prompt information configured for the sample voice interaction scenario; performing signal conversion on the complete sample audio signal collected in the sample voice interaction scenario to obtain at least one sample audio segment feature vector; annotating the complete sample audio signal to obtain sample annotation text matching the complete sample audio signal; and performing phased training on the original speech recognition model based on the sample scene description information, the sample recognition guidance prompt information, and the sample annotation text to obtain the speech recognition model. In each phase of training, some model parameters within the trained speech recognition model are adjusted based on the training output results. The training output results are used to indicate the difference between the output text of the trained speech recognition model and the sample annotation text.

[0018] According to another aspect of the present invention, a speech recognition device is also provided, comprising: a first acquisition unit, configured to acquire scene description information corresponding to a speech interaction scenario, and recognition guidance prompt information configured for the speech interaction scenario; a first conversion unit, configured to perform signal conversion on the audio signal acquired in the speech interaction scenario to obtain at least one audio segment feature vector; a recognition unit, configured to, upon acquiring the first i-1 text segments obtained by speech recognition of the first i-1 audio segment feature vectors, recognize the i-th audio segment feature vector based on the first i-1 text segments, the scene description information, and the recognition guidance prompt information to obtain the i-th text segment, wherein i is a natural number greater than 1 and less than or equal to N, and N is the total number of the at least one audio segment feature vector; and a generation unit, configured to generate speech recognition text corresponding to the audio signal acquired in the speech interaction scenario based on the acquired N text segments.

[0019] As an optional implementation, the recognition unit further includes: an acquisition module, used to acquire a scene keyword vector matching the scene description information and a recognition guidance text vector matching the recognition guidance prompt information, wherein the scene keywords indicated by the scene keyword vector are used to assist in recognizing the speech interaction domain information corresponding to the audio segment feature vector, and the recognition guidance text vector is used to guide the recognition of the result text; and a recognition module, in a speech recognition model, under the guidance constraints indicated by the recognition guidance text vector, based on the scene keyword vector and the first i-1 text segments, to recognize the i-th audio segment feature vector and obtain the i-th text segment, wherein the speech recognition model is a neural network model obtained by step-by-step training of the model network parameters for recognizing the text information corresponding to the audio segment feature vector.

[0020] As an optional implementation, the above recognition module includes: a first generation submodule, used to generate first recognition reference information based on the scene keyword vector and the first i-1 text segments in the above speech recognition model under the above guiding constraints; and a first decoding submodule, used to decode the feature vector of the i-th audio segment according to the first recognition reference information to obtain the i-th text segment.

[0021] As an optional implementation, the above recognition module further includes: a second generation submodule, used to generate second recognition reference information based on the above scene keyword vector, the above first i-1 text segments, and the feature vector of the above ith audio segment in the above speech recognition model under the above guiding constraints; and a second decoding submodule, used to decode the feature vector of the above ith audio segment according to the above second recognition reference information to obtain the above ith text segment.

[0022] As an optional implementation, the above-mentioned recognition module further includes: a third generation submodule, used to generate third recognition reference information based on the above-mentioned scenario keyword vector, the above-mentioned first i-1 text segments, and the above-mentioned reference pre-recognition text in the above-mentioned speech recognition model under the above-mentioned guiding constraints when a reference pre-recognition text matching the complete audio signal collected in the above-mentioned voice interaction scenario is obtained; and a third decoding submodule, used to decode the feature vector of the above-mentioned i-th audio segment according to the above-mentioned third recognition reference information to obtain the above-mentioned i-th text segment.

[0023] As an optional implementation, the above recognition module further includes: a recognition submodule, which recognizes the feature vector of the i-th audio segment based on the above scene keyword vector and the above first i-1 text segments to obtain an intermediate text segment; and an adjustment submodule, which adjusts the above intermediate text segment to the above i-th text segment that satisfies the above guidance constraint conditions.

[0024] As an optional implementation, the first conversion unit includes: a first encoding module for encoding the audio signal to obtain an audio encoding matrix; a first mapping module for mapping the audio encoding matrix to a text space according to the recognition guidance prompt information to obtain a candidate text vector group; and a determination module for treating each candidate text vector in the candidate text vector group as a feature vector of the audio segment.

[0025] As an optional implementation, the first conversion unit further includes: a segmentation module for segmenting the audio signal to obtain multiple audio segments; a second encoding module for encoding the multiple audio segments respectively to obtain multiple audio segment encoding vectors; and a second mapping module for mapping the multiple audio segment encoding vectors to a text space according to the recognition guidance prompt information to obtain at least one audio segment feature vector.

[0026] As an optional implementation, the above-mentioned apparatus further includes at least one of the following: an adding unit, used to add a spacing symbol for dividing the sentence structure to the i-th text segment; a removing unit, used to remove tone words from the i-th text segment; a processing unit, used to perform reverse text formatting on the text content that belongs to strings identified in the i-th text segment; and a tagging unit, used to tag the voice interaction events that appear in the i-th text segment to obtain the i-th text segment carrying event tags.

[0027] As an optional implementation, the above-mentioned apparatus further includes: a decoding unit, configured to, under the above-mentioned guiding constraints, decode the first audio segment feature vector based on the above-mentioned scene keyword vector to obtain the first text segment when the feature vector of the i-th audio segment is the feature vector of the first audio segment.

[0028] As an optional implementation, the above apparatus further includes: a second acquisition unit, configured to acquire sample scene description information corresponding to the sample voice interaction scene, and sample recognition guidance prompt information configured for the sample voice interaction scene; a second conversion unit, configured to perform signal conversion on the complete sample audio signal collected in the sample voice interaction scene to obtain at least one sample audio segment feature vector; an annotation unit, configured to annotate the complete sample audio signal to obtain sample annotation text matching the complete sample audio signal; and a training unit, configured to perform phased training on the original speech recognition model based on the sample scene description information, the sample recognition guidance prompt information, and the sample annotation text to obtain the speech recognition model, wherein, during each stage of training, some model parameters within the trained speech recognition model are adjusted based on the training output results, and the training output results are used to indicate the difference between the output text of the trained speech recognition model and the sample annotation text.

[0029] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer-readable storage medium, and the computer program is configured to execute the above-described speech recognition method when it is run.

[0030] According to another aspect of the embodiments of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned speech recognition method.

[0031] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to execute the above-described speech recognition method through the computer program.

[0032] In this embodiment of the invention, after obtaining the scene description information corresponding to the voice interaction scenario and the recognition guidance prompt information configured for the voice interaction scenario, the audio signal collected in the voice interaction scenario is converted to obtain at least one audio segment feature vector; when the first i-1 text segments obtained by voice recognition of the first i-1 audio segment feature vectors are obtained, the i-th audio segment feature vector is recognized based on the first i-1 text segments, the scene description information and the recognition guidance prompt information to obtain the i-th text segment, where i is a natural number greater than 1 and less than or equal to N, and N is the total number of at least one audio segment feature vector; based on the obtained N text segments, the voice recognition text corresponding to the audio signal collected in the voice interaction scenario is generated, thereby realizing that when the first i-1 text segments obtained by voice recognition of the first i-1 audio segments are obtained, the first i-1 text segments can provide information guidance for the ongoing decoding process, and thus can make full use of the audio's voice information when performing voice recognition, thereby achieving the purpose of improving the accuracy of voice recognition and solving the technical problem of poor voice recognition results caused by ignoring the complete voice information of the audio. Attached Figure Description

[0033] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0034] Figure 1 This is a schematic diagram of the hardware environment of an optional speech recognition method according to an embodiment of this application;

[0035] Figure 2 This is a flowchart of an optional speech recognition method according to an embodiment of this application;

[0036] Figure 3 This is a schematic diagram of the structure of an optional speech recognition model according to an embodiment of this application;

[0037] Figure 4 This is a schematic diagram of the structure of another optional speech recognition model according to an embodiment of this application;

[0038] Figure 5 This is a flowchart of another optional speech recognition method according to an embodiment of this application;

[0039] Figure 6 This is a flowchart of another optional speech recognition method according to an embodiment of this application;

[0040] Figure 7 This is a flowchart of another optional speech recognition method according to an embodiment of this application;

[0041] Figure 8 This is a flowchart of another optional speech recognition method according to an embodiment of this application;

[0042] Figure 9 This is a flowchart of another optional speech recognition method according to an embodiment of this application;

[0043] Figure 10 This is an optional text processing flowchart according to an embodiment of this application;

[0044] Figure 11 This is a schematic diagram of the structure of another optional speech recognition model according to an embodiment of this application;

[0045] Figure 12 This is a flowchart of another optional speech recognition method according to an embodiment of this application;

[0046] Figure 13 This is a schematic diagram of the structure of an optional speech recognition device according to an embodiment of this application;

[0047] Figure 14 This is a computer system architecture block diagram of an optional electronic device according to an embodiment of this application;

[0048] Figure 15 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0049] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0050] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0051] According to an embodiment of this application, a speech recognition method is provided. Optionally, in this embodiment, the speech recognition system can be applied to various online or offline voice interaction scenarios, such as online meetings, online classes, and telephone recordings. Furthermore, the speech recognition system can also be applied to various voice interactive scenarios, such as question-and-answer voice response scenarios in smart home control, or question-and-answer smart service scenarios. During speech recognition, the speech recognition system can acquire text data and audio to be recognized through devices such as smart wearable devices, smartphones, and smart terminals, and recognize the audio to obtain the recognized text of the audio content. It should be noted that the audio to be recognized can be categorized into short audio, long audio, or ultra-long audio based on its duration. For example, audio with a duration of less than 10 minutes is considered short audio, audio with a duration of 10 minutes to 1 hour is considered long audio, and audio with a duration of more than 1 hour is considered ultra-long audio. In addition, audio can also be classified based on its size. For example, audio with a size of less than 100MB is considered short audio, audio with a size of 100MB to 1GB is considered long audio, and audio with a size of more than 1GB is considered ultra-long audio. Short audio files offer fast processing speeds, making them suitable for real-time applications. Speech recognition models can quickly recognize them with low memory consumption. Long audio files contain more contextual information, helping speech recognition models to understand semantics more accurately and improving recognition accuracy. However, they also place higher demands on the performance of speech recognition models. Ultra-long audio files contain extremely rich contextual information, helping speech recognition models understand words and sentences in richer contexts and enabling deeper content analysis. However, recognizing ultra-long audio files places extremely high demands on the stability and efficiency of the model, while also requiring effective error correction and context management mechanisms.

[0052] In the embodiments of this application, the above-described speech recognition method can be applied to, but is not limited to, situations such as... Figure 1The speech recognition system in the hardware environment shown may include, but is not limited to, an information acquisition device 102, an information processing server 104, a speech recognition server 106, and a terminal device 108. The information acquisition device 102 is used to acquire audio signals in a voice interaction scenario, obtain scene description information for the voice interaction scenario, and recognition guidance information configured for the voice interaction scenario. Subsequently, the information acquisition device 102 sends the acquired audio, along with the acquired scene description information and recognition guidance information, to the information processing server 104. The information processing server 104 is used to execute steps S1042 to S1048, performing signal conversion on the audio signals acquired in the voice interaction scenario to obtain at least one audio segment feature vector; and processing the scene description information and recognition guidance information. The speech recognition server 106 here is used to execute steps S1062 to S1064, receiving N audio segment feature vectors, processed recognition guidance information, and processed scene description information from the information processing server 104. Given the first i-1 text segments obtained by speech recognition of the first i-1 audio segment feature vectors, the server recognizes the i-th audio segment feature vector based on the first i-1 text segments, scene description information, and recognition guidance information to obtain the i-th text segment, where i is a natural number greater than 1 and less than or equal to N, and N is the total number of feature vectors of at least one audio segment. The terminal device 108 here is used to generate speech-recognized text corresponding to the audio signals collected in the speech interaction scenario based on the obtained N text segments, and displays the generated speech-recognized text.

[0053] As an optional implementation, assuming the aforementioned speech recognition system is applied to an online meeting, the information acquisition device is used to collect the voice of each speaker in the meeting, while simultaneously acquiring scene description information such as the meeting topic, participant list, and meeting schedule. Depending on the nature of the meeting, the information acquisition device may be equipped with some professional terms or proper nouns that may be related to the meeting topic. Furthermore, before the meeting begins, the information acquisition device can acquire recognition guidance prompts, such as "Recognize the meeting audio sentence by sentence according to the meeting content." The information acquisition device sends the collected audio signals, scene description information, and recognition guidance information to an information processing server via a network. The information processing server converts the received audio signals into feature vectors for at least one audio segment and converts the scene description information and recognition guidance prompts from text form to vector form. Subsequently, the information processing server sends the feature vectors of at least one audio segment, along with the vectorized scene description information and recognition guidance prompts, to a speech recognition server via a network. The speech recognition server recognizes the feature vectors of at least one audio segment one by one based on the received information to gradually obtain N recognized text segments, each of which is a complete sentence. After completing the feature vector recognition for all audio segments, the speech recognition server sends the resulting N text segments to the terminal device via the network. The speech recognition server can send a single text segment immediately after the feature vector recognition for each audio segment is completed, or it can send all N text segments to the terminal device after the feature vector recognition process for all audio segments is finished. The terminal device can display the received single text segments as subtitles to the participants in real time, or it can organize all N text segments into a document or other format after the meeting and display it to the participants. After the meeting, participants can edit and correct the generated speech recognition text using the terminal device 108, or archive it as meeting minutes.

[0054] Optionally, in this embodiment, the terminal device can be a terminal device configured with a target client, which may include, but is not limited to, at least one of the following: mobile phone (such as Android phone, iOS phone, etc.), laptop computer, tablet computer, PDA, MID (Mobile Internet Devices), PAD, desktop computer, smart TV, etc. The target client may be a video client, instant messaging client, browser client, educational client, etc. The network may include, but is not limited to, wired network and wireless network, wherein the wired network includes: local area network, metropolitan area network and wide area network, and the wireless network includes: Bluetooth, WIFI and other networks that enable wireless communication. The server may be a single server, a server cluster composed of multiple servers, or a cloud server. The above is only an example, and no limitation is made in this embodiment.

[0055] Alternatively, as an optional implementation, such as Figure 2 As shown, the above speech recognition method includes:

[0056] S202, obtain the scene description information corresponding to the voice interaction scene, as well as the recognition guidance prompt information configured for the voice interaction scene;

[0057] Optionally, in this embodiment, the aforementioned voice interaction scenario may be, but is not limited to, an ongoing or concluded meeting or lecture. These meetings or lectures can be conducted online or in person. The aforementioned scenario description information may be, but is not limited to, multiple words used to describe the aforementioned voice interaction scenario. For example, when the aforementioned voice interaction scenario is a meeting, the scenario description information may be the topic of the meeting and the host; when the aforementioned voice interaction scenario is a lecture, the scenario description information may be the speaker of the lecture, the lecture topic, and technical terms related to the lecture's field. The aforementioned recognition guidance prompt information may include, but is not limited to, a complete sentence used to guide and constrain the aforementioned voice interaction scenario. For example, the aforementioned recognition guidance prompt information may be "Based on the given full-text pre-recognition text, auxiliary information, and the entire voice content, perform voice re-recognition sentence by sentence."

[0058] S204, perform signal conversion on the audio signal collected in the voice interaction scenario to obtain at least one audio segment feature vector;

[0059] Optionally, in this embodiment, the acquired audio signal may be, but is not limited to, a complete long audio file (e.g., the complete audio of a meeting) or the audio of a sentence (e.g., the speech of a participant in a meeting). The audio segment feature vector may, but is not limited to, have a one-to-one correspondence with the sentences included in the acquired audio signal, that is, a sentence in the audio signal corresponds to an audio segment feature vector here.

[0060] S206, after obtaining the first i-1 text segments obtained by speech recognition of the feature vectors of the first i-1 audio segments, based on the first i-1 text segments, scene description information and recognition guidance prompt information, the feature vector of the i-th audio segment is recognized to obtain the i-th text segment, where i is a natural number greater than 1 and less than or equal to N, and N is the total number of feature vectors of at least one audio segment.

[0061] Optionally, in this embodiment, the i-th text segment can be, but is not limited to, a complete text obtained by recognizing the feature vector of the i-th audio segment, and the complete text here has undergone text processing (e.g., adding punctuation, inverse text normalization, etc.).

[0062] S208, based on the acquired N text fragments, generates speech recognition text corresponding to the audio signals collected in the speech interaction scenario.

[0063] Optionally, in this embodiment, the aforementioned speech-recognized text may be, but is not limited to, a complete text containing at least one sentence obtained by connecting the acquired N text segments in chronological order.

[0064] Through the embodiments provided in this application, after obtaining the scene description information corresponding to the voice interaction scene and the recognition guidance prompt information configured for the voice interaction scene, the audio signal collected in the voice interaction scene is converted into a signal to obtain at least one audio segment feature vector; when the first i-1 text segments obtained by speech recognition of the first i-1 audio segment feature vectors are obtained, the i-th audio segment feature vector is recognized based on the first i-1 text segments, the scene description information and the recognition guidance prompt information to obtain the i-th text segment, where i is a natural number greater than 1 and less than or equal to N, and N is the total number of at least one audio segment feature vector; based on the obtained N text segments, the speech recognition text corresponding to the audio signal collected in the voice interaction scene is generated, thereby realizing that when the first i-1 text segments obtained by the recognition of the first i-1 audio segments are obtained, the first i-1 text segments can provide information guidance for the ongoing decoding process, thereby making full use of the audio's speech information during speech recognition, achieving the goal of improving speech recognition accuracy, and solving the technical problem of poor speech recognition accuracy caused by ignoring the complete speech information of the audio.

[0065] As an optional approach, based on the first i-1 text segments, scene description information, and recognition guidance prompts, the feature vector of the i-th audio segment is identified, resulting in the i-th text segment including:

[0066] S1, obtain the scene keyword vector that matches the scene description information and the recognition guidance text vector that matches the recognition guidance prompt information. The scene keywords indicated by the scene keyword vector are used to assist in recognizing the speech interaction domain information corresponding to the audio segment feature vector, and the recognition guidance text vector is used to guide the recognition result text.

[0067] Optionally, in this embodiment, the aforementioned scenario keywords may be, but are not limited to, relevant words in the scenario description information that can assist in identifying the aforementioned voice interaction scenario or voice interaction content. When the aforementioned voice interaction scenario is a technical lecture or academic conference, the aforementioned scenario keywords may be words related to the conference theme, information about the participants, or content related to the lecture theme (such as lecture introduction, speaker information, lecture organizer information, etc.); when the aforementioned voice interaction scenario is a daily conversation, the aforementioned scenario keywords may be words that frequently appear in the conversation. For example, when the aforementioned voice interaction scenario is an online educational lecture, the aforementioned scenario description information is: "Speaker: Zhang San", "Lecture Topic: Education", "Thinking", "Diversified Education", "Ideological Construction", "Cultivation of Cultural Literacy", and the aforementioned scenario keywords may be "Thinking", "Diversified Education", "Ideological Construction", and "Cultivation of Cultural Literacy" in the aforementioned scenario description information. The aforementioned scenario keyword vector and recognition guidance text vector may be, but are not limited to, vectors obtained by processing the acquired scenario keywords and recognition guidance information using natural language processing technology.

[0068] The aforementioned scenario keyword vectors and recognition guidance text vectors can be obtained by vectorizing the acquired scenario keywords and recognition guidance information using a text vectorization model. Word2Vec is a word embedding model applied in the field of natural language processing. In this embodiment, it can be used to convert words in text into numerical vectors. These vectors can not only capture the semantic information of words, but also reflect the grammatical relationships between words.

[0069] Word2Vec has two model architectures: the CBOW model and the Skip-gram model. For example... Figure 3 As shown, the CBOW model includes an input layer 302, a hidden layer 304, and an output layer 306. The input layer 302 acquires multiple words from the input and converts them into vector representations with consistent dimensionality. These vector representations indicate the semantic information contained in the words. The hidden layer 304 integrates the semantic information in the words and calculates the word vectors from the input layer to predict the target word. The output layer 306 obtains the target word vector by calculating the predicted probability distribution of all words. Figure 3 As shown, input word x 1k x 2k x ck Equal to the weight matrix W V*N Multiplying these results in multiple 1*N vectors. Averaging these vectors yields an N-dimensional vector h. i Then multiply by the coefficient matrix W ′ N*V The output vector y is obtained. j .

[0070] As an optional implementation method, such as Figure 4 As shown, the Skip-gram model includes an input layer 402, a hidden layer 404, and an output layer 406. The input layer 402 acquires individual words from the input and converts them into vector representations, where the vector representations indicate the semantic information contained in the words. The hidden layer 404 calculates the word vectors of the input layer based on the semantic information in the words. The output layer 406 calculates the predicted probability distribution of the words to obtain at least one target word vector. Figure 3 As shown, input word x k In the hidden layer, a dimensionality transformation is performed to obtain a 1*N vector, which is then multiplied by the coefficient matrix W. ′ N*V The output vector y is obtained. 1j y 2j ... y Cj .

[0071] S2, in the speech recognition model, under the guidance constraints indicated by the recognition guidance text vector, based on the scene keyword vector and the first i-1 text segments, the feature vector of the i-th audio segment is recognized to obtain the i-th text segment. Here, the speech recognition model is a neural network model used to recognize the text information corresponding to the feature vector of the audio segment after the model network parameters are trained step by step.

[0072] Optionally, in this embodiment, the speech recognition model described above can be, but is not limited to, a neural network model capable of speech recognition of audio segments, such as the Whisper model, the Transformer network, etc. In this embodiment, the neural network model is used to learn the contextual information in the first i-1 text segments, and to complete the recognition of the feature vector of the i-th audio segment based on the contextual information and the scene keyword vector.

[0073] Through the embodiments provided in this application, after obtaining the scene keyword vector that matches the scene description information and the recognition guidance text vector that matches the recognition guidance prompt information, in the speech recognition model, under the guidance constraints indicated by the recognition guidance text vector, the feature vector of the i-th audio segment is recognized based on the scene keyword vector and the first i-1 text segments to obtain the i-th text segment. The scene keywords and recognition guidance prompt information can provide information constraints and scene guidance for the recognition process of the speech recognition model, so as to achieve faster and more accurate speech recognition, thereby improving the efficiency and accuracy of speech recognition.

[0074] As an alternative approach, in the speech recognition model, under the guidance constraints indicated by the guiding text vector, the feature vector of the i-th audio segment is recognized based on the scene keyword vector and the first i-1 text segments, resulting in the i-th text segment including:

[0075] S1, In the speech recognition model, under guided constraints, the first recognition reference information is generated based on the scene keyword vector and the first i-1 text segments;

[0076] S2, decode the feature vector of the i-th audio segment according to the first recognition reference information to obtain the i-th text segment.

[0077] As an optional implementation method, such as Figure 5 As shown, after obtaining the scene keyword vector and the first i-1 text segments, the speech recognition model, under the guidance constraints indicated by the recognition guidance text vector, executes step S502 to generate first recognition reference information based on the scene keyword vector and the first i-1 text segments. This recognition reference information is used to provide assistance or information related to the speech interaction scenario for the speech recognition process. For example, the recognition reference information can be technical terms that are difficult to recognize among the scene keywords, to assist the speech recognition model in accurately recognizing professional terms related to the speech interaction field during the recognition process. Subsequently, the speech recognition model executes step S504 to decode the feature vector of the i-th audio segment according to the first recognition reference information, obtaining the i-th text segment.

[0078] For example, when performing audio recognition on an ongoing academic conference with the theme of "biotechnology," the scene keywords could be "gene" or "cell." Assuming the guidance message in this voice interaction scenario is "Based on the given scene keywords and voice content, perform speech recognition with reference to the previous audio recognition results," then the guidance constraints could be "Based on the given scene keywords" and "Refer to the previous audio recognition results." Under these constraints, the first recognition reference information generated based on the scene keyword vector and the first i-1 text segments can include, but is not limited to, text segments or words containing the aforementioned scene keywords in the first i-1 text segments, such as "transgenic technology," "gene editing," and "cell structure." When the speech recognition model decodes the i-th audio segment based on the first recognition reference information, if it encounters words from the first recognition reference information, it can accurately and quickly obtain the recognition result.

[0079] Through the embodiments provided in this application, in the speech recognition model, under guided constraints, first recognition reference information is generated based on the scene keyword vector and the first i-1 text segments, and the feature vector of the i-th audio segment is decoded according to the first recognition reference information to obtain the i-th text segment. Thus, the i-th audio segment can be directly decoded based on the reference to the first i-1 text segments and scene keywords, thereby improving decoding efficiency and optimizing speech recognition effect, making the recognized i-th text more consistent with the actual voice interaction scenario.

[0080] As an alternative approach, in the speech recognition model, under the guidance constraints indicated by the guiding text vector, the feature vector of the i-th audio segment is recognized based on the scene keyword vector and the first i-1 text segments, resulting in the i-th text segment including:

[0081] S1, In the speech recognition model, under guided constraints, second recognition reference information is generated based on the scene keyword vector, the first i-1 text segments, and the feature vector of the i-th audio segment;

[0082] S2, decode the feature vector of the i-th audio segment according to the second recognition reference information to obtain the i-th text segment.

[0083] As an optional implementation method, such as Figure 6 As shown, after obtaining the scene keyword vector and the first i-1 text segments, the speech recognition model, under the guidance constraints indicated by the recognition guidance text vector, executes step S602 to generate second recognition reference information based on the scene keyword vector, the feature vector of the i-th audio segment, and the first i-1 text segments. This second recognition reference information provides assistance or information related to the speech interaction scenario for the speech recognition process. For example, the second recognition reference information could be frequently occurring words in the scene keywords to assist the speech recognition model in quickly recognizing professional terms related to the speech interaction field. Subsequently, the speech recognition model executes step S604 to decode the feature vector of the i-th audio segment based on the second recognition reference information, obtaining the i-th text segment.

[0084] For example, when performing audio recognition on an ongoing academic conference with the theme of "education," the scene keywords could be "school," "quality education," or "teacher." Assuming the guidance prompt in this voice interaction scenario is "Based on the given scene keywords and voice content, combine this audio segment with the previous audio recognition results to perform speech recognition," then the guidance constraint could be "Based on the given scene keywords," and "Combined with the previous audio recognition results." Under these guidance constraints, the second recognition reference information generated based on the scene keyword vector and the first i-1 text segments can include, but is not limited to, text segments or words containing the aforementioned scene keywords and appearing frequently in the first i-1 text segments, such as "teacher competence," "school development," and "implementing quality education." When the speech recognition model decodes the i-th audio segment based on the second recognition reference information, it can accurately and quickly obtain the recognition result if it encounters words from the second recognition reference information. Furthermore, the second recognition reference information can also include the semantic information contained in the first i-1 text segments to help the speech recognition model more accurately decode the feature vector of the i-th audio segment by referring to the context information.

[0085] Through the embodiments provided in this application, in the speech recognition model, under guided constraints, second recognition reference information is generated based on the scene keyword vector, the first i-1 text segments, and the feature vector of the i-th audio segment. The feature vector of the i-th audio segment is then decoded according to the second recognition reference information to obtain the i-th text segment. This achieves the full utilization of the contextual information in the first i-1 text segments and improves the decoding accuracy of the model.

[0086] As an optional approach, given that a reference pre-recognition text matching the complete audio signal collected in the voice interaction scenario is obtained, in the speech recognition model, under the guidance constraints indicated by the recognition guidance text vector, based on the scene keyword vector and the first i-1 text segments, the feature vector of the i-th audio segment is recognized to obtain the i-th text segment, which also includes:

[0087] S1, In the speech recognition model, under guided constraints, a third recognition reference information is generated based on the scene keyword vector, the first i-1 text fragments, and the reference pre-recognition text;

[0088] S2, decode the feature vector of the i-th audio segment according to the third recognition reference information to obtain the i-th text segment.

[0089] Optionally, in this embodiment, the aforementioned reference pre-recognition text may be, but is not limited to, the full-text pre-recognition text of the audio to be recognized. This full-text pre-recognition text may be, but is not limited to, obtained by performing preliminary text recognition on the audio to be recognized using a conventional speech recognition system.

[0090] As an optional implementation method, such as Figure 7 As shown, after obtaining the scene keyword vector, the reference pre-recognition text, and the first i-1 text segments, the speech recognition model, under the guidance constraints indicated by the recognition guidance text vector, executes step S702 to generate third recognition reference information based on the scene keyword vector, the feature vector of the i-th audio segment, and the first i-1 text segments. This second recognition reference information is used to provide assistance or information related to the speech interaction scenario for the speech recognition process. For example, the third recognition reference information could be frequently occurring words in the scene keywords to assist the speech recognition model in quickly recognizing professional terms related to the speech interaction field. Subsequently, the speech recognition model executes step S704 to decode the feature vector of the i-th audio segment based on the third recognition reference information, obtaining the i-th text segment.

[0091] For example, when performing audio recognition on an ended meeting with the theme "Game Upgrade," the reference pre-recognition text can be the text obtained after processing the audio of the meeting using a speech recognition tool. Scene keywords could include "user," "game level," "game character," etc. Assuming the recognition guidance prompt in the above voice interaction scenario is "Based on the given scene keywords, reference pre-recognition text, and voice content, perform speech recognition on the audio to be recognized sentence by sentence," then the guidance constraints could be "Based on the given scene keywords," "reference pre-recognition text," and "Perform speech recognition on the audio to be recognized sentence by sentence." Under these guidance constraints, the third recognition reference information generated based on the scene keyword vector, the reference pre-recognition text, and the first i-1 text segments can include, but is not limited to, contextual information in the reference pre-recognition text, and text segments or words containing the aforementioned scene keywords in the first i-1 text segments, such as "teacher ability," "school development," and "implementing quality education." When the speech recognition model decodes the i-th audio segment based on the above third recognition reference information, if it encounters words in the third recognition reference information, it can accurately and quickly obtain the recognition result. Furthermore, in the aforementioned third set of identification reference information, the contextual information in the reference pre-identification text can be the core content of the meeting, for example, the core content of the meeting being "how to enrich gameplay and enable players to interact more during the game." Based on this, when decoding the feature vector of the i-th audio segment, the speech recognition model can more accurately distinguish some polyphonic characters or more accurately determine the meaning of some more abstract words based on the contextual information in the reference pre-identification text. For example, when the speech recognition model recognizes the word "skin," it can know from the context in the reference pre-identification text that "skin" in the aforementioned meeting does not refer to human skin, but rather to a virtual game character.

[0092] Through the embodiments provided in this application, in the speech recognition model, under guided constraints, a third recognition reference information is generated based on the scene keyword vector, the first i-1 text segments, and the reference pre-recognition text. The feature vector of the i-th audio segment is then decoded according to the third recognition reference information to obtain the i-th text segment. This enables the decoding of the feature vector of the i-th audio segment based on the complete speech information in the audio, avoiding illusions when the model performs speech recognition on long audio files and improving the quality of the speech recognition results.

[0093] As an optional approach, under the guidance constraints indicated by the guiding text vector, the feature vector of the ith audio segment is identified based on the scene keyword vector and the first i-1 text segments, resulting in the ith text segment including:

[0094] S1, based on the scene keyword vector and the first i-1 text segments, identify the feature vector of the i-th audio segment to obtain the intermediate text segment;

[0095] S2, adjust the intermediate text segment to the i-th text segment that satisfies the guiding constraints.

[0096] As an optional implementation, when the guiding constraint is "provide speech recognition results in segments", after recognizing the i-th audio segment and obtaining the intermediate text segment, the speech recognition model adds a corresponding sequence number before the intermediate text segment to obtain the adjusted i-th text segment that satisfies the guiding constraint. For example, if the intermediate text segment obtained from speech recognition of the 10th audio segment is "What are we eating tonight?", then the adjusted 10th text segment is "10. What are we eating tonight?", thus satisfying the guiding constraint of "providing speech recognition results in segments".

[0097] The embodiments provided in this application identify the feature vector of the i-th audio segment based on the scene keyword vector and the first i-1 text segments to obtain the intermediate text segment. The intermediate text segment is then adjusted to satisfy the i-th text segment that meets the guiding constraints. This fully utilizes the guiding constraints indicated by the identified guiding text vector to adjust the speech recognition result, thereby making the text obtained from speech recognition more consistent with the constraints of the identified guiding text and optimizing the speech recognition result.

[0098] As an optional approach, signal conversion is performed on the audio signals acquired in the voice interaction scenario to obtain at least one audio segment feature vector, including:

[0099] S1, Encode the audio signal to obtain the audio coding matrix;

[0100] S2, based on the recognition guidance prompts, map the audio encoding matrix to the text space to obtain candidate text vector groups;

[0101] S3, treat each candidate text vector in the candidate text vector group as an audio segment feature vector.

[0102] As an optional implementation method, such as Figure 8As shown, the acquired audio signal is input to the encoder to execute step S802, obtaining an audio encoding matrix. The encoder here can be a pre-trained speech encoder, such as the Wav2vec model or the Whisper model, or a neural network structure capable of encoding audio to obtain an audio encoding matrix, such as the Transformer network structure. Since the text space and audio space are inconsistent, the dimensions of the audio encoding matrix and the recognition guidance text vector may not be the same. Therefore, the audio encoding matrix needs to be mapped to the text space, as in step S804, to obtain a candidate text vector group, where each candidate text vector in the candidate text vector group has the same dimension as the recognition guidance text vector. This mapping operation can be performed using basic matrix transformation methods or through a neural network used for matrix transformation. For example, the audio encoding matrix N = (x1, x2, ..., x5), where x1 to x5 are all 1×10 vectors, while the recognition guidance text vector y is a 1×8 vector. Therefore, the candidate text vector group obtained after mapping the audio encoding matrix to the text space is N. ′ =(x1) ′ x2 ′ ,…,x5 ′ ), where x1 ′ Up to x5 ′ Each candidate text vector is a 1×8 vector. Subsequently, each candidate text vector in the candidate text vector group is used as an audio segment feature vector, as in step S806.

[0103] According to the embodiments provided in this application, after encoding the audio signal to obtain an audio encoding matrix, the audio encoding matrix is ​​mapped to the text space according to the recognition guidance prompt information to obtain a candidate text vector group. Each candidate text vector in the candidate text vector group is used as an audio segment feature vector to obtain an audio segment feature vector with the same dimension as the recognition guidance text vector. This facilitates the speech recognition model to decode the audio segment feature vector, thereby improving the speech recognition efficiency.

[0104] As an optional approach, signal conversion is performed on the audio signals acquired in the voice interaction scenario to obtain at least one audio segment feature vector, including:

[0105] S1, the audio signal is segmented to obtain multiple audio segments;

[0106] S2, encode multiple audio segments separately to obtain multiple audio segment encoding vectors;

[0107] S3. Based on the recognition guidance prompts, map the encoding vectors of multiple audio segments to the text space respectively to obtain at least one audio segment feature vector.

[0108] Optionally, in this embodiment, the above-mentioned segmentation operation can be implemented by processing the audio signal through an audio segmentation system, but is not limited to. The audio segmentation system can be, but is not limited to, a model applied in the field of audio processing that can segment a complete audio signal sentence by sentence.

[0109] As an optional implementation method, such as Figure 9 As shown, the acquired audio signal is input to an audio segmentation system, which segments the acquired audio signal into multiple audio segments, as in step S902. The acquired audio signal is a long audio file containing multiple complete sentences, and each audio segment obtained after segmentation corresponds to a complete sentence in the aforementioned long audio file. Furthermore, the audio segmentation system can be a conventional system for segmenting audio, such as a Voice Activity Detection (VAD) system. Subsequently, the multiple audio segments are input to an encoder to execute step S904, obtaining multiple audio segment encoding vectors. The encoder here can be a pre-trained speech encoder, such as the Wav2vec model or the Whisper model, or a neural network structure capable of encoding audio to obtain an audio encoding matrix, such as a Transformer network structure. Since the text space and audio space are inconsistent, the dimensions of the audio segment encoding vectors and the recognition guidance text vectors may not be the same. Therefore, the audio segment encoding vectors need to be mapped to the text space, as in step S906, to obtain at least one audio segment feature vector, where each audio segment feature vector has the same dimension as the recognition guidance text vector. The mapping operation described above can be performed using basic matrix transformations or through neural networks used for matrix transformations. For example, the multiple audio segment encoding vectors mentioned above are x1, x2, ..., x5, where x1 to x5 are all 1×10 vectors, and the recognition guidance text vector y is a 1×8 vector. Therefore, the audio segment feature vector obtained after mapping the audio segment encoding vectors to the text space is x1. ′ x2 ′ ,…,x5 ′ , where x1 ′ Up to x5 ′ Both are 1×8 vectors.

[0110] Through the embodiments provided in this application, after segmenting the audio signal to obtain multiple audio segments, the multiple audio segments are encoded separately to obtain multiple audio segment encoding vectors. Based on the recognition guidance prompt information, the multiple audio segment encoding vectors are mapped to the text space to obtain at least one audio segment feature vector. This enables the encoding of long audio by encoding multiple audio segments separately, thereby improving encoding efficiency, saving encoding resources, and avoiding unnecessary deviations caused by excessively long audio during the encoding process.

[0111] As an optional approach, after identifying the feature vector of the i-th audio segment to obtain the i-th text segment, the following steps are also included:

[0112] Perform at least one of the following processing tasks on the i-th text segment:

[0113] Add a separator symbol to the i-th text segment to divide the sentence structure;

[0114] Remove tone-expressing words from the i-th text segment;

[0115] Perform inverse text formatting on the text content that belongs to the string in the i-th text segment;

[0116] Tag the voice interaction events that appear in the i-th text segment to obtain the i-th text segment with event tags.

[0117] As an optional implementation method, such as Figure 10 As shown, after obtaining the i-th text segment, in order to make the recognized text more accurate, the speech recognition model performs at least one of steps S1002-1 to S1002-4 on the i-th text segment:

[0118] S1002-1, Add a separator to the i-th text segment to divide the sentence structure. Optionally, the separator used to divide the sentence structure may be, but is not limited to, punctuation marks used in written text to express tone or pauses. For example, if the original i-th text segment is "The weather is so nice today, let's go on a picnic," then the i-th text segment after adding punctuation is "The weather is so nice today, let's go on a picnic!".

[0119] S1002-2 is to remove modal expressions from the i-th text segment. Optionally, the modal expressions mentioned above may include, but are not limited to, modal expressions with unclear meanings such as "ah" or "ah".

[0120] S1002-3, perform inverse text formatting on the text content that belongs to the string identified in the i-th text segment. Optionally, the above inverse text formatting is used to convert the text from spoken form to written form, for example, converting "password is 130" to "password is 130".

[0121] S1002-4, tagged the voice interaction events appearing in the i-th text segment to obtain the i-th text segment carrying event tags. Optionally, the aforementioned voice interaction events may refer to, but are not limited to, tagging special sounds in the audio that are different from human voices, such as bird calls, applause, coughs, etc., or tagging special events in the audio, such as changes in the speaker's identity.

[0122] During speech recognition, the speech recognition model can perform one of steps S1002-1 to S1002-4 on the i-th text segment, or it can perform multiple steps S1002-1 to S1002-4 on the i-th text segment in sequence. For example, after removing interjections from the i-th text segment, the speech recognition model can then perform reverse text formatting on the i-th text segment after removing interjections to obtain the processed i-th text segment.

[0123] Through the embodiments provided in this application, after recognizing the feature vector of the i-th audio segment to obtain the i-th text segment, the i-th text segment is further processed to achieve end-to-end speech recognition, reduce the cumulative error caused by the cascaded system in the speech recognition process, and improve the efficiency and accuracy of speech recognition.

[0124] As an optional approach, after performing signal conversion on the audio signals acquired in the voice interaction scenario to obtain at least one audio segment feature vector, the method further includes:

[0125] When the feature vector of the i-th audio segment is the feature vector of the first audio segment, under the guidance constraint, the feature vector of the first audio segment is decoded based on the scene keyword vector to obtain the first text segment.

[0126] Through the embodiments provided in this application, when the feature vector of the i-th audio segment is the feature vector of the first audio segment, under the guidance constraint, the feature vector of the first audio segment is directly decoded based on the scene keyword vector to obtain the first text segment, thereby saving model resource consumption and reducing speech recognition cost.

[0127] As an optional approach, before obtaining the scene description information corresponding to the voice interaction scenario and the recognition guidance prompts configured for the voice interaction scenario, the following steps are also included:

[0128] S1, obtain the sample scene description information corresponding to the sample voice interaction scene, as well as the sample recognition guidance prompt information configured for the sample voice interaction scene;

[0129] S2, perform signal conversion on the complete sample audio signal collected in the sample voice interaction scenario to obtain at least one sample audio segment feature vector;

[0130] S3, annotate the complete sample audio signal to obtain sample annotation text that matches the complete sample audio signal;

[0131] S4. Based on the sample scene description information, sample recognition guidance prompts and sample annotation text, the original speech recognition model is trained in stages to obtain a speech recognition model. In each stage of training, some model parameters in the speech recognition model are adjusted based on the training output results. The training output results are used to indicate the difference between the output text of the speech recognition model and the sample annotation text.

[0132] Optionally, in this embodiment, the above-mentioned sample annotation text may be, but is not limited to, obtained by manually annotating the collected complete sample audio signals.

[0133] As an optional implementation, the model structure of the original speech recognition model described above is as follows: Figure 11 As shown in the diagram. The preprocessing layer 1102 preprocesses the received recognition guidance text vector, scene keyword vector, and feature vector of the i-th audio segment. The multi-head self-attention layer 1104, upon receiving the preprocessed recognition guidance text vector, scene keyword vector, and feature vector of the i-th audio segment, captures the dependencies between the scene keyword vector and the i-th audio segment feature vector, generates feature representations between these vectors, and learns the semantic information in the recognition guidance text vector to obtain the guidance constraints indicated by the recognition guidance text vector. The feedforward network layer 1106, upon receiving the feature representations learned by the multi-head self-attention layer, further captures the complex feature relationships between the feature representations under the aforementioned guidance constraints to generate recognition reference information. The autoregressive decoding layer 1108 decodes the i-th audio segment feature vectors using autoregressive decoding based on the recognition reference information to obtain the i-th preliminary recognition text, and further processes the i-th preliminary recognition text, such as adding punctuation and removing colloquial words, to obtain the i-th text segment. The output layer 1110 performs format checks on the i-th text segment and outputs it.

[0134] When training the model based on sample scene description information, sample recognition guidance prompts, and sample labeled text, the training output results mentioned above can be, but are not limited to, obtained based on loss function calculations. The aforementioned staged training can, but is not limited to, progressively training each layer of the original speech recognition model. For example, when training a model based on... Figure 11 The model shown is trained by first training the preprocessing layer based on the loss function, then training the multi-head self-attention layer based on the trained preprocessing layer, and so on, until the output layer is trained. The loss function remains unchanged throughout this phased training process.

[0135] The embodiments provided in this application train the original speech recognition model step by step before the speech recognition process, thereby reducing errors during model training, improving the accuracy of model training, and optimizing the effect of model parameter adjustment by gradually completing the training of the model.

[0136] Specific combination Figure 12 To illustrate the speech recognition process of actual long audio files using the embodiments of this application:

[0137] S1202, the obtained text guidance "based on the given full-text pre-recognition text and auxiliary information, the entire speech is recognized sentence by sentence" is converted into vector form through the Word2vec model to obtain the recognition guidance vector.

[0138] S1204 converts the acquired auxiliary information into vectors to obtain the scene keyword vectors for the voice interaction scenario. Here, the voice interaction scenario is a completed meeting, and the auxiliary information includes the meeting's topic, participants, and several professional terms related to the meeting topic.

[0139] S1206: The complete audio of this meeting is segmented and encoded to obtain multiple audio encoding vectors. The segmentation operation here is performed using a conventional Voice Activity Detection (VAD) system.

[0140] S1208, convert multiple audio encoding vectors to text space respectively to obtain N audio segment feature vectors with the same dimension as the recognition guide text vector, where N is a natural number greater than 1.

[0141] In step S1210, the obtained full-text pre-recognition text, the recognition guidance vectors obtained in steps S1202 to S1208, the N audio segment feature vectors, and the scene keyword vectors are input into the speech recognition model for speech recognition. First, the speech recognition model decodes the feature vector of the first audio segment among the N audio segment feature vectors to obtain the first recognized text. Then, based on the scene keyword vector, the full-text pre-recognition text, and the first recognized text, recognition reference information is generated. The feature vector of the second audio segment is decoded according to the recognition reference information to obtain the second recognized text, and so on, until the feature vector of the Nth audio segment is decoded. After the decoding process, the speech recognition model performs punctuation addition, de-colloquialization, inverse text normalization, and audio event tagging on the N decoded recognized texts to obtain the processed N text segments.

[0142] S1212, the N text segments obtained from the above speech recognition process are concatenated in chronological order to obtain the full-text recognition text.

[0143] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0144] According to an embodiment of this application, a speech recognition device for implementing the above-described speech recognition method is also provided, such as... Figure 13 As shown, the device includes:

[0145] The first acquisition unit 1302 is used to acquire scene description information corresponding to the voice interaction scene, as well as recognition guidance prompt information configured for the voice interaction scene;

[0146] The first conversion unit 1304 is used to convert the audio signal collected in the voice interaction scenario to obtain at least one audio segment feature vector.

[0147] The recognition unit 1306 is used to, when it has obtained the first i-1 text segments by speech recognition of the feature vectors of the first i-1 audio segments, recognize the feature vector of the i-th audio segment based on the first i-1 text segments, scene description information and recognition guidance prompt information, to obtain the i-th text segment, where i is a natural number greater than 1 and less than or equal to N, and N is the total number of feature vectors of at least one audio segment.

[0148] The generation unit 1308 is used to generate speech recognition text corresponding to the audio signals collected in the speech interaction scenario based on the acquired N text fragments.

[0149] Optionally, the embodiments in this solution may be, but are not limited to, the embodiments of the reference method, which will not be described in detail here.

[0150] As an optional solution, the identification unit 1306 includes:

[0151] The acquisition module is used to acquire scene keyword vectors that match scene description information and recognition guidance text vectors that match recognition guidance prompts. The scene keywords indicated by the scene keyword vectors are used to assist in identifying the speech interaction domain information corresponding to the audio segment feature vectors, and the recognition guidance text vectors are used to guide the recognition result text.

[0152] The recognition module, in the speech recognition model, under the guidance constraints indicated by the recognition guidance text vector, recognizes the feature vector of the i-th audio segment based on the scene keyword vector and the first i-1 text segments to obtain the i-th text segment. The speech recognition model is a neural network model used to recognize the text information corresponding to the feature vector of the audio segment after step-by-step training of the model network parameters.

[0153] Optionally, the embodiments in this solution may be, but are not limited to, the above-described reference method embodiments, which will not be repeated here.

[0154] As an optional solution, the identification module includes:

[0155] The first generation submodule is used in the speech recognition model to generate first recognition reference information based on scene keyword vectors and the first i-1 text fragments under guided constraints.

[0156] The first decoding submodule is used to decode the feature vector of the i-th audio segment based on the first recognition reference information to obtain the i-th text segment.

[0157] Optionally, the embodiments in this solution may be, but are not limited to, the above-described reference method embodiments, which will not be repeated here.

[0158] As an optional solution, the recognition module also includes:

[0159] The second generation submodule is used to generate second recognition reference information in the speech recognition model under guided constraints, based on the scene keyword vector, the first i-1 text segments, and the feature vector of the i-th audio segment.

[0160] The second decoding submodule is used to decode the feature vector of the i-th audio segment according to the second recognition reference information to obtain the i-th text segment.

[0161] Optionally, the embodiments in this solution may be, but are not limited to, the embodiments of the reference method, which will not be described in detail here.

[0162] As an optional solution, the recognition module also includes:

[0163] The third generation submodule is used to generate third recognition reference information in the speech recognition model under guided constraints, based on the scene keyword vector, the first i-1 text segments and the reference pre-recognition text, when a reference pre-recognition text matching the complete audio signal collected in the voice interaction scenario is obtained.

[0164] The third decoding submodule is used to decode the feature vector of the i-th audio segment based on the third recognition reference information to obtain the i-th text segment.

[0165] Optionally, the embodiments in this solution may be, but are not limited to, the above-described reference method embodiments, which will not be repeated here.

[0166] As an optional solution, the recognition module also includes:

[0167] The recognition submodule, based on the scene keyword vector and the first i-1 text segments, recognizes the feature vector of the i-th audio segment to obtain the intermediate text segment;

[0168] The adjustment submodule is used to adjust the intermediate text fragment to the i-th text fragment that satisfies the guiding constraints.

[0169] Optionally, the embodiments in this solution may be, but are not limited to, the embodiments of the reference method, which will not be described in detail here.

[0170] As an optional solution, the first conversion unit 1304 includes:

[0171] The first encoding module is used to encode the audio signal to obtain an audio encoding matrix;

[0172] The first mapping module is used to map the audio encoding matrix to the text space based on the recognition guidance prompts to obtain candidate text vector groups;

[0173] The determination module is used to treat each candidate text vector in the candidate text vector group as an audio segment feature vector.

[0174] Optionally, the embodiments in this solution may be, but are not limited to, the embodiments of the reference method, which will not be described in detail here.

[0175] As an optional solution, the first conversion unit 1304 further includes:

[0176] The segmentation module is used to segment the audio signal into multiple audio segments;

[0177] The second encoding module is used to encode multiple audio segments separately to obtain multiple audio segment encoding vectors;

[0178] The second mapping module is used to map the encoding vectors of multiple audio segments to the text space according to the recognition guidance prompts, so as to obtain at least one audio segment feature vector.

[0179] Optionally, the embodiments in this solution may be, but are not limited to, the embodiments of the reference method, which will not be described in detail here.

[0180] As an optional embodiment, the above-mentioned device further includes at least one of the following:

[0181] Add a unit to add a separator symbol for dividing the sentence structure to the i-th text segment;

[0182] The removal unit is used to remove tone-expressing words from the i-th text segment;

[0183] The processing unit is used to perform inverse text formatting on the text content that belongs to the string in the i-th text segment;

[0184] The tagging unit is used to tag the voice interaction events that appear in the i-th text segment, so as to obtain the i-th text segment carrying event tags.

[0185] Optionally, the embodiments in this solution may be, but are not limited to, the embodiments of the reference method, which will not be described in detail here.

[0186] As an optional solution, the above-mentioned device further includes:

[0187] The decoding unit is used to decode the feature vector of the first audio segment based on the scene keyword vector under the guidance constraint, when the feature vector of the i-th audio segment is the feature vector of the first audio segment, to obtain the first text segment.

[0188] Optionally, the embodiments in this solution may be, but are not limited to, the embodiments of the reference method, which will not be described in detail here.

[0189] As an optional solution, the above-mentioned device further includes:

[0190] The second acquisition unit is used to acquire sample scene description information corresponding to the sample voice interaction scene, as well as sample recognition guidance prompt information configured for the sample voice interaction scene.

[0191] The second conversion unit is used to convert the complete sample audio signal collected in the sample voice interaction scenario to obtain at least one sample audio segment feature vector.

[0192] The annotation unit is used to annotate the complete sample audio signal to obtain sample annotation text that matches the complete sample audio signal;

[0193] The training unit is used to train the original speech recognition model in stages based on sample scene description information, sample recognition guidance prompts and sample annotation text to obtain a speech recognition model. In each stage of training, some model parameters in the speech recognition model are adjusted based on the training output results. The training output results are used to indicate the difference between the output text of the speech recognition model and the sample annotation text.

[0194] According to one aspect of this application, a computer-readable storage medium is provided, from which a processor of a computer device reads computer instructions, and the processor executes the computer instructions, causing the computer device to perform the method provided in various alternative implementations of the above-described speech recognition method.

[0195] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps:

[0196] S1, obtain the scene description information corresponding to the voice interaction scene, as well as the recognition guidance prompt information configured for the voice interaction scene;

[0197] S2, perform signal conversion on the audio signals collected in the voice interaction scenario to obtain at least one audio segment feature vector;

[0198] S3, after obtaining the first i-1 text segments obtained by speech recognition of the feature vectors of the first i-1 audio segments, based on the first i-1 text segments, scene description information and recognition guidance prompt information, the feature vector of the i-th audio segment is recognized to obtain the i-th text segment, where i is a natural number greater than 1 and less than or equal to N, and N is the total number of feature vectors of at least one audio segment.

[0199] S4 generates speech recognition text corresponding to the audio signals collected in the voice interaction scenario based on the acquired N text fragments.

[0200] According to one aspect of this application, a computer program product is provided, the computer program product comprising a computer program / instructions containing program code for performing the above-described speech recognition method.

[0201] Figure 14A schematic block diagram of a computer system architecture for implementing an electronic device according to embodiments of the present application is shown.

[0202] It should be noted that, Figure 14 The computer system of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0203] like Figure 14 As shown, the computer system includes a central processing unit (CPU) 1401, which performs various appropriate actions and processes based on programs stored in read-only memory (ROM) 1402 or programs loaded from storage section 1408 into random access memory (RAM) 1403. The RAM 1403 also stores various programs and data required for system operation. The CPU 1401, ROM 1402, and RAM 1403 are interconnected via a bus 1404. An input / output interface 1405 (I / O interface) is also connected to the bus 1404.

[0204] The following components are connected to the input / output interface 14014: an input section 1406 including a keyboard, mouse, etc.; an output section 1407 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1408 including a hard disk, etc.; and a communication section 1409 including a network interface card such as a local area network card, modem, etc. The communication section 1409 performs communication processing via a network such as the Internet. A drive 1410 is also connected to the input / output interface 14014 as needed. Removable media 1411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on the drive 1410 as needed so that computer programs read from them can be installed into the storage section 1408 as needed.

[0205] Specifically, according to embodiments of this application, the processes described in the various method flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1409, and / or installed from removable medium 1411. When the computer program is executed by central processing unit 1401, it performs various functions defined in the system of this application.

[0206] According to another aspect of the embodiments of this application, an electronic device for implementing the above-described object recognition method is also provided. For example... Figure 15 As shown, the electronic device includes a memory 1502 and a processor 1504. The memory 1502 stores a computer program, and the processor 1504 is configured to execute the steps of any of the above method embodiments via the computer program.

[0207] Optionally, in this embodiment, the aforementioned electronic device may be located in at least one of a plurality of network devices in a computer network.

[0208] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0209] S1, obtain the scene description information corresponding to the voice interaction scene, as well as the recognition guidance prompt information configured for the voice interaction scene;

[0210] S2, perform signal conversion on the audio signals collected in the voice interaction scenario to obtain at least one audio segment feature vector;

[0211] S3, after obtaining the first i-1 text segments obtained by speech recognition of the feature vectors of the first i-1 audio segments, based on the first i-1 text segments, scene description information and recognition guidance prompt information, the feature vector of the i-th audio segment is recognized to obtain the i-th text segment, where i is a natural number greater than 1 and less than or equal to N, and N is the total number of feature vectors of at least one audio segment.

[0212] S4 generates speech recognition text corresponding to the audio signals collected in the voice interaction scenario based on the acquired N text fragments.

[0213] Alternatively, as those skilled in the art will understand, Figure 15The structure shown is for illustrative purposes only. Electronic devices can also be smartphones (such as Android phones, iOS phones, etc.), tablets, PDAs, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 15 This does not limit the structure of the aforementioned electronic devices. For example, the electronic device may also include components that are more... Figure 15 The more or fewer components shown (such as network interfaces, etc.), or having the same Figure 15 The different configurations shown.

[0214] The memory 1502 can be used to store software programs and modules, such as the program instructions / modules corresponding to the speech recognition method and device in this embodiment. The processor 1504 executes various functional applications and data processing by running the software programs and modules stored in the memory 1502, thereby realizing the aforementioned speech recognition method. The memory 1502 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1502 may further include memory remotely located relative to the processor 1504, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Specifically, the memory 1502 may be used, but is not limited to, for information such as acquired audio signals. As an example, such as... Figure 15 As shown, the memory 1502 may include, but is not limited to, the first acquisition unit 1302, the first conversion unit 1304, the recognition unit 1306, and the generation unit 1308 in the speech recognition device. Furthermore, the memory 1502 may also include, but is not limited to, other module units in the speech recognition device, which will not be elaborated upon in this example.

[0215] Optionally, the transmission device 1506 described above is used to receive or send data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 1506 includes a Network Interface Controller (NIC), which can be connected to other network devices and a router via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 1506 is a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0216] In addition, the aforementioned electronic device also includes: a display 1508 for displaying the aforementioned voice-recognized text and a connection bus 1510 for connecting the various module components in the aforementioned electronic device.

[0217] In other embodiments, the aforementioned terminal device or server can be a node in a distributed system, wherein the distributed system can be a blockchain system, which is a distributed system formed by connecting multiple nodes through network communication. The nodes can form a point-to-point network, and any form of computing device, such as a server, terminal, or other electronic device, can become a node in the blockchain system by joining this point-to-point network.

[0218] Optionally, in embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0219] Optionally, in this embodiment, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0220] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0221] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0222] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0223] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0224] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0225] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A speech recognition method, characterized in that, include: Obtain scene description information corresponding to the voice interaction scenario, as well as recognition guidance prompts configured for the voice interaction scenario; The audio signals collected in the voice interaction scenario are converted to obtain at least one audio segment feature vector; Given the first i-1 text segments obtained by speech recognition of the feature vectors of the first i-1 audio segments, based on the first i-1 text segments, the scene description information and the recognition guidance prompt information, the feature vector of the i-th audio segment is recognized to obtain the i-th text segment, where i is a natural number greater than 1 and less than or equal to N, and N is the total number of feature vectors of the at least one audio segment. Based on the acquired N text fragments, the speech recognition text corresponding to the audio signal collected in the voice interaction scenario is generated.

2. The method according to claim 1, characterized in that, The step of recognizing the feature vector of the i-th audio segment based on the first i-1 text segments, the scene description information, and the recognition guidance prompt information, to obtain the i-th text segment includes: Obtain a scene keyword vector that matches the scene description information and an identification guidance text vector that matches the identification guidance prompt information. The scene keywords indicated by the scene keyword vector are used to assist in identifying the voice interaction domain information corresponding to the audio segment feature vector, and the identification guidance text vector is used to guide the identification of the result text. In the speech recognition model, under the guidance constraints indicated by the recognition guidance text vector, the feature vector of the i-th audio segment is recognized based on the scene keyword vector and the first i-1 text segments to obtain the i-th text segment. The speech recognition model is a neural network model for recognizing text information corresponding to the feature vector of the audio segment, obtained by step-by-step training of the model network parameters.

3. The method according to claim 2, characterized in that, In the speech recognition model, under the guidance constraints indicated by the recognition guidance text vector, the feature vector of the i-th audio segment is recognized based on the scene keyword vector and the first i-1 text segments, resulting in the i-th text segment including: In the speech recognition model, under the guided constraints, first recognition reference information is generated based on the scene keyword vector and the first i-1 text segments; The feature vector of the i-th audio segment is decoded based on the first identification reference information to obtain the i-th text segment.

4. The method according to claim 2, characterized in that, In the speech recognition model, under the guidance constraints indicated by the recognition guidance text vector, the feature vector of the i-th audio segment is recognized based on the scene keyword vector and the first i-1 text segments, resulting in the i-th text segment including: In the speech recognition model, under the guided constraints, second recognition reference information is generated based on the scene keyword vector, the first i-1 text segments, and the feature vector of the i-th audio segment; The feature vector of the i-th audio segment is decoded based on the second identification reference information to obtain the i-th text segment.

5. The method according to claim 2, characterized in that, In the case of obtaining a reference pre-recognition text that matches the complete audio signal collected in the voice interaction scenario, in the speech recognition model, under the guidance constraints indicated by the recognition guidance text vector, based on the scenario keyword vector and the first i-1 text segments, the feature vector of the i-th audio segment is recognized to obtain the i-th text segment, which further includes: In the speech recognition model, under the guidance constraints, a third recognition reference information is generated based on the scene keyword vector, the first i-1 text segments, and the reference pre-recognition text; The feature vector of the i-th audio segment is decoded based on the third identification reference information to obtain the i-th text segment.

6. The method according to claim 2, characterized in that, Under the guidance constraints indicated by the recognition guidance text vector, based on the scene keyword vector and the first i-1 text segments, the feature vector of the i-th audio segment is recognized to obtain the i-th text segment, which includes: Based on the scene keyword vector and the first i-1 text segments, the feature vector of the i-th audio segment is identified to obtain the intermediate text segment; The intermediate text fragment is adjusted to the i-th text fragment that satisfies the guiding constraint conditions.

7. The method according to claim 1, characterized in that, The step of converting the audio signals collected in the voice interaction scenario to obtain at least one audio segment feature vector includes: The audio signal is encoded to obtain an audio encoding matrix; Based on the recognition guidance prompts, the audio encoding matrix is ​​mapped to the text space to obtain candidate text vector groups; Each candidate text vector in the candidate text vector group is used as a feature vector for the audio segment.

8. The method according to claim 1, characterized in that, The step of converting the audio signals collected in the voice interaction scenario to obtain at least one audio segment feature vector includes: The audio signal is segmented to obtain multiple audio segments; The multiple audio segments are encoded separately to obtain multiple audio segment encoding vectors; Based on the identification guidance prompt information, the encoded vectors of the multiple audio segments are mapped to the text space respectively to obtain the feature vector of the at least one audio segment.

9. The method according to claim 1, characterized in that, After identifying the feature vector of the i-th audio segment to obtain the i-th text segment, the process further includes: Perform at least one of the following processing tasks on the i-th text segment: Add a separator symbol to the i-th text segment to divide the sentence structure; Remove tone-expressing words from the i-th text segment; Perform inverse text formatting on the text content that belongs to the string in the i-th text segment; Tag the voice interaction events that appear in the i-th text segment to obtain the i-th text segment carrying event tags.

10. The method according to claim 2, characterized in that, After performing signal conversion on the audio signals acquired in the voice interaction scenario to obtain at least one audio segment feature vector, the method further includes: When the feature vector of the i-th audio segment is the feature vector of the first audio segment, under the guidance constraint, the feature vector of the first audio segment is decoded based on the scene keyword vector to obtain the first text segment.

11. The method according to any one of claims 2 to 10, characterized in that, Before obtaining the scene description information corresponding to the voice interaction scene and the recognition guidance prompt information configured for the voice interaction scene, the method further includes: Obtain the sample scene description information corresponding to the sample voice interaction scene, as well as the sample recognition guidance prompt information configured for the sample voice interaction scene; The complete sample audio signal collected in the sample voice interaction scenario is converted into a signal to obtain at least one sample audio segment feature vector; The complete sample audio signal is annotated to obtain sample annotation text that matches the complete sample audio signal; Based on the sample scene description information, the sample recognition guidance prompt information, and the sample annotation text, the original speech recognition model is trained in stages to obtain the speech recognition model. In each stage of training, some model parameters in the speech recognition model are adjusted based on the training output results. The training output results are used to indicate the difference between the output text of the speech recognition model in training and the sample annotation text.

12. A voice recognition device, characterized in that, include: The first acquisition unit is used to acquire scene description information corresponding to the voice interaction scene, as well as recognition guidance prompt information configured for the voice interaction scene; The first conversion unit is used to perform signal conversion on the audio signal collected in the voice interaction scenario to obtain at least one audio segment feature vector. The recognition unit is configured to, upon obtaining the first i-1 text segments obtained by speech recognition of the feature vectors of the first i-1 audio segments, recognize the feature vector of the i-th audio segment based on the first i-1 text segments, the scene description information, and the recognition guidance prompt information, to obtain the i-th text segment, where i is a natural number greater than 1 and less than or equal to N, and N is the total number of feature vectors of the at least one audio segment; The generation unit is used to generate speech recognition text corresponding to the audio signal collected in the voice interaction scenario based on the acquired N text fragments.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method described in any one of claims 1 to 11.

14. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 11.

15. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method described in any one of claims 1 to 11 through the computer program.

Images