Method and apparatus for processing voice data, storage medium, and electronic device

Abstract

This application discloses a method and apparatus for processing voice data, a storage medium, and an electronic device, relating to the field of smart home / intelligent home technology. The method includes: acquiring a current interactive statement and a statement to be transcribed corresponding to the current interactive statement, wherein the current interactive statement is an interactive statement extracted from voice data collected by a voice device during the current round of voice interaction, and the statement to be transcribed is an interactive statement acquired before the current round of voice interaction; performing word segmentation on the statement to be transcribed to obtain a set of first words corresponding to the statement to be transcribed; inputting the statement to be transcribed and the current interactive statement into a target transcription model to obtain a target editing matrix, wherein the target editing matrix is ​​used to indicate the characters to be transcribed in the statement to be transcribed; and verifying the target editing matrix according to the set of first words to obtain a verified target editing matrix.

Description

Technical Field

[0001] This application relates to the field of smart home / intelligent home, and more specifically, to a method and apparatus for processing voice data, a storage medium, and an electronic device. Background Technology

[0002] Currently, voice interaction functions can be configured on smart devices. For example, a voice interaction assistant can be run to control the smart device or its associated devices via voice commands. Correspondingly, a smart device configured with voice interaction functions is called a voice device. Because different users have different speaking habits, there are usually a lot of pronouns and ellipsis words in the process of users interacting with voice devices, and the complete meaning needs to be understood by relying on the context.

[0003] In related technologies, multi-turn voice interaction capabilities can be added to voice devices. A transcription model is used to transcribe the current interaction statement based on the previous interaction statement, thereby splicing the context with the current sentence to obtain a rewritten, semantically complete sentence.

[0004] However, transcription models are usually based on word-level transcription. During the transcription process, it is easy to only transcribe a portion of the multiple words that need to be transcribed in the previous interactive statement, which will cause changes in the semantics of the transcription result and reduce the accuracy of the statement transcription.

[0005] It is evident that the speech data processing methods in related technologies suffer from low accuracy in speech transcription due to the tendency for missing content during transcription. Summary of the Invention

[0006] This application provides a method and apparatus for processing voice data, a storage medium, and an electronic device to at least solve the problem of low accuracy in speech transcription caused by the easy occurrence of missing transcription content in related technologies.

[0007] According to one aspect of the embodiments of this application, a method for processing voice data is provided, comprising: acquiring a current interactive statement and a statement to be transcribed corresponding to the current interactive statement, wherein the current interactive statement is an interactive statement extracted from voice data collected by a voice device during a current round of voice interaction, and the statement to be transcribed is an interactive statement acquired before the current round of voice interaction; performing word segmentation on the statement to be transcribed to obtain a set of first words corresponding to the statement to be transcribed; inputting the statement to be transcribed and the current interactive statement into a target transcription model to obtain a target editing matrix, wherein the target editing matrix is ​​used to indicate the characters to be transcribed in the statement to be transcribed; verifying the target editing matrix according to the set of first words to obtain a verified target editing matrix, wherein all characters in the first word to which each character to be transcribed indicated by the verified target editing matrix belongs are characters to be transcribed; inserting the characters to be transcribed in the statement to be transcribed into the current interactive statement according to the verified target editing matrix to obtain a target interactive statement.

[0008] According to another aspect of the embodiments of this application, a speech data processing apparatus is also provided, comprising: an acquisition unit, configured to acquire a current interactive statement and a statement to be transcribed corresponding to the current interactive statement, wherein the current interactive statement is an interactive statement extracted from speech data collected by a speech device during a current round of speech interaction, and the statement to be transcribed is an interactive statement acquired before the current round of speech interaction; a processing unit, configured to perform word segmentation processing on the statement to be transcribed to obtain a set of first words corresponding to the statement to be transcribed; an input unit, configured to input the statement to be transcribed and the current interactive statement into a target transcription model to obtain a target editing matrix, wherein the target editing matrix is ​​used to indicate the characters to be transcribed in the statement to be transcribed; a first verification unit, configured to verify the target editing matrix according to the set of first words to obtain a verified target editing matrix, wherein all characters in the first word to which each character to be transcribed indicated by the verified target editing matrix belongs are characters to be transcribed; and an insertion unit, configured to insert the characters to be transcribed in the statement to be transcribed into the current interactive statement according to the verified target editing matrix to obtain a target interactive statement.

[0009] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer-readable storage medium, and the computer program is configured to execute the above-described voice data processing method when running.

[0010] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the above-described voice data processing method through the computer program.

[0011] In this embodiment, the editing matrix is ​​validated based on the word segmentation results of the sentence to be transcribed. This is achieved by obtaining the current interactive sentence and the corresponding sentence to be transcribed. The current interactive sentence is extracted from the voice data collected by the voice device during the current round of voice interaction, while the sentence to be transcribed is the interactive sentence obtained before the current round of voice interaction. The sentence to be transcribed is then segmented to obtain a set of first words corresponding to it. The sentence to be transcribed and the current interactive sentence are input into the target transcription model to obtain the target editing matrix, which indicates the characters to be transcribed in the sentence. The target editing matrix is ​​then validated according to the set of first words. The verified target edit matrix is ​​obtained, where all characters in the first word to which each character to be transcribed belongs, as indicated by the verified target edit matrix, are characters to be transcribed. The characters to be transcribed in the sentence to be transcribed are inserted into the current interactive sentence according to the verified target edit matrix, resulting in the target interactive sentence. Because the edit matrix is ​​verified based on the word segmentation results of the sentence to be transcribed, the keywords in the sentence to be transcribed can be completely inserted into the current sentence, avoiding semantic changes caused by inserting only parts of the keywords into the current sentence. This achieves the technical effect of improving the accuracy of sentence transcription, thereby solving the problem of low accuracy in speech data processing methods in related technologies due to the easy occurrence of missing transcription content. Attached Figure Description

[0012] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a schematic diagram of the hardware environment for a voice data processing method according to an embodiment of this application;

[0015] Figure 2 This is a flowchart illustrating an optional voice data processing method according to an embodiment of this application;

[0016] Figure 3 This is a flowchart illustrating another optional voice data processing method according to an embodiment of this application;

[0017] Figure 4 This is a structural block diagram of an optional voice data processing apparatus according to an embodiment of this application;

[0018] Figure 5 This is a structural block diagram of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0019] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0020] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0021] According to one aspect of the embodiments of this application, a method for processing voice data is provided. This voice data processing method is widely used in whole-house intelligent digital control application scenarios such as smart homes, smart home ecosystems, and intelligence house ecosystems. Optionally, in this embodiment, the above-mentioned voice data processing method can be applied to, for example... Figure 1 The hardware environment shown consists of terminal device 102 and server 104. For example... Figure 1As shown, server 104 is connected to terminal device 102 via a network and can be used to provide services (such as application services) to the terminal or clients installed on the terminal. A database can be set up on the server or independently of the server to provide data storage services for server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data processing services for server 104.

[0022] The aforementioned network may include, but is not limited to, at least one of the following: wired network, wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: wide area network, metropolitan area network, local area network. The aforementioned wireless network may include, but is not limited to, at least one of the following: Wi-Fi (Wireless Fidelity), Bluetooth. The terminal device 102 may not be limited to PC, mobile phone, tablet computer, smart air conditioner, smart range hood, smart refrigerator, smart oven, smart stove, smart washing machine, smart water heater, smart washing equipment, smart dishwasher, smart projector, smart TV, smart clothes rack, smart curtains, smart audio-visual equipment, smart socket, smart speaker, smart speaker box, smart fresh air equipment, smart kitchen and bathroom equipment, smart bathroom equipment, smart robot vacuum cleaner, smart window cleaning robot, smart mopping robot, smart air purifier, smart steam oven, smart microwave oven, smart water heater, smart air purifier, smart water dispenser, smart door lock, etc.

[0023] The voice data processing method of this application embodiment can be executed by server 104, terminal device 102, or jointly by server 104 and terminal device 102. Alternatively, the voice data processing method of this application embodiment can be executed by a client installed on terminal device 102.

[0024] Taking the voice data processing method in this embodiment as an example, which is executed by a voice device (a type of terminal device 102), Figure 2 This is a flowchart illustrating an optional voice data processing method according to an embodiment of this application, as shown below. Figure 2 As shown, the process of this method may include the following steps:

[0025] Step S202: Obtain the current interactive statement and the corresponding statement to be transcribed. The current interactive statement is the interactive statement extracted from the voice data collected by the voice device during the current round of voice interaction, and the statement to be transcribed is the interactive statement obtained before the current round of voice interaction.

[0026] The voice data processing method in this embodiment can be applied to scenarios where voice data collected by a voice device is processed. The voice device can be an intelligent voice device that can collect voice data during voice interaction with the user and perform corresponding device operations or respond to the user's interactive voice by recognizing the user's intent. The device type of the voice device can be a smart home device, such as a smart washing machine, a smart speaker, a smart air conditioner, etc.

[0027] During voice interaction with a voice device, a user's speech typically contains numerous pronouns and ellipsis, requiring the device to understand the complete meaning based on the context. For example, in a multi-turn voice interaction, a user says, "I want to listen to a song," and the voice device asks, "What song?" The user replies, "Singer A's." The user's complete intent is "I want to listen to singer A's songs," but the current statement "Singer A's" omits the subject, predicate, and object. If, in a subsequent voice interaction, the user says, "Play his song B," the complete meaning is "Play singer A's song B," where the demonstrative pronoun "him" refers to "singer A." In a conversation, the human brain can automatically understand the current statement based on the context, but voice devices can only understand the semantics of the current statement during voice interaction. This leads to semantic gaps, an inability to correctly identify the user's intent, and consequently, an inability to perform corresponding device operations or provide accurate responses.

[0028] To address the aforementioned technical challenges, voice devices can be configured with multi-turn voice interaction capabilities, meaning they can leverage the context of these interactions to identify the user's intent. One approach to multi-turn voice interaction is to infer the response using an end-to-end algorithm or model based on the context and the current statement. Responses inferred from context and the current statement are highly relevant to the business logic, exhibiting a strong coupling between natural language and business requirements. When business logic changes (e.g., the response needs modification), the model needs to be retrained. Furthermore, this approach fails to address the issue of different voice devices potentially providing different responses to the same context.

[0029] Alternatively, dialogue rewriting can be used. This involves rewriting the current statement based on the preceding text, eliminating pronouns, and completing omitted components to create a semantically complete statement. Then, semantic parsing is performed on the current statement to generate a response. Dialogue rewriting can include pronoun elimination and omission completion. The dialogue rewriting process can be divided into two steps: first, pronoun elimination, then omission completion, and finally, the two results are integrated. However, this method has a lengthy processing flow, and the two results may not be well integrated.

[0030] To shorten the processing flow and improve the accuracy of dialogue rewriting, the following dialogue rewriting method can also be adopted: concatenate the context and the current sentence, that is, splice the context and the current sentence together, and directly generate the rewritten sentence through the model. The above dialogue rewriting method can be divided into the following two schemes:

[0031] Scheme 1: Use a pure generative model to directly concatenate the previous text and the current sentence to generate the rewritten sentence. The pure generative model can be BertForMLM (Bert (Bidirectional Encoder Representations, a language representation model) from Transformer For Masked Language Model, a language representation model that masks some information) or GPT (Generative Pre-Training, a generative pre-training model). However, since the words generated by the pure generative model are random, the randomly generated sentences have poor effects and may not be relevant to the current dialogue, and are not applicable to fields with strong directivity such as the household appliance control field.

[0032] Scheme 2: Concatenate the previous text and the current sentence through a model (a transcribing model) to generate an editing matrix, and identify in the editing matrix the words in the previous text that need to be inserted into the current sentence and the words in the current sentence that need to be inserted or replaced, so as to insert or replace the words in the previous text into the current sentence. The sentences rewritten by the above method are strongly relevant to the current dialogue and are applicable to the household appliance control field.

[0033] For example, in the process of multi-turn voice interaction, the collected interaction sentences are "Turn on the air conditioner" and "Set it to 24 degrees" in sequence, then the generated editing matrix is shown in Table 1:

[0034] Table 1

[0035] beat 0 0 0 0 open 0 0 0 0 null 1 0 0 0 Adjust 1 0 0 0 set up for 24 Spend

[0036] Among them, 0 represents no operation, and 1 represents inserting in front of the current position. The above editing matrix means inserting "air" and "conditioner" in front of "set", and keeping other words unchanged.

[0037] However, current transcription models are based on word-level granularity. That is, the sentence to be transcribed is broken down into characters, and each character is used to determine which characters from the preceding text need to be transcribed into the current sentence. During the transcription process, keywords may not be fully supplemented or replaced. For example, keywords from the preceding text might be segmented, and only parts of the keywords might be transcribed into the current sentence, resulting in a change in the semantics of the transcribed sentence. For instance, if "XYZ" appears in the preceding text, only "XY" might be transcribed into the current sentence. In the field of home appliance control, keywords are crucial; missing keywords can lead to a final parsing result that is completely different from the user's intent.

[0038] To at least partially address the aforementioned issues, this embodiment optimizes the method of rewriting sentences using a model. By combining the word segmentation results with the keyword integrity optimization of the edit matrix calculated by the model, the keywords in the generated sentence remain complete, avoiding the transcribing of incomplete keywords into the current sentence. At the same time, it also avoids the situation where the transcribed keywords are transcribed in the wrong position, thereby improving the accuracy of sentence transcribing.

[0039] When interaction with a voice device is required, the user can send interactive voice messages. The voice device may be equipped with voice acquisition components, such as microphones, microphone arrays, or pickup units. The voice device can acquire the current voice data collected by its voice acquisition components. The voice device (e.g., its processor) can extract the current interactive statement from the current voice data. Here, the current voice data is the voice data acquired by the voice device during the current round of voice interaction. The current round of voice interaction can be the Nth round, where N is a positive integer greater than or equal to 2. In addition to the current interactive statement, the corresponding statement to be transcribed can also be acquired.

[0040] Here, the statement to be transcribed is the interactive statement to be transcribed into the current interactive statement, or in other words, the interactive statement used for the current interactive statement. The statement to be transcribed is the interactive statement obtained before the current round of voice interaction. It can be the previous interactive statement or the interactive statement obtained after rewriting the statement. In this embodiment, there is no limitation on the statement to be transcribed.

[0041] Step S204: Perform word segmentation on the sentence to be transcribed to obtain a first set of words corresponding to the sentence to be transcribed.

[0042] In this embodiment, the voice device can perform word segmentation on the statement to be transcribed to obtain a set of first words corresponding to the statement. The word segmentation operation can be performed after the statement to be transcribed is obtained, or after the current interactive statement is obtained, or after it is determined that the current interactive statement needs to be rewritten. The word segmentation operation can be performed using a word segmentation tool. For example, the word segmentation tool can be an open-source word segmentation tool or a custom word segmentation tool. For instance, a custom keyword dictionary can be added to the word segmentation tool's lexicon.

[0043] Step S206: Input the statement to be transcribed and the current interactive statement into the target transcription model to obtain the target editing matrix, wherein the target editing matrix is ​​used to indicate the words to be transcribed in the statement to be transcribed.

[0044] After obtaining the current statement to be transcribed and the current interactive statement, the statement to be transcribed and the current interactive statement can be input into the target transcription model. The target transcription model outputs the target editing matrix. The target editing matrix identifies the characters to be transcribed in the statement to be transcribed based on the character granularity. For example, if "turn on the air conditioner" is taken as the statement to be transcribed and "set to 24 degrees" is taken as the current interactive statement, the aforementioned editing matrix can be obtained.

[0045] Optionally, the target transcription model can be a transcription model based on RUN (Rewritten U-shaped Network) or other types of transcription models. In some examples of this embodiment, the target transcription model is illustrated using a transcription model based on RUN.

[0046] Step S208: Verify the target editing matrix according to a set of first words to obtain the verified target editing matrix. In this matrix, all characters in the first word to which each character to be transcribed belongs are characters to be transcribed.

[0047] Since the target edit matrix segments the sentence to be transcribed based on word granularity, the transcription identifier of a word in the sentence to be transcribed may be different. For example, for the keyword "air conditioner", the target edit matrix may set the word "empty" or "adjust" as the word to be transcribed, but not the other word. If the transcription is performed according to the target edit matrix, only the word "empty" or "adjust" will be transcribed into the current sentence, which may cause the semantics of the current interactive sentence to change or be missing.

[0048] In this embodiment, the target editing matrix can be verified based on the word segmentation results of the sentence to be transcribed, so that all characters in the first word to which the character to be transcribed belongs are set as characters to be transcribed. This can avoid semantic loss or semantic change caused by transcribing some characters in the keywords of the sentence to be transcribed into the current interactive sentence.

[0049] Step S210: Insert the characters to be transcribed from the target edit matrix into the current interactive statement according to the target edit matrix after verification, and obtain the target interactive statement.

[0050] In the edit matrix, different markers can be used to identify words to be transcribed and words that do not need to be transcribed. For example, a first marker (e.g., the number 1) can be used to identify words to be transcribed, and a second marker can be used to identify words that do not need to be transcribed. Furthermore, the first marker is set to the matrix position where the word to be transcribed matches the word to be inserted in the current interactive statement (e.g., before or after this word), while the second marker is set to the other matrix positions. An example of an edit matrix is ​​shown above.

[0051] For the statement to be transcribed and the current interactive statement, the characters to be transcribed in the statement to be transcribed and the insertion positions in the current interactive statement can be determined based on the target editing matrix. Here, the insertion position is the location in the current interactive statement where the characters to be transcribed are to be inserted, which can be indicated by specifying the characters before or after the insertion position. The characters to be transcribed in the statement to be transcribed are then inserted into the insertion positions in the current interactive statement to obtain the target interactive statement. Here, if consecutive characters (possibly belonging to the same keyword) are transcribed into the current interactive statement, the transcribed order is consistent with the order in the statement to be transcribed.

[0052] Through steps S202 to S210, the current interactive statement and the corresponding statement to be transcribed are obtained. The current interactive statement is extracted from the speech data collected by the voice device during the current round of voice interaction, and the statement to be transcribed is the interactive statement obtained before the current round of voice interaction. The statement to be transcribed is segmented to obtain a set of first words corresponding to the statement to be transcribed. The statement to be transcribed and the current interactive statement are input into the target transcription model to obtain the target editing matrix, which is used to indicate the characters to be transcribed in the statement to be transcribed. The target editing matrix is ​​verified according to the set of first words to obtain the verified target editing matrix. All characters in the first word to which each character to be transcribed indicated by the verified target editing matrix belong are characters to be transcribed. The characters to be transcribed in the statement to be transcribed are inserted into the current interactive statement according to the verified target editing matrix to obtain the target interactive statement. This solves the problem of low accuracy in speech transcription caused by the easy occurrence of missing transcription content in the speech data processing methods of related technologies, and improves the accuracy of speech transcription.

[0053] In one exemplary embodiment, the target edit matrix is ​​validated according to a set of first words to obtain a validated target edit matrix, including:

[0054] S11, Based on the target editing matrix, determine each word to be transcribed in the sentence to be transcribed;

[0055] S12, determine the first word to which each character to be transcribed belongs in a set of first words;

[0056] S13, if there is a non-transcribing character in the first target word to which the first character to be transcribed belongs in the sentence to be transcribed, the target editing matrix is ​​verified to obtain the verified target editing matrix.

[0057] When validating the target editing matrix, each character to be transcribed in the statement can be determined by traversing the target editing matrix. During traversal, the matrix position corresponding to each character in the statement to be transcribed can be determined. If the first identifier is encountered, the currently traversed character is identified as the character to be transcribed. At most one matrix position with the first identifier exists corresponding to a character in the statement to be transcribed. If more than two matrix positions with the first identifier exist corresponding to a character in the statement to be transcribed, it indicates an error in the editing matrix, and an error is directly reported, without further statement rewriting operations.

[0058] Based on the word segmentation results of the sentence to be transcribed, it can be determined whether the first word to which each character to be transcribed belongs, and all characters in that first word, are also characters to be transcribed. If none of the characters to which the first word to which all characters to be transcribed belong in the sentence to be transcribed are characters not to be transcribed, then the target edit matrix is ​​not modified. If the first word to which the character to be transcribed belongs contains characters not to be transcribed, for example, if the first target word to which the first character to be transcribed belongs contains characters not to be transcribed, then the target edit matrix can be verified to obtain a verified target edit matrix. The characters in the first target word indicated by the verified target edit matrix are all characters to be transcribed, and the insertion position of the first target word in the current interactive sentence is the insertion position of the first character to be transcribed in the current interactive sentence.

[0059] In this embodiment, by traversing the editing matrix, it is determined whether there are any words in the sentence to be transcribed that simultaneously contain words to be transcribed and words not to be transcribed. For keywords that simultaneously contain words to be transcribed and words not to be transcribed, the editing matrix is ​​verified, which can improve the accuracy of the editing matrix verification.

[0060] In one exemplary embodiment, determining each word to be transcribed in the statement to be transcribed, based on the target editing matrix, includes:

[0061] S21, for each character in the sentence to be transcribed, treat it as the current character and perform the following steps to determine whether the current character is the character to be transcribed:

[0062] Iterate through the target editing matrix, identifying the matrix positions corresponding to the current character;

[0063] If a target value exists at the matrix position corresponding to the current character, the current character is determined to be a character to be transcribed.

[0064] If there is no target value at the matrix position corresponding to the current character, it is determined that the current character is not the character to be transcribed.

[0065] To determine the characters to be transcribed in a given statement, each character in the statement can be considered as the current character. This is done by traversing all matrix positions in the target editing matrix corresponding to the current character. If a target value exists at a matrix position corresponding to the current character (as in the example of the first identifier mentioned earlier), the current character is determined to be a character to be transcribed. If no target value exists at any matrix position corresponding to the current character in the editing matrix, the current character is determined not to be a character to be transcribed; that is, the current character is determined to be a non-transcribed character.

[0066] For example, as shown in Table 1, in the editing matrix, the number 1 (i.e., the target value) indicates that it needs to be transcribed, and the number 0 indicates that it does not need to be transcribed. By checking whether the number 1 exists in the matrix position (i.e., the corresponding row) of a character in the previous text, it can be determined whether the character is a character to be transcribed.

[0067] Through this embodiment, by traversing the matrix positions corresponding to each character in the statement to be transcribed in the editing matrix and determining whether the currently traversed character is a character to be transcribed based on whether the target value is found, the convenience of finding the characters to be transcribed is improved, and the efficiency of statement transcription is increased.

[0068] In one exemplary embodiment, verifying the target editing matrix to obtain the verified target editing matrix includes:

[0069] S31. According to the target editing matrix, determine the reference character corresponding to the first character to be transcribed in the current interaction statement. In the target editing matrix, the matrix positions corresponding to the first character to be transcribed and the reference character are the target value, and the reference character is used to indicate the position where the first character to be transcribed in the current interaction statement is to be inserted;

[0070] S32. Set the values of the matrix positions corresponding to the non-characters to be transcribed and the reference character in the first target word in the target editing matrix to the target value to obtain the verified target editing matrix.

[0071] When verifying the target editing matrix, there may be multiple matrix positions corresponding to the first character to be transcribed in the target editing matrix. Among them, the matrix positions corresponding to the first character to be transcribed and the reference character are the target value. Here, the reference character belongs to the current interaction statement, and it can be used to indicate the position where the first character to be transcribed in the current interaction statement is to be inserted. For example, the above text (i.e., the statement to be transcribed) is "Turn on the air conditioner", and the current statement (i.e., the current interaction statement) is "Set to 24 degrees". After inputting the two into the RUN model, the obtained editing matrix is shown in Table 2:

[0072] Table 2[[ID=,18]]

[0073] beat 0 0 0 0 open 0 0 0 0 null 0 0 0 0 Adjust 1 0 0 0 set up for 24 Spend

[0074] Among them, the "iao" in the previous sentence is the character to be transcribed, and its corresponding reference character is "set", that is, the character "iao" is inserted in front of the character "set". If the statement is rewritten according to the above editing matrix, it may be incorrectly rewritten as "iao set to 24 degrees".

[0075] The first word to which the first character to be transcribed belongs is the first target word, and the first target word contains non-characters to be transcribed. The values of the matrix positions corresponding to the non-characters to be transcribed and the reference character in the first target word in the target editing matrix can be set to the target value to obtain the verified target editing matrix.

[0076] For example, the word segmentation result for the phrase "turn on the air conditioner" is "open" and "air conditioner". The values ​​in the matrix positions corresponding to "adjust" and "set" are 1, while the values ​​in the matrix positions corresponding to "empty" and "set" are 0, as shown in Table 2. Based on the word segmentation results, the editing matrix is ​​validated by setting the values ​​in the matrix positions corresponding to "empty" and "set" to 1. The validated editing matrix is ​​shown in Table 3.

[0077] Table 3

[0078] beat 0 0 0 0 open 0 0 0 0 null 1 0 0 0 Adjust 1 0 0 0 set up for 24 Spend

[0079] Based on the verified edit matrix, the current sentence is rewritten, and the transliteration result is "Air conditioner set to 24 degrees". It can be seen that by performing secondary processing on the edit matrix, the keywords can be completed, preventing the rewritten sentence from containing incomplete keywords.

[0080] This embodiment improves the convenience of editing matrix verification by using reference characters in the current statement to identify non-transcribing characters that need adjustment in the statement to be transcribed in the editing matrix.

[0081] In one exemplary embodiment, the target transcription model includes a pre-trained model, a long short-term memory network, a convolutional neural network, and a fully connected layer; the statement to be transcribed and the current interactive statement are input into the target transcription model to obtain a target editing matrix, including:

[0082] S41, Input the statement to be transcribed and the current interactive statement into the target transcription model;

[0083] S42, the inputs of each model network are processed sequentially using the pre-trained model of the target transcription model, the long short-term memory network, the convolutional neural network, and the fully connected layer to obtain the target editing matrix output by the fully connected layer.

[0084] In this embodiment, the target transcription model may include a multi-layer network structure (i.e., a model network). The multi-layer network structure may include, but is not limited to, a pre-trained model, a long short-term memory network model, a convolutional neural network model, and a fully connected layer. For example, the target transcription model is a RUN model, which may include network structures such as BERT, LSTM (Long Short Term Memory), CNN (Convolutional Neural Networks), and FC (Full Connect).

[0085] For the statement to be transcribed and the current interactive statement, they can be input into the network structure of the target transcription model and connected sequentially. Correspondingly, in the target transcription model, the inputs of each model network can be processed sequentially using the pre-trained model, the long short-term memory network, the convolutional neural network, and the fully connected layer. The input of each model network can be the input of the target transcription model (e.g., the input of the pre-trained model) or the output of the previous model network. Finally, the output of the fully connected layer is the target edit matrix.

[0086] In this embodiment, the multi-layer network structure in the transcription model is used to process the statement to be transcribed and the current interactive statement to obtain the corresponding editing matrix, which can improve the accuracy of editing matrix acquisition.

[0087] In an exemplary embodiment, the characters to be transcribed in the statement to be transcribed are inserted into the current interactive statement according to the verified target edit matrix to obtain the target interactive statement, including:

[0088] S51, according to the verified target editing matrix, determine the insertion position of each character to be transcribed in the current interactive statement;

[0089] S52, when the insertion position of the second character to be transcribed in the sentence to be transcribed is the middle position of the second target word in a group of second words, the insertion position of the second character to be transcribed is adjusted to obtain the adjusted insertion position of the second character to be transcribed. Here, the group of second words is obtained by segmenting the current interactive sentence, and the adjusted insertion position of the second character to be transcribed is not the middle position of any second word.

[0090] S53. According to the insertion position of each character to be transcribed, insert each character to be transcribed into the current interactive statement to obtain the target interactive statement.

[0091] Based on the verified target edit matrix, the characters to be transcribed in the sentence to be transcribed and the insertion position of each character to be transcribed in the current interactive sentence can be determined. If the insertion position of a character to be transcribed in the sentence to be transcribed is in the middle of a word in the current interactive sentence, it may disrupt the semantics of the current interactive sentence. Therefore, the insertion position of the character to be transcribed can be adjusted to avoid inserting the character to be transcribed in the middle of a word in the current interactive sentence.

[0092] In this embodiment, the current interactive statement can be segmented into a group of second words. The segmentation operation can be performed at any time after the current interactive statement is obtained. After verifying the edit matrix, if it is determined from the verified edit matrix that the character to be transcribed (i.e., the second character to be transcribed) in the statement to be transcribed needs to be inserted into the middle position of a word (i.e., the second target word) in the group of second words in the current interactive statement, the insertion position of the second character to be transcribed can be adjusted. The adjusted insertion position of the second character to be transcribed does not depend on the middle position of any second word in the current interactive statement.

[0093] By inserting each character to be transcribed into its corresponding insertion position in the current interactive statement according to the adjusted insertion position, the target interactive statement can be obtained.

[0094] By adjusting the insertion position of the character to be transcribed in this embodiment, it is possible to avoid inserting the character to be transcribed into the middle of the words in the current interactive statement, thereby improving the accuracy of semantic parsing of the interactive statement.

[0095] In one exemplary embodiment, adjusting the insertion position of the second character to be transcribed to obtain the adjusted insertion position of the second character to be transcribed includes:

[0096] S61, adjust the insertion position of the second character to be transcribed to before or after the second target word to obtain the adjusted insertion position of the second character to be transcribed.

[0097] The methods for adjusting the insertion position of the second character to be transcribed may include, but are not limited to: adjusting the insertion position of the second character to be transcribed to before or after the second target word. Optionally, the specific adjustment method may be determined based on the semantic parsing result of the current interactive statement. The second character to be transcribed may be inserted before the second target word and after the second target word, respectively. After performing semantic parsing on the interactive statements obtained by the two adjustment methods, the insertion position corresponding to the interactive statement that can be parsed for semantics is determined as the insertion position corresponding to the second character to be transcribed. Alternatively, the insertion position of the second character to be transcribed may be directly specified, for example, adjusting the insertion position of the second character to be transcribed to before the second target word.

[0098] Here, the second character to be transcribed can belong to a word in the sentence to be transcribed. When adjusting the insertion position of the second character to be transcribed, the insertion positions of other characters to be transcribed in the same word can be adjusted accordingly, without having to adjust the insertion position of each character to be transcribed separately. This improves both the efficiency and accuracy of adjusting the characters to be transcribed.

[0099] This embodiment improves the ease of adjusting the insertion position of the character to be transcribed by inserting it before or after the target word.

[0100] In an exemplary embodiment, after inserting the characters to be transcribed from the statement to be transcribed into the current interactive statement according to the verified target edit matrix to obtain the target interactive statement, the above method further includes:

[0101] S71, Perform semantic integrity verification on the target interaction statement and obtain the integrity verification result of the target interaction statement;

[0102] S72, if the integrity verification result indicates that the target interactive statement is semantically complete, control the voice device to perform device operations that match the target interactive statement.

[0103] After obtaining the target interaction statement, semantic parsing and intent recognition can be performed directly on it. However, since the semantics of the transcribed interaction statement may not be complete, directly performing semantic parsing and intent recognition on the target interaction statement can easily lead to semantic parsing errors and intent recognition failures.

[0104] In this embodiment, in order to improve the reliability of semantic parsing, after obtaining the target interaction statement, semantic integrity prediction can be performed on the target interaction statement to obtain the integrity verification result of the target interaction statement. Semantic integrity prediction can be performed using a semantic integrity prediction model. The semantic integrity prediction model can be obtained by training the initial semantic integrity prediction model using a training statement with semantic integrity annotation results, or other methods can be used for semantic integrity prediction. This embodiment does not limit this.

[0105] If the integrity check result indicates that the target interaction statement is semantically complete, then the target interaction statement can be semantically parsed to obtain the device operation matching the target interaction statement, and the voice device can be controlled to execute the device operation matching the target interaction statement. However, if the integrity check result indicates that the target interaction statement is semantically incomplete, then the process can continue to wait for the next round of voice interaction until a semantically complete interaction statement is obtained, or the user stops engaging in voice interaction.

[0106] Here, the voice data recognition operation, the input of interactive sentences into the transcription model, and the sentence integrity prediction operation can be performed by a voice device or a server. That is, the voice device transmits voice data to the server, transmits the interactive sentences recognized from the voice data to the server, and the server performs the subsequent processing operations. Alternatively, the voice data and the server may each perform some operations. This embodiment does not limit this.

[0107] Through this embodiment, semantic integrity prediction is performed on the rewritten interaction statement, and the matching device operation is executed when it is determined that the rewritten interaction statement is semantically complete, which can improve the efficiency of voice interaction.

[0108] The processing method of voice data in the embodiments of the present application will be explained below with reference to optional examples. In this optional example, the current interaction statement is the current statement, the statement to be transcribed is the above text, the target training model is the RUN model, and the target word is the keyword.

[0109] This optional example provides a dialogue transcription scheme with keyword integrity verification. When using the RUN model for dialogue transcription, the editing matrix is processed twice using word segmentation to complete the keywords and prevent the rewritten sentence from containing incomplete keywords. Combining Figure 3 , the process of the processing method of voice data in this optional example may include the following steps:

[0110] Step 1, obtain the above text and the current statement. The above text can be "Turn on the air conditioner", and the current statement can be "Set it to 24 degrees".

[0111] Step 2, input the above text and the current statement into the RUN model to obtain the editing matrix corresponding to the above text and the current statement.

[0112] Step 3, perform word segmentation on the above text. A word segmentation tool can be used to perform word segmentation on the above text in combination with a custom keyword dictionary to obtain the word segmentation result. The word segmentation result of "Turn on the air conditioner" is "Turn on" and "air conditioner".

[0113] Step 4, input the above text and the current statement into the RUN model to obtain the editing matrix output by the RUN model. The output editing matrix can be shown in Table 2. Based on the editing matrix, the characters to be transcribed into the current statement are part of the keyword "air conditioner", which is an incomplete keyword.

[0114] Step 5, verify the editing matrix according to the word segmentation result. If the value of a certain character in a word is not 0, then set the values of the other characters in the word to the same value. If the values of some characters in a keyword are 1 and the values of the other characters are not 0, then set the values of the other characters in the keyword in the editing matrix to the same value to obtain an editing matrix with complete keywords. For the editing matrix shown in Table 2, according to the word segmentation result, set the operation of the character "kong" to 1 to obtain the editing matrix shown in Table 3.

[0115] Step 6, rewrite the current statement according to the verified editing matrix to restore the complete statement and obtain the final rewritten statement.

[0116] This optional example demonstrates how, when using the RUN model for dialogue transcription, word segmentation is used to perform secondary processing on the edit matrix to complete the keywords, preventing the rewritten sentences from containing incomplete keywords and improving the accuracy of speech transcription.

[0117] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0118] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM (Read-Only Memory) / RAM (Random Access Memory), magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0119] According to another aspect of the embodiments of this application, a voice data processing apparatus for implementing the above-described voice data processing method is also provided, which can be applied to a smart device. Figure 4 This is a structural block diagram of an optional voice data processing apparatus according to an embodiment of this application, such as... Figure 4 As shown, the device may include:

[0120] The acquisition unit 402 is used to acquire the current interactive statement and the corresponding statement to be transcribed. The current interactive statement is the interactive statement extracted from the voice data collected by the voice device during the current round of voice interaction, and the statement to be transcribed is the interactive statement acquired before the current round of voice interaction.

[0121] The processing unit 404, connected to the acquisition unit 402, is used to perform word segmentation on the sentence to be transcribed, and obtain a set of first words corresponding to the sentence to be transcribed.

[0122] The input unit 406, connected to the processing unit 404, is used to input the statement to be transcribed and the current interactive statement into the target transcription model to obtain the target editing matrix, wherein the target editing matrix is ​​used to indicate the characters to be transcribed in the statement to be transcribed;

[0123] The first verification unit 408 is connected to the input unit 406 and is used to verify the target editing matrix according to a set of first words to obtain the verified target editing matrix. In this matrix, all characters in the first word to which each character to be transcribed belongs are characters to be transcribed.

[0124] The insertion unit 410 is connected to the first verification unit 408 and is used to insert the characters to be transcribed in the statement to be transcribed into the current interactive statement according to the verified target editing matrix, so as to obtain the target interactive statement.

[0125] It should be noted that the acquisition unit 402 in this embodiment can be used to execute the above step S202, the processing unit 404 in this embodiment can be used to execute the above step S204, the input unit 406 in this embodiment can be used to execute the above step S206, the first verification unit 408 in this embodiment can be used to execute the above step S208, and the insertion unit 410 in this embodiment can be used to execute the above step S210.

[0126] Through the above modules, the current interactive statement and the corresponding statement to be transcribed are obtained. The current interactive statement is extracted from the speech data collected by the voice device during the current round of voice interaction, and the statement to be transcribed is the interactive statement obtained before the current round of voice interaction. The statement to be transcribed is segmented to obtain a set of first words corresponding to it. The statement to be transcribed and the current interactive statement are input into a target transcription model to obtain a target editing matrix, which indicates the characters to be transcribed in the statement to be transcribed. The target editing matrix is ​​validated according to the set of first words to obtain a validated target editing matrix, where all characters in the first word to which each character to be transcribed belongs are characters to be transcribed. The characters to be transcribed in the statement to be transcribed are inserted into the current interactive statement according to the validated target editing matrix to obtain the target interactive statement. This solves the problem of low accuracy in speech transcription caused by missing transcription content in related technologies, thus improving the accuracy of speech transcription.

[0127] In one exemplary embodiment, the first verification unit includes:

[0128] The first determination module is used to determine each word to be transcribed in the sentence to be transcribed based on the target editing matrix;

[0129] The second determining module is used to determine the first word to which each character to be transcribed belongs in a set of first words;

[0130] The verification module is used to verify the target editing matrix when there is a non-transferable character in the first target word to which the first character to be transcribed belongs in the sentence to be transcribed. The result is a verified target editing matrix, in which the first target word indicated by the verified target editing matrix is ​​a character to be transcribed, and the insertion position of the first target word in the current interactive sentence is the insertion position of the first character to be transcribed in the current interactive sentence.

[0131] In one exemplary embodiment, the first determining module includes:

[0132] The first execution submodule is used to perform the following steps on each character in the statement to be transcribed, treating it as the current character, to determine whether the current character is the character to be transcribed:

[0133] Iterate through the target editing matrix, identifying the matrix positions corresponding to the current character;

[0134] If a target value exists at the matrix position corresponding to the current character, the current character is determined to be a character to be transcribed.

[0135] If there is no target value at the matrix position corresponding to the current character, it is determined that the current character is not the character to be transcribed.

[0136] In one exemplary embodiment, the verification module includes:

[0137] The determination submodule is used to determine the reference word corresponding to the first character to be transcribed in the current interactive statement based on the target editing matrix. In the target editing matrix, the matrix position corresponding to the first character to be transcribed and the reference word is the target value. The reference word is used to indicate the position where the first character to be transcribed should be inserted in the current interactive statement.

[0138] The second execution submodule is used to set the values ​​in the matrix positions corresponding to the non-to-translate characters and reference characters in the first target word in the target editing matrix to the target values, so as to obtain the verified target editing matrix.

[0139] In one exemplary embodiment, the target transcription model includes a pre-trained model, a long short-term memory network, a convolutional neural network, and a fully connected layer; the input unit includes:

[0140] The input module is used to input the statement to be transcribed and the current interactive statement into the target transcription model;

[0141] The processing module is used to process the inputs of each model network sequentially using the pre-trained model of the target transcription model, the long short-term memory network, the convolutional neural network, and the fully connected layer, to obtain the target editing matrix output by the fully connected layer.

[0142] In one exemplary embodiment, the insertion unit includes:

[0143] The third determination module is used to determine the insertion position of each character to be transcribed in the current interactive statement according to the verified target editing matrix;

[0144] The adjustment module is used to adjust the insertion position of the second character to be transcribed in the sentence to be transcribed when the insertion position of the second target word in a group of second words is the middle position of the second target word. The adjusted insertion position of the second character to be transcribed is obtained by segmenting the current interactive sentence.

[0145] The insertion module is used to insert each character to be transcribed into the current interactive statement according to the insertion position of each character to be transcribed, so as to obtain the target interactive statement.

[0146] In one exemplary embodiment, the adjustment module includes:

[0147] The adjustment submodule is used to adjust the insertion position of the second character to be transcribed to before or after the second target word, so as to obtain the adjusted insertion position of the second character to be transcribed.

[0148] In one exemplary embodiment, the above-described apparatus further includes:

[0149] The second verification unit is used to insert the characters to be transcribed in the target editing matrix into the current interactive statement according to the verified target editing matrix, and then perform semantic integrity verification on the target interactive statement to obtain the integrity verification result of the target interactive statement.

[0150] The control unit is used to control the voice device to perform device operations that match the target interactive statement when the integrity verification result indicates that the target interactive statement is semantically complete.

[0151] According to another aspect of the embodiments of this application, a storage medium is also provided, which can be located on a smart device. Optionally, in this embodiment, the storage medium can be used to execute program code for any of the voice data processing methods described in the embodiments of this application.

[0152] Optionally, in this embodiment, the storage medium may be located on at least one of the network devices in the network shown in the above embodiment.

[0153] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps:

[0154] S1, obtain the current interaction statement and the corresponding statement to be transcribed. The current interaction statement is the interaction statement extracted from the voice data collected by the voice device during the current round of voice interaction, and the statement to be transcribed is the interaction statement obtained before the current round of voice interaction.

[0155] S2, perform word segmentation on the sentence to be transcribed to obtain a first set of words corresponding to the sentence to be transcribed;

[0156] S3, input the statement to be transcribed and the current interactive statement into the target transcription model to obtain the target editing matrix, where the target editing matrix is ​​used to indicate the words to be transcribed in the statement to be transcribed;

[0157] S4. Verify the target editing matrix according to a set of first words to obtain the verified target editing matrix. In the verified target editing matrix, all characters in the first word to which each character to be transcribed belongs are characters to be transcribed.

[0158] S5. According to the verified target editing matrix, insert the characters to be transcribed in the statement to be transcribed into the current interactive statement to obtain the target interactive statement.

[0159] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated in this embodiment.

[0160] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, ROMs, RAMs, portable hard drives, magnetic disks, or optical disks.

[0161] According to another aspect of the embodiments of this application, an electronic device for implementing the above-described voice data processing method is also provided. The electronic device may be a server, a terminal, or a combination thereof.

[0162] Figure 5 This is a structural block diagram of an optional electronic device according to an embodiment of this application, such as... Figure 5 As shown, it includes a processor 502, a communication interface 504, a memory 506, and a communication bus 508. The processor 502, communication interface 504, and memory 506 communicate with each other via the communication bus 508.

[0163] Memory 506 is used to store computer programs;

[0164] When processor 502 executes a computer program stored in memory 506, it performs the following steps:

[0165] S1, obtain the current interaction statement and the corresponding statement to be transcribed. The current interaction statement is the interaction statement extracted from the voice data collected by the voice device during the current round of voice interaction, and the statement to be transcribed is the interaction statement obtained before the current round of voice interaction.

[0166] S2, perform word segmentation on the sentence to be transcribed to obtain a first set of words corresponding to the sentence to be transcribed;

[0167] S3, input the statement to be transcribed and the current interactive statement into the target transcription model to obtain the target editing matrix, where the target editing matrix is ​​used to indicate the words to be transcribed in the statement to be transcribed;

[0168] S4. Verify the target editing matrix according to a set of first words to obtain the verified target editing matrix. In the verified target editing matrix, all characters in the first word to which each character to be transcribed belongs are characters to be transcribed.

[0169] S5. According to the verified target editing matrix, insert the characters to be transcribed in the statement to be transcribed into the current interactive statement to obtain the target interactive statement.

[0170] Optionally, the communication bus can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 The symbol is represented by a single thick line, but this does not indicate that there is only one bus or one type of bus. The communication interface is used for communication between the aforementioned electronic device and other devices.

[0171] The memory may include RAM, or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0172] As an example, the memory 506 described above may include, but is not limited to, the acquisition unit 402, processing unit 404, input unit 406, first verification unit 408, and insertion unit 410 from the voice data processing device described above. Furthermore, it may include, but is not limited to, other module units from the voice data processing device described above, which will not be elaborated upon in this example.

[0173] The processors mentioned above can be general-purpose processors, including but not limited to: CPU (Central Processing Unit), NP (Network Processor), etc.; they can also be DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0174] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated here.

[0175] Those skilled in the art will understand that Figure 5 The structure shown is for illustrative purposes only. The device that implements the above voice data processing method can be a terminal device, such as a smartphone (e.g., Android phone, iOS phone), tablet computer, PDA, mobile Internet device (MID), PAD, etc. Figure 5 This does not limit the structure of the aforementioned electronic device. For example, the electronic device may also include components that are more... Figure 5 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 5 The different configurations shown.

[0176] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, ROM, RAM, disk or optical disk, etc.

[0177] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0178] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0179] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0180] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0181] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the solution provided in this embodiment, depending on actual needs.

[0182] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or at least two units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0183] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for processing voice data, characterized in that, include: Obtain the current interaction statement and the corresponding statement to be transcribed, wherein the current interaction statement is an interaction statement extracted from the voice data collected by the voice device during the current round of voice interaction, and the statement to be transcribed is an interaction statement obtained before the current round of voice interaction. The sentence to be transcribed is segmented into words to obtain a first set of words corresponding to the sentence to be transcribed. The statement to be transcribed and the current interactive statement are input into the target transcription model to obtain the target editing matrix, wherein the target editing matrix is ​​used to indicate the characters to be transcribed in the statement to be transcribed; The target editing matrix is ​​verified according to the first set of words to obtain the verified target editing matrix, wherein all characters in the first word to which each character to be transcribed belongs, as indicated by the verified target editing matrix, are characters to be transcribed. According to the verified target editing matrix, the characters to be transcribed in the statement to be transcribed are inserted into the current interactive statement to obtain the target interactive statement.

2. The method according to claim 1, characterized in that, The step of validating the target edit matrix according to the first set of words to obtain the validated target edit matrix includes: Based on the target editing matrix, determine each word to be transcribed in the sentence to be transcribed; Determine the first word to which each character to be transcribed belongs in the first set of words; If there is a non-transcribing character in the first target word to which the first character to be transcribed belongs in the sentence to be transcribed, the target editing matrix is ​​verified to obtain the verified target editing matrix. In the verified target editing matrix, the first target word indicated by the first target word is the character to be transcribed, and the insertion position of the first target word in the current interactive sentence is the insertion position of the first character to be transcribed in the current interactive sentence.

3. The method according to claim 2, characterized in that, The step of determining each word to be transcribed in the sentence to be transcribed based on the target editing matrix includes: For each character in the sentence to be transcribed, treat it as the current character and perform the following steps to determine whether the current character is the character to be transcribed: Iterate through the target editing matrix to find the matrix positions corresponding to the current character; If a target value exists at the matrix position corresponding to the current character, the current character is determined to be a character to be transcribed. If there is no target value at the matrix position corresponding to the current character, it is determined that the current character is not a character to be transcribed.

4. The method according to claim 2, characterized in that, The step of verifying the target edit matrix to obtain the verified target edit matrix includes: Based on the target editing matrix, a reference character corresponding to the first character to be transcribed is determined in the current interactive statement. In the target editing matrix, the matrix position corresponding to the first character to be transcribed and the reference character is the target value. The reference character is used to indicate the position where the first character to be transcribed is to be inserted in the current interactive statement. In the target editing matrix, the values ​​at the matrix positions corresponding to the non-to-translate characters in the first target word and the reference characters are set to the target values ​​to obtain the verified target editing matrix.

5. The method according to claim 1, characterized in that, The target transcription model includes a pre-trained model, a long short-term memory network, a convolutional neural network, and a fully connected layer; The step of inputting the statement to be transcribed and the current interactive statement into the target transcription model to obtain the target editing matrix includes: The statement to be transcribed and the current interactive statement are input into the target transcription model; The inputs of each model network are processed sequentially using the pre-trained model of the target transcription model, the long short-term memory network, the convolutional neural network, and the fully connected layer to obtain the target editing matrix output by the fully connected layer.

6. The method according to claim 1, characterized in that, The step of inserting the characters to be transcribed from the target edit matrix into the current interactive statement according to the verified target edit matrix to obtain the target interactive statement includes: Based on the verified target editing matrix, determine the insertion position of each character to be transcribed in the current interactive statement; When the insertion position of the second character to be transcribed in the sentence to be transcribed is the middle position of the second target word in a group of second words, the insertion position of the second character to be transcribed is adjusted to obtain the adjusted insertion position of the second character to be transcribed. Here, the group of second words is obtained by segmenting the current interactive sentence, and the adjusted insertion position of the second character to be transcribed is not the middle position of any second word. According to the insertion position of each character to be transcribed, each character to be transcribed is inserted into the current interactive statement to obtain the target interactive statement.

7. The method according to claim 6, characterized in that, The step of adjusting the insertion position of the second character to be transcribed, to obtain the adjusted insertion position of the second character to be transcribed, includes: The insertion position of the second character to be transcribed is adjusted to be before or after the second target word, resulting in the adjusted insertion position of the second character to be transcribed.

8. The method according to any one of claims 1 to 7, characterized in that, After inserting the characters to be transcribed from the target edit matrix into the current interactive statement according to the verified target edit matrix to obtain the target interactive statement, the method further includes: Perform semantic integrity verification on the target interaction statement to obtain the integrity verification result of the target interaction statement; If the integrity verification result indicates that the target interactive statement is semantically complete, the voice device is controlled to perform a device operation that matches the target interactive statement.

9. A voice data processing apparatus, characterized in that, include: The acquisition unit is used to acquire the current interactive statement and the statement to be transcribed corresponding to the current interactive statement, wherein the current interactive statement is an interactive statement extracted from the voice data collected by the voice device during the current round of voice interaction, and the statement to be transcribed is an interactive statement acquired before the current round of voice interaction. The processing unit is used to perform word segmentation on the statement to be transcribed to obtain a first set of words corresponding to the statement to be transcribed. An input unit is used to input the statement to be transcribed and the current interactive statement into a target transcription model to obtain a target editing matrix, wherein the target editing matrix is ​​used to indicate the characters to be transcribed in the statement to be transcribed; The first verification unit is used to verify the target editing matrix according to the set of first words to obtain the verified target editing matrix, wherein all characters in the first word to which each character to be transcribed belongs, as indicated by the verified target editing matrix, are characters to be transcribed. An insertion unit is used to insert the characters to be transcribed in the statement to be transcribed into the current interactive statement according to the verified target editing matrix, so as to obtain the target interactive statement.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method of any one of claims 1 to 7.

11. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method of any one of claims 1 to 7 through the computer program.

Images