Speech recognition error correction method and related equipment

Abstract

The present application proposes a speech recognition error correction method and related equipment. In a multi-person conversation application scenario, after obtaining the speech recognition results of each participating object, that is, the context feature information of each object in the text to be processed, the present application will cause the identity of the participating object in the scenario, process the context feature information and the identity feature information of the corresponding participating object, and obtain the target feature information of the corresponding object by synthesizing the obtained conversation feature information and context feature information. Based on the target feature information, high-accuracy and reliable error correction processing of the text to be processed is achieved, thereby improving the accuracy of speech recognition error correction.

Description

Technical Field

[0001] The present application mainly relates to the field of speech recognition, and more specifically to a speech recognition error correction method and related equipment. Background Art

[0002] With the development of artificial intelligence technology, speech recognition technology in artificial intelligence has been widely used in various fields. For example, in a multi-person conversation scenario, the speech recognition system can recognize the voice signals of each user and obtain the text conversation content between users for subsequent reference.

[0003] However, during the speech recognition process, the speech recognition system may be affected by factors such as the user's pronunciation, the recognition accuracy of the speech recognition model, and the application environment, resulting in some errors in the speech recognition results and reducing the accuracy of the speech recognition results. Summary of the invention

[0004] In view of this, the present application provides a speech recognition error correction method, the method comprising:

[0005] Acquire context feature information of each object in the text to be processed and identity feature information of the participating objects; the text to be processed is obtained by performing speech recognition on the speech signals of the corresponding participating objects;

[0006] Processing the context feature information and the identity feature information of the corresponding participating object to obtain the conversation feature information of the corresponding object in the text to be processed;

[0007] Obtaining target feature information of a corresponding object based on the context feature information and the session feature information of the same object;

[0008] According to the target feature information of each object in the text to be processed, error correction processing is performed on the text to be processed to obtain a target text.

[0009] Optionally, the obtaining of context feature information of each object in the file to be processed includes:

[0010] Performing speech recognition on the speech signals of each participant to obtain the text to be processed of the corresponding participant;

[0011] Obtaining object feature information of each object in the text to be processed;

[0012] The object feature information of each object is input into the first encoding network of the error correction model, and the context feature information of the corresponding object is output.

[0013] Optionally, the processing of the context feature information and the identity feature information of the corresponding participating object to obtain the conversation feature information of the corresponding object in the text to be processed includes:

[0014] The context feature information and the identity feature information of the corresponding participating objects are input into the second encoding network of the error correction model, and the conversation feature information of the corresponding objects in the text to be processed is output.

[0015] Optionally, obtaining target feature information of a corresponding object based on the context feature information and the session feature information of the same object includes:

[0016] Performing weighted fusion processing on the context feature information and the session feature information of the same object to obtain target feature information of the corresponding object;

[0017] The respective weights of the context feature information and the conversation feature information of the same object are obtained by fitting the training sample data of the error correction model.

[0018] Optionally, the training process of the error correction model includes:

[0019] Acquire multiple training sample speech signals generated by conversations between multiple sample objects, and sample identity feature information corresponding to the sample objects;

[0020] Input the training sample speech signal into a speech recognition model and output a corresponding training sample text;

[0021] Input the training sample text and the corresponding sample identity feature information into an initial error correction network, and output the predicted probability and replacement object of each training object in the training sample text belonging to the object to be corrected;

[0022] Obtaining the error correction loss of the corresponding training object according to the predicted probability of each training object and the replacement object;

[0023] According to the error correction loss, adjusting the network parameters of the error correction network, and continuing to train the error correction network with the adjusted network parameters until a training termination condition is met, thereby obtaining an error correction model;

[0024] Among them, the network parameters include the first weight of the first coding vector for different training objects output by the first coding network in the error correction network, and the second weight of the second coding vector for different training objects output by the second coding network; the first weight and the second weight represent the prediction influence of the corresponding coding vector on the corresponding training object belonging to the object to be corrected.

[0025] Optionally, performing error correction processing on the text to be processed based on the target feature information of each object in the text to be processed to obtain a target text includes:

[0026] Inputting the target feature information of each object in the to-be-processed text into the classification network of the error correction model, and obtaining the to-be-corrected object whose predicted probability is greater than a probability threshold and the corresponding replacement object according to the obtained predicted probability that the corresponding object belongs to the to-be-corrected object;

[0027] The replacement object is used to replace the corresponding object to be corrected in the text to be processed to obtain the target text.

[0028] Optionally, obtaining identity feature information of the participant includes:

[0029] Obtain the identity registration information of each participant registered on the conversation platform;

[0030] Feature extraction is performed on the body registration information to obtain identity feature information of the corresponding participant.

[0031] The present application also proposes a speech recognition error correction device, the device comprising:

[0032] A context feature information acquisition module is used to acquire context feature information of each object in the text to be processed; the text to be processed is obtained by performing speech recognition on the speech signal of the corresponding participating object;

[0033] An identity feature information acquisition module is used to acquire the identity feature information of the participant.

[0034] A session characteristic information obtaining module, used to process the context characteristic information and the identity characteristic information of the corresponding participating object to obtain session characteristic information of the corresponding object in the text to be processed;

[0035] A target feature information acquisition module, used to obtain target feature information of a corresponding object based on the context feature information and the session feature information of the same object;

[0036] The error correction processing module is used to perform error correction processing on the text to be processed according to the target feature information of each object in the text to be processed to obtain a target text.

[0037] The present application also proposes a computer device, the computer device comprising: at least one memory and at least one processor;

[0038] The memory is used to store a program for implementing the above-mentioned speech recognition error correction method;

[0039] The processor is used to load and execute the program stored in the memory to implement the speech recognition error correction method as described above.

[0040] The present application also proposes a computer-readable storage medium on which a computer program is stored. The computer program is called and executed by a processor to implement the speech recognition error correction method as described above.

[0041] It can be seen that the present application provides a speech recognition error correction method and related equipment. In a multi-person conversation application scenario, speech recognition is performed on the collected voice signals of each participating object. After the context feature information of each object in the text to be processed is obtained, the present application will cause the identity of the participating object in the scenario, process the context feature information and the identity feature information of the corresponding participating object, and obtain the target feature information of the corresponding object by synthesizing the obtained conversation feature information and context feature information. Based on the target feature information, high-accuracy and reliable error correction processing of the text to be processed is achieved, thereby improving the accuracy of speech recognition error correction. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0043] Figure 1 A schematic diagram of a system structure of an optional application environment suitable for the speech recognition error correction method proposed in this application;

[0044] Figure 2 A schematic diagram of the hardware structure of an optional example of a computer device applicable to the speech recognition error correction method proposed in this application;

[0045] Figure 3 A schematic diagram of the hardware structure of another optional example of a computer device applicable to the speech recognition error correction method proposed in the present application;

[0046] Figure 4 A flowchart of an optional example of the speech recognition error correction method proposed in this application;

[0047] Figure 5 A flowchart of another optional example of the speech recognition error correction method proposed in this application;

[0048] Figure 6 A flowchart of another optional example of the speech recognition error correction method proposed in this application;

[0049] Figure 7A flowchart of another optional example of the speech recognition error correction method proposed in this application;

[0050] Figure 8 A flowchart of an optional example of a method for training an error correction model in the speech recognition error correction method proposed in this application;

[0051] Fig. 9 This is a schematic diagram of the structure of an optional example of the speech recognition error correction device proposed in this application. DETAILED DESCRIPTION

[0052] In response to the technical problems described in the background technology part, in order to improve the accuracy of speech recognition results, it is proposed to use a speech recognition error correction model to correct the recognition results. That is to say, after performing speech recognition on the collected speech signal and obtaining the corresponding text information, it can be input into a pre-trained error correction model to determine the object to be corrected contained in the text information (such as a certain character or word contained in the text information) and its replacement object, and the replacement object replaces the object to be corrected in the text information to obtain more accurate target text information.

[0053] Among them, the error correction processing of speech recognition results can be regarded as a sequence labeling problem. The text sequence obtained by speech recognition is labeled with categories, that is, whether each object in the text sequence belongs to the object to be corrected or the correct object that does not need to be corrected is labeled, the category label of each object is predicted, and the object to be corrected is corrected to what object.

[0054] However, for multi-person conversation application scenarios, such as online conference scenarios and voice question-and-answer scenarios, the speech recognition results are often not a single text sequence, but are presented in the form of multiple rounds of conversations, and the identity of the speaker and other information will be distinguished, which increases the difficulty of speech recognition in such scenarios. If the speech recognition results of the speech signals generated by each participant are directly input into the error correction model as independent text information to be corrected, the accuracy of the error correction results is often relatively low.

[0055] In order to improve the error correction accuracy of speech recognition results in application scenarios of multi-person conversations, the present application proposes that when performing error correction processing on speech recognition results, the identity information of different participants can be entered to distinguish the contexts of different participants, and a two-layer encoding mechanism can be used to realize error correction of speech recognition results of multiple rounds of conversations, thereby improving the error correction efficiency and accuracy of speech recognition results of multi-person conversations.

[0056] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0057] Reference Figure 1 , is a schematic diagram of a system structure of an optional application environment applicable to the speech recognition error correction method proposed in this application. In this application environment, Figure 1 As shown, the system may include a terminal 11 and a server 12, wherein:

[0058] Terminal 11 can be a user terminal that supports voice remote communication, voice recognition and other functions. It may include but is not limited to smart phones, tablet computers, wearable devices (such as smart watches, smart bracelets, etc.), augmented reality technology (AR) devices, virtual reality (VR) devices, vehicle-mounted devices, smart speakers, robots, smart home devices, smart transportation equipment, smart medical equipment, etc. This application does not limit the product type of the terminal 11, which can be determined according to the application scenario requirements.

[0059] In the multi-person conversation application scenarios listed above, different participants (i.e., users participating in the conversation) can use the communication client of terminal 11 to realize multi-person online voice conversation. In this case, the voice collector of terminal 11 itself or other voice collection devices in the environment can collect voice of the participants' speeches or questions. In order to record the content of the conversation, applications such as voice recognition engines can be used to perform voice recognition on the voice signals of the participant and the voice signals received from other participants to obtain the text to be processed corresponding to each voice signal.

[0060] It can be understood that in the application scenario of multi-person online voice calls described above, the voice recognition process for non-participating objects can be implemented by the voice recognition engine running on the terminal that collects or receives the voice signal, or the communication server supporting this communication can perform voice recognition on the voice signals of each participating object and record the obtained multi-round conversation text information, etc. This application does not impose any restrictions on the executor of the voice recognition, which can be determined according to the circumstances.

[0061] In some other embodiments, for multi-person conversation application scenarios, multiple users may have a conversation in the same environment. In order to record the content of the conversation, the voice acquisition device in the environment collects the voice signals of each participant and sends them to the server or local electronic device for voice recognition, error correction, etc. This application does not limit the method for obtaining the voice recognition results to be corrected, including but not limited to the implementation methods of each scenario described in this application.

[0062] The server 12 can be a service device that supports a multi-person online voice communication service of the terminal 11 and implements voice recognition and error correction services. It can be an independent physical server, a server cluster consisting of multiple physical servers, or a cloud server that can implement cloud computing services, etc. It can realize data interaction with each terminal 11 through the Internet. The specific interaction process can be determined in combination with the application scenario of voice recognition and error correction, and will not be described in detail in the embodiments of the present application.

[0063] It should be understood that Figure 1 The system structure shown does not constitute a limitation of the system architecture applicable to the application environment of the speech recognition error correction method proposed in this application. In actual applications, in different application environments, the system may include more devices, such as databases, other application servers, etc., which are not listed one by one in this application.

[0064] Reference Figure 2 , is a schematic diagram of a hardware structure of an optional example of a computer device applicable to the speech recognition error correction method proposed in the present application, the computer device may be the above-mentioned terminal or server, the computer device may include at least one memory 21 and at least one processor 22, wherein:

[0065] The memory 21 can be used to store a program for implementing the speech recognition error correction method as described above; the processor 22 can be used to load and execute the program stored in the memory 21 to implement the various steps of the speech recognition error correction method described in any of the above method embodiments. The specific implementation process can refer to the description of the corresponding part of the embodiment below, and this embodiment will not be described in detail here.

[0066] In the embodiment of the present application, the memory 21 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device. The processor 22 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, etc.

[0067] It should be understood that Figure 2 The structure of the computer device shown does not constitute a limitation on the computer device in the embodiment of the present application. In practical applications, the computer device may include Figure 2 More components shown, or some components are combined, such as various communication interfaces, etc. And in the case where the computer device is a terminal, such as Figure 3 As shown, the computer may also include at least one input device such as a camera, a microphone, etc.; at least one output device such as a display, a speaker, etc.; a sensor module composed of various sensors; a power management module; an antenna, etc., which are not listed one by one in this application.

[0068] Reference Figure 4 , is a flow chart of an optional example of the speech recognition error correction method proposed in this application. The method can be executed by a server or a terminal, or by a terminal and a server configuration, which can be determined according to the application scenario requirements. This application embodiment takes the server execution as an example for explanation. Figure 4 As shown, the speech recognition error correction method proposed in this embodiment may include:

[0069] Step S11, obtaining context feature information of each object in the text to be processed, and identity feature information of the participating objects;

[0070] In an embodiment of the present application, the text to be processed can be obtained by performing speech recognition on the speech signals of the corresponding participating objects, such as inputting the collected speech signals into a speech recognition model, outputting the corresponding text information, and inputting the text to be processed into an error correction model for error correction processing. The present application does not describe the speech recognition process in detail.

[0071] Combined with the above description of the speech recognition result correction process, in order to determine the category label of each object contained in the text to be processed, that is, to determine whether each object is an object to be corrected, the context analysis of each object contained in the text to be processed can be performed to determine whether the semantics are smooth, whether the logic is correct, etc. Therefore, after obtaining any text to be processed, the context feature information of each object contained therein can be obtained, and this application does not limit the method for obtaining context information.

[0072] In addition, in order to improve the accuracy of error correction of speech recognition results, especially the error correction of speech recognition results of each participating object obtained in a multi-person conversation scenario, the present application proposes to introduce the identity information of different participating objects, distinguish different participating objects, and use it as a reference dimension information to realize the analysis of whether each object in the text to be processed is the object to be corrected.

[0073] Therefore, before performing error correction processing on the text to be processed, the present application can also obtain the identity feature information of each participant, such as obtaining the identity registration information of each participant registered on the conversation platform, and obtaining the identity feature information of the corresponding participant by performing feature extraction on the body registration information, etc. The present application does not limit the method for obtaining the identity feature information.

[0074] Step S12, processing the context feature information and the identity feature information of the corresponding participating object to obtain the conversation feature information of the corresponding object in the text to be processed;

[0075] Following the above analysis, the present application proposes to introduce the identity feature information of the participating objects to perform targeted error correction on the errors in the speech recognition results (i.e., the text to be processed) of different participating objects. Therefore, after obtaining the context feature information of each object contained in the text to be processed, the present application can perform encoding processing in combination with the identity feature information of the corresponding participating objects, and use the obtained encoding vector as the session feature information of the corresponding object in the text to be processed, which represents another kind of context information that is different from the context feature information described above. The present application does not describe in detail the process of obtaining the session feature information.

[0076] Step S13, obtaining target feature information of a corresponding object based on the context feature information and session feature information of the same object;

[0077] In combination with the above description of the technical concept of the present application, compared with directly predicting and replacing the correction object based on the context feature information, the present application will obtain the context information obtained by encoding the identity information of the participants in the multi-person conversation, that is, the conversation feature information. Afterwards, for each object in the text to be processed, the influence of the two feature vectors, the context feature information and the conversation feature information, corresponding to the object, on the prediction of whether the classification label of the object is the object to be corrected can be analyzed, and then the two feature vectors can be fused according to their respective influences to obtain the target feature information of the object.

[0078] It can be seen that the target feature information of each object contained in the text to be processed obtained in the present application not only includes the contextual information of the object features, but also introduces the identity feature information of the corresponding participating objects (i.e., speakers) (i.e., the identities of the participating objects in multi-person conversation scenarios). In this way, in speech recognition and error correction applications in conversation scenarios where the identities of the participating objects affect the content of their speeches, subsequent processing combined with the above-mentioned conversation feature information will help improve the prediction accuracy of object category labels.

[0079] It should be noted that the processing of context information by the identity of the participating object is not necessarily a positive guidance effect, but may also be a reverse guidance effect, the latter of which will reduce the prediction accuracy of the object to be corrected. Therefore, this application not only realizes category label prediction based on the conversation feature information that introduces the identity characteristic information of the participating object, but also realizes category label prediction based on the context feature information that does not consider the identity characteristic information of the participating object, so as to be applicable to a variety of dialogue application scenarios. Therefore, this application will use the target feature information obtained in the above manner to enable it to more accurately characterize the features of each object that constitutes the text to be processed, thereby improving the prediction accuracy of the category label of the speech recognition result in different dialogue scenarios.

[0080] Step S14, performing error correction processing on the text to be processed according to the target feature information of each object in the text to be processed to obtain the target text.

[0081] In the embodiment of the present application, after obtaining the target feature information of each object in the text to be processed according to the above method, that is, the target feature vector, a pre-trained classification network, that is, a binary classifier for identifying whether the object is an object to be corrected, can be used to predict the category label of the target feature information. For objects whose prediction results are the category label of the object to be corrected, it is also possible to analyze what object the object should be corrected to, that is, the replacement object of the object to be corrected. The present application does not elaborate on how to use the target feature vectors of each object to perform a method for predicting and identifying the object to be corrected and its replacement object.

[0082] Afterwards, for one or more objects to be corrected contained in the predicted text to be processed and their corresponding replacement objects, the corresponding objects to be corrected in the text to be processed can be directly replaced by the replacement objects to obtain the target text.

[0083] In summary, in an embodiment of the present application, in a multi-person conversation application scenario, speech recognition is performed on the collected voice signals of each participant to obtain a text to be processed. In order to detect the objects to be corrected contained in the text to be processed of each participant in multiple rounds of conversations, more accurate replacement objects are used to replace them to obtain a more accurate speech recognition result, i.e., the target text. The present application takes into account the identity of the participant in the conversation scenario and the impact of the content of their speech. After obtaining the contextual feature information of each object contained in the text to be processed, the identity feature information of the object and the corresponding participant is re-encoded, thereby obtaining the target feature information of each object in the text to be processed based on the obtained conversation feature information that integrates the identity of the participant and the above feature information that does not consider the identity of the participant, thereby achieving high-accuracy and reliable error correction processing of the text to be processed, and obtaining a target text that is basically consistent with the content of the participant's speech.

[0084] Reference Figure 5, which is a flow chart of another optional example of the speech recognition error correction method proposed in the present application. The embodiment of the present application may be an optional refined implementation method of the speech recognition error correction method described above, such as describing how to implement error correction processing on the processed text based on the pre-trained error correction model to obtain a high-precision speech recognition result, but is not limited to the implementation method described in this embodiment, and the embodiment of the present application does not describe in detail the training implementation method of the error correction model, such as Figure 5 As shown, the speech recognition error correction method proposed in this embodiment may include:

[0085] Step S21, obtaining the voice signal and identity registration information of each participant in the dialogue scene;

[0086] Step S22, inputting the speech signal into the speech recognition model, and outputting the to-be-processed text of the corresponding participant;

[0087] In multi-person conversation scenarios such as meetings, voice collection equipment is used to collect the voice of the speaker (i.e., the participant). After obtaining the corresponding voice signal, the voice signal of each participant is recognized using applications such as a voice recognition engine to obtain the corresponding text to be processed by the participant, i.e., the voice recognition result of each participant.

[0088] The identity registration information may be entered by the participant when registering on the conversation platform. For example, in an online voice conference scenario, in order to facilitate the participants to know each other's identities, personal identity information, i.e., identity registration information, is usually noted in the conference group. It should be understood that in different conversation scenarios, the identity registration information of the same participant may be different, and in the same conversation scenario, the same participant may have multiple identities, etc. This application does not restrict the method of obtaining the identity registration information of each participant and its content, which may be determined according to the circumstances.

[0089] Step S23, extracting features from the identity registration information to obtain identity feature information of the corresponding participant;

[0090] In order to introduce the identity registration information of the participating objects into the error correction model for processing, the present application can vectorize the various forms of identity registration information obtained, and in the model application, there are usually certain requirements for parameters such as the length of the input vector. For example, if the input vector is required to have a specific length, when the present application vectorizes the obtained identity feature information, it can extract features from it to obtain identity feature information with a specific length, namely, an identity feature vector. The implementation process is not described in detail in this application.

[0091] Step S24, obtaining object feature information of each object in the text to be processed;

[0092] Regarding the object feature information of each object in the text to be processed, the object feature vector of each recognized object can be extracted during the process of performing speech recognition on the corresponding input speech signal using the speech recognition model, such as extracting the word vector of each word in the text to be processed. Optionally, the present application can perform word segmentation processing on the text to be processed based on a neural network, such as word segmentation based on statistical sequence annotation or mechanical word segmentation based on a dictionary, to obtain multiple objects contained in the text to be processed, such as words formed by a single character, or words composed of multiple characters, etc. After that, the word vector of each object, that is, the object feature information, can be constructed based on the correlation of the semantic space of each object, thereby improving the semantic accuracy of the word vector and helping to accurately determine whether the object is an object to be corrected.

[0093] Optionally, the present application can utilize pre-trained models such as Word2vec, making full use of the context and semantic information of words, mapping each object in the text to be processed into a k-dimensional real vector, and characterizing the semantic similarity between the objects by the distance between the objects, to meet the subsequent detection needs of objects to be corrected. In another possible implementation, according to the scenario requirements, the present application can also perform semantic analysis on the text to be processed based on a deep learning algorithm to obtain object feature information of each object, etc. The present application does not limit the above-mentioned word vector, i.e., the method for extracting object feature information.

[0094] In some other embodiments, the present application may also filter the object feature information corresponding to each object in the to-be-processed text from the object feature database, and the present application does not limit the method for obtaining the above object feature information. Among them, the object feature database may include a large number of mapping relationships between objects and object feature information. The present application may extract object features from a large number of texts (such as historical texts obtained by past speech recognition downloaded from a third-party application platform and stored on the present application platform, and the present application does not limit the method for obtaining a large number of texts), obtain corresponding object feature information, and construct a mapping relationship between the object feature information and the corresponding object.

[0095] Step S25, inputting the identity feature information corresponding to each participating object and the object feature information of each object in the corresponding text to be processed into the error correction model, and outputting the objects to be corrected contained in each text to be processed and their corresponding replacement objects;

[0096] Step S26, using the replacement object to replace the object to be corrected in the corresponding text to be processed, to obtain the target text.

[0097] In combination with the above description of the technical concept of the present application, the present application introduces the identity feature information of each participant in the current multi-round dialogue application scenario into the error correction model for feature analysis, fully considering the identity of the participant in the dialogue scenario, and the targeted error correction influence on its speech recognition results, thereby improving the accuracy and reliability of error correction.

[0098] Among them, according to the description of each step of the above embodiment, the pre-trained error correction model of the present application can adopt a two-level encoding mechanism to construct a coding network, such as using a bidirectional neural network, and perform feature extraction on the object feature information and identity feature information of multiple rounds of conversations according to the method described above, so as to more accurately and reliably obtain the context information of each object in the text to be processed in the multi-round conversation scenario. The implementation process is not described in detail in this application.

[0099] In the process of context information extraction, the present application can consider the context information between objects in a text to be processed. In a multi-round dialogue scenario, the object context information of the text to be processed of two adjacent speakers can also be considered, that is, the correlation between the text content of multiple rounds of dialogue is considered to improve the accuracy and reliability of feature extraction. The implementation process can refer to but is not limited to the description of the corresponding part of the following embodiment, which will not be described in detail in this embodiment.

[0100] It can be seen that the texts of multiple rounds of dialogue obtained in a multi-round dialogue scenario, that is, multiple related texts to be processed, can be directly input into the error correction model after obtaining their respective multiple object feature information and identity feature information in the above manner, so that the error correction model can perform feature extraction according to the above analysis method to obtain the target feature information of each object in the corresponding text to be processed, that is, the contextual information that can better reflect the characteristics of the dialogue scenario. Based on this, the objects to be corrected and their replacement objects in the corresponding text to be processed can be determined more accurately, thereby improving the error correction accuracy and reliability of the speech recognition results.

[0101] Reference Figure 6 , which is a flowchart of another optional example of the speech recognition error correction method proposed in this application. The embodiment of this application can be an optional detailed implementation method of the speech recognition error correction method described above. In this embodiment, the error correction model can include a two-level coding network (recorded as the first coding network and the second coding network) and a classification network, but this application does not limit the network structure of each network, and can be based on but not limited to the neural network construction. The embodiment of this application will not be described in detail here. Based on this, Figure 6 As shown, the speech recognition error correction method proposed in this embodiment may include:

[0102] Step S31, obtaining object feature information of each object in the text to be processed, and identity feature information of the participating objects;

[0103] Regarding the implementation process of step S31, reference may be made to the description of the corresponding part above, and this embodiment will not elaborate on it.

[0104] Step S32, inputting object feature information of each object in the text to be processed into a first encoding network, and outputting context feature information of the corresponding object;

[0105] In the embodiment of the present application, the first encoding network has the ability to extract context information of objects such as words, in order to be applicable to multi-round dialogue scenarios, such as Figure 7 The two-stage encoding mechanism network structure of the error correction model shown in the figure, the present application can use a bidirectional neural network to construct a first encoding network to realize the context information extraction of each object in the processed text, but is not limited to this network structure.

[0106] Optionally, the above-mentioned bidirectional neural network can be a bidirectional LSTM (Long Short-Term Memory) recurrent neural network, but is not limited to a bidirectional neural network of this structure. According to the above method, after obtaining the object feature information of each of the multiple objects in the text to be processed, it can be input into the bidirectional neural network to output the context feature information of each object. The processing process can be determined based on the operation principle of the bidirectional neural network, which is not described in detail in this application.

[0107] Step S33, inputting the context feature information of each object and the identity feature information of the corresponding participating object into the second encoding network, and outputting the session feature information of the corresponding object;

[0108] In response to the need for speech recognition error correction in multi-round dialogue application scenarios, this application proposes to introduce speaker features to perform targeted error correction on speech recognition result errors of different speakers, such as Figure 5 As shown, the present application can use the context feature information of each object output by the first coding network and the identity feature information of the corresponding participating objects as the input of the second coding network. The second coding network fuses the identity feature information of the participating objects to perform context semantic analysis to obtain new context information, namely, session feature information.

[0109] The second encoding network is similar to the first encoding network, and a bidirectional neural network can also be used to implement contextual semantic analysis, such as Figure 7 As shown, in the process of obtaining the conversation feature information of each object in each text to be processed, while analyzing the semantic spatial correlation between different objects in the corresponding text to be processed, the identity characteristics of the participating objects in the output text to be processed can also be considered, and the impact on the semantics of each object. The implementation process can be determined based on the operation principle of the bidirectional neural network, which will not be described in detail in this application.

[0110] Step S34, performing weighted fusion processing on the context feature information and the session feature information of the same object to obtain target feature information of the corresponding object;

[0111] In the embodiment of the present application, the context feature information F of each object in the text to be processed obtained above can be context and session feature information F speaker These two feature vectors jointly determine whether the object needs to be corrected, that is, determine whether the object is an object to be corrected. However, for different objects in the text to be processed, the influence of these two feature vectors on determining whether it is an object to be corrected may be different. Therefore, in the training stage of the error correction model, the present application can obtain the weights of the context feature information and the conversation feature information of each object by fitting the training sample data. The weight is used as a model parameter of the error correction model to obtain the context feature information F of any object. context and session feature information F speaker After that, the two feature vectors can be weighted fused according to the pre-trained weight a to obtain the target feature information of the object, that is, the target feature vector F, that is, F = a*F context +(1-a)F speaker .

[0112] The weight a can be a value between 0 and 1. The context feature information F of the same object in different texts to be processed, or different objects in the same text to be processed, can be context and session feature information F speaker The weights may be different and can be adaptively adjusted according to actual conditions. The implementation process will not be described in detail in this application.

[0113] Exemplarily, in a multi-round dialogue scenario, the to-be-processed text of the multi-round dialogue obtained by speech recognition may be:

[0114] The speech recognition result of participant A is: How to repair a business computer?

[0115] The speech recognition result of participant B is: There is no one on duty in the computer maintenance department this morning. Can you tell me your specific situation first?

[0116] The speech recognition result of participant A is: My laptop motherboard broke down this morning and it can't be turned on. Or: I won't go there this morning, I'll go in the afternoon.

[0117] According to the error correction processing mentioned above, for the object "morning" in the text to be processed of participant A, "My laptop motherboard broke down one morning and couldn't be turned on", its conversation feature information has a greater influence on the prediction result of the error correction object than the context feature information. Therefore, for the object "morning" in the text to be processed, the value of the above weight a is often less than 0.5, and the specific value is not limited.

[0118] For the object "morning" in the text to be processed of participant A, "I won't go there in the morning, I will go there in the afternoon", its context feature information has a greater influence on the prediction result of the error correction object than the conversation feature information, and the content of the previous round of dialogue, that is, the text to be processed of participant B, also includes the object "morning". Therefore, for the object "morning" in "I won't go there in the morning, I will go there in the afternoon", the value of the weight a when splicing its context feature information and conversation feature information is often greater than 0.5, and the specific value is not limited in this application.

[0119] Step S35, inputting the target feature information of each object in the to-be-processed text into the classification network to obtain the predicted probability that the corresponding object belongs to the to-be-corrected object;

[0120] Step S36, determining the object whose predicted probability is greater than the probability threshold as the object to be corrected, and obtaining a replacement object for the object to be corrected;

[0121] Step S37, using the replacement object to replace the corresponding object to be corrected in the corresponding text to be processed, to obtain the target text.

[0122] According to the description of the corresponding part above, the classification network in the embodiment of the present application can be a classifier that predicts two types of labels: the object to be corrected and the correct object. Combined with the method for obtaining the target feature information described above, in the process of class label prediction, the contextual semantics of the object in the text to be processed can be considered at the same time, and the influence of the identity of the participating objects in the multi-round dialogue scene can also be considered, thereby improving the accuracy of the predicted probability of whether the object belongs to the object to be corrected. The predicted probability can be obtained based on, but not limited to, activation functions such as softmax, and the present application does not elaborate on the process of obtaining the predicted probability.

[0123] It should be noted that, based on the target feature information of each object in the text to be processed, while predicting the probability that the corresponding object belongs to the object to be corrected, a replacement object for the object to be corrected can also be obtained. This application does not limit the method for obtaining the replacement object.

[0124] Since the larger the above-mentioned predicted probability is, the more likely the corresponding object is to be the object to be corrected, the present application can pre-define the object error detected by speech recognition, that is, the probability threshold of the object to be corrected. In this way, after obtaining the predicted probability of each object in the above manner, it is compared with the probability threshold, and the object with a predicted probability greater than the probability threshold is determined as the object to be corrected. Otherwise, it can be considered to be the correct object and no correction adjustment is required.

[0125] Among them, the above-mentioned probability threshold can be obtained through multiple trainings during the error correction model training stage, such as the probability threshold corresponding to the case where the loss function value of the error correction model converges; of course, it can also be determined based on experience. This application does not limit the numerical size of the above-mentioned probability threshold and its acquisition method.

[0126] In some embodiments, after the target feature information of each object in the text to be processed of multiple rounds of dialogue is input into the classification network, the object to be corrected contained in the corresponding text to be processed and its corresponding replacement object can be directly output, or the target object obtained by replacing the corresponding object to be corrected with the replacement object can be directly output. This application does not impose any restrictions on this and it can be determined according to the circumstances.

[0127] In some other embodiments, according to the method described above, after using the pre-trained error correction model to obtain the prediction results of the objects to be processed, the error correction model can be further trained based on the prediction results, and the model parameters of the error correction model can be adjusted, that is, the error correction model is optimized to improve the error correction accuracy of the error correction model. Afterwards, the optimized error correction model is used to perform error correction processing on the speech recognition results according to the method described above. The implementation process is not described in detail in this application.

[0128] Exemplarily, the multi-round dialogue scenario between participant A and participant B described above is still used as an example for explanation. According to the error correction processing method described above, it can be determined that the object "morning" in the to-be-processed text of participant A "The motherboard of my laptop broke this morning and it couldn't be turned on" is the object to be corrected, and its replacement object can be "business". The target text thus obtained can be "The motherboard of my business laptop broke down and it couldn't be turned on"; and the object "morning" in the to-be-processed text "I won't go there in the morning, I'll go in the afternoon" is not the object to be corrected.

[0129] The following will describe the training implementation process of the error correction model in the above embodiment, but is not limited to the training method of the error correction model of the above network structure described in this application. Figure 8 As shown, it is a flowchart of an optional example of a method for training an error correction model in the speech recognition error correction method proposed in this application. The method can be executed by a server, such as Figure 8 As shown, the method may include:

[0130] Step S41, obtaining a plurality of training sample speech signals generated by a conversation between a plurality of sample objects, and sample identity feature information of the corresponding sample objects;

[0131] In order to make the error correction model suitable for speech recognition error correction processing in multi-round dialogue scenarios, the present application can obtain training sample speech signals of sample object (i.e., sample user) dialogues from the dialogue content generated in various multi-round dialogue scenarios. Similarly, test samples for the error correction model can be obtained. The implementation process is not described in detail in this application.

[0132] Regarding the method for obtaining the sample identity feature information of different sample objects, reference may be made to the method for obtaining the identity feature information of the participating objects described in the above embodiment, and this embodiment will not elaborate on it.

[0133] Step S42, inputting the training sample speech signal into the speech recognition model, and outputting the corresponding training sample text;

[0134] Step S43, inputting the training sample text and the corresponding sample identity feature information into the initial error correction network, and outputting the predicted probability and replacement object of each training object in the training sample text belonging to the object to be corrected;

[0135] In the embodiment of the present application, in combination with the two-level coding mechanism of the error correction model described above, the initial error correction network constructed in the embodiment of the present application may include two-level initial coding networks, namely, a first coding network and a second coding network, and a classification network for predicting whether a sample object is a sample object to be corrected based on the sample feature information of each sample object output by the coding network. Among them, the two coding networks may be bidirectional neural networks, and the network structure of the initial error correction network and its construction method are not described in detail in the present application.

[0136] It should be noted that the network parameters that need to be trained in the initial error correction network may include but are not limited to: the first weight of the first coding vector (such as context feature information) output by the first coding network for different training objects, and the second weight of the second coding vector (such as session feature information) output by the second coding network for different training objects; combined with the relevant description of the weights of the context feature information and session feature information of the same object in the above embodiment, the first weight and the second weight represent the predictive influence of the corresponding coding vector on the corresponding training object belonging to the object to be corrected. Normally, the sum of the first weight and the second weight is 1. The present application can adjust the first weight and the second weight of the output of each layer in the coding network through continuous iterative training of the error correction network to meet the termination condition of the model training.

[0137] In the case where the above-mentioned coding network is a bidirectional neural network, the training object feature information of different training objects input into the coding network can be input into different units in the bidirectional neural network (these units have similar structures and are interrelated), and different units process the input training object feature information, or process the identity feature information and the context feature information output by the previous unit, and the context feature information and session feature information of the training object obtained by different units are input into the next unit. In this processing process, the parameters used by the intermediate layer units of the bidirectional neural network to process the input feature information can be determined as the weights of the corresponding input feature information. It can be seen that the above-mentioned first weight and second weight can be the processing parameters of the intermediate layer of the coding network, which can be determined according to the operation principle of the bidirectional neural network.

[0138] During each training process, the error correction network can refer to the error correction model in the above embodiment to process the text to be processed and the identity feature information of the participating objects, realize the processing of the input training sample text and the corresponding sample identity feature information, and obtain the predicted probability and replacement object of each training object (word, etc.) in the training sample text belonging to the object to be corrected. The implementation process will not be repeated in the embodiment of this application.

[0139] It can be understood that the present application can also obtain the training object feature information of each training object contained in the training sample text according to the feature extraction method described above, and then input the training object feature information of each training object and the training identity feature information of the corresponding sample object into the error correction network for processing. The implementation process is similar to the error correction processing process described in the above embodiment, and the embodiments of the present application will not be described in detail.

[0140] Step S44, obtaining the error correction loss of the corresponding training object according to the prediction probability of each training object and the replacement object;

[0141] Step S45, adjusting the network parameters of the error correction network according to the error correction loss, and continuing to train the error correction network with the adjusted network parameters until the training termination condition is met to obtain an error correction model.

[0142] Among them, the training termination conditions may include but are not limited to: the number of iterative training times reaches the number threshold; the error correction loss of the error correction network converges; the total loss obtained based on the error correction loss of the error correction network and the speech recognition loss of the speech recognition model converges, etc. Therefore, after each training to obtain the error correction result, the preset loss function can be used to obtain the loss value of the error correction result (i.e. the above-mentioned error correction loss), and then analyze whether the error correction loss of the error correction results obtained by multiple trainings converges. The implementation process is not described in detail in this application, and this application does not limit the category of the preset loss function, which can be determined according to the situation.

[0143] After obtaining the error correction loss after each training according to the above method, the network parameters of the error correction network can be adjusted based on but not limited to optimizers such as stochastic gradient descent. After that, the error correction network with the adjusted network parameters can be used to continue training according to the method described above.

[0144] In the application of the embodiment of the present application, after the error correction model is trained according to the above method, a test sample can be obtained to test the error correction model. The test process is similar to the error correction process described above, and the identities of different speakers can be determined. The implementation process is not described in detail in this application. If the test result does not meet the test requirements, the error correction model can be optimized according to the model training method described above until the test requirements are met. The test requirements may include an error correction accuracy greater than a threshold, etc. This application does not limit the content of the test requirements.

[0145] Reference Fig. 9 , is a schematic diagram of an optional example of a speech recognition error correction device proposed in this application, and the device may include:

[0146] The context feature information acquisition module 31 is used to acquire context feature information of each object in the text to be processed; the text to be processed is obtained by performing speech recognition on the speech signal of the corresponding participating object;

[0147] The identity feature information acquisition module 32 is used to acquire the identity feature information of the participant.

[0148] A session characteristic information obtaining module 33 is used to process the context characteristic information and the identity characteristic information of the corresponding participating object to obtain session characteristic information of the corresponding object in the text to be processed;

[0149] A target feature information obtaining module 34, configured to obtain target feature information of a corresponding object based on the context feature information and the session feature information of the same object;

[0150] The error correction processing module 35 is used to perform error correction processing on the text to be processed according to the target feature information of each object in the text to be processed to obtain a target text.

[0151] Optionally, the context feature information acquisition module 31 may include:

[0152] A speech recognition unit, used to perform speech recognition on the speech signals of each participant to obtain the text to be processed of the corresponding participant;

[0153] An object feature information acquisition unit, used to acquire object feature information of each object in the text to be processed;

[0154] The first encoding processing unit is used to input the object feature information of each object into the first encoding network of the error correction model, and output the context feature information of the corresponding object.

[0155] Optionally, the identity feature information acquisition module 32 may include:

[0156] An identity registration information acquisition unit, used to acquire the identity registration information of each participant registered on the conversation platform;

[0157] The identity feature information acquisition unit is used to extract features from the body registration information to obtain identity feature information of the corresponding participant.

[0158] Optionally, the session feature information obtaining module 33 may include:

[0159] The second encoding processing unit is used to input the context feature information and the identity feature information of the corresponding participating objects into the second encoding network of the error correction model, and output the conversation feature information of the corresponding objects in the text to be processed.

[0160] Optionally, the target feature information obtaining module 34 may include:

[0161] A weighted fusion processing unit, used to perform weighted fusion processing on the context feature information and the session feature information of the same object to obtain target feature information of the corresponding object;

[0162] The respective weights of the context feature information and the conversation feature information of the same object are obtained by fitting the training sample data of the error correction model.

[0163] Optionally, the error correction processing module 35 may include:

[0164] A classification prediction unit, used for inputting the target feature information of each object in the to-be-processed text into a classification network of an error correction model, and obtaining an object to be corrected whose prediction probability is greater than a probability threshold and a corresponding replacement object according to the obtained prediction probability that the corresponding object belongs to the object to be corrected;

[0165] The text error correction unit is used to use the replacement object to replace the corresponding object to be corrected in the text to be processed to obtain a target text.

[0166] In some other embodiments proposed in the present application, in order to implement error correction model training, the above-mentioned device may further include:

[0167] A training sample acquisition module, used to acquire a plurality of training sample speech signals generated by a conversation between a plurality of sample objects, and sample identity feature information corresponding to the sample objects;

[0168] A training sample text acquisition module, used to input the training sample speech signal into a speech recognition model and output a corresponding training sample text;

[0169] A training object error correction processing module, used to input the training sample text and the corresponding sample identity feature information into an initial error correction network, and output a predicted probability and a replacement object for each training object in the training sample text belonging to an object to be corrected;

[0170] An error correction loss obtaining module, used for obtaining the error correction loss of the corresponding training object according to the predicted probability of each training object and the replacement object;

[0171] A network parameter adjustment module, used to adjust the network parameters of the error correction network according to the error correction loss, and continue to train the error correction network with the adjusted network parameters until a training termination condition is met to obtain an error correction model;

[0172] Among them, the network parameters include the first weight of the first coding vector for different training objects output by the first coding network in the error correction network, and the second weight of the second coding vector for different training objects output by the second coding network; the first weight and the second weight represent the prediction influence of the corresponding coding vector on the corresponding training object belonging to the object to be corrected.

[0173] It should be noted that the various modules, units, etc. in the above-mentioned device embodiments can be stored in the memory as program modules, and the processor executes the above-mentioned program modules stored in the memory to implement the corresponding functions. For the functions implemented by each program module and its combination, as well as the technical effects achieved, please refer to the description of the corresponding parts of the above-mentioned method embodiments, which will not be repeated in this embodiment.

[0174] The present application also provides a computer-readable storage medium on which a computer program can be stored. The computer program can be called and loaded by a processor to implement the various steps of the speech recognition error correction method described in the above embodiment.

[0175] Finally, it should be noted that, with respect to the above embodiments, unless the context clearly indicates an exception, the words "a", "an", "a kind" and / or "the" do not specifically refer to the singular, but may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of clearly identified steps and elements, and these steps and elements do not constitute an exclusive list, and the method or device may also include other steps or elements. The elements defined by the sentence "includes a..." do not exclude the existence of other identical elements in the process, method, commodity or device that includes the elements.

[0176] In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0177] The terms such as "first", "second", etc. used in this application are only used for descriptive purposes to distinguish one operation, unit or module from another operation, unit or module, and do not necessarily require or imply any actual relationship or order between these units, operations or modules. They cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0178] In this specification, each embodiment is described in a progressive or parallel manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the devices, systems, and computer equipment disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part.

[0179] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A speech recognition error correction method, the method comprising: Obtaining context feature information of each object in the text to be processed, as well as identity feature information of the participating objects; The text to be processed is obtained by performing speech recognition on the speech signals of the corresponding participating objects; the objects are the participating objects in a multi-person conversation scenario; Processing the context feature information and the identity feature information of the corresponding participating object to obtain the conversation feature information of the corresponding object in the text to be processed; Obtaining target feature information of a corresponding object based on the context feature information and the session feature information of the same object; According to the target feature information of each object in the text to be processed, error correction processing is performed on the text to be processed to obtain a target text.

2. According to the method of claim 1, the step of obtaining context feature information of each object in the text to be processed comprises: Performing speech recognition on the speech signals of each participant to obtain the text to be processed of the corresponding participant; Obtaining object feature information of each object in the text to be processed; The object feature information of each object is input into the first encoding network of the error correction model, and the context feature information of the corresponding object is output.

3. According to the method of claim 1, the processing of the context feature information and the identity feature information of the corresponding participating object to obtain the conversation feature information of the corresponding object in the text to be processed comprises: The context feature information and the identity feature information of the corresponding participating objects are input into the second encoding network of the error correction model, and the conversation feature information of the corresponding objects in the text to be processed is output.

4. The method according to any one of claims 1 to 3, wherein obtaining target feature information of a corresponding object based on the context feature information and the session feature information of the same object comprises: Performing weighted fusion processing on the context feature information and the session feature information of the same object to obtain target feature information of the corresponding object; The respective weights of the context feature information and the conversation feature information of the same object are obtained by fitting the training sample data of the error correction model.

5. According to the method of claim 4, the training process of the error correction model comprises: Acquire a plurality of training sample speech signals generated by a conversation between a plurality of sample objects, and sample identity feature information corresponding to the sample objects; Inputting the training sample speech signal into a speech recognition model and outputting corresponding training sample text; Input the training sample text and the corresponding sample identity feature information into an initial error correction network, and output the predicted probability and replacement object of each training object in the training sample text belonging to the object to be corrected; Obtaining the error correction loss of the corresponding training object according to the predicted probability of each training object and the replacement object; According to the error correction loss, adjusting the network parameters of the error correction network, and continuing to train the error correction network with the adjusted network parameters until a training termination condition is met, thereby obtaining an error correction model; Among them, the network parameters include the first weight of the first coding vector for different training objects output by the first coding network in the error correction network, and the second weight of the second coding vector for different training objects output by the second coding network; the first weight and the second weight represent the prediction influence of the corresponding coding vector on the corresponding training object belonging to the object to be corrected.

6. The method according to claim 4, wherein the error correction process is performed on the text to be processed based on the target feature information of each object in the text to be processed to obtain the target text, comprising: Inputting the target feature information of each object in the to-be-processed text into the classification network of the error correction model, and obtaining the to-be-corrected object whose predicted probability is greater than a probability threshold and the corresponding replacement object according to the obtained predicted probability that the corresponding object belongs to the to-be-corrected object; The replacement object is used to replace the corresponding object to be corrected in the text to be processed to obtain the target text.

7. According to the method of claim 1, the step of obtaining identity feature information of a participant comprises: Obtain the identity registration information of each participant registered on the conversation platform; Feature extraction is performed on the identity registration information to obtain identity feature information of the corresponding participating object.

8. A speech recognition error correction device, the device comprising: A context feature information acquisition module is used to acquire context feature information of each object in the text to be processed; The text to be processed is obtained by performing speech recognition on the speech signals of the corresponding participating objects; the objects are the participating objects in a multi-person conversation scenario; An identity feature information acquisition module is used to acquire the identity feature information of the participant. A session characteristic information obtaining module, used to process the context characteristic information and the identity characteristic information of the corresponding participating object to obtain session characteristic information of the corresponding object in the text to be processed; A target feature information acquisition module, used to obtain target feature information of a corresponding object based on the context feature information and the session feature information of the same object; The error correction processing module is used to perform error correction processing on the text to be processed according to the target feature information of each object in the text to be processed to obtain a target text.

9. A computer device, comprising: at least one memory and at least one processor; The memory is used to store a program for implementing the speech recognition error correction method according to any one of claims 1 to 7; The processor is used to load and execute the program stored in the memory to implement the speech recognition error correction method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, wherein the computer program is called and executed by a processor to implement the speech recognition error correction method according to any one of claims 1 to 7.

Images