Information processing method, apparatus, device, storage medium, and program product

Abstract

An information processing method, device, equipment, storage medium and program product are disclosed, the method comprising: obtaining a to-be-detected target sentence of a user, the to-be-detected target sentence being composed of multiple characters; determining corrected characters according to the multiple characters through an error correction model; the error correction model being obtained by training a pre-trained language model, a semantic aggregation layer network and a classifier, the pre-trained language model being used to generate character vectors, the semantic aggregation layer network being used to generate error correction vectors according to the character vectors, and the classifier being used to generate the corrected characters according to the error correction vectors; wherein the corrected characters compose a corrected sentence, which is used to support voice interaction with the user. The present application can solve the problem of complex processing and low accuracy of error detection in the prior art voice error correction method.

Description

Technical Field

[0001] This invention relates to the field of information processing technology, and in particular to an information processing method, apparatus, device, storage medium, and program product. Background Technology

[0002] Intelligent voice systems are an indispensable part of human-computer interaction systems, widely used in electronic products such as mobile phones, smartwatches, speakers, and televisions, supporting various voice interaction scenarios including casual conversation, question-and-answer, and voice control. The accuracy of Acceleration Slip Regulation (ASR) in intelligent voice systems is a key constraint affecting the development of intelligent voice products. During voice interaction, user requests are typically converted from voice commands by the ASR system. However, due to technical limitations, the text generated by the ASR system may contain errors, leading to deviations in the subsequent understanding of user intent. Therefore, how to preprocess and correct the request text from the ASR system has become a pressing problem.

[0003] Most current speech error correction technologies are based on homophone rules or obfuscated character sets. Homophone rule-based correction requires cumbersome rule formulation and is prone to miscorrection if contextual information is not considered. The effectiveness of obfuscated character set construction largely depends on the quantity and quality of manually collected obfuscated characters. Since different scenarios require different obfuscated sets, different business units still need to invest significant manpower and time to label sufficient data. Furthermore, the speech used in the same scenario may vary. Therefore, obfuscated character set-based error correction involves processing large amounts of data, wastes resources, and has a relatively low accuracy rate in error detection.

[0004] Therefore, existing voice error correction methods are complex to process and have low accuracy in error detection. Summary of the Invention

[0005] The main objective of this invention is to provide an information processing method, apparatus, device, storage medium, and program product, which aims to solve the problems of complex processing and low accuracy of error detection in existing speech error correction methods.

[0006] To achieve the above objectives, the present invention provides an information processing method, the method comprising:

[0007] Obtain the user's target sentence to be detected, which consists of multiple characters;

[0008] Based on the multiple characters, the corrected characters are determined through an error correction model. The error correction model is obtained by training a pre-trained language model, a semantic aggregation layer network, and a classifier. The pre-trained language model is used to generate character vectors, the semantic aggregation layer network is used to generate error correction vectors based on the character vectors, and the classifier is used to generate the corrected characters based on the error correction vectors.

[0009] The corrected characters form a corrected sentence, which is used to support voice interaction with the user.

[0010] Optionally, determining the corrected character based on the plurality of characters using an error correction model includes:

[0011] Based on the multiple characters, the original vectors and semantic vectors of the multiple characters are obtained through the trained pre-trained language model;

[0012] Based on the original vector and the semantic vector, the error correction vector is determined through the trained semantic aggregation layer network;

[0013] The error correction vector is input into the trained classifier to obtain the corrected character.

[0014] Optionally, obtaining the original vectors and semantic vectors of the multiple characters based on the multiple characters through a pre-trained language model includes: inputting the multiple characters into a pre-trained language model, and mapping the original vectors corresponding to the multiple characters through the character vector matrix in the pre-trained language model;

[0015] Based on the original vectors, semantic vectors related to the context of the multiple characters are obtained through the multi-layer attention mechanism in the pre-trained language model.

[0016] Optionally, determining the error correction vector based on the original vector and the semantic vector through a trained semantic aggregation layer network includes:

[0017] The original vector and the semantic vector are normalized by the trained semantic aggregation layer network to obtain the normalized original vector and the normalized semantic vector.

[0018] The difference vector between the normalized original vector and the normalized semantic vector is calculated.

[0019] Based on the normalized original vector, the normalized semantic vector, and the difference vector, the error correction vector is determined through the trained semantic aggregation layer network.

[0020] Optionally, the method further includes:

[0021] The system uses an ASR (Automatic Speech Recognition) system to obtain the sentence to be detected corresponding to the labeled historical audio and the correct sentence corresponding to the historical audio.

[0022] The sentence to be detected and the corresponding correct sentence are used as training samples to train the pre-trained language model, the semantic aggregation layer network and the classifier respectively, so as to obtain the trained language model, the trained semantic aggregation layer network and the trained classifier.

[0023] The trained language model, the trained semantic aggregation layer network, and the trained classifier constitute the error correction model.

[0024] Optionally, obtaining the user's target sentence to be detected includes:

[0025] Obtain the user's voice commands;

[0026] Based on the user's voice command, the target sentence to be detected is obtained through the ASR system.

[0027] Optionally, the method further includes:

[0028] Based on the corrected sentence, search for the corresponding response information;

[0029] The response information is displayed or read aloud.

[0030] The present invention also provides an information processing apparatus, the apparatus comprising:

[0031] The acquisition module is used to acquire the user's target sentence to be detected, which consists of multiple characters;

[0032] The processing module is used to determine the corrected character based on the multiple characters using an error correction model. The error correction model is obtained by training a pre-trained language model, a semantic aggregation layer network, and a classifier. The pre-trained language model is used to generate character vectors, the semantic aggregation layer network is used to generate error correction vectors based on the character vectors, and the classifier is used to generate the corrected character based on the error correction vectors.

[0033] The corrected characters form a corrected sentence, which is used to support voice interaction with the user.

[0034] The present invention also provides an information processing device, the information processing device comprising: a memory, a processor, and an information processing program stored in the memory and executable on the processor, wherein the information processing program, when executed by the processor, implements the steps of the information processing method as described in any of the preceding claims.

[0035] The present invention also provides a computer-readable storage medium storing an information processing program, which, when executed by a processor, implements the steps of the information processing method as described in any of the preceding claims.

[0036] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the method described in any of the preceding claims.

[0037] In this invention, a semantic error correction model is used to perform semantic error correction on the acquired target sentence to be detected, thereby accurately understanding the user's intent, achieving a higher level of voice interaction with the user, and improving the user experience. Specifically, the target sentence to be detected is first acquired from the user. Then, the sentence, composed of multiple characters, is semantically analyzed using a pre-trained language model to generate character vectors, providing a data foundation for semantic-level voice error correction. Furthermore, semantic analysis, compared to text analysis alone, improves error correction capabilities. Next, a pre-trained semantic aggregation layer network is used to concatenate the character vectors, generating an error correction vector. The error correction vector, after semantic aggregation, achieves a higher accuracy rate in error detection. Finally, the error correction vector is passed through a pre-trained classifier to obtain new characters, i.e., the corrected characters, forming the corrected sentence. The processing is simple and does not require manual collection of obfuscated characters or the specification of numerous error correction rules. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of the present invention;

[0039] Figure 2 A flowchart illustrating an information processing method provided in an embodiment of the present invention;

[0040] Figure 3 A flowchart illustrating an information processing method provided in another embodiment of the present invention;

[0041] Figure 4 A flowchart illustrating an information processing method according to another embodiment of the present invention;

[0042] Figure 5 This is a schematic diagram of the structure of an information processing device provided in an embodiment of the present invention;

[0043] Figure 6 This is a schematic diagram of the structure of an information processing device provided in an embodiment of the present invention.

[0044] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0045] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the invention and to fully convey the invention to those skilled in the art.

[0046] Currently, most speech error correction technologies are based on homophone rules or obfuscated character sets. Homophone rule-based correction requires complex rule formulation and is prone to miscorrection if contextual information is not considered. Other methods involve manually or machine-assisted construction of obfuscated vocabularies (or obfuscated character sets), followed by rule-based modeling or statistical language modeling for error correction. Essentially, these methods are still closely related to the construction of obfuscated vocabularies; a well-constructed vocabulary is crucial for improved performance. However, the effectiveness of obfuscated character set construction largely depends on the quantity and quality of manually collected obfuscated characters. Since different scenarios require different obfuscated sets, different business units still need to invest significant manpower and time in labeling sufficient data. Furthermore, the speech used in the same scenario varies. These methods are becoming increasingly inadequate for the ever-growing volume of audio data. Therefore, obfuscated character set-based error correction involves processing large amounts of data, is resource-intensive, and has low error detection accuracy. Thus, existing speech error correction methods are complex in process and have low error detection accuracy.

[0047] To address this issue, the technical concept of this invention is to utilize an error correction model composed of a pre-trained language model, a semantic aggregation layer network, and a classifier to achieve speech error correction. The processing is simple, and the machine learning model ensures the accuracy of speech recognition, thereby enabling accurate understanding of the user's intent and achieving a high level of voice interaction.

[0048] The following detailed description of some embodiments of the present invention is provided in conjunction with the accompanying drawings. Where there is no conflict between the embodiments, the following embodiments and features thereof can be combined with each other.

[0049] It should be noted that the term "multiple" in this application is used to mean two or more, and the specific number is not specifically limited here.

[0050] Figure 1 This is a schematic diagram illustrating an application scenario provided by an embodiment of the present invention. For example... Figure 1As shown, the executing entity of this information processing method can be an information processing device. This information processing device can be configured in the intelligent voice system of the human-computer interaction system. The intelligent voice system can be applied to smart electronic products such as mobile phones, wristbands, speakers, and televisions, or to question-and-answer robots provided by banks, shopping malls, etc. There are no specific limitations here. The intelligent voice system realizes the speech conversion of the user's voice request, and then performs semantic error correction on the converted text, so as to accurately understand the user's intention and provide the user with a more accurate response, thereby realizing a high level of voice interaction.

[0051] For example, combined Figure 1 As shown, taking a question-and-answer robot as an example, in the application scenario of bank consultation, the intelligent voice system in the question-and-answer robot can realize the conversion between voice commands and text, as well as semantic error correction of the text. For example, the speech content recognition (Acceleration Slip Regulation) system in the intelligent voice system can convert user request instructions (i.e., queries) (or voice commands) into text. Since the text generated by ASR may contain errors, it will lead to deviations in the subsequent understanding of user intent. Therefore, an error correction model can be constructed by pre-trained language models, semantic aggregation layer networks, and classifiers to obtain the corrected results, realizing semantic error correction of the text. This results in a higher accuracy rate of error correction, thus more accurately understanding the user's intent and providing the user with responses that meet the user's needs, completing voice interaction at a high level, and improving the user experience. Therefore, for the speech error correction scenario, a pre-trained language model is introduced for semantic error correction. At the same time, a simple and effective semantic aggregation layer network is proposed to achieve semantic error correction.

[0052] This invention employs an error correction model trained using a pre-trained language model, a semantic aggregation layer network, and a classifier to semantically model user request commands. By learning the logic and common-sense reasoning of language through data, it achieves data-driven analysis and correction of grammatical errors and typos in sentences. The processing is simple, and the machine learning model ensures the accuracy of speech recognition, enabling accurate understanding of user intent and achieving a high level of voice interaction.

[0053] Figure 2 This is a flowchart illustrating an information processing method provided in an embodiment of the present invention. Figure 2 As shown, the method in this embodiment may include:

[0054] S201. Obtain the user's target sentence to be detected, wherein the target sentence to be detected consists of multiple characters.

[0055] The target sentence to be detected can be obtained by converting the user's request instruction (or voice command) into an ASR system.

[0056] Optionally, obtaining the user's target sentence for detection can be achieved through the following steps:

[0057] Step a1: Obtain the user's voice commands.

[0058] Step a2: Based on the user's voice command, obtain the target sentence to be detected through the ASR system.

[0059] Specifically, users input voice commands into the human-computer interaction system via voice dialogue. The ASR (Automatic Speech Recognition) system within the system is trained using pre-labeled audio samples. Therefore, when there is audio input to be detected (receiving the user's voice command, which can be considered as the audio to be detected), the ASR system can perform speech content recognition on the audio to obtain the predicted text corresponding to the audio. When there is a difference between the predicted text and the actual text, in order to more accurately understand the user's intention and provide a suitable and accurate response, the human-computer interaction system corrects the content of the predicted text through an information processing device. Here, the predicted text is considered as the target sentence to be detected, composed of multiple characters.

[0060] S202. Based on the multiple characters, determine the corrected characters using an error correction model.

[0061] The error correction model is obtained by training a pre-trained language model, a semantic aggregation layer network, and a classifier. The pre-trained language model is used to generate character vectors, the semantic aggregation layer network is used to generate error correction vectors based on the character vectors, and the classifier is used to generate corrected characters based on the error correction vectors.

[0062] Specifically, an error correction model is obtained by training a pre-trained language model, a semantic aggregation layer network, and a classifier model. Then, multiple characters corresponding to the target sentence to be detected are input into the error correction model to obtain the corrected characters, thereby realizing semantic error correction of the user's speech content recognition during voice interaction.

[0063] By using a pre-trained language model, a corresponding character vector can be generated for each character. Then, a semantic aggregation layer network preprocesses and concatenates these character vectors to obtain an error correction vector. This error correction vector is then input into a multi-classifier to determine the correct character, thus generating the corrected character. The corrected character then forms a corrected sentence to support voice interaction with the user.

[0064] By acquiring the user's target sentence for detection, and then performing semantic analysis on the sentence composed of multiple characters using a pre-trained language model to generate character vectors, a data foundation is provided for semantic-level speech error correction. Furthermore, semantic analysis, compared to text analysis alone, improves error correction capabilities. Then, a pre-trained semantic aggregation layer network is used to concatenate the character vectors, generating error correction vectors. These semantically aggregated error correction vectors result in higher error detection accuracy. Finally, the error correction vectors are passed through a pre-trained classifier to obtain new characters—the corrected characters—forming the corrected sentence. The process is simple and eliminates the need for manual collection of obfuscated characters or the specification of numerous error correction rules.

[0065] Optionally, this embodiment, based on the above embodiments, provides a detailed explanation of how to determine the corrected character. Determining the corrected character based on the plurality of characters using an error correction model can be achieved through the following steps:

[0066] Step b1: Based on the multiple characters, obtain the original vectors and semantic vectors of the multiple characters through the trained pre-trained language model.

[0067] Step b2: Based on the original vector and the semantic vector, determine the error correction vector through the trained semantic aggregation layer network.

[0068] Step b3: Input the error correction vector into the trained classifier to obtain the corrected character.

[0069] Semantic analysis is performed using a pre-trained language model to generate character vectors. A trained semantic aggregation layer network is then used to concatenate these character vectors to generate error correction vectors. Finally, these error correction vectors are passed through a trained classifier to obtain new characters, i.e., the corrected characters, thus improving the accuracy of speech recognition.

[0070] Specifically, a pre-trained language model (Bidirectional Encoder Representation from Transformers, BERT) can be used to semantically model user queries, obtaining a semantic vector for each character. Since semantic vectors are heavily influenced by their context, when a sentence contains a typo, its semantics become illogical with the context, resulting in a greater change in the semantic vector than the vector at the correct position. Therefore, a semantic aggregation layer network is used to capture these changes in the semantic vector of typos, generating error correction vectors to detect and correct errors in sentences. Finally, a classifier identifies and determines the correct character.

[0071] Optionally, based on the multiple characters, the original vectors and semantic vectors of the multiple characters are obtained through a pre-trained language model, which is achieved through the following steps:

[0072] Step c1: Input the multiple characters into the pre-trained language model, and obtain the original vectors corresponding to the multiple characters by mapping through the character vector matrix in the pre-trained language model.

[0073] Step c2: Based on the original vector, obtain the semantic vector related to the context of the multiple characters through the multi-layer attention mechanism in the pre-trained language model.

[0074] For example, combined Figure 3 As shown, given a user query(T), T = {T1, T2, ..., T} i ,…,T n} is a sentence consisting of n characters, where T i This refers to the index of the i-th character. When these characters are input into a pre-trained language model, the original vector E = {E1, E2, ..., E...} of the character can be found through the character vector matrix in the pre-trained language model. i , ..., E n The original vectors, after passing through multiple layers of a self-attention neural network (i.e., the multi-layer attention mechanism in a pre-trained language model), can yield a context-dependent semantic vector C = {C1, C2, ..., C}. i C n}

[0075] The sentence to be detected (containing multiple characters) requires a start character CLS and an end character T during the actual input process. ESP Enter them together. Accordingly, E [CLS] The original vector representing the start character CLS, C [CLS] The semantic vector representing the start character CLS; E [ESP] The end character T indicates the end of the string. ESP The original vector, C [ESP] The end character T indicates the end of the string. ESP The semantic vector.

[0076] By using a pre-trained language model, the original vector and semantic vector are obtained respectively, enabling semantic-level error correction and improving the accuracy of error correction.

[0077] Optionally, the error correction vector can be determined based on the original vector and the semantic vector through the trained semantic aggregation layer network, which can be achieved through the following steps:

[0078] Step d1: Normalize the original vector and the semantic vector using the trained semantic aggregation layer network to obtain the normalized original vector and the normalized semantic vector.

[0079] Step d2: Calculate the difference vector between the normalized original vector and the normalized semantic vector.

[0080] Step d3: Based on the normalized original vector, the normalized semantic vector, and the difference vector, determine the error correction vector through the trained semantic aggregation layer network.

[0081] To ensure convergence and network stability, given the characteristics of the network, vector normalization is necessary, specifically for both the original vector and the semantic vector. To improve error correction accuracy and enhance its capability, the three types of feature data (the normalized original vector, the normalized semantic vector, and the difference vector) can be concatenated to achieve semantic aggregation. This aggregated vector is then used as the error correction vector. By using data containing multiple feature classes as input to the classifier, inaccurate outputs due to biases in a single feature are avoided. Therefore, this method is more accurate than inputting only one type of feature data (either the original vector or the semantic vector) as the error correction vector.

[0082] The semantic composition layer obtains the enhanced error correction vector by concatenating the original character vector E, the semantic vector C, and the difference vector CV. Specifically, to highlight the changes in the character vector, the semantic composition layer first calculates the difference between the original character vector (the context-independent character vector) and the semantic vector output by BERT, and then concatenates the original character vector, the semantic vector, and the difference vector.

[0083] Furthermore, the difference vector calculation is exemplary. Besides calculating the difference vector to obtain the error correction vector, one can also calculate the standard deviation or variance vector of the normalized original vector and the normalized semantic vector after obtaining them; or, assign weights to the normalized original vector and the normalized semantic vector respectively, and calculate the target difference vector. Then, the normalized original vector, the normalized semantic vector, and the standard deviation vector are passed through a trained semantic aggregation layer network to determine the error correction vector; or, the normalized original vector, the normalized semantic vector, and the standard deviation vector are passed through a trained semantic aggregation layer network to determine the error correction vector; or, the normalized original vector, the normalized semantic vector, and the target difference vector are passed through a trained semantic aggregation layer network to determine the error correction vector. The aim is to concatenate the three types of feature data to obtain the error correction vector, thereby improving the accuracy of error correction.

[0084] For example, combined Figure 3 As shown, the semantic aggregation layer network processes the original vector E = {E1, E2, ..., E...} respectively. i , ..., E n} and semantic vector C = {C1, C2, ..., C i Normalize the vectors {C1, ..., Cn} to obtain the normalized original vector v. E ={v1 E v2 E , ..., v i E , ..., v n E} and the normalized semantic vector v C ={v1 C v2 C , ..., v i C , ..., v n C Then calculate the difference vector v. i C -v i E and for v E v C and (v) i C -v i E The semantic vectors of misspelled words are concatenated to obtain the error correction vector. Therefore, by using a semantic aggregation layer network to capture the changes in the semantic vectors of misspelled words, errors in sentences can be detected and corrected, greatly improving the capabilities of the semantic error correction model.

[0085] Finally, the error correction vector is input into the multi-classifier (i.e., the correction layer), and the output is a new character index, T' = {T1, T2, ..., T}. i ',…,Tn},where T i Not equal to T i , indicating T i 'This is the result after model correction.'

[0086] The classifier for the error correction vector input consists of a three-layer fully connected neural network. The number of neurons in the first layer is the same as the dimension of the error correction vector. The number of neurons in the second layer is 2 / 3 of that in the first layer, and the number of neurons in the third layer is 1 / 3 of that in the first layer (the number of neurons decreases from the first layer to the third layer). Dropout regularization is applied to the input of the classifier, and the activation function of each neuron is the ReLU activation function.

[0087] Optionally, model training can be performed using a data-driven approach, enabling the model to learn from low-cost labeled data and identify speech errors based on semantic information. This can be achieved through the following steps:

[0088] Step e1: Using an ASR (Automatic Speech Recognition) system, obtain the sentence to be detected corresponding to the labeled historical audio and the correct sentence corresponding to the historical audio.

[0089] Step e2: Using the sentence to be detected and the corresponding correct sentence as training samples, train the pre-trained language model, semantic aggregation layer network and classifier respectively to obtain the trained language model, the trained semantic aggregation layer network and the trained classifier.

[0090] The trained language model, the trained semantic aggregation layer network, and the trained classifier constitute the error correction model.

[0091] Specifically, training the error correction model requires the sentence to be detected as input and the correct sentence as the label. The entire process does not require manually constructing a confusion set, resulting in low data costs. For example, inputting pre-annotated audio into the ASR system yields the sentence to be detected (i.e., the sentence to be detected corresponding to the annotated historical audio and the corresponding correct sentence). Since the audio already has correct annotations, these annotations can still serve as training labels for the error correction model (i.e., the sentence to be detected and the corresponding correct sentence are used as training samples for the error correction model). In this way, pre-annotated speech data can be fully utilized, achieving efficient acquisition of error correction data. In other words, the training process of the error correction model utilizes the data used by the ASR system itself, i.e., the pre-annotated audio data, as training labels, achieving efficient acquisition of error correction data.

[0092] Optionally, the user's voice interaction process can be implemented through the following steps:

[0093] Step f1: Based on the corrected sentence, search for the response information corresponding to the corrected sentence.

[0094] Step f2: Display or verbally announce the response information.

[0095] For example, a user can first wake up the question-and-answer robot with a specific phrase, such as "Hello, A," and the robot will respond, "Hello! How can I help you?" The user can then begin asking questions to the robot, achieving voice interaction. See, for example... Figure 4 As shown, during voice interaction, the user issues the voice command "I want to consult A" to the question-and-answer robot. The ASR system of the intelligent voice system in the question-and-answer robot receives and parses the user's voice command, obtaining the converted text "I want to consult A'". Then, the error correction model of the intelligent voice system corrects the converted text, resulting in "I want to consult A". This voice error correction process utilizes the characteristics of the pre-trained language model itself. By combining context, it can achieve semantic analysis, generate character vectors, and use a semantic aggregation layer network to generate error correction vectors based on character vectors, which can greatly improve the error correction model's error correction capability. Then, through a classifier, a new character, i.e., the corrected character, i.e., the corrected result, is output. The human-computer interaction system searches for response information based on the corrected result and outputs the response information. Here, the response information can be displayed directly on the human-computer interaction system or broadcast by voice, realizing voice interaction. In addition, generating error correction data through the ASR system does not require a large amount of manpower to collect confusion sets and rules, and the model can be transferred to different scenarios at low cost.

[0096] Figure 5 This is a schematic diagram of the structure of an information processing device provided in an embodiment of the present invention. Figure 5 As shown, the information processing device may include:

[0097] The acquisition module 501 is used to acquire the user's target sentence to be detected, which consists of multiple characters;

[0098] The processing module 502 is used to determine the corrected character based on the multiple characters using an error correction model. The error correction model is obtained by training a pre-trained language model, a semantic aggregation layer network, and a classifier. The pre-trained language model is used to generate character vectors, the semantic aggregation layer network is used to generate error correction vectors based on the character vectors, and the classifier is used to generate the corrected character based on the error correction vectors.

[0099] The corrected characters form a corrected sentence, which is used to support voice interaction with the user.

[0100] Optionally, the processing module includes a first processing unit, a second processing unit, and a third processing unit; the first processing unit is used to obtain the original vectors and semantic vectors of the multiple characters based on the multiple characters through a pre-trained language model; the second processing unit is used to determine the error correction vector based on the original vectors and the semantic vectors through a trained semantic aggregation layer network; the third processing unit is used to input the error correction vector into a trained classifier to obtain the corrected characters.

[0101] Optionally, the first processing unit is specifically used for:

[0102] The multiple characters are input into a pre-trained language model, and the original vectors corresponding to the multiple characters are obtained by mapping through the character vector matrix in the pre-trained language model.

[0103] Based on the original vectors, semantic vectors related to the context of the multiple characters are obtained through the multi-layer attention mechanism in the pre-trained language model.

[0104] Optionally, the second processing unit is specifically used for:

[0105] The original vector and the semantic vector are normalized by the trained semantic aggregation layer network to obtain the normalized original vector and the normalized semantic vector.

[0106] The difference vector between the normalized original vector and the normalized semantic vector is calculated.

[0107] Based on the normalized original vector, the normalized semantic vector, and the difference vector, the error correction vector is determined through the trained semantic aggregation layer network.

[0108] Optionally, the processing module further includes a fourth processing unit and a fifth processing unit;

[0109] The fourth processing unit is used to obtain the sentence to be detected corresponding to the labeled historical audio and the correct sentence corresponding to the historical audio through the speech content recognition ASR system;

[0110] The fifth processing unit is used to train a pre-trained language model, a semantic aggregation layer network, and a classifier using the sentence to be detected and the corresponding correct sentence as training samples, respectively, to obtain a trained training language model, a trained semantic aggregation layer network, and a trained classifier; wherein, the trained training language model, the trained semantic aggregation layer network, and the trained classifier constitute the error correction model.

[0111] Optionally, the acquisition module is specifically used for:

[0112] Obtain the user's voice commands;

[0113] Based on the user's voice command, the target sentence to be detected is obtained through the ASR system.

[0114] Optionally, the processing module further includes a sixth processing unit; the sixth processing unit is used to search for response information corresponding to the corrected sentence based on the corrected sentence; and to display or verbally announce the response information.

[0115] The information processing apparatus provided in any of the foregoing embodiments is used to execute the technical solutions of any of the foregoing method embodiments. Its implementation principle and technical effect are similar, and will not be repeated here.

[0116] Figure 6 This is a schematic diagram of the structure of an information processing device provided in an embodiment of the present invention. Figure 6 As shown, the device may include: a memory 601, a processor 602, and an information processing program stored in the memory 601 and executable on the processor 602. When the information processing program is executed by the processor 602, it implements the steps of the information processing method as described in any of the foregoing embodiments.

[0117] Alternatively, the memory 601 can be either standalone or integrated with the processor 602.

[0118] The implementation principle and technical effects of the device provided in this embodiment can be found in the foregoing embodiments, and will not be repeated here.

[0119] This invention also provides a computer-readable storage medium storing an information processing program, which, when executed by a processor, implements the steps of the information processing method as described in any of the foregoing embodiments.

[0120] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the methods described in any of the foregoing embodiments.

[0121] In the several embodiments provided by this invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed.

[0122] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of the present invention.

[0123] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0124] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk or optical disc, etc.

[0125] The aforementioned storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.

[0126] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. Both the processor and the storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components in an electronic device or host device.

[0127] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0128] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0129] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0130] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. An information processing method, characterized in that, The method includes: Obtain the user's target sentence to be detected, which consists of multiple characters; Based on the multiple characters, the original vectors and semantic vectors of the multiple characters are obtained through the trained pre-trained language model; The original vector and the semantic vector are normalized by the trained semantic aggregation layer network to obtain the normalized original vector and the normalized semantic vector. The difference vector between the normalized original vector and the normalized semantic vector is calculated. Based on the normalized original vector, the normalized semantic vector, and the difference vector, the error correction vector is determined through the trained semantic aggregation layer network. The error correction vector is input into the trained classifier to obtain the corrected character; wherein, the trained pre-trained language model, the trained semantic aggregation layer network, and the trained classifier constitute the error correction model, the pre-trained language model is used to generate character vectors, the semantic aggregation layer network is used to generate error correction vectors based on character vectors, and the classifier is used to generate corrected characters based on error correction vectors. The corrected characters form a corrected sentence, which is used to support voice interaction with the user.

2. The method according to claim 1, characterized in that, The step of obtaining the original vectors and semantic vectors of the multiple characters using a pre-trained language model includes: The multiple characters are input into a pre-trained language model, and the original vectors corresponding to the multiple characters are obtained by mapping through the character vector matrix in the pre-trained language model. Based on the original vectors, semantic vectors related to the context of the multiple characters are obtained through the multi-layer attention mechanism in the pre-trained language model.

3. The method according to any one of claims 1-2, characterized in that, The method further includes: The system uses an ASR (Automatic Speech Recognition) system to obtain the sentence to be detected corresponding to the labeled historical audio and the correct sentence corresponding to the historical audio. The sentence to be detected and the corresponding correct sentence are used as training samples to train a pre-trained language model, a semantic aggregation layer network, and a classifier, respectively, to obtain the trained pre-trained language model, the trained semantic aggregation layer network, and the trained classifier.

4. The method according to claim 3, characterized in that, The process of obtaining the user's target sentence to be detected includes: Obtain the user's voice commands; Based on the user's voice command, the target sentence to be detected is obtained through the ASR system.

5. The method according to any one of claims 1-2, characterized in that, The method further includes: Based on the corrected sentence, search for the corresponding response information; The response information is displayed or read aloud.

6. An information processing device, characterized in that, The device includes: The acquisition module is used to acquire the user's target sentence to be detected, which consists of multiple characters; The processing module is configured to: obtain the original vectors and semantic vectors of the multiple characters using a pre-trained language model; normalize the original vectors and semantic vectors using a trained semantic aggregation layer network to obtain normalized original vectors and normalized semantic vectors; calculate the difference vector between the normalized original vectors and the normalized semantic vectors; determine an error correction vector using the trained semantic aggregation layer network based on the normalized original vectors, the normalized semantic vectors, and the difference vector; and input the error correction vector into a trained classifier to obtain the corrected character. The pre-trained language model, the trained semantic aggregation layer network, and the trained classifier constitute an error correction model. The pre-trained language model generates character vectors, the semantic aggregation layer network generates error correction vectors based on the character vectors, and the classifier generates the corrected character based on the error correction vectors. The corrected characters form a corrected sentence, which is used to support voice interaction with the user.

7. An information processing device, characterized in that, The information processing device includes: a memory, a processor, and an information processing program stored in the memory and executable on the processor, wherein when the information processing program is executed by the processor, it implements the steps of the information processing method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an information processing program, which, when executed by a processor, implements the steps of the information processing method as described in any one of claims 1-5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1-5.

Images