Text error correction method for voice recognition result

Abstract

A text error correction method for a speech recognition result belongs to the technical field of computers, aims to provide an efficient and accurate text error correction solution and improve the quality and reliability of the text result after speech recognition, and is characterized in that a text error correction model network structure comprises a BERT layer, a full connection layer, a desection layer and a correction layer, the full connection layer is used for error detection, and the correction layer is used for error correction. And weighting the loss of the convergence layer and the correction layer to obtain the final loss for training the model.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology and relates to natural language processing, in particular to a text error correction method and a text post-processing method in natural language processing technology. Background Art

[0002] Speech recognition technology has been widely used in modern society and can convert speech input into text form. However, due to factors such as noise, accent, and pronunciation errors in the speech recognition process, the recognition results often contain certain errors. These errors will cause the quality of the transcribed text to deteriorate, causing trouble for subsequent processing and application.

[0004] Although some Chinese text error correction technologies already exist, there are still some challenges in accurately correcting text after speech recognition. First, the characteristics of speech recognition errors are complex and diverse, involving spelling, grammar, punctuation and other aspects. Second, speech recognition errors are not exactly the same as normal typing errors because they are usually related to the characteristics of speech input. Summary of the invention

[0005] The present invention aims to provide an efficient and accurate text error correction solution to improve the quality and reliability of text results after speech recognition. In one embodiment of the present invention, a text error correction method for speech recognition results includes:

[0006] S10. Obtaining the text of the speech recognition result;

[0007] S20. Correct the errors in the text.

[0008] In a text error correction method for speech recognition results in one embodiment of the present invention, the errors in the text are corrected in step S20, which is implemented by a text error correction model network structure.

[0009] In a text correction method for speech recognition results in one embodiment of the present invention, the text correction model network structure in step S20 includes a BERT layer, a fully connected layer, a detection layer, and a correction layer, wherein the fully connected layer is used for error detection, and the losses of the detection layer and the correction layer are weighted to obtain a final loss for training the model.

[0010] In an embodiment of the present invention, the text error correction method for speech recognition results, the error correction method in step S20 includes:

[0011] S21. Analyze the text after speech recognition;

[0012] S22. Detecting and locating errors in the text according to the analysis;

[0013] S23. Generate a candidate set of error corrections based on the location of the error;

[0014] S24. Using an evaluation method based on a machine learning algorithm and a language model, candidates are weighed and judged through a text correction model network structure, and errors in the original text are corrected using correct words in the correction layer of the text correction model network structure.

[0015] In a text error correction method for speech recognition results in one embodiment of the present invention, the text after speech recognition in step S21 is analyzed, including lexical analysis, syntactic analysis and context understanding;

[0016] In step S22, errors in the text are detected and located according to the analysis, including using natural language processing and machine learning algorithms, combined with domain knowledge and corpus resources, to identify spelling errors, grammatical errors and other problems;

[0017] The step S23 generates a set of error correction candidates according to the location of the error, including possible replacement words, phrases or sentences. When generating the error correction candidates, context information, language model and domain knowledge are comprehensively considered to ensure the accuracy and legitimacy of the error correction candidates, and the characteristics and common errors of the speech recognition engine are considered to provide options for generating the error correction candidates;

[0018] The step S24 uses an evaluation method based on a machine learning algorithm and a language model to weigh and judge the candidates, taking into account the candidate's language fluency, contextual coherence, semantic consistency, and the error factors of the speech recognition engine, so as to select the candidate that is most likely to be correctly corrected as the final output.

[0019] In a text error correction method for speech recognition results in one embodiment of the present invention, the loss function of the model is:

[0020] loss = w·loss det +(1-w)·loss cor .

[0021] In an embodiment of the present invention, the text correction method for speech recognition results, the text of the speech recognition results is obtained in step S10, including:

[0022] Extract audio files from video files;

[0023] Identify a first text corresponding to the audio in the audio file, the first text is a set of corresponding texts of the audio in the audio file, and the first text includes an audio time mark;

[0024] Classifying different sound features in the audio in the audio file through voiceprint recognition;

[0025] Determine different speakers and audio intervals of different speakers according to the feature classification;

[0026] Identify a second text corresponding to the audio of the audio interval, the second text is a collection of texts corresponding to the audio of each of the audio intervals, and the second text includes a text interval tag of the text interval corresponding to each audio interval and a speaker tag corresponding to the text interval, and the second text also includes an audio time tag;

[0027] According to the audio time mark, the second text is matched with the first text, the text corresponding to the audio of each audio interval in the second text is mapped to the corresponding text position of the first text, and the speaking object of the corresponding text interval of the first text is marked according to the text interval mark of the second text, and the first text of the speaking object of the marked text interval is the file obtained by the speech transcription, wherein the audio interval and the text interval are corresponding representations of the time interval of the audio time.

[0028] In an embodiment of the present invention, a text error correction method for speech recognition results is used to identify valid text content in an audio interval, identify sound features and feature classification of the audio interval, so that the text interval of this type of text in the second text has no corresponding text content, but the text interval corresponds to a speaker tag, and this type of text is a special text in the second text;

[0029] According to the audio time mark, the second text is matched with the first text, wherein the special text in the second text corresponds to the corresponding text position in the first text according to the audio time mark, and the speaking object of the corresponding text interval mark of the first text is matched according to the text interval mark of the special text in the second text.

[0030] In one embodiment of the present invention, a text error correction method for speech recognition results includes:

[0031] For video files in law enforcement recorders or monitors, use automated editing tools to extract the audio from the video and fix the format. The extracted audio format is fixed to 16kHz, 16-bit mono audio;

[0032] The method for classifying different sound features in the audio in the audio file by voiceprint recognition includes:

[0033] Preprocessing the audio signal of the audio in the audio interval;

[0034] Perform MFCC feature extraction on the audio signal of the preprocessed audio interval, convert each frame waveform into a multidimensional vector containing sound information, and obtain a feature vector in the audio of the audio interval;

[0035] The acoustic model classifies the feature vector according to the acoustic features of the speech to obtain a sound feature classification in the audio in the audio file;

[0036] Identify the second text corresponding to the audio of the audio interval, including

[0037] The acoustic model classifies the acoustic features into phonemes and / or word units;

[0038] The obtained phonemes and / or word units are decoded into a complete sentence through a language model, and the complete sentence is the second text of the text interval corresponding to the audio of the audio interval.

[0039] In a text error correction method for speech recognition results in one embodiment of the present invention, the preprocessing includes any one of frame cutting, pre-emphasis on the high-frequency part of the speech, and increasing the high-frequency resolution of the speech, or a combination thereof.

[0040] An embodiment of the present application also provides an electronic device, comprising: one or more processors, a memory, and one or more programs; wherein the one or more programs are stored in the memory, and the one or more programs include instructions, and when the instructions are executed by the electronic device, the electronic device executes any possible technical solution of the first aspect of the embodiment of the present application.

[0041] An embodiment of the present application also provides a computer-readable storage medium, which includes a computer program. When the computer program runs on an electronic device, the electronic device executes any possible technical solution of the first aspect of the embodiment of the present application.

[0042] Beneficial effects of the present invention:

[0043] The error correction method of the present invention is intended to improve the accuracy and reliability of error correction. The technology uses natural language processing and machine learning algorithms, combined with professional knowledge and corpus resources in the field of speech recognition, to detect and correct errors in the text after speech recognition, and provide more accurate text output.

[0044] The present invention can effectively handle spelling errors, grammatical errors, missing punctuation and other problems in speech recognition text. By analyzing the context information, language model and domain rules of the text, errors are identified and corrected, and accurate text results are finally provided.

[0045] In the judicial field, this patented technology can be applied to automated speech recognition document processing systems, providing legal workers with efficient and accurate text correction tools and improving the quality and reliability of legal documents.

[0046] With respect to the speech recognition method of the present invention, based on the two requirements of manpower and time, the present invention proposes a method that can extract the conversation content of the video in the law enforcement recorder or monitor, and save the text of the conversation, thereby reducing the demand for manpower and improving the speed of storing law enforcement information data.

[0047] In order to recognize text in a video, the present invention uses speech signal processing and recognition to extract and initially recognize the audio. In order to distinguish different speakers in the audio, the present invention uses voiceprint recognition to classify different sound features.

[0048] The present invention matches the second texts of different speakers with the first text obtained by overall recognition, so that, on the one hand, the first text can have text partitions and corresponding marks of speakers, and the first text with text partitions and corresponding marks of speakers is used as the final transcription text. On the other hand, the speech that cannot be recognized in the second text of segmented recognition due to reasons such as short audio time is compensated by the first text of overall recognition, thus avoiding the phenomenon that segmented recognition has some speech that cannot be recognized due to reasons such as short audio time. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a schematic diagram of the network structure of a text error correction model in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The present invention relates to natural language processing in the computer field, and specifically to a text error correction method and a text post-processing method in natural language processing, aiming to provide an efficient and accurate text error correction solution, and improve the quality and reliability of text results after speech recognition. The embodiments of the present application are described in detail below with reference to the accompanying drawings, and examples of the embodiments are shown in the accompanying drawings. The present application provides a method, a device, and a computer-readable storage medium, aiming to provide an efficient and accurate text error correction solution, and improve the quality and reliability of text results after speech recognition. Among them, the method, the device, and the computer-readable storage medium are based on the same technical concept, and the principles of solving the problem are similar, so the implementation of each subject can refer to each other, and the repetitions will not be repeated.

[0051] In the first embodiment, Figure 1 This is a diagram of the network structure of the text error correction model. Figure 1 As shown, the speech error correction method comprises the following steps:

[0052] 1. Text analysis after speech recognition:

[0053] Analyze the text after speech recognition to detect errors and inconsistencies. The analysis process includes lexical analysis, syntactic analysis and context understanding to accurately grasp the language features and semantic information in the text.

[0054] 2. Error detection and location:

[0055] After the text after speech recognition is analyzed, errors in the text can be detected and located. Common problems such as spelling errors and grammatical errors can be identified by using natural language processing and machine learning algorithms, combined with domain knowledge and corpus resources.

[0056] 3. Error correction candidate generation:

[0057] Generate a set of error correction candidates based on the location of the error, including possible replacement words, phrases or sentences. When generating error correction candidates, context information, language model and domain knowledge are comprehensively considered to ensure the accuracy and legitimacy of the error correction candidates. In addition, the characteristics of the speech recognition engine and common errors are also considered to provide more accurate selection for generating error correction candidates.

[0058] 4. Evaluation and selection of error correction candidates:

[0059] After generating the error correction candidates, the evaluation method based on machine learning algorithms and language models is used to weigh and judge the candidates. The candidate's language fluency, contextual coherence, semantic consistency, and the error of the speech recognition engine are comprehensively considered to select the candidate that is most likely to be the correct correction as the final output.

[0060] In the specific implementation, the BERT series pre-trained model in the field of natural language processing is used, and the original model structure is modified. There are two main ideas: using a discriminator and an error corrector or using only an error corrector. The main reason is that one of the pre-training tasks of the pre-trained model during training is the masked language model (MLM), which is very suitable for text error correction tasks. The network structure of the text error correction model is as follows: Figure 1 As shown in the figure, based on the pre-trained model, a fully connected layer is added as error detection, and the final loss is obtained by weighting the loss of the detection layer and the correction layer to train the model. In the prediction stage, the correct words in the correction layer are used to correct the errors in the original text.

[0061] By using a pre-trained model, you can analyze the text after speech recognition and detect and locate errors. The range of error correction candidates is generally the content of the vocabulary, which can be expanded according to needs.

[0062] The error correction method for Chinese text after speech recognition of the present invention can effectively detect and correct errors in speech recognition text, thereby improving the quality, accuracy and reliability of the text, by combining natural language processing and machine learning algorithms, as well as domain expertise and corpus resources.

[0063] In another embodiment, the error correction method comprises the following steps:

[0064] First, it is necessary to prepare a large-scale text data set after speech recognition and pre-process the data for subsequent training and analysis; then build a model based on the above-mentioned text error correction model network structure, and use the prepared corpus for training, and save the trained model weights; then the model can be integrated into a complete Chinese text error correction system. The system can be used as an independent application or integrated into a speech recognition system to realize the function of automatic error correction of text after speech recognition. By implementing the above-mentioned specific implementation method, an efficient and accurate error correction function for Chinese text after speech recognition can be realized, and the quality and reliability of speech recognition text can be improved.

[0065] Based on the above embodiments, an embodiment of the present application also provides an electronic device, which includes: one or more processors, a memory, and one or more programs; wherein the one or more programs are stored in the memory, and the one or more programs include instructions, and when the instructions are executed by the electronic device, the electronic device executes the method provided in the above embodiments.

[0066] Based on the above embodiments, the embodiments of the present application further provide a computer storage medium, in which a computer program is stored. When the computer program is executed by a computer, the computer executes the method provided in the above embodiments.

[0067] The storage medium may be any available medium that can be accessed by a computer. For example, but not limited to, a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer.

[0068] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0069] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0070] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0071] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0072] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A text error correction method for speech recognition results, characterized in that: include S10. Obtaining the text of the speech recognition result; S20. Correct the errors in the text.

2. The text error correction method for speech recognition results according to claim 1, characterized in that: In step S20, the errors in the text are corrected, which is implemented by the text error correction model network structure.

3. The text error correction method for speech recognition results according to claim 2, characterized in that: The text error correction model network structure in step S20 includes a BERT layer, a fully connected layer, a detection layer, and a correction layer, wherein the fully connected layer is used for error detection, and the losses of the detection layer and the correction layer are weighted to obtain the final loss for training the model.

4. The text error correction method for speech recognition results according to claim 3, characterized in that: The error correction method in step S20 includes: S21. Analyze the text after speech recognition; S22. Detecting and locating errors in the text according to the analysis; S23. Generate a candidate set of error corrections based on the location of the error; S24. Using an evaluation method based on a machine learning algorithm and a language model, candidates are weighed and judged through a text correction model network structure, and errors in the original text are corrected using correct words in the correction layer of the text correction model network structure.

5. The text error correction method for speech recognition results according to claim 4, characterized in that: The step S21 includes analyzing the text after speech recognition, including lexical analysis, syntactic analysis and context understanding; In step S22, errors in the text are detected and located according to the analysis, including using natural language processing and machine learning algorithms, combined with domain knowledge and corpus resources, to identify spelling errors, grammatical errors and other problems; The step S23 generates a set of error correction candidates according to the location of the error, including possible replacement words, phrases or sentences. When generating the error correction candidates, context information, language model and domain knowledge are comprehensively considered to ensure the accuracy and legitimacy of the error correction candidates, and the characteristics and common errors of the speech recognition engine are considered to provide options for generating the error correction candidates; The step S24 uses an evaluation method based on a machine learning algorithm and a language model to weigh and judge the candidates, taking into account the candidate's language fluency, contextual coherence, semantic consistency, and the error factors of the speech recognition engine, so as to select the candidate that is most likely to be correctly corrected as the final output.

6. The text error correction method for speech recognition results according to claim 5, characterized in that: The loss function of the model is: loss=w·loss det +(1-w)·loss cor 。 7. The text error correction method for speech recognition results according to claim 1, characterized in that: The text of the speech recognition result obtained in step S10 includes Extract audio files from video files; Identify a first text corresponding to the audio in the audio file, the first text is a set of corresponding texts of the audio in the audio file, and the first text includes an audio time mark; Classifying different sound features in the audio in the audio file through voiceprint recognition; Determine different speakers and audio intervals of different speakers according to the feature classification; Identify a second text corresponding to the audio of the audio interval, the second text is a collection of texts corresponding to the audio of each of the audio intervals, and the second text includes a text interval tag of the text interval corresponding to each audio interval and a speaker tag corresponding to the text interval, and the second text also includes an audio time tag; According to the audio time mark, the second text is matched with the first text, the text corresponding to the audio of each audio interval in the second text is mapped to the corresponding text position of the first text, and the speaking object of the corresponding text interval of the first text is marked according to the text interval mark of the second text, and the first text of the speaking object of the marked text interval is the file obtained by the speech transcription, wherein the audio interval and the text interval are corresponding representations of the time interval of the audio time.

8. The text error correction method for speech recognition results according to claim 7, characterized in that: No valid text content is recognized in the audio interval, and the sound features and feature classification of the audio interval are recognized, so that the text interval of such text in the second text has no corresponding text content, but the text interval corresponds to the speaker tag, and such text is a special text in the second text; According to the audio time mark, the second text is matched with the first text, wherein the special text in the second text corresponds to the corresponding text position in the first text according to the audio time mark, and the speaking object of the corresponding text interval mark of the first text is matched according to the text interval mark of the special text in the second text.

9. The text error correction method for speech recognition results according to claim 8, characterized in that: include For video files in law enforcement recorders or monitors, use automated editing tools to extract the audio from the video and fix the format. The extracted audio format is fixed to 16kHz, 16-bit mono audio; The method for classifying different sound features in the audio in the audio file by voiceprint recognition includes: Preprocessing the audio signal of the audio in the audio interval; Perform MFCC feature extraction on the audio signal of the preprocessed audio interval, convert each frame waveform into a multidimensional vector containing sound information, and obtain a feature vector in the audio of the audio interval; The acoustic model classifies the feature vector according to the acoustic features of the speech to obtain a sound feature classification in the audio in the audio file; Identify the second text corresponding to the audio of the audio interval, including The acoustic model classifies the acoustic features into phonemes and / or word units; The obtained phonemes and / or word units are decoded into a complete sentence through a language model, and the complete sentence is the second text of the text interval corresponding to the audio of the audio interval.

10. The text error correction method for speech recognition results according to claim 9, characterized in that: The preprocessing includes any one of frame cutting, pre-emphasizing the high frequency part of the speech, and increasing the high frequency resolution of the speech, or a combination thereof.

11. An electronic device, comprising: One or more processors, a memory, and one or more programs; wherein the one or more programs are stored in the memory, and the one or more programs include instructions, and when the instructions are executed by the electronic device, the electronic device executes any one of the methods of claims 1 to 10.

12. A computer-readable storage medium, comprising a computer program, and when the computer program is executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 10.

Images