Speech recognition error correction method and terminal equipment

Abstract

The invention provides a voice recognition error correction method and terminal equipment, relates to the technical field of voice recognition, and can improve the voice recognition accuracy and accelerate the processing efficiency of voice recognition error correction. The method comprises the steps that the terminal equipment acquires first voice data; the terminal device recognizes the first voice data based on the voice recognition system and outputs a first recognition result; the terminal device determines a target recognition result according to whether the user performs error correction operation on the first recognition result; if the user does not perform error correction operation on the first recognition result, the terminal equipment determines that the target recognition result is the first recognition result; if the user performs error correction operation on the first recognition result, the terminal equipment corrects the first recognition result into a second recognition result, and determines that the target recognition result is the second recognition result; and the terminal device updates the special pronunciation association building library according to at least one of the first voice data, the first recognition result and the second recognition result. The method and the device are applied to a speech recognition process.

Description

Technical Field

[0001] The present application relates to the field of speech recognition technology, and in particular to a speech recognition error correction method and terminal device. Background Art

[0002] At present, speech recognition technology mainly trains speech recognition systems based on massive speech data and text data, and realizes modeling of speech data to text data through signal processing, feature extraction, mathematical statistics and other methods. In this way, the speech recognition system converts the input speech content into matching text content. However, the accuracy of speech recognition technology is affected by factors such as the speaker's pronunciation accuracy, environmental noise, and data volume. For example, the speaker's pronunciation accuracy is affected by the speaker's age, gender, accent, and pathological state. Or, the speech recognition system has few training samples, and the training samples do not cover specialized vocabulary or uncommon characters in specific fields. There are often errors in recognizing the input speech content as text content with the same pronunciation but different characters, which in turn affects the user's reading and understanding. Summary of the invention

[0003] The present application provides a speech recognition error correction method and terminal device, which can optimize the speech recognition strategy, improve the accuracy of speech recognition, and speed up the processing efficiency of speech recognition error correction.

[0004] In order to achieve the above objectives, this application provides the following technical solutions:

[0005] In the first aspect, the present application provides a speech recognition error correction method, which is applied to a terminal device. The method includes: the terminal device obtains first speech data; the terminal device recognizes the first speech data based on the speech recognition system and outputs a first recognition result; the terminal device determines the target recognition result according to whether the user performs an error correction operation on the first recognition result; if the user does not perform an error correction operation on the first recognition result, the terminal device determines that the target recognition result is the first recognition result; if the user performs an error correction operation on the first recognition result, the terminal device corrects the first recognition result to a second recognition result and determines that the target recognition result is the second recognition result; the terminal device updates the dedicated pronunciation association library according to at least one of the first speech data, the first recognition result, and the second recognition result.

[0006] In this method, during the speech recognition process, the dedicated pronunciation association library in the speech recognition system is updated in real time according to the error correction information fed back by the user, so that after the terminal device detects the same error correction information again, the error can be corrected in real time according to the dedicated pronunciation association library in the updated speech recognition system, and it will take effect in time. It can speed up the processing efficiency of speech recognition error correction, reduce the frequency of secondary processing of speech recognition results by users, optimize the speech recognition system, improve user experience, and make it more practical.

[0007] According to the first aspect, the first voice data is voice data collected and acquired by the terminal device.

[0008] In some examples, the terminal device obtains the first voice data uttered by the current user when speaking through a voice collection device.

[0009] In other examples, the terminal device obtains the first voice data through a media file.

[0010] In the embodiment of the present application, before the terminal device obtains the first voice data, the terminal device has started the voice recognition function.

[0011] According to the first aspect, or any implementation of the first aspect above, the speech recognition system includes a dedicated pronunciation association library, which is used to correct the candidate text corresponding to the first speech data to output a first recognition result.

[0012] In the embodiment of the present application, the speech recognition system is a system for performing speech recognition and recognition error correction.

[0013] According to the first aspect, or any one of the implementations of the first aspect above, the dedicated pronunciation association library includes error correction phrases, error correction words, incorrect pronunciations of error correction words, and correct pronunciations of error correction words.

[0014] In some examples, the dedicated pronunciation association database is a database including error correction information text and pronunciation, which is built based on the error correction information fed back by the user. The dedicated pronunciation association database acts on a specific user and is bound to the user account. Each user has its own corresponding and unique dedicated pronunciation association database.

[0015] In some examples, the error correction operation includes preset operations such as editing and modification by the user.

[0016] In some examples, if the terminal device does not receive a correction operation from the user within a preset time, it is determined that the first recognition result obtained by the terminal device does not need to be corrected, and the first recognition result is the correct text corresponding to the first voice data.

[0017] In some examples, the terminal device accepts a correction operation from a user and, in response to the correction operation, corrects the first recognition result to a second recognition result.

[0018] According to the first aspect, or any implementation method of the first aspect above, the terminal device updates the dedicated pronunciation association library based on at least one of the first voice data, the first recognition result, and the second recognition result, including: the terminal device associates at least one of the first voice data, the first recognition result, and the second recognition result to determine the error correction information; the terminal device updates the dedicated pronunciation association library based on the error correction information.

[0019] According to the first aspect, or any one of the implementation methods of the first aspect above, the terminal device links the first voice data, the first recognition result and the second recognition result to determine the error correction information, including: the terminal device determines the error correction phrase and / or the error correction word according to the first recognition result and the second recognition result; the terminal device determines the incorrect pronunciation of the error correction word according to the first voice data; the terminal device obtains the correct pronunciation of the error correction word; the terminal device establishes a mapping relationship between the error correction phrase, the error correction word, the incorrect pronunciation of the error correction word and the correct pronunciation of the error correction word, and generates error correction information.

[0020] In some examples, the terminal device determines the correction phrase and / or the correction word in the correction phrase by comparing the first recognition result and the second recognition result.

[0021] In some examples, the terminal device determines a correction word based on the user's correction operation, and determines a correction phrase including the correction word through word segmentation processing.

[0022] In this way, the terminal device responds to the user's error correction operation, associates the information before and after the error correction, and stores the correspondence between various types of information before and after the error correction, so that when the same error occurs again in the future, it can automatically correct the error based on the stored information, thereby improving the accuracy of speech recognition and speeding up the processing efficiency of speech recognition error correction.

[0023] According to the first aspect, or any implementation method of the first aspect above, the terminal device recognizes the first voice data based on the voice recognition system and outputs a first recognition result, including: the terminal device recognizes the first voice data based on the voice recognition system to obtain N candidate texts; wherein N is an integer greater than 0; the terminal device corrects and reorders the N candidate texts according to a dedicated pronunciation association library and a preset decoding method to determine the first recognition result.

[0024] In some examples, the speech recognition system of the terminal device generates multiple corresponding text candidates based on speech feature decoding of the input first speech data, and selects text sequences with higher scores as candidate texts based on acoustic scores and language scores, that is, the first recognition result.

[0025] In some examples, the preset decoding method is a decoding method based on a weighted finite state transition machine.

[0026] In some examples, after the speech recognition system in the terminal device obtains N candidate texts, the N candidate texts output by the first recognition of the speech recognition system are corrected, re-scored and re-sorted through a dedicated pronunciation association library and a decoding method based on a weighted finite state transition machine.

[0027] In this way, the voice recognition system in the terminal device recognizes and corrects the first data input by the user, and displays the first recognition result. The special pronunciation association library in the voice recognition system includes the correction information fed back by the user. During the recognition process of the voice recognition system, corrections are automatically made according to the user's personal pronunciation habits. A special pronunciation association library corresponding to each user is generated based on the corrections and the pronunciation characteristics of each user. Based on the special pronunciation association library, real-time personalized corrections are made to improve the accuracy of voice recognition.

[0028] According to the first aspect, or any implementation of the first aspect above, after the terminal device corrects the first recognition result to the second recognition result and determines that the target recognition result is the second recognition result, the method also includes: the terminal device obtains second voice data, the second voice data includes a first incorrect pronunciation in the first voice data, the first incorrect pronunciation corresponds to a first error correction word, and the dedicated pronunciation association library updated based on the first voice data includes the correspondence between the first incorrect pronunciation and the first error correction word; the terminal device recognizes the second voice data based on the voice recognition system and outputs a third recognition result, and the third recognition result is the correct voice recognition result corresponding to the second voice recognition data.

[0029] In some examples, the first erroneous speech may be an erroneous pronunciation corresponding to an error correction phrase and / or an error correction word during the user's error correction operation on the first recognition result. The updated dedicated pronunciation association library includes the above error correction information.

[0030] In this way, the voice recognition system in the terminal device can automatically recognize and correct the second voice data, and directly obtain the correct voice recognition result, that is, the third recognition result, without the need for manual error correction, thereby reducing the frequency of users' secondary processing of the voice recognition results and optimizing the voice recognition system.

[0031] According to the first aspect, or any implementation of the first aspect above, the speech recognition system also includes a common pronunciation association library corresponding to multiple users; the common pronunciation association library and the special pronunciation association library are used to correct the candidate texts corresponding to the first speech data to output a first recognition result; the terminal device outputs the first recognition result based on the speech recognition system to recognize the first speech data, including: the terminal device recognizes the first speech data based on the speech recognition system to obtain N candidate texts; wherein N is an integer greater than 0; the terminal device corrects and reorders the N candidate texts according to the special pronunciation association library, the common pronunciation association library and the preset decoding method to determine the first recognition result.

[0032] In some examples, the terminal device can obtain the universal pronunciation association library from the cloud server.

[0033] In some examples, the user can authorize the dedicated pronunciation association library generated by the terminal device to the cloud server of the speech recognition system. The cloud server can update the general pronunciation association library in the speech recognition system in the cloud server according to the data in the dedicated pronunciation association library uploaded by the authorized users, and synchronize the updated general pronunciation association library to the terminal device, so that the terminal device can perform speech recognition error correction based on the latest general pronunciation association library, thereby speeding up the recognition error correction efficiency.

[0034] According to the first aspect, or any implementation of the first aspect above, the method further includes: if the dedicated pronunciation association library and the general pronunciation association library contain the same error correction phrase, then for the same error correction phrase, the priority of the dedicated pronunciation association library is higher than that of the general pronunciation association library.

[0035] In this way, the terminal device generates a dedicated pronunciation association library unique to each user based on the user's error correction feedback, so as to determine the user's unique pronunciation habits based on the dedicated pronunciation association library, so that the speech recognition system can adapt to users with different pronunciation habits, and the speech recognition system can adaptively identify and personalize error corrections based on the characteristics of each user. The speech recognition system automatically matches users without changing the user's pronunciation habits, which can improve the user's experience. Moreover, the speech recognition system based on matching the user's pronunciation habits can more accurately and effectively recognize the user's voice, and improve the recognition efficiency. Moreover, during the error correction process, the data in the general pronunciation association library can also be used for error correction, which can improve the accuracy of the speech recognition results.

[0036] In a second aspect, the present application provides a terminal device, comprising: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the terminal device executes a method as described in the first aspect or any one of the embodiments of the first aspect.

[0037] In a third aspect, the present application provides a chip system comprising at least one processor and at least one interface circuit, wherein the at least one interface circuit is used to perform transceiver functions and send instructions to at least one processor, and when the at least one processor executes the instructions, the at least one processor executes a method as described in the first aspect or any one of the embodiments of the first aspect.

[0038] In a fourth aspect, the present application provides a computer-readable storage medium, which includes a computer program or instructions. When the computer program or instructions are run on a computer, the computer executes a method as described in the first aspect or any one of the embodiments of the first aspect.

[0039] In a fifth aspect, the present application provides a computer program product, which includes: a computer program or instructions, which, when the computer program or instructions are run on a computer, enable the computer to execute a method as described in the first aspect or any one of the embodiments of the first aspect.

[0040] It should be noted that the technical effects brought about by any design in the above-mentioned second to fifth aspects can refer to the technical effects brought about by the corresponding design in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A flowchart of a system-preset voice recognition automatic error correction solution provided in an embodiment of the present application;

[0042] Figure 2 A flowchart of a speech recognition error correction solution based on manual editing provided in an embodiment of the present application;

[0043] Figure 3 A schematic diagram of a terminal device provided in an embodiment of the present application Figure 1 ;

[0044] Figure 4 A schematic diagram of a speech recognition error correction system architecture provided in an embodiment of the present application Figure 1 ;

[0045] Figure 5 A schematic diagram of a speech recognition error correction system architecture provided in an embodiment of the present application Figure 2 ;

[0046] Figure 6 A flow chart of a speech recognition error correction method provided in an embodiment of the present application Figure 1 ;

[0047] Figure 7 A scenario example of a speech recognition error correction method provided in an embodiment of the present application Figure 1 ;

[0048] Figure 8 A scenario example of a speech recognition error correction method provided in an embodiment of the present application Figure 2 ;

[0049] Fig. 9 A scenario example of a speech recognition error correction method provided in an embodiment of the present application Figure 3 ;

[0050] Fig.10 A scenario example of a speech recognition error correction method provided in an embodiment of the present application Figure 4 ;

[0051] Fig.11 A scenario example of a speech recognition error correction method provided in an embodiment of the present application Figure 5 ;

[0052] Fig.12 A flow chart of a speech recognition error correction method provided in an embodiment of the present application Figure 2 ;

[0053] Fig.13 A flow chart of a speech recognition error correction method provided in an embodiment of the present application Figure 3 ;

[0054] Fig.14 A schematic diagram of a terminal device provided in an embodiment of the present application Figure 2 ;

[0055] Fig.15 A schematic diagram of the structure of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "described", "above", "the" and "this" are intended to also include, for example, "one or more" such expressions, unless there is an explicit contrary indication in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one or more than two (including two). The character " / " generally represents that the front and back associated objects are a kind of "or" relationship.

[0057] References to "one embodiment" or "some embodiments" etc. described in this specification mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Thus, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways. The term "connected" includes direct and indirect connections, unless otherwise stated.

[0058] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0059] In the embodiments of the present application, the words "exemplarily" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a specific way.

[0060] In some examples, the speech recognition error correction is implemented based on the speech recognition automatic error correction scheme preset by the system. Figure 1 As shown, it is a flow chart of the automatic error correction scheme of speech recognition based on the system preset. In this example, the speech recognition system of the terminal device is pre-set with a basic speech recognition system and a preset error correction library. Among them, the speech recognition system is used to identify the speech data input by the error correction user and obtain the text data corresponding to the speech data. Specifically, the basic speech recognition system is used to perform speech recognition on the speech data input by the user based on the basic vocabulary in the system to obtain the corresponding text data. The preset error correction library is used to correct the text data recognized by the basic speech recognition system to obtain the final speech-to-text recognition result (which can also be described as the target text). For example, the preset error correction library includes data such as easy-to-mistake words and easy-to-mistake pronunciations. The user inputs speech data to the terminal device, and the speech recognition system in the terminal device first recognizes the speech data based on the basic speech recognition system to obtain multiple candidate texts, and the speech recognition system in the terminal device then corrects and sorts the multiple candidate texts based on the preset error correction library to obtain the target text. Subsequently, the terminal device displays the target text.

[0061] For example, the preset error correction library includes various hot words (which can be understood as hot words or easy-to-make mistakes with similar pronunciations, etc.), which can realize the hot word recognition function. Based on the preset error correction library, the terminal device identifies easy-to-make mistakes by similar pronunciations, and then replaces and modifies the easy-to-make mistakes by pronunciation matching, and finally sorts the modified text, and determines the text with a higher language model score as the target text. For another example, the preset error correction library includes various sensitive words, which can realize the text regularization function. Based on the preset error correction library, the terminal device determines the sensitive words in the candidate text by text matching and deletes the sensitive words in the candidate text.

[0062] However, the automatic error correction scheme based on system preset speech recognition depends on the vocabulary coverage of the preset error correction library. It can only correct the words included in the preset error correction library, and cannot effectively correct the words outside the preset error correction library. Moreover, all users use the same speech recognition system and perform error correction based on the same preset error correction library. When the terminal device corrects errors according to the preset error correction library, it uses a fixed error correction logic and cannot perform adaptive error correction processing for different users. It has poor adaptability and may cause miscorrection or missed correction.

[0063] In other examples, speech recognition error correction is implemented based on manually edited speech recognition error correction schemes. Specifically, Figure 2 The figure is a flow chart of a speech recognition error correction scheme based on manual editing. In this example, the speech recognition system of the terminal device is only pre-set with a basic speech recognition system. The user inputs speech data to the terminal device, and the speech recognition system in the terminal device recognizes the speech data based on the basic speech recognition system to obtain multiple candidate texts, and sorts the multiple candidate texts ( Figure 2 (not shown in the figure), the text with a higher language model score is determined as the output text. The terminal device displays the output text and provides an editing entry to the user so that the user can edit the output text independently to correct errors. The terminal device responds to the user's editing operation (such as modification operation, etc.) on the output text, and corrects the output text to obtain the target text. Subsequently, the terminal device displays the target text.

[0064] The speech recognition error correction scheme based on manual editing corrects the recognition results output by the speech recognition system through manual intervention, but this error correction method is inefficient. The speech recognition system outputs the same text data for the same speech data. If there is an error in the text data, the error information in the recognized text data needs to be modified every time the user inputs the speech data. The user repeats the same error correction operation, and the user experience is poor. Moreover, the user needs to frequently modify the recognition results (i.e. the above-mentioned output text), and the recognition efficiency is low.

[0065] In order to improve the technical problems described above, the embodiments of the present application provide a method for speech recognition error correction. During the speech recognition process, the dedicated pronunciation association library in the speech recognition system is updated in real time according to the error correction information fed back by the user, so that after the terminal device detects the same error correction information again, the error can be corrected in real time according to the dedicated pronunciation association library in the updated speech recognition system, and it takes effect in time. The method provided by the embodiments of the present application speeds up the processing efficiency of speech recognition error correction, reduces the frequency of secondary processing of speech recognition results by users, optimizes the speech recognition system, improves the user experience, and is more practical.

[0066] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.

[0067] The technical solution provided in the embodiment of the present application can be applied to the terminal device 100, or to a system including the terminal device 100.

[0068] In the embodiments of the present application, the terminal device may be a mobile phone, a computer, a two-in-one notebook, a two-in-one tablet computer, a laptop computer, a smart TV, a tablet computer (Pad), a computer with wireless transceiver function, a wearable device, a vehicle-mounted device, a monitoring device, a smart screen, a smart speaker, an augmented reality (AR) terminal device\virtual reality (VR) device, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), an artificial intelligence (AI) terminal, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, etc. The embodiments of the present application do not limit the specific technology and specific device form adopted by the terminal device.

[0069] The terminal device may also be called a terminal, user equipment (UE), a mobile station (MS), a mobile terminal (MT), etc.

[0070] For example, Figure 3The schematic diagram of the structure of the terminal device 100 is shown. The terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a USB interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a radio frequency module 150, a communication module 160, an audio module 170, a sensor module 180, a camera 193, a display screen 194, and a SIM card interface 195, etc.

[0071] The structure illustrated in the embodiment of the present invention does not constitute a limitation on the terminal device 100. It may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware. The embodiment of the present application does not limit the structure and form of the terminal device 100.

[0072] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0073] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor is a high-speed cache memory, which can store instructions or data that the processor has just used or cyclically used. If the processor needs to use the instruction or data again, it can be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor, and thus improves the efficiency of the system.

[0074] The USB interface 130 may be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface may be used to connect a charger to charge the terminal device 100, or may be used to transmit data between the terminal device 100 and a peripheral device. The USB interface may also be used to connect an earphone to play audio through the earphone.

[0075] The charging management module 140 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. While the charging management module is charging the battery, it can also power the terminal device 100 through the power management module 141.

[0076] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module receives input from the battery and / or the charging management module to power the processor, internal memory, external memory and display screen, etc. The power management module can also be used to monitor parameters such as battery capacity and battery health status (leakage, impedance).

[0077] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, a cellular network antenna can be reused as a wireless local area network diversity antenna. In some embodiments, the antenna can be used in combination with a tuning switch.

[0078] The RF module 150 can provide a communication processing module for wireless communication solutions including 2G-5G etc. applied to the terminal device 100. The RF module receives electromagnetic waves from the antenna 1, and performs filtering, amplification and other processing on the received electromagnetic waves, and transmits them to the modem for demodulation.

[0079] The communication module 160 can provide a communication processing module for wireless communication solutions including wireless local area networks (WLAN), Bluetooth (BT), global navigation satellite system (GNSS), near field communication (NFC), infrared (IR), etc., which are applied to the terminal device 100. The communication module 160 can be one or more devices integrating at least one communication processing module. The communication module receives electromagnetic waves via the antenna 2, modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor.

[0080] In some embodiments, the antenna 1 of the terminal device 100 is coupled to the radio frequency module, and the antenna 2 is coupled to the communication module, so that the terminal device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC and / or IR technology, etc. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS) and / or a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS), etc.

[0081] The display screen 194 is used to display images, videos, etc. The display screen includes a display panel. The display panel can be LCD (liquid crystal display), OLED (organic light-emitting diode), active matrix organic light emitting diode or active matrix organic light emitting diode (AMOLED), flexible light emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diodes (quantum dot light-emitting diodes, QLED), etc. In some embodiments, the terminal device 100 may include 1 or N display screens, where N is a positive integer greater than 1. In an embodiment of the present application, the display screen 194 can display a first recognition result, and the user can perform an error correction operation on the first recognition result in the display screen 194. In response to the user's error correction operation, the terminal device 100 displays a second recognition result (the recognition result after the first recognition result is corrected) in the display screen 194.

[0082] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. In some embodiments, the terminal device 100 may include 1 or N cameras, where N is a positive integer greater than 1.

[0083] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device 100. The external memory card communicates with the processor through the external memory interface to implement a data storage function, such as storing music, video and other files in the external memory card.

[0084] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the terminal device 100 by running the instructions stored in the internal memory 121. The memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function, etc. The data storage area may store data created during the use of the terminal device 100, etc. In addition, the memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, other volatile solid-state storage devices, a universal flash storage (UFS), etc.

[0085] The audio module 170 is used to convert digital audio information into analog audio signal output, and also to convert analog audio input into digital audio signal. The audio module can also be used to encode and decode audio signals.

[0086] The sensor module 180 of the terminal device 100 may specifically include: a pressure sensor, a distance sensor, an ambient light sensor, a fingerprint sensor, a temperature sensor, and a touch sensor, etc. The touch sensor is also called a "touch panel" and may be provided on a display screen to detect a touch operation on or near it.

[0087] The terminal device 100 may further include components such as a battery 142 and a SIM card interface 195 , to which the embodiment of the present application does not impose any limitation.

[0088] The speech recognition error correction provided by the embodiments of the present application can be applied to any speech recognition scenario. Optionally, the speech recognition scenario includes a human-computer dialogue scenario (for example, a smart home scenario, a banking business scenario, a smart device interaction scenario, etc.), a speech input scenario (for example, a speech input scenario in an input method application, a speech input scenario in an entertainment game application, or a speech input scenario in a meeting, etc.), a speech-to-text scenario in an instant messaging application, a speech search scenario in a search application, etc. The embodiments of the present application do not impose any special restrictions on the specific form of the speech recognition scenario.

[0089] For ease of understanding, Figure 4 The following is a schematic diagram of a speech recognition error correction system architecture applicable to the speech recognition error correction method provided in the embodiment of the present application. The speech recognition error correction system includes a terminal device 401 and a server 402.

[0090] In the embodiment of the present application, the terminal device 401 may be a terminal device capable of display and having a voice recognition and error correction function. Exemplarily, the terminal device may be a mobile phone, a computer, a tablet, a smart screen, etc.

[0091] In the embodiment of the present application, the server 402 may be a Linux server, a Windows server or other server device that can provide simultaneous access to multiple devices, or may be a server cluster consisting of multiple regions, multiple computer rooms, and multiple servers.

[0092] Exemplarily, the terminal device 401 may be connected to the server 402 via a wired network or a wireless network, such as a local area network, a cellular network, wireless fidelity (Wi-Fi), or the like.

[0093] In the embodiment of the present application, the terminal device 401 includes a collection module and an editing and display module.

[0094] In the embodiment of the present application, the server 402 includes a speech recognition system and a pronunciation association module.

[0095] Specifically, the acquisition module can be used to collect voice data acquired by the terminal device, such as the first voice data and the second voice data described below. The acquisition module can also be used to send the collected voice data to the voice recognition system. The acquisition module is also used to send the collected voice data to the pronunciation association module after the user performs an error correction operation. Exemplarily, the acquisition module is a microphone.

[0096] The editing and display module can be used to display the recognition results, such as displaying the first recognition result and the third recognition result obtained by the speech recognition system described below; or, displaying the second recognition result obtained after the user's error correction operation (which can also be understood as the manual error correction result). The editing and display module can also be used for users to edit recognition results, such as receiving the user's error correction operation as described below, and correcting the first recognition result to the second recognition result in response to the error correction operation. The editing and display module is also used to send the recognition result after error correction (such as the second recognition result described below) to the pronunciation association module. Exemplarily, the editing and display module provides a user editing entrance and a display interface.

[0097] The speech recognition system can be used to recognize the speech data sent by the acquisition module to obtain the recognition result corresponding to the speech data, such as the speech recognition system as described below recognizes the first speech data and outputs the first recognition result. The speech recognition system is also used to send the recognition result to the editing and display module so that the editing and display module displays the recognition result. The speech recognition system is also used to send the recognition result obtained by the speech recognition system before the user performs the error correction operation (such as the first recognition result described below) to the pronunciation association module after the user performs the error correction operation. The speech recognition system includes a dedicated pronunciation association library.

[0098] The pronunciation association module performs association according to the data sent by other modules, performs pronunciation association on the voice data sent by the acquisition module, the recognition result sent by the voice recognition system and the recognition result after error correction sent by the editing and display module, and determines the error correction information. The special pronunciation association library in the pronunciation association module is updated according to the error correction information. The pronunciation association module includes a special pronunciation association library. The pronunciation association module is also used to synchronize the updated special pronunciation association library to the voice recognition system so that the voice recognition system performs voice recognition error correction according to the updated special pronunciation association library.

[0099] Understandably, Figure 4 The voice recognition error correction system shown is only an example. The embodiment of the present application does not limit the module division method in the terminal device 401 and the server 402, and the specific functions of each module.

[0100] In some other embodiments of the present application, the terminal device 401 may also include a pronunciation association module, which includes a dedicated pronunciation association library. The pronunciation association module on the terminal side can be used to detect the user's error correction operation, obtain the user's voice data from the acquisition module, obtain the recognition result obtained by the voice recognition system before the user performs the error correction operation from the voice recognition system (such as the first recognition result described below) and obtain the recognition result after the user's error correction from the editing and display module (such as the second recognition result described below), perform pronunciation association according to the acquired information, determine the error correction information, and update the dedicated pronunciation association library according to the error correction information. The pronunciation association module in the terminal device 401 is also used to synchronize the updated dedicated pronunciation association library to the pronunciation association module of the server (cloud side). Correspondingly, the pronunciation association module on the cloud side includes a dedicated pronunciation association library and a general pronunciation association library. The voice recognition system on the cloud side performs voice recognition error correction according to the dedicated pronunciation association library and the general pronunciation association library in the pronunciation association module on the cloud side.

[0101] In the embodiment of the present application, the terminal device 401 may further include an end-side storage and computing module, which is used to store data generated or acquired by the end-side.

[0102] In an embodiment of the present application, server 402 may also include a cloud-side storage and computing module, which is used to store data generated on the cloud side, the computing process of the cloud-side speech recognition system for speech recognition error correction, and the update process of the end-cloud collaborative update of the dedicated pronunciation association library.

[0103] It is understandable that the above embodiments are described by taking the speech recognition error correction method applied to a speech recognition error correction system including a terminal device and a server as an example, and the speech recognition error correction method provided in the embodiments of the present application can also be applied to the terminal device itself. That is, the speech recognition error correction system only includes the terminal device, and the speech recognition error correction method is completed by the terminal device itself.

[0104] For example, Figure 5 , which is a schematic diagram of another speech recognition error correction system architecture applicable to the speech recognition error correction method provided in the embodiment of the present application. The speech recognition error correction system includes a terminal device 501.

[0105] In the embodiment of the present application, the terminal device 501 includes a collection module, a speech recognition module, an editing and display module, a pronunciation association module and a speech error correction module.

[0106] Specifically, the acquisition module can be used to collect voice data acquired by the terminal device, such as the first voice data and the second voice data described below. Exemplarily, the user's voice data is collected and processed through a microphone. The acquisition module can also be used to send the collected voice data to the voice recognition module. The acquisition module can also be used to send the collected voice data to the pronunciation association module after the user performs an error correction operation.

[0107] The speech recognition module can be used to identify the text content corresponding to the speech data sent by the generation acquisition module to obtain a recognition result, such as the speech recognition system described below recognizes the first speech data and outputs the first recognition result. The speech recognition module is also used to send the recognition result to the editing and display module so that the editing and display module displays the recognition result. The speech recognition module is also used to send the recognition result to the pronunciation association module after the user performs an error correction operation. The speech recognition module is also used to receive a dedicated pronunciation association library sent by the speech error correction module.

[0108] The editing and display module can be used to display the recognition results, such as displaying the first recognition result and the third recognition result obtained by the speech recognition system described below; or, displaying the second recognition result obtained after the user's error correction operation. The editing and display module can also be used for the user to edit the recognition results, such as receiving the user's error correction operation as described below, and correcting the first recognition result to the second recognition result (that is, the final recognition result) in response to the error correction operation. The editing and display module is also used to send the recognition result after error correction (such as the second recognition result described below) to the pronunciation association module.

[0109] Optionally, the editing and display module can also be used to display the correct pronunciation prompt corresponding to the error-corrected word.

[0110] The pronunciation association module can be used to perform pronunciation association according to the voice data sent by the acquisition module, the recognition result sent by the voice recognition module (such as the first recognition result described below) and the recognition result after error correction sent by the editing and display module (such as the second recognition result described below), and determine the error correction information. The dedicated pronunciation association library in the pronunciation association module is updated according to the error correction information, and the pronunciation association module is also used to synchronize the updated dedicated pronunciation association library to the voice error correction module.

[0111] The speech error correction module takes effect when the speech recognition module generates recognition results, and intervenes and corrects the recognition results based on the dedicated pronunciation association library.

[0112] It is understandable that the above Figure 4 and Figure 5 The system architecture diagram shown is only an example. In actual applications, the terminal device 401 or the terminal device 501 may include more or fewer modules, and the server 402 may also include more or fewer modules. The embodiment of the present application does not limit the division of modules in the terminal device and the server. For example, the speech recognition module and the speech error correction module in the terminal device 501 can be integrated into one module, such as a speech recognition error correction system.

[0113] The system architecture and business scenarios described in this application are intended to more clearly illustrate the technical solutions of this application, and do not constitute the sole limitation on the technical solutions provided by this application. A person of ordinary skill in the art will appreciate that, with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided by this application are equally applicable to similar technical problems.

[0114] It is understandable that in the embodiment of the present application, the terminal device can perform some or all of the steps in the embodiment of the present application, and these steps or operations are only examples. The embodiment of the present application can also perform other operations or variations of various operations. In addition, the various steps can be performed in different orders presented in the embodiment of the present application, and it is possible that not all operations in the embodiment of the present application need to be performed.

[0115] Exemplarily, the technical solutions involved in the following embodiments can all be implemented in the terminal device 100 described above. The speech recognition error correction method provided in the embodiments of the present application is described in detail below in conjunction with the accompanying drawings and application scenarios.

[0116] The following is an application example of the present invention. Figure 5 The speech recognition error correction system shown is used as an example for explanation.

[0117] In the embodiments of the present application, Figure 6 FIG. 5 is a flow chart of a speech recognition error correction method provided by an embodiment of the present application, the method comprising the following steps S601 to S607:

[0118] S601. The terminal device obtains first voice data.

[0119] In the embodiment of the present application, the first voice data is voice data collected and acquired by the terminal device.

[0120] In a possible implementation, the terminal device acquires the first voice data emitted by the current user when speaking through a voice collection device. For example, the microphone of the terminal device collects the voice data input by the user in real time in the input method application of the terminal device. The embodiment of the present application does not limit the voice collection device.

[0121] In another possible implementation, the terminal device obtains the first voice data through a media file. For example, a user uploads a conference recording file to the terminal device, and the terminal device obtains the first voice data based on the conference recording file. The first voice data may be part of the voice data or all of the voice data in the conference recording file. The embodiment of the present application does not limit the specific implementation method of the terminal device obtaining the first voice data.

[0122] It is understandable that before the terminal device acquires the first voice data, the terminal device has already started the voice recognition function. In some embodiments, the terminal device can start the voice recognition function when the voice assistant installed thereon is started, and display a prompt message on the currently displayed page to prompt the user that the voice recognition function is started.

[0123] There are many ways to start the voice assistant. For example, the mobile phone can start the voice assistant according to the user's preset operation on the power button. Or, the mobile phone can start the voice assistant when it determines that the wake-up word input by the user's voice is the same as the pre-stored wake-up word. Or, the mobile phone can start the voice assistant according to the user's preset operation on the currently displayed page. The embodiment of the present application does not limit the implementation method of starting the voice assistant. The embodiment of the present application also does not limit the implementation method of starting the voice recognition function of the terminal device.

[0124] S602: The terminal device recognizes the first voice data based on a voice recognition system and outputs a first recognition result, wherein the voice recognition system includes a dedicated pronunciation association library.

[0125] In the embodiment of the present application, the speech recognition system is a system for performing speech recognition and recognition error correction. Exemplarily, the speech recognition system can also be described as a speech recognition error correction system.

[0126] In the embodiment of the present application, the first recognition result is the recognition text output by the terminal device after the first voice data is recognized based on the voice recognition system.

[0127] In an embodiment of the present application, the dedicated pronunciation association library is a database including error correction information text and pronunciation, which is constructed based on the error correction information fed back by the user himself. The dedicated pronunciation association library is a database used for speech recognition error correction. The dedicated pronunciation association library acts on a specific user and is bound to a user account (industrial design, ID). Each user has a corresponding and unique dedicated pronunciation association library. The dedicated pronunciation association library is stored on the terminal device.

[0128] Exemplarily, as shown in Table 1, it is a pronunciation association mapping table in the dedicated pronunciation association library provided in an embodiment of the present application.

[0129] Table 1

[0130] Error correction phrase Error correction word Correct pronunciation Wrong pronunciation Hot Blazing chì zhì download load zài zǎi Mysophobia addiction pǐ pì …… …… …… ……

[0131] As shown in Table 1, only part of the information is shown in Table 1, which includes the error correction phrases, error correction words, correct pronunciation of error correction words and incorrect pronunciation of error correction words of the error correction information fed back by the user. Among them, the error correction phrases and error correction words in the error correction information are the objects for the user to correct. The correct pronunciation is the correct pronunciation of the error correction word in the error correction phrase, and the incorrect pronunciation is the pronunciation corresponding to the error correction word in the voice data input by the user.

[0132] It is understandable that the pronunciation association mapping table in the above example includes error correction phrases, error correction words, correct pronunciations of error correction words, and incorrect pronunciations of error correction words. In practical applications, the pronunciation association mapping table may include more or less content, and the embodiment of the present application does not limit the specific content contained in the dedicated pronunciation association library.

[0133] In one possible implementation, the terminal device recognizes the first voice data based on a voice recognition system to obtain N candidate texts. The terminal device corrects and reorders the N candidate texts according to a preset decoding method and a dedicated pronunciation association library to determine a first recognition result. The terminal device displays the first recognition result.

[0134] Specifically, the voice recognition system of the terminal device generates (i.e., one-step decoding) corresponding multiple text candidates (i.e., N candidate texts) based on the voice feature decoding of the input first voice data, and selects text sequences with higher scores as candidate texts based on the acoustic scores and language scores, i.e., the first recognition result.

[0135] Among them, the N candidate texts can be the N recognition results with the highest confidence ranking among the recognition results output by the speech recognition system, and N is an integer greater than 0.

[0136] In some examples, after acquiring the first voice data, the terminal device sends the first voice data to the voice recognition system. The voice recognition system in the terminal device first preprocesses the first voice data, such as noise elimination, noise reduction, audio gain, etc. Then, the voice recognition system extracts features from the preprocessed first voice data, and recognizes the features according to the voice recognition model in the voice recognition system to obtain N candidate texts.

[0137] For example, in traditional speech recognition solutions based on hidden Markov models, the acoustic model in the speech recognition system calculates the probability distribution of each phoneme or syllable based on the features. Subsequently, the language model in the speech recognition system calculates the probability of the next word based on the generated text sequence and context information. Finally, the decoder uses search techniques such as the Viterbi algorithm to search for text sequences with a high probability ranking as candidate texts under the guidance of the acoustic model and language model.

[0138] As another example, based on an end-to-end speech recognition solution, the speech recognition system includes a trained neural network model, and speech recognition is performed on input speech data based on the neural network model to obtain candidate texts corresponding to the speech data.

[0139] The preset decoding method may be a decoding method based on a weighted finite state transducer (WFST).

[0140] In other examples, after the speech recognition system in the terminal device obtains N candidate texts, a decoding method based on a weighted finite state transition machine is used to correct, re-score and re-sort the N candidate texts output by the first recognition of the speech recognition system.

[0141] Specifically, the speech recognition system includes a pre-constructed weighted finite state machine. The weighted finite state machine is constructed based on the speech model, acoustic model and other relevant information, and contains various word sequences, word sequence pronunciations and weights corresponding to each word sequence. The weighted finite state machine in the speech recognition system decodes and corrects N candidate texts. For example, by calculating the degree of match between each candidate text and the weighted finite state machine, and combining the word sequence pronunciation in the weighted finite state machine and the weight corresponding to each word sequence, the candidate text is corrected, and the score of each candidate text after correction is recalculated.

[0142] For example, in a traditional speech recognition solution based on a hidden Markov model, after one decoding, the candidate texts and pronunciation information corresponding to the speech data are obtained. The speech recognition system reorders each candidate text according to its score after error correction, and determines the candidate text with the highest ranking as the first recognition result. For example, the N candidate texts after error correction are reordered in descending order.

[0143] For another example, based on an end-to-end speech recognition solution, after one decoding, there are only candidate texts corresponding to speech data. It is necessary to first generate the pronunciation corresponding to the decoded candidate texts through a grapheme-to-phoneme (G2P) method, and then reorder the candidate texts through a weighted finite state transition machine to determine the first recognition result.

[0144] It is understandable that the specific implementation of determining the first recognition result in the above examples refers to the prior art and will not be repeated here. Moreover, the embodiment of the present application does not limit the specific implementation of the terminal device outputting the first recognition result based on the speech recognition system recognizing the first speech data.

[0145] In an embodiment of the present application, a dedicated pronunciation association library and a weighted finite state transition machine are stored in the speech recognition system of the terminal device. When the dedicated pronunciation association library is updated, the weighted finite state transition machine before the update is operated according to the update data (such as the phrase corrected by the user), and the mapping relationship between the pronunciation and text of the update data is constructed in the weighted finite state transition machine, and the weight of the update data is set. In this way, the weighted finite state transition machine is updated based on the updated dedicated pronunciation establishment library, so that when speech recognition is performed subsequently, it can be recognized based on the latest data, thereby improving the accuracy and efficiency of speech recognition. That is, the speech recognition system in the present application will automatically correct the recognition result according to the data in the dedicated pronunciation association library while performing speech recognition.

[0146] For example, Figure 7 , which is a schematic diagram of the state corresponding to the phrase “download” shown in Table 1 in the weighted finite state transition machine. Figure 7The nodes in it represent states, the connections between nodes represent the transitions between states, the pinyin and data on the connections are the labels and weights for each transition, the bold circles represent the initial state, and the two circles represent the terminal states. The solid line between the initial state and the state corresponding to the "1" node represents the correct pronunciation of "xia" in "downloading" and the weight of this correct pronunciation. The solid line between the state corresponding to the "1" node and the terminal state represents the correct pronunciation of "zai" in "download" and the weight of this correct pronunciation. The dashed line between the state corresponding to the "1" node and the terminal state represents the incorrect pronunciation of "zai" in "download" and the weight of this incorrect pronunciation in the user input voice data obtained through the arc pasting operation. In this way, the phrase is determined according to the pronunciation and weight of each character, and then the text corresponding to the voice data is determined.

[0147] Exemplarily, as Figure 8 shown, taking the correct text actually corresponding to the user input voice data "Send me the file you downloaded. I need to correct it." as an example for illustration. The dedicated pronunciation association library in the speech recognition system includes the pronunciation association mapping table shown in Table 1. The weighted finite state transducer in the speech recognition system is updated and constructed based on the language model using the dedicated pronunciation association library containing Table 1. The terminal device receives the first voice data input by the user (such as the voice data), but the user has an accent resulting in inaccurate pronunciation. The terminal device uses the speech recognition system to recognize the first voice data and obtains the syllables corresponding to the first voice data as "bǎnǐxiàzǎi dēwēn jiàn fāgěiwǒwǒyào xiào zhèng yíxià". The speech recognition system uses the syllables and its own speech recognition model to recognize and obtain 2 candidate texts "Send me the file you downloaded. I need to make a minor adjustment." and "Send me the file you downloaded. I need to make a minor correction.". After obtaining the two candidate texts, error correction is performed on the two candidate texts according to the decoding method based on the dedicated pronunciation association library shown in Table 1 and the weighted finite state transducer, and "downloaded" is corrected to "downloaded" in the two candidate texts. Thus, two corrected candidate texts "Send me the file you downloaded. I need to make a minor adjustment." and "Send me the file you downloaded. I need to make a minor correction." are obtained, and the score of each error-corrected candidate text is recalculated according to the weighted finite state transducer. The "Send me the file you downloaded. I need to make a minor correction." with a higher score is determined as the first recognition result and displayed on the terminal interface.

[0148] It can be understood that in the embodiment of the present application, the speech recognition system performs the first recognition on the first voice data, but the recognition result may be incorrect or incomplete. Therefore, after the speech recognition system outputs N candidate texts in the first recognition, error correction, re-scoring, and re-ranking are performed on the N candidate texts through the decoding scheme based on the weighted finite state transducer, so as to improve the recognition accuracy and efficiency and provide a recognition result that more conforms to the actual situation.

[0149] In other embodiments of the present application, the speech recognition system also includes a universal pronunciation association library. The universal pronunciation association library is a database containing error correction information text and pronunciation, which is constructed based on the data in the special pronunciation association library fed back by all users. That is, the universal pronunciation association library is iteratively updated according to the special pronunciation association library. The universal pronunciation association library is a database for speech recognition error correction. The universal pronunciation association library acts on all users and is a database common to all users. The universal pronunciation association library is stored on the terminal device and / or on the cloud server.

[0150] It is understandable that the data in the dedicated pronunciation association library and the general pronunciation association library gradually increase with the use of users. When a user uses it for the first time, the dedicated pronunciation association library corresponding to the user may not contain any data, or contain some basic error correction data, etc.; as the user uses it, the data in the dedicated pronunciation association library continues to increase. Correspondingly, when the speech recognition system is used for the first time, the general pronunciation association library may not contain any data, or contain some basic error correction data, etc.; as different users use it, the data in the general pronunciation association library gradually increases.

[0151] In other embodiments of the present application, if the speech recognition system includes a general pronunciation association library and a special pronunciation association library, the terminal device recognizes the first speech data based on the speech recognition system to obtain N candidate texts, the terminal device corrects and reorders the N candidate texts according to a preset decoding method, the special pronunciation association library and the general pronunciation association library, determines a first recognition result, and the terminal device displays the first recognition result.

[0152] In some instances, when both the dedicated pronunciation association library and the general pronunciation association library contain a correction phrase in the candidate text, the dedicated pronunciation association library has a higher priority than the general pronunciation association library, and the candidate text is corrected and reordered based on the dedicated pronunciation association library.

[0153] In a possible implementation, the user can authorize the dedicated pronunciation association library generated by the terminal device to the cloud server of the speech recognition system. Subsequently, the cloud server can update the general pronunciation association library in the speech recognition system in the cloud server according to the data in the dedicated pronunciation association library uploaded by the authorized users, and synchronize the updated general pronunciation association library to the terminal device, so that the terminal device can perform speech recognition error correction based on the latest general pronunciation association library, thereby speeding up the recognition error correction efficiency.

[0154] In some examples, the terminal device obtains a universal pronunciation association library from a cloud server. When the terminal device performs speech recognition for the first time, it obtains a first recognition result based on the speech recognition system and the universal pronunciation association library. If the user performs an error correction operation on the first recognition result, the first recognition result is corrected to a second recognition result according to the error correction operation, and the dedicated pronunciation association library on the terminal side is updated according to the first speech data, the first recognition result, and the second recognition result. When the terminal device performs speech recognition again subsequently, the second speech data input by the user is recognized based on the speech recognition system, the universal pronunciation association library, and the dedicated pronunciation association library.

[0155] In some embodiments of the present application, after the cloud server obtains the data of the dedicated pronunciation association library uploaded by different terminal devices, if the amount of data of the dedicated pronunciation association library uploaded by the terminal device exceeds a preset threshold, the data in the different dedicated pronunciation association libraries are sorted and counted, and the universal pronunciation association library in the cloud server is updated based on the data.

[0156] In other embodiments of the present application, the cloud server periodically obtains data of the dedicated pronunciation association library uploaded by different terminal devices, and updates the general pronunciation association library in the cloud server according to the high-frequency pronunciation association data in the data. The embodiments of the present application do not limit the specific implementation method of updating the general pronunciation association library.

[0157] For example, Fig. 9 As shown, it is a schematic diagram of updating a universal pronunciation association library provided by an embodiment of the present application. Terminal device 1, terminal device 2 and terminal device 3 respectively obtain a speech recognition system containing a universal pronunciation association library from a cloud server, so as to perform speech recognition and error correction (that is, automatic error correction by the system) according to the speech recognition system, and obtain a first recognition result. Terminal device 1, terminal device 2 and terminal device 3 respectively obtain the user's error correction operation on the first recognition result (that is, manual error correction by the user), and construct corresponding special pronunciation association library 1, special pronunciation association library 2 and special pronunciation association library 3 according to the error correction operation and the first recognition result. Terminal device 1, terminal device 2 and terminal device 3 send the special pronunciation association library 1, special pronunciation association library 2 and special pronunciation association library 3 collected and generated by each of them to the cloud server through end-cloud collaboration. The cloud server updates its own universal pronunciation association library according to the special pronunciation association library 1, special pronunciation association library 2 and special pronunciation association library 3, and sends the updated universal pronunciation association library to terminal device 1, terminal device 2 and terminal device 3.

[0158] It can be understood that by using the end-cloud collaborative mechanism to count the error correction information in the dedicated pronunciation association library of a large number of users, and updating the general pronunciation association library on the cloud side based on the statistical results, it is possible to fully collect and count the common and easily erroneous pronunciations of the public, and to improve the general speech recognition effect of the speech recognition system.

[0159] In this way, the terminal device generates a unique pronunciation association library for each user based on the user's error correction feedback, so as to determine the user's unique pronunciation habits based on the dedicated pronunciation association library, so that the speech recognition system can adapt to users with different pronunciation habits. The speech recognition system can adaptively identify and personalize error correction based on the characteristics of each user. The speech recognition system automatically matches users without changing the user's pronunciation habits, which can improve the user experience. Moreover, the speech recognition system based on matching the user's pronunciation habits can more accurately and effectively recognize the user's voice and improve recognition efficiency.

[0160] It can be understood that the above steps S601-S602 are the recognition and error correction of the first data input by the user by the voice recognition system in the terminal device, and the display of the first recognition result. After the terminal device displays the first recognition result, the user can further determine whether the recognition result is wrong. If the user determines that the first recognition result is wrong and needs to correct the wrong text, the following steps S603-S605 can be continued. The terminal device responds to the user's error correction operation and corrects the first recognition result to the second recognition result. The terminal device updates the dedicated pronunciation association library in the voice recognition system according to the first voice data, the first recognition result, and the second recognition result. So that the subsequent voice recognition system can recognize and correct the second voice data input by the user next time according to the updated dedicated pronunciation association library. If the second voice data input next time includes the error correction information in the dedicated pronunciation association library, the voice recognition and error correction are automatically performed according to the information in the dedicated pronunciation association library, without the need for the user to manually edit and correct the error again. That is, "remember and modify at the same time". The dedicated pronunciation association library in the speech recognition system is updated in time according to the error correction information fed back by users. When the same error occurs again, the error correction information can take effect immediately, reducing the frequency of users' secondary processing of speech recognition results and optimizing the speech recognition system.

[0161] S603: The terminal device corrects the first recognition result into a second recognition result in response to the user's error correction operation.

[0162] In an embodiment of the present application, the error correction operation may be a preset operation such as editing or modifying the first recognition result performed by the user.

[0163] The above preset operations include preset quick operations or voice commands. Optionally, the preset operation can be certain quick operations of the user, such as quick operations in the form of gestures or key combinations. Specifically, when the terminal device is a computer, the user can use the mouse to drag the cursor to select the single word to be corrected in the first recognition result, and input the correct text through the keyboard to replace the single word to be corrected. When the terminal device is a terminal device such as a mobile phone or a tablet computer with a touch display screen, the user can use a stylus or a finger to select the single word to be corrected in the first recognition result by means of gestures (such as double-clicking or long-pressing on a certain word) or dragging the cursor, and input the correct text by handwriting or keyboard to replace the single word to be corrected. In this way, the correction operation is completed.

[0164] The above preset operation can also be a voice command of the user, such as下达 commands to the terminal device to select and modify a certain word through a voice assistant. The embodiments of the present application do not limit the specific implementation form of the preset operation.

[0165] In the embodiments of the present application, the second recognition result is the text data after the user corrects the first recognition result.

[0166] In some embodiments of the present application, the terminal device accepts the correction operation of the user, and in response to the correction operation, corrects the first recognition result to the second recognition result.

[0167] Exemplarily, based on the above Figure 8 example. As Fig.10 shown in (a) of, the text display area 1001 displays "Send me the file you downloaded. I want to correct it a bit". When the user reviews the first recognition result "Send me the file you downloaded. I want to correct it a bit", and finds that the "小" in the first recognition result is incorrect, the user moves the cursor before "小" and drags the cursor backward to select the word "小". Fig.10 The "小" selected by the user is shown in a square box in (a) of. The mobile phone receives the operation of the user and determines that the "小" selected by the user is the single word to be corrected this time. Subsequently, as Fig.10 shown in (b) of, the user handwrites a character "校" in the half-screen handwriting keyboard in the text editing area 1002. Correspondingly, the "校" in the text editing area 1002 replaces the "小". In this way, the terminal device corrects "Send me the file you downloaded. I want to correct it a bit" to "Send me the file you downloaded. I want to proofread it" according to the correction operation of the user. That is, the second recognition result is "Send me the file you downloaded. I want to proofread it".

[0168] Optionally, the text editing area 1002 may also display only the content input by the user when the user inputs. If the user has finished inputting and the content input by the user is already displayed in the text display area 1001, the text editing area 1002 may be restored to a blank page to facilitate the user to input new content.

[0169] Optionally, if the terminal device does not receive a user's error correction operation within a preset time after receiving the user's error correction operation, it is determined that the user has completed the error correction of the first recognition result this time.

[0170] Optionally, the terminal device may also provide a function control. When the user triggers the function control, it is determined that the user has ended the error correction of the first recognition result this time. For example, as Fig.10 shown in (a) of, the terminal device provides a confirmation button 1003 in the text editing area 1002. If the user clicks the confirmation button 1003, it is considered that the user has ended the error correction of the first recognition result this time.

[0171] In some other embodiments of the present application, if the terminal device does not receive a user's error correction operation within a preset time, it is determined that the first recognition result recognized by the terminal device does not need to be corrected, and this first recognition result is the correct text corresponding to the first voice data.

[0172] S604. The terminal device associates the first voice data, the first recognition result, and the second recognition result to determine error correction information.

[0173] In the embodiments of the present application, the terminal device determines the error correction single words in the error correction phrase based on the first recognition result and the second recognition result, obtains the incorrect pronunciation corresponding to the error correction single word in the first voice data, and the correct pronunciation corresponding to the error correction single word. The terminal device establishes a mapping relationship among the error correction phrase, the error correction single word, the incorrect pronunciation, and the correct pronunciation to generate error correction information.

[0174] In a possible implementation manner, the terminal device determines the error correction phrase and / or the error correction single words in the error correction phrase by comparing the first recognition result and the second recognition result.

[0175] In another possible implementation manner, the terminal device determines the error correction single word according to the user's error correction operation, and determines the error correction phrase containing the error correction single word through word segmentation processing.

[0176] Exemplarily, based on the above Fig.10 example, when the user performs an error correction operation, change the "xiao" in the first recognition result to "jiao". In this way, it is determined that the error correction single words in the first recognition result and the second recognition result are "xiao" and "jiao" respectively, and then the error correction phrases are determined as "xiaozheng" and "jiaozheng" through word segmentation processing.

[0177] In a possible implementation, the terminal device determines the incorrect pronunciation corresponding to the error correction word according to the first recognition result and the first voice data.

[0178] Specifically, the terminal device performs forced alignment on the first recognition result and the first voice data to correspond each phrase or phoneme in the first recognition result to the specific time of the audio. The terminal device determines the incorrect pronunciation corresponding to the error correction word in the first recognition result according to the alignment result.

[0179] Exemplarily, based on the above Fig.10 example, the terminal device generates the pronunciation corresponding to the error correction word through the forced alignment method of the hidden Markov model or the end-to-end grapheme-to-phoneme (G2P) method, and determines that the incorrect pronunciation of the error correction word "小" in the first recognition result is "xiào".

[0180] It can be understood that the traditional speech recognition scheme based on the hidden Markov model performs pronunciation prediction during one-pass decoding, and the speech data, as well as the pronunciation and text corresponding to the speech data, can be obtained after one-pass decoding. While the end-to-end speech recognition scheme recognizes the text corresponding to the speech data according to the neural network model during one-pass decoding. Therefore, the end-to-end speech recognition scheme needs to obtain the pronunciation corresponding to the text through methods such as G2P during two-pass decoding.

[0181] In a possible implementation, the terminal device generates the pronunciation sequence corresponding to the error correction word in the error correction phrase through the grapheme-to-phoneme (G2P) algorithm to obtain the correct pronunciation of the error correction word.

[0182] Among them, the G2P algorithm is usually based on linguistic knowledge and statistical modeling methods. Specifically, the G2P algorithm can use linguistic rules, auxiliary dictionaries, and training data to infer pronunciations. The training data of the algorithm includes a large number of phrases and their correct pronunciation annotations. By analyzing these data, the model can learn the potential relationship between the letter or character sequence and the pronunciation to determine the pronunciation sequence of the character.

[0183] Exemplarily, based on the above Fig.10 example, the terminal device determines that the correct pronunciation of the error correction word "校" in the second recognition result in the error correction phrase "校正" is "jiào" through the G2P algorithm.

[0184] In a possible implementation, the terminal device combines at least one piece of information among the error correction phrase, the error correction word, the incorrect pronunciation, and the correct pronunciation into error correction information. The embodiments of the present application do not limit the data types included in the error correction information.

[0185] Exemplarily, an association relationship is established between the corrected single-character error correction, error-corrected phrase, incorrect pronunciation, and correct pronunciation to generate error correction information. The specific form can be: correction - correct - jiào - xiào.

[0186] It can be understood that in the above example, the terminal device determines the error correction information based on the first voice data, the first recognition result, and the second recognition result. In actual use, the terminal device can establish a connection based on at least one of the first voice data, the first recognition result, and the second recognition result to determine the error correction information.

[0187] S605. The terminal device updates the dedicated pronunciation connection library according to the error correction information.

[0188] In the embodiment of the present application, the terminal device adds the error correction information to the dedicated pronunciation connection library to update the dedicated pronunciation connection library.

[0189] Exemplarily, the terminal device adds the error correction information to the pronunciation connection mapping table in the dedicated pronunciation connection library shown in Table 1 to obtain the updated pronunciation connection mapping table in the dedicated pronunciation connection library shown in Table 2 (only part of the information is shown in Table 2).

[0190] Table 2

[0191] Error correction phrase Error correction word Correct pronunciation Wrong pronunciation Hot Blazing chì zhì download load zài zǎi Mysophobia addiction pǐ pì …… …… …… …… Correction school jiào xiaoào

[0192] As shown in Table 2, the error correction information determined according to the user's error correction operation in the above steps S603 - S604 is newly added in Table 2: correction - correct - jiào - xiào. In this way, when the terminal device performs the next voice recognition, it can perform voice recognition and error correction according to the data in the updated dedicated pronunciation connection library shown in Table 2.

[0193] It can be understood that the above steps S601-S605 are specific implementations of the terminal device obtaining the correct text corresponding to the first voice data. First, the first voice data is automatically identified and corrected according to the voice recognition system to obtain the first recognition result. If the user determines that there are still errors in the first recognition result and performs an error correction operation, the terminal device responds to the user's error correction operation and corrects the first recognition result to the second recognition result (that is, the correct text corresponding to the first voice data). Further, the terminal device obtains the feedback information before and after the user's correction (including the first voice data, the first recognition result and the second recognition result) to determine the error correction information generated by this error correction operation, and updates the special pronunciation association library in the voice recognition system in real time according to the error correction information. When the user inputs the second voice data next time, the following steps S606-S607 can be continued. So that when performing subsequent voice recognition, it is timely judged whether the second voice data input next time includes the error correction information in the updated special pronunciation association library, and if it is included, it will take effect immediately, reducing the user's secondary processing frequency of the voice recognition result and optimizing the voice recognition system.

[0194] It can be understood that, in actual applications, the terminal device can update the dedicated pronunciation association library according to at least one of the first voice data, the first recognition result, and the second recognition result.

[0195] S606: The terminal device obtains second voice data, where the second voice data includes the erroneous pronunciation in the above error correction information.

[0196] In an embodiment of the present application, the second voice data is voice data acquired by the terminal device after acquiring the first voice data.

[0197] The specific implementation method for the terminal device to obtain the second voice data in the embodiment of the present application can be found in S601 above, which will not be repeated here.

[0198] S607: The terminal device recognizes the second voice data based on a voice recognition system and outputs a third recognition result, wherein the voice recognition system includes an updated dedicated pronunciation association library.

[0199] In the embodiment of the present application, the third recognition result is the recognition text output by the terminal device after the second voice data is recognized by the voice recognition system.

[0200] It is understandable that the updated dedicated pronunciation association library includes the above-mentioned error correction information. Therefore, the voice recognition system in the terminal device can automatically recognize and correct the error correction information without manual correction.

[0201] It can be understood that if the user determines that the third recognition result is incorrect (the error is not included in the updated dedicated pronunciation association library), the terminal device can obtain the error correction information feedback by the user according to the above steps S603 - S605, and update the pronunciation association library again, which will not be elaborated here.

[0202] In the embodiment of the present application, when the terminal device displays the third recognition result, it prompts the user with the correct pronunciation corresponding to the error-corrected single character, so as to correct the user's pronunciation habit and improve the accuracy of speech recognition.

[0203] In some examples, when displaying the third recognition result, the terminal device can mark the correct pronunciation near the error-corrected single character.

[0204] In other examples, when displaying the third recognition result, the terminal device can highlight the error-corrected single character, such as by changing the color, increasing the font size, making the font bold, underlining the error-corrected single character, etc. And display the pronunciation mark of the correct pronunciation near the error-corrected single character, such as icons like a speaker or a player. After the user touches the pronunciation mark, the correct pronunciation of the error-corrected single character is played. The embodiment of the present application does not limit the specific implementation manner of the user device to prompt the user with the correct pronunciation corresponding to the error-corrected single character.

[0205] Exemplarily, as Fig.11 shown, it is a schematic diagram of the scenario where the terminal device performs speech recognition on the conference recording file. The user plays the conference recording file, and the terminal device distinguishes different roles according to the sound waves in the conference recording file. The specific implementation manner for the terminal device to distinguish different roles in the conference recording file refers to the prior art and will not be elaborated here.

[0206] The terminal device recognizes the first speech data of Role 1 and obtains the first recognition result "Role 1: Today we have a meeting to discuss the small positive method of remote sensing images". As Fig.11 shown in (a) below, after the terminal device displays the first recognition result, it receives the operation of the user to correct "small positive" to "correction". In response to this error correction operation, the terminal device displays the second recognition result "Today we have a meeting to discuss the correction method of remote sensing images." (as Fig.11 shown in (b) below). And, the terminal device determines the error correction information generated by this error correction operation according to this error correction operation and the feedback information before and after the user's error correction (including the first speech data, the first recognition result, and the second recognition result), and updates the dedicated pronunciation association library in the speech recognition system in real time according to this error correction information. Fig.11). The terminal device then continues to voice recognize the conference recording file in chronological order. For example, the terminal device recognizes the second voice data of character 2 to obtain the second recognition result "Who will give a relevant introduction?" At this time, the user performs an error correction operation on the second recognition result, and the terminal device recognizes the third voice data of character 1, and the third voice data includes the wrong pronunciation in the above error correction information. The voice recognition system in the terminal device can automatically recognize and correct the third voice data based on the updated dedicated pronunciation association library to obtain the third recognition result "Correction of remote sensing images includes radiation correction and geometric correction." (such as Fig.11 As shown in (c) in FIG. 1 ). In this way, when the terminal device detects the same error correction information again, it can correct the error in real time according to the dedicated pronunciation association library in the updated speech recognition system, and the error takes effect in time, thereby speeding up the processing efficiency of speech recognition error correction.

[0207] In the above-mentioned embodiments of the present application, the speech recognition error correction method is applied to a speech recognition error correction system that only includes the terminal device itself as an example. The speech recognition error correction method provided in the embodiments of the present application can also be applied to a speech recognition error correction system that includes a terminal device and a server.

[0208] The following is an example of an embodiment of the present application being applied to a speech recognition error correction system including a terminal device and a server.

[0209] like Fig.12 As shown, a speech recognition error correction method provided in an embodiment of the present application includes the following steps:

[0210] S1200. The terminal device obtains first voice data.

[0211] The specific implementation method for the terminal device to obtain the first voice data in the embodiment of the present application can be found in S601 above, which will not be repeated here.

[0212] S1201. The terminal device sends first voice data to the server.

[0213] In a possible implementation, the terminal device generates a first voice recognition request according to the first voice data, and sends the first voice recognition request to the server, where the first voice recognition request includes the first voice data.

[0214] Optionally, the terminal device may send the first voice recognition request to the server based on a network protocol such as the Hypertext Transfer Protocol (HTTP) or a security-oriented HTTP channel (HTTPS).

[0215] S1202: The server recognizes the first voice data based on a voice recognition system and outputs a first recognition result. The voice recognition system includes a dedicated pronunciation association library.

[0216] In a possible implementation, the server obtains a dedicated pronunciation association library corresponding to each terminal device connected to the server. When the server receives the first voice data sent by a terminal device, the voice recognition system performs voice recognition based on the dedicated pronunciation association library corresponding to the terminal device and outputs a first recognition result.

[0217] The specific implementation method of the server in the embodiment of the present application recognizing the first voice data and outputting the first recognition result based on the voice recognition system can be found in S602 above, which will not be repeated here.

[0218] S1203: The server returns the first recognition result to the terminal device.

[0219] S1204. The terminal device displays the first recognition result.

[0220] S1205. The terminal device corrects the first recognition result into a second recognition result in response to the user's error correction operation.

[0221] In the embodiment of the present application, the specific implementation method of the terminal device correcting the first recognition result to the second recognition result in response to the user's error correction operation can be found in S603 above, which will not be repeated here.

[0222] S1206. The terminal device links the first voice data, the first recognition result, and the second recognition result, determines error correction information, and sends the error correction information to the server.

[0223] In the embodiment of the present application, the terminal device establishes a connection between the first voice data, the first recognition result and the second recognition result, and the specific implementation method for determining the error correction information can be found in S604 above, which will not be repeated here.

[0224] S1207. The server updates the dedicated pronunciation association library according to the error correction information.

[0225] The specific implementation method of the server updating the dedicated pronunciation association library according to the error correction information in the embodiment of the present application can be found in S605 above, which will not be repeated here.

[0226] S1208. The terminal device obtains second voice data, where the second voice data includes the erroneous pronunciation in the above error correction information.

[0227] In the embodiment of the present application, the terminal device obtains the second voice data, and the specific implementation method of the second voice data including the erroneous pronunciation in the above-mentioned error correction information can be referred to in step S606, which will not be repeated here.

[0228] S1209. The terminal device sends second voice data to the server.

[0229] S1210. The server recognizes the second voice data based on a voice recognition system and outputs a third recognition result. The voice recognition system includes an updated dedicated pronunciation association library.

[0230] In this embodiment, the server recognizes the second voice data based on the voice recognition system and outputs a third recognition result. The specific implementation method of the voice recognition system including the updated dedicated pronunciation association library can be found in S607 above and will not be repeated here.

[0231] The speech recognition error correction method provided in the embodiment of the present application updates the dedicated pronunciation association library in the speech recognition system in real time according to the error correction information fed back by the user during the speech recognition process, so that after the terminal device detects the same error correction information again, it can perform real-time error correction according to the dedicated pronunciation association library in the updated speech recognition system, and the error takes effect in time, thereby speeding up the processing efficiency of speech recognition error correction, reducing the frequency of secondary processing of speech recognition results by users, optimizing the speech recognition system, improving user experience, and making it more practical.

[0232] For example, Fig.13 As shown, another speech recognition error correction method provided in an embodiment of the present application includes the following steps S1301-S1305.

[0233] S1301. The terminal device obtains first voice data.

[0234] The specific implementation method for the terminal device to obtain the first voice data in the embodiment of the present application can be found in S601 above and will not be repeated here.

[0235] S1302. The terminal device recognizes the first voice data based on a voice recognition system and outputs a first recognition result; the voice recognition system includes a dedicated pronunciation association library, which is used to correct the candidate text corresponding to the first voice data to output the first recognition result.

[0236] In the embodiment of the present application, the dedicated pronunciation association library includes error correction phrases, error correction words, incorrect pronunciations of the error correction words, and correct pronunciations of the error correction words.

[0237] In some embodiments of the present application, the terminal device recognizes the first voice data based on a voice recognition system and outputs a first recognition result, including: the terminal device recognizes the first voice data based on the voice recognition system to obtain N candidate texts; wherein N is an integer greater than 0; the terminal device corrects and reorders the N candidate texts according to a dedicated pronunciation association library and a preset decoding method to determine the first recognition result.

[0238] In some other embodiments of the present application, the speech recognition system also includes a universal pronunciation association library corresponding to multiple users, and the universal pronunciation association library and the dedicated pronunciation association library are used to correct the candidate text corresponding to the first speech data to output a first recognition result. Then the terminal device outputs the first recognition result based on the speech recognition system recognizing the first speech data, including: the terminal device obtains N candidate texts based on the speech recognition system recognizing the first speech data; wherein N is an integer greater than 0; the terminal device corrects and reorders the N candidate texts according to the dedicated pronunciation association library, the universal pronunciation association library and the preset decoding method to determine the first recognition result.

[0239] In other embodiments of the present application, if the dedicated pronunciation association library and the general pronunciation association library contain the same error correction phrase, the priority of the dedicated pronunciation association library is higher than that of the general pronunciation association library for the same error correction phrase.

[0240] It is understandable that the user's pronunciation is mainly used as the basis for terminal device recognition. The dedicated pronunciation association library corresponds to the user using the terminal device, while the general pronunciation association library corresponds to multiple users. When the dedicated pronunciation association library and the general pronunciation association library contain the same error correction phrase, the data in the dedicated pronunciation association library shall be used as the basis for recognition and error correction.

[0241] In the embodiment of the present application, the specific implementation method in which the terminal device recognizes the first voice data based on the voice recognition system and outputs the first recognition result can be found in S602 above and will not be repeated here.

[0242] S1303: The terminal device determines a target recognition result according to whether the user performs an error correction operation on the first recognition result.

[0243] It is understandable that the first recognition result is the text data obtained after the voice recognition system in the terminal device recognizes and corrects the first voice data. However, if the user's pronunciation is inaccurate, the first recognition result obtained by the voice recognition system may not be the correct text data corresponding to the user's first voice data. The user is also required to manually correct the first recognition result in order to obtain the correct recognition result (i.e., the target recognition result).

[0244] S1304: If the user does not perform an error correction operation on the first recognition result, the terminal device determines that the target recognition result is the first recognition result.

[0245] In the embodiment of the present application, if the user does not perform an error correction operation on the first recognition result, it indicates that the first recognition result obtained by the speech recognition system is correct, and the terminal device determines that the target recognition result is the first recognition result.

[0246] S1305. If the user performs an error correction operation on the first recognition result, the terminal device corrects the first recognition result to the second recognition result, and determines that the target recognition result is the second recognition result; the terminal device updates the dedicated pronunciation association library according to at least one of the first voice data, the first recognition result and the second recognition result.

[0247] In the embodiment of the present application, if the user performs an error correction operation on the first recognition result, it indicates that the first recognition result obtained by the speech recognition system is incorrect, and the terminal device corrects the first recognition result to the second recognition result in response to the user's error correction operation. The terminal device determines that the target recognition result is the second recognition result.

[0248] In an embodiment of the present application, the terminal device updates the dedicated pronunciation association library based on at least one of the first voice data, the first recognition result, and the second recognition result, including: the terminal device associates at least one of the first voice data, the first recognition result, and the second recognition result to determine error correction information; the terminal device updates the dedicated pronunciation association library based on the error correction information.

[0249] In an embodiment of the present application, the terminal device links the first voice data, the first recognition result and the second recognition result to determine the error correction information, including: the terminal device determines the error correction phrase and / or the error correction word according to the first recognition result and the second recognition result; the terminal device determines the incorrect pronunciation of the error correction word according to the first voice data; the terminal device obtains the correct pronunciation of the error correction word; the terminal device establishes a mapping relationship between the error correction phrase, the error correction word, the incorrect pronunciation of the error correction word and the correct pronunciation of the error correction word to generate error correction information.

[0250] In an embodiment of the present application, after the terminal device corrects the first recognition result to the second recognition result and determines that the target recognition result is the second recognition result, the method also includes: the terminal device obtains second voice data, the second voice data includes a first incorrect pronunciation in the first voice data, the first incorrect pronunciation corresponds to a first error correction word, and the dedicated pronunciation association library updated based on the first voice data includes the correspondence between the first incorrect pronunciation and the first error correction word; the terminal device recognizes the second voice data based on the voice recognition system and outputs a third recognition result, and the third recognition result is the correct voice recognition result corresponding to the second voice recognition data.

[0251] In an embodiment of the present application, the terminal device performs an error correction operation, corrects the first recognition result to the second recognition result, and updates the dedicated pronunciation association library. The specific implementation method can be found in S603-S605 above; the terminal device obtains the second voice data, recognizes the second voice data and outputs the third recognition result. The specific implementation method can be found in S606-S607 above, which will not be repeated here.

[0252] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. It is understandable that, in order to realize the above functions, the electronic device includes a hardware structure and / or software module corresponding to the execution of each function. In combination with the units and algorithm steps of each example described in the embodiment disclosed in this application, the embodiment of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer-driven hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiment of the present application.

[0253] The present application is an embodiment that can divide the functional modules of the electronic device according to the above method example. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of units in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0254] like Fig.14 1400 is a schematic diagram of the structure of another terminal device provided in an embodiment of the present application. The terminal device 1400 can be used to implement the methods described in the above method embodiments. Exemplarily, the terminal device 1400 may specifically include: a processing module 1401, an acquisition module 1402, and a display module 1403.

[0255] The processing module 1401 is used to execute the support terminal device 1400 to execute Figures 6 to 13 The processing function of any one of the items.

[0256] The acquisition module 1402 is used to execute the support terminal device 1400 to execute Figures 6 to 13 The acquisition function of any item in. For example, obtaining the currently displayed image, etc.

[0257] The display module 1403 may be used to display a screen according to the display driving data, etc. And / or, the display module 1403 may also be used to support the terminal device 1400 to perform other display operations performed by the terminal device in the embodiment of the present application.

[0258] Optional, Fig.14 The terminal device 1400 shown may also include a communication module ( Fig.14 (not shown), the communication module is used to support the terminal device 1400 to execute the steps of communication between the terminal device and other devices in the embodiment of the present application.

[0259] Optional, Fig.14 The terminal device 1400 shown may also include a storage module ( Fig.14 (not shown in the figure), the storage module stores a program or instruction. When the processing module 1401 executes the program or instruction, Fig.14 The terminal device 1400 shown can execute the method shown in the above method embodiment.

[0260] Fig.14 The technical effects of the terminal device 1400 shown can refer to the technical effects of the method described in the above method embodiment, and will not be repeated here. Fig.14 The processing module 1401 involved in the terminal device 1400 shown can be implemented by a processor or a processor-related circuit component, which can be a processor or a processing module. The communication module can be implemented by a transceiver or a transceiver-related circuit component, which can be a transceiver or a transceiver module. The display module 1403 can be implemented by display screen-related components.

[0261] The present application also provides a chip system, such as Fig.15 As shown, the chip system 1500 includes at least one processor 1501 and at least one interface circuit 1502. As an example, when the chip system 1500 includes one processor and one interface circuit, the one processor may be Fig.15 The processor 1501 shown in the solid line frame (or the processor 1501 shown in the dotted line frame) may be Fig.15 The interface circuit 1502 shown in the solid line frame (or the interface circuit 1502 shown in the dotted line frame). When the chip system 1500 includes two processors and two interface circuits, the two processors include Fig.15 The processor 1501 shown in the solid line frame and the processor 1501 shown in the dotted line frame, the two interface circuits include Fig.15 The interface circuit 1502 shown in the solid line frame and the interface circuit 1502 shown in the dotted line frame are not limited to this.

[0262] The processor 1501 and the interface circuit 1502 can be interconnected via a line. For example, the interface circuit 1502 can be used to receive a signal. For another example, the interface circuit 1502 can be used to send a signal to another device (such as the processor 1501). Exemplarily, the interface circuit 1502 can read an instruction stored in a memory and send the instruction to the processor 1501. When the instruction is executed by the processor 1501, the various steps in the above embodiment can be executed. Of course, the chip system can also include other discrete devices, which is not specifically limited in the embodiments of the present application.

[0263] Optionally, the processor in the chip system may be one or more. The processor may be implemented by hardware or by software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented by software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

[0264] Optionally, the chip system may also include a memory ( Fig.15 As shown in the figure, the memory may be one or more, and the memory may be integrated with the processor or may be separately arranged from the processor, which is not limited in the present application. Exemplarily, the memory may be a non-transient processor, such as a read-only memory (ROM), which may be integrated with the processor on the same chip or may be arranged on different chips, and the present application does not specifically limit the type of memory and the arrangement of the memory and the processor.

[0265] Exemplarily, the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD) or other integrated chips.

[0266] It should be understood that each step in the above method embodiment can be completed by an integrated logic circuit of hardware in a processor or by instructions in the form of software. The method steps disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.

[0267] An embodiment of the present application also provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on a terminal device, the terminal device executes the method described in the above method embodiment.

[0268] Computer-readable storage media include, but are not limited to, any of the following: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and other media that can store program codes.

[0269] An embodiment of the present application provides a computer program product, which includes: a computer program or instructions, when the computer program or instructions are executed on a computer, the computer executes the method described in the above method embodiment.

[0270] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory so that the device executes the methods in the above-mentioned method embodiments.

[0271] Among them, the terminal device, computer storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0272] The steps of the method or algorithm described in conjunction with the disclosure of the embodiments of the present application can be implemented in a hardware manner, or can be implemented by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in a random access memory (RAM), a flash memory, a read-only memory, an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a register, a hard disk, a mobile hard disk, a compact disc read-only memory (CD-ROM) or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC).

[0273] Through the description of the above implementation methods, those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed; that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0274] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The various embodiments can be combined with each other or referenced to each other without conflict. The device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0275] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0276] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0277] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0278] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A speech recognition error correction method, characterized in that: The method comprises: The terminal device obtains the first voice data; The terminal device recognizes the first voice data based on a voice recognition system and outputs a first recognition result; the voice recognition system includes a dedicated pronunciation association library, and the dedicated pronunciation association library is used to correct the candidate text corresponding to the first voice data to output the first recognition result; The terminal device determines a target recognition result according to whether the user performs an error correction operation on the first recognition result; If the user does not perform an error correction operation on the first recognition result, the terminal device determines that the target recognition result is the first recognition result; If the user performs an error correction operation on the first recognition result, the terminal device corrects the first recognition result to a second recognition result, and determines that the target recognition result is the second recognition result; the terminal device updates the dedicated pronunciation association library based on at least one of the first voice data, the first recognition result and the second recognition result.

2. The method according to claim 1, characterized in that The dedicated pronunciation association library includes error correction phrases, error correction words, incorrect pronunciations of the error correction words, and correct pronunciations of the error correction words.

3. The method according to claim 1 or 2, characterized in that: The terminal device updates the dedicated pronunciation association library according to at least one of the first voice data, the first recognition result, and the second recognition result, including: The terminal device associates at least one of the first voice data, the first recognition result, and the second recognition result to determine error correction information; The terminal device updates the dedicated pronunciation association library according to the error correction information.

4. The method according to claim 3, characterized in that The terminal device associates the first voice data, the first recognition result, and the second recognition result to determine error correction information, including: The terminal device determines the error correction phrase and / or the error correction word according to the first recognition result and the second recognition result; The terminal device determines the incorrect pronunciation of the error correction word according to the first voice data; The terminal device obtains the correct pronunciation of the error correction word; The terminal device establishes a mapping relationship between the error correction phrase, the error correction word, the incorrect pronunciation of the error correction word, and the correct pronunciation of the error correction word to generate the error correction information.

5. The method according to any one of claims 1 to 4, characterized in that: The terminal device recognizes the first voice data based on the voice recognition system and outputs a first recognition result, including: The terminal device recognizes the first voice data based on the voice recognition system to obtain N candidate texts; wherein N is an integer greater than 0; The terminal device corrects and reorders the N candidate texts according to the dedicated pronunciation association library and the preset decoding method to determine the first recognition result.

6. The method according to any one of claims 1 to 5, characterized in that: After the terminal device corrects the first recognition result to a second recognition result and determines that the target recognition result is the second recognition result, the method further includes: The terminal device acquires second voice data, the second voice data includes a first erroneous pronunciation in the first voice data, the first erroneous pronunciation corresponds to a first error correction word, and the dedicated pronunciation association library updated based on the first voice data includes a correspondence between the first erroneous pronunciation and the first error correction word; The terminal device recognizes the second voice data based on a voice recognition system and outputs a third recognition result, where the third recognition result is a correct voice recognition result corresponding to the second voice recognition data.

7. The method according to claim 1, characterized in that The speech recognition system further includes a common pronunciation association library corresponding to a plurality of users; the common pronunciation association library and the special pronunciation association library are used to correct the candidate text corresponding to the first speech data to output the first recognition result; The terminal device recognizes the first voice data based on the voice recognition system and outputs a first recognition result, including: The terminal device recognizes the first voice data based on the voice recognition system to obtain N candidate texts; wherein N is an integer greater than 0; The terminal device corrects and reorders the N candidate texts according to the dedicated pronunciation association library, the general pronunciation association library and a preset decoding method to determine the first recognition result.

8. The method according to claim 7, characterized in that The method further comprises: If the dedicated pronunciation association library and the general pronunciation association library contain the same error correction phrase, the dedicated pronunciation association library has a higher priority than the general pronunciation association library for the same error correction phrase.

9. A terminal device, characterized in that: include: one or more processors; Memory; One or more computer programs are stored in the memory, and the one or more computer programs include instructions. When the instructions are executed by the terminal device, the terminal device executes the method according to any one of claims 1 to 8.

10. A chip system, characterized in that: It includes at least one processor and at least one interface circuit, wherein the at least one interface circuit is used to perform transceiver functions and send instructions to the at least one processor, and the at least one processor executes the instructions, and the at least one processor executes the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a computer program or instructions, and when the computer program or instructions are run on a computer, the computer is caused to execute the method according to any one of claims 1 to 8.

12. A computer program product, characterized in that The computer program product comprises: a computer program or instructions, and when the computer program or instructions are run on a computer, the computer is caused to execute the method according to any one of claims 1 to 8.

Images