Speech Recognition Method and Device

Abstract

The present invention discloses a voice recognition method and apparatus. The method includes: receiving voice data of a speaker and obtaining text materials related to the voice data; extracting keywords from the text materials; performing voice recognition on the voice data using a decoding network, adjusting scores of decoding paths in the decoding network according to the keywords, and obtaining a voice recognition text based on scores of the respective decoding paths. By using the solution of the present invention, the accuracy of voice recognition results can be greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition, and particularly to a speech recognition method and device. Background Art

[0002] Currently, with the development of artificial intelligence technology, machine simultaneous interpretation has emerged. The greatest advantage of machine simultaneous interpretation is that the translation rate will not decrease due to fatigue. Machine simultaneous interpretation is limited by speech recognition technology and machine translation technology, and the quality of machine translation depends on the quality of the speech recognition text. Therefore, in order to improve the accuracy of machine simultaneous interpretation, a high-quality speech recognition system is required. In some machine simultaneous interpretation application scenarios, problems often occur in the recognition and translation of entity words or professional nouns such as personal names, place names, and product names, and these content words often play an important role in the on-site effect.

[0003] In the prior art, speech recognition models usually adopt general acoustic models and language models, and preferentially recognize common words and words with a higher occurrence probability in the corpus. For some specific application scenarios, in order to improve the recognition accuracy, it is often necessary to customize a model for the specific application scenario. The training of the customized model requires a large amount of relevant corpus of the application scenario in advance. However, for some scenarios such as meetings, since the specific content of the speeches of the participants cannot be obtained in advance, customization cannot be carried out. Summary of the Invention

[0004] Embodiments of the present invention provide a speech recognition method and device to improve the accuracy of speech recognition results.

[0005] For this purpose, the present invention provides the following technical solutions:

[0006] A speech recognition method, the method comprising:

[0007] Receiving speech data of a speaker and obtaining text materials related to the speech data;

[0008] Extracting keywords from the text materials;

[0009] Performing speech recognition on the speech data by using a decoding network, adjusting the scores of the decoding paths in the decoding network according to the keywords, and obtaining a speech recognition text according to the scores of the respective decoding paths.

[0010] Optionally, the text materials related to the speech data include any one or more of the following: promotional materials related to the speech site, speech materials displayed at the speech site.

[0011] Optionally, the obtaining of the text materials related to the speech data includes:

[0012] Obtaining pictures related to the speech data through an image device;

[0013] The text in the picture is recognized by OCR to obtain text materials related to the speech data.

[0014] Optionally, the keyword includes any one or more of the following: entity words, proper nouns.

[0015] Optionally, the method further includes:

[0016] Present the speech recognition text;

[0017] Receive corrections to the recognition errors in the speech recognition text;

[0018] Adjust the scores of the decoding paths in the decoding network according to the corrections.

[0019] A speech recognition device, the device includes:

[0020] A speech receiving module, configured to receive the speech data of the speaker;

[0021] A text information acquisition module, configured to acquire text materials related to the speech data;

[0022] A keyword extraction module, configured to extract keywords from the text materials;

[0023] A speech recognition module, configured to perform speech recognition on the speech data by using a decoding network, adjust the scores of the decoding paths in the decoding network according to the keywords, and obtain a speech recognition text according to the scores of each decoding path.

[0024] Optionally, the text materials related to the speech data include any one or more of the following: promotional materials related to the speech site, speech materials displayed at the speech site.

[0025] Optionally, the text information acquisition module includes:

[0026] A picture acquisition unit, configured to acquire a picture related to the speech data;

[0027] A text recognition unit, configured to recognize the text in the picture by OCR to obtain text materials related to the speech data.

[0028] Optionally, the keyword includes any one or more of the following: entity words, proper nouns.

[0029] Optionally, the device further includes:

[0030] A presentation module, configured to present the speech recognition text;

[0031] A correction module, configured to provide a correction interface for receiving corrections to recognition errors in the speech recognition text;

[0032] The speech recognition module is further configured to adjust the scores of the decoding paths in the decoding network according to the correction.

[0033] An electronic device includes: one or more processors and a memory;

[0034] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the method described above.

[0035] A readable storage medium has instructions stored thereon, and the instructions are executed to implement the method described above.

[0036] The speech recognition method and apparatus provided by the embodiments of the present invention, during the speech recognition process, obtain text materials related to the speech data to be recognized, extract keywords therefrom, adjust the scores of the decoding paths in the decoding network according to the keywords, and obtain the speech recognition text according to the scores of each decoding path, thereby effectively improving the speech recognition effect in some application scenarios where the model cannot be pre-customized. Especially for entity words such as personal names, place names, product names, etc. contained in the speech data, as well as some professional terms, the recognition accuracy of these words can be greatly improved.

[0037] Furthermore, real-time corrections to the speech recognition text can be received, so as to adjust the scores of the decoding paths according to the corrections, and further improve the recognition effect of subsequent speech data. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings.

[0039] Figure 1 is a flowchart of a speech recognition method according to an embodiment of the present invention;

[0040] Figure 2 is another flowchart of a speech recognition method according to an embodiment of the present invention;

[0041] Figure 3 is a structural block diagram of a speech recognition apparatus according to an embodiment of the present invention;

[0042] Figure 4 is another structural block diagram of a speech recognition apparatus according to an embodiment of the present invention;

[0043] Figure 5 is a block diagram of an apparatus for an input method shown according to an exemplary embodiment;

[0044] Figure 6 is a schematic structural diagram of a server in an embodiment of the present invention. Detailed implementation manners

[0045] In order to enable those skilled in the art to better understand the solutions of the embodiments of the present invention, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and implementation manners.

[0046] In some application scenarios where models cannot be pre-customized, such as meeting scenarios, if a general language model is used for speech recognition, the recognition effect is not ideal. Especially for entity words such as personal names, place names, and product names contained in speech data, as well as some professional terms, recognition errors often occur. Due to speech recognition errors, subsequent machine translation errors will occur.

[0047] For example: Regarding Gauss, whether Gauss here refers to a personal name or the Gauss function in data, the translations are completely different;

[0048] For example: For the Xiaoai Speaker, when it appears for the first time, is it Xiaoai or Xiaoi;

[0049] For example: In a competition, it is more appropriate to translate game as "one inning, two innings", etc., and in other scenarios, it is more appropriate to translate it as "game, play";

[0050] For example: In a tennis match, fifteen love is a score and should be translated as 15:0, while the traditional love should be translated as love or like.

[0051] In view of the above problems, the embodiments of the present invention provide a speech recognition method and apparatus, which make full use of some text materials synchronously played at the meeting site, extract keywords from the text materials, and during the decoding process, adjust the scores of the decoding paths according to the keywords, and obtain the speech recognition text according to the scores of each decoding path, effectively improving the accuracy of the speech recognition result.

[0052] As Figure 1 shown, is a flowchart of a speech recognition method according to an embodiment of the present invention, including the following steps:

[0053] Step 101, receive the speech data of the speaker and obtain the text materials related to the speech data.

[0054] It should be noted that the text materials related to the speech data may be, for example, but not limited to any one or more of the following: promotional materials related to the speaking site, speech materials displayed at the speaking site.

[0055] Among them, the promotional materials related to the speaking site can be obtained before or during the meeting.

[0056] The text materials can be in various forms, such as, picture + text, video + text, etc. The text format can be PPT format, WORD format, etc.

[0057] The acquisition of the text materials can first obtain the pictures related to the voice data through an image device such as a camera, and then recognize the text in the pictures through OCR (Optical Character Recognition), so as to obtain the text materials related to the voice data.

[0058] Step 102, extract the keywords in the text materials.

[0059] It should be noted that in practical applications, the categories of the keywords can be set. For example, they can be entity words and / or professional nouns, etc. Of course, according to the characteristics of the voice to be recognized in the application scenario, the keywords can also include some other categories of words, and the embodiments of the present invention do not limit this.

[0060] The extraction of keywords can adopt the existing technology, and will not be described in detail here.

[0061] Step 103, perform speech recognition on the voice data by using a decoding network, adjust the scores of the decoding paths in the decoding network according to the keywords, and obtain the speech recognition text according to the scores of each decoding path.

[0062] In the embodiments of the present invention, the language model and the acoustic model in the decoding network can adopt general language models and acoustic models, but the scores of the decoding paths in the decoding network need to be adjusted according to the keywords. Specifically, the scores of the decoding paths passing through the keywords can be increased. The specific adjustment method can determine the increased scores of the decoding paths according to the number of times or ratios of the keywords appearing in the text materials, etc.

[0063] By adjusting the scores of the decoding paths, the scores of the decoding paths where the keywords are located are increased, so that the possibility of the decoding selecting this path is increased, and some keywords in the speech recognition text obtained by the decoding are more matched with the content of the materials, improving the accuracy of the speech recognition text.

[0064] It can be seen that the speech recognition method provided by the embodiments of the present invention makes full use of some text materials related to the meeting site, extracts the keywords in the text materials, adjusts the scores of the decoding paths according to the keywords during the decoding process, and obtains the speech recognition text according to the scores of each decoding path, effectively improving the accuracy of the speech recognition result.

[0065] As Figure 2 shown, it is another flowchart of the speech recognition method according to an embodiment of the present invention, including the following steps:

[0066] Step 200: Receive the speech data of the speaker, and perform speech recognition on the speech data by using a decoding network to obtain a speech recognition text;

[0067] Step 201: Obtain text materials related to the speech data, and extract keywords from the text materials;

[0068] Step 202: Present the speech recognition text obtained in step 200;

[0069] Step 203: Receive corrections to the recognition errors in the speech recognition text, and record the correction information.

[0070] It should be noted that when performing speech recognition on the received language data in step 200, the scores of the decoding paths in the decoding network can be adjusted according to the keywords obtained in step 201 and the correction information recorded in step 203.

[0071] For application scenarios that require real-time speech recognition such as a conference site, in the starting stage, the decoding network in the recognition process uses a general language model to perform speech recognition processing on the received speech data. As the text materials are read, keywords in the text materials can be obtained. Correspondingly, when performing recognition on the currently received speech data in step 200, the scores of the corresponding decoding paths can be adjusted according to the keywords extracted in step 201. As speech recognition progresses, after obtaining the correction information in step 203, the speech recognition processing in step 200 can not only adjust the scores of the decoding paths according to the keywords, but also further adjust the scores of the decoding paths according to the correction information, thereby effectively improving the accuracy of the speech recognition result. Especially for scenarios with real-time decoding requirements and where the received speech data contains entity words such as personal names, place names, product names, etc., and some professional terms, the speech recognition effect can be significantly improved by using the solution of the present invention.

[0072] Correspondingly, an embodiment of the present invention further provides a speech recognition device, as Figure 3 shown, which is a schematic structural diagram of the device.

[0073] In this embodiment, the device includes the following modules:

[0074] A speech receiving module 301, configured to receive the speech data of the speaker;

[0075] A text information obtaining module 302, configured to obtain text materials related to the speech data;

[0076] A keyword extraction module 303 is configured to extract keywords from the text material.

[0077] A speech recognition module 304 is configured to perform speech recognition on the speech data by using a decoding network, adjust the scores of decoding paths in the decoding network according to the keywords, and obtain a speech recognition text based on the scores of the decoding paths.

[0078] It should be noted that the text material related to the speech data may be, for example, but not limited to any one or more of the following: promotional materials related to the speaking site, speech materials displayed at the speaking site. Among them, the promotional materials related to the speaking site can be obtained before or during the meeting.

[0079] The text material may be in various forms, such as picture + text, video + text, etc. The text format may be PPT format, WORD format, etc.

[0080] The text information acquisition module 302 may include a picture acquisition unit and a text recognition unit. Among them, the picture acquisition unit may be a camera, etc., to acquire pictures related to the speech data; the text recognition unit is configured to recognize the text in the pictures through OCR to obtain text material related to the speech data.

[0081] The keyword extraction module 303 may extract keywords from the text material by using existing technologies. For example, by using a corresponding dictionary and matching with the entries in the dictionary, the keywords in the text material are obtained.

[0082] It should be noted that in practical applications, the categories of the keywords may be set. For example, they may be entity words and / or professional nouns, etc. Of course, according to the characteristics of the speech to be recognized in the application scenario, the keywords may also include some other categories of words, which are not limited in the embodiments of the present invention.

[0083] When the speech recognition module 304 recognizes the received speech data, it may adopt a general language model and acoustic model, and adjust the scores of the decoding paths in the decoding network according to the keywords. Specifically, the scores of the decoding paths passing through the keywords may be increased. The specific adjustment method may determine the increased scores of the decoding paths according to the number of times or ratios of the keywords appearing in the text material, etc.

[0084] By adjusting the scores of the decoding paths, the scores of the decoding paths where the keywords are located are increased, so that the possibility of the decoding selecting this path is increased, and some keywords in the obtained speech recognition text are more matched with the material content, improving the accuracy of the speech recognition text.

[0085] The voice recognition device provided by the embodiment of the present invention can make full use of some text materials played synchronously at the meeting site, extract the keywords in the text materials, and during the decoding process, adjust the score of the decoding path according to the keywords, and obtain the voice recognition text according to the scores of each decoding path, effectively improving the accuracy of the voice recognition result.

[0086] As Figure 4 shown, it is another structural block diagram of the voice recognition device according to the embodiment of the present invention.

[0087] Different from the embodiment Figure 3 shown, in this embodiment, the device further includes: a display module 401 and a correction module 402. Among them:

[0088] The display module 401 is used to display the voice recognition text;

[0089] The correction module 402 is used to provide an artificial correction interface to receive corrections to the recognition errors in the voice recognition text.

[0090] Correspondingly, in this embodiment, the voice recognition module 304 not only adjusts the decoding path score according to the keywords output by the keyword extraction module 303, but also further adjusts the decoding path score according to the correction of the recognition errors in the voice recognition text by the correction module 402, thereby effectively improving the accuracy of the voice recognition result. Especially for scenarios with real-time decoding requirements, and the received voice data contains entity words such as names of people, places, product names, and some professional terms, the voice recognition effect can be significantly improved by using the solution of the present invention.

[0091] The voice recognition method and device provided by the embodiment of the present invention can not only be applied to the voice transcription system in the meeting scenario, but also be applied to the machine simultaneous interpretation system to improve the voice recognition and translation effects.

[0092] Figure 5 FIG. is a block diagram of a device 800 for determining the weight of search terms according to an exemplary embodiment. For example, the device 800 can be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0093] Referring to Figure 5 , the device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0094] The processing component 802 generally controls the overall operation of the device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing element 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above methods. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0095] The memory 804 is configured to store various categories of data to support the operation of the device 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0096] The power component 806 provides power to the various components of the device 800. The power component 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the device 800.

[0097] The multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can not only sense the boundaries of touch or swipe actions, but also detect the duration and pressure associated with the touch or swipe operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have focal length and optical zoom capabilities.

[0098] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC) that is configured to receive external audio signals when the device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 further includes a speaker for outputting audio signals.

[0099] The I / O interface 812 provides an interface between the processing component 802 and peripheral interface modules, and the peripheral interface modules may be a keyboard, a click wheel, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a power button, and a lock button.

[0100] The sensor component 814 includes one or more sensors for providing an assessment of various aspects of the state of the device 800. For example, the sensor component 814 can detect the on / off state of the device 800, the relative positioning of components, such as the display and keypad of the device 800. The sensor component 814 can also detect a change in the position of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration / deceleration of the device 800, and a change in the temperature of the device 800. The sensor component 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor component 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 814 may further include an acceleration sensor, a gyro sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0101] The communication component 816 is configured to facilitate communication between the device 800 and other devices in a wired or wireless manner. The device 800 can access a wireless network based on communication standards, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0102] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above method.

[0103] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the above instructions can be executed by a processor 820 of the apparatus 800 to complete the above method for correcting accidental touch of keys. For example, the non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0104] The present invention also provides a non-transitory computer-readable storage medium. When the instructions in the storage medium are executed by a processor of a mobile terminal, the mobile terminal can execute all or part of the steps in the method embodiments of the present invention described above.

[0105] Figure 6 FIG. 10 is a schematic structural diagram of a server in an embodiment of the present invention. The server 1900 may vary greatly due to different configurations or performances, and may include one or more central processing units (CPUs) 1922 (for example, one or more processors) and a memory 1932, and one or more storage media 1930 (for example, one or more mass storage devices) for storing application programs 1942 or data 1944. Among them, the memory 1932 and the storage medium 1930 may be transient storage or persistent storage. The program stored in the storage medium 1930 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Further, the central processor 1922 may be configured to communicate with the storage medium 1930 and execute a series of instruction operations in the storage medium 1930 on the server 1900.

[0106] The server 1900 may further include one or more power supplies 1926, one or more wired or wireless network interfaces 1950, one or more input / output interfaces 1958, one or more keyboards 1956, and / or one or more operating systems 1941, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0107] Other embodiments of the present invention will be readily apparent to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include known common general knowledge or conventional technical means in the technical field not disclosed in this disclosure. The specification and examples are only illustrative, and the true scope and spirit of the present invention are pointed out by the following claims.

[0108] It should be understood that the present invention is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

[0109] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A voice recognition method, characterized in that, The method includes: Receiving the speech data of the speaker, and obtaining the text materials related to the speech data in the meeting; extracting the keywords in the text materials; Performing speech recognition on the speech data by using a decoding network, and in real time adjusting the scores of the decoding paths in the decoding network according to the number of occurrences or ratio of the keywords in the text materials, and obtaining the speech recognition text according to the scores of the respective decoding paths; Displaying the speech recognition text, receiving corrections to the recognition errors in the speech recognition text, and in real time adjusting the scores of the decoding paths in the decoding network according to the corrections.

2. The method according to claim 1, characterized in that, The text materials related to the speech data include any one or more of the following: promotional materials related to the speaking site, speech materials displayed at the speaking site.

3. The method according to claim 1, characterized in that The obtaining of the text materials related to the speech data includes: Obtaining the pictures related to the speech data through an image device; Performing OCR recognition on the text in the pictures to obtain the text materials related to the speech data.

4. The method according to claim 1, wherein The keywords include any one or more of the following: entity words, professional nouns.

5. A voice recognition device, characterized in that, The device includes: A speech receiving module, configured to receive the speech data of the speaker; A text information obtaining module, configured to obtain the text materials related to the speech data in the meeting; A keyword extraction module, configured to extract the keywords in the text materials; A speech recognition module, configured to perform speech recognition on the speech data by using a decoding network, and in real time adjusting the scores of the decoding paths in the decoding network according to the number of occurrences or ratio of the keywords in the text materials, and obtaining the speech recognition text according to the scores of the respective decoding paths; A display module, configured to display the speech recognition text; A correction module, configured to provide a correction interface to receive corrections to the recognition errors in the speech recognition text; The speech recognition module is further configured to in real time adjust the scores of the decoding paths in the decoding network according to the corrections.

6. The device according to claim 5, characterized in that, The text information obtaining module includes: A picture obtaining unit, configured to obtain the pictures related to the speech data; A text recognition unit, configured to perform OCR recognition on the text in the pictures to obtain the text materials related to the speech data.

7. An electronic device, characterized in that, Comprising: One or more processors, a memory; The memory is used for storing computer-executable instructions, and the processor is used for executing the computer-executable instructions to implement the method according to any one of claims 1 to 4.

8. A readable storage medium, on which instructions are stored, and the instructions are executed to implement the method according to any one of claims 1 to 4.

Images