Text regularization method and device
By extracting features from speech data and recognition text and deleting colloquialized words using text regularization model, the semantic incompleteness and repetition problems in machine simultaneous transmission are solved, the translation quality and accuracy are improved, and the input of standardized text is realized.
Patent Information
- Application Number
- CN201910023851.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-01-10
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2039-01-10
AI Technical Summary
Existing machine simultaneous products have reduced the translation quality due to incomplete semantics, repetitions and modal words when dealing with improvised speeches, and the existing methods have poor effect in removing these expressions, which affects the quality of subsequent translations.
By extracting pronunciation features and text features from speech data and identification text, using a pre-constructed text regularization model, the label of word units is determined and semantic incomplete, repetitive and modular words are removed, and normalized text is obtained, thereby improving the accuracy of machine translation. Rate.
Effectively remove colloquial words in speech recognition text, improve translation quality and readability, and significantly improve the accuracy and user experience of machine translation.
Smart Images

Figure CN111435595B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of text processing, and in particular to a text regularization method and device. Background Art
[0002] Simultaneous interpretation, or "simultaneous interpretation" for short, refers to the uninterrupted and real-time translation of speech content to the audience without interrupting the speaker. Its biggest feature is its high efficiency, and the audience can obtain information in a timely manner. It is widely used in important occasions such as international conferences and diplomatic negotiations. At present, with the development of artificial intelligence technology, machine simultaneous interpretation has emerged. The biggest advantage of machine simultaneous interpretation is that it will not cause the translation rate to drop due to fatigue. However, since machine simultaneous interpretation is limited by speech recognition and machine translation technology, the overall translation quality of machine simultaneous interpretation is still far behind that of manual simultaneous interpretation. Especially in some speech occasions, speakers often improvise according to the topic and ideas. During the speech, there will be colloquial expressions such as incomplete semantics, repetition, and modal particles. This phenomenon will affect the subsequent translation quality and even cause complete translation errors. In response to this situation, existing machine simultaneous interpretation products usually use rule-based methods to remove some of the colloquial expressions, but the effect is not good, which will greatly affect the effect of subsequent machine simultaneous interpretation. Summary of the invention
[0003] The embodiments of the present invention provide a text regularization method and device to make speech recognition text more understandable and standardized in expression.
[0004] To this end, the present invention provides the following technical solutions:
[0005] A text regularization method, the method comprising:
[0006] Acquire voice data and recognized text corresponding to the voice data;
[0007] Obtain word units in the recognized text and word vectors corresponding to the word units;
[0008] Extracting regular features, specifically including extracting voice features from the voice data and extracting text features from the recognized text;
[0009] Input the word vector and the feature vector corresponding to the regularization feature into a pre-built text regularization model, and obtain a label corresponding to the word unit according to the output of the text regularization model, wherein the label at least includes: deletion;
[0010] The recognized text is regularized according to the labels corresponding to the word units to obtain a regularized speech recognition text.
[0011] Optionally, the text features include any one or more of the following: part-of-speech features, Ngram repetition features, and adjacent word fuzzy matching scores.
[0012] Optionally, the speech feature includes any one or more of the following: fundamental frequency feature, pause feature, energy feature.
[0013] Optionally, the regularizing the recognized text according to the label corresponding to the word unit to obtain the regularized text includes:
[0014] The word units with the label of "delete" in the recognized text are deleted to obtain a regularized text.
[0015] Optionally, before obtaining each word unit in the recognized text and the word vector corresponding to the word unit, the method further includes:
[0016] Modal particles in the recognized text are filtered out.
[0017] Optionally, the method further comprises:
[0018] Acquiring speaker mouth shape information synchronized with the voice data;
[0019] The extracting regular features also includes:
[0020] Extracting lip shape features from the lip shape information.
[0021] A text regularization device, comprising:
[0022] A voice receiving module, used to obtain voice data;
[0023] A recognition text acquisition module, used to acquire the recognition text corresponding to the voice data;
[0024] A word unit acquisition module, used to acquire each word unit in the recognized text and the word vector corresponding to the word unit;
[0025] A feature extraction module is used to extract regular features; the feature extraction module includes: a speech feature extraction unit and a text feature extraction unit; the speech feature extraction unit is used to extract speech features from the speech data; the text feature extraction unit is used to extract text features from the recognition text;
[0026] A judgment module is used to input the word vector and the feature vector corresponding to the regularization feature into a pre-built text regularization model, and obtain a label corresponding to the word unit according to the output of the text regularization model, wherein the label at least includes: deletion;
[0027] The regularization module is used to regularize the recognition text according to the labels corresponding to the word units to obtain the regularized speech recognition text.
[0028] Optionally, the text features include any one or more of the following: part-of-speech features, Ngram repetition features, and adjacent word fuzzy matching scores.
[0029] Optionally, the speech feature includes any one or more of the following: fundamental frequency feature, pause feature, energy feature.
[0030] Optionally, the regularization module is specifically used to delete word units with a label of deletion in the recognized text to obtain a regularized text.
[0031] Optionally, the device further comprises:
[0032] The filtering module is used to filter out modal particles in the recognized text before the word unit acquisition module acquires each word unit in the recognized text and the word vector corresponding to the word unit.
[0033] Optionally, the device further comprises:
[0034] A lip shape information acquisition module, used to acquire the speaker's lip shape information synchronized with the voice data;
[0035] The feature extraction module comprises:
[0036] The lip shape feature extraction unit is used to extract the lip shape features from the lip shape information.
[0037] An electronic device, comprising: one or more processors and a memory;
[0038] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the aforementioned method.
[0039] A readable storage medium stores instructions thereon, wherein the instructions are executed to implement the above-mentioned method.
[0040] The text regularization method and device provided by the embodiment of the present invention respectively extract voice features and text features from the current voice data of the speaker and the corresponding recognized text, and based on the word vectors in the recognized text, the feature vectors corresponding to the voice features and the feature vectors corresponding to the text features, use a pre-built text regularization model to determine the labels corresponding to the word units in the recognized text, thereby determining the word units that need to be deleted in the recognized text, and then regularize the recognized text according to the labels, including deleting the word units in the recognized text with labels for deletion, to obtain the regularized speech recognition text.
[0041] Furthermore, it is also possible to obtain speaker mouth shape information synchronized with the voice data, and extract lip shape features from it, and use the lip shape features together with the aforementioned voice features and text features as regularization features for regularizing the recognized text, and input the word vectors in the recognized text and the feature vectors corresponding to the regularization features into the text regularization model. The labels corresponding to each word unit are determined based on the output of the text regularization model, which can effectively improve the accuracy of the labels.
[0042] By utilizing the scheme of the present invention, colloquial word units such as semantically incomplete words, repeated words, modal particles, etc. that appear in the recognized text obtained by speech recognition can be deleted to obtain a de-colloquialized, standardized text. This text can be used as input text for machine translation, which can greatly improve the accuracy of machine translation. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.
[0044] Figure 1 It is a schematic diagram of a topological structure of a text regularization model in an embodiment of the present invention;
[0045] Figure 2 It is a flow chart of the text regularization method according to an embodiment of the present invention;
[0046] Figure 3 is another flow chart of the text regularization method according to an embodiment of the present invention;
[0047] Figure 4 It is a structural block diagram of a text regularization device according to an embodiment of the present invention;
[0048] Figure 5 It is another structural block diagram of the text regularization device according to an embodiment of the present invention;
[0049] Figure 6 is a block diagram of a device for input method according to an exemplary embodiment;
[0050] Figure 7 It is a schematic diagram of the structure of the server in the embodiment of the present invention. DETAILED DESCRIPTION
[0051] In order to enable persons skilled in the art to better understand the solutions of the embodiments of the present invention, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings and implementation modes.
[0052] In view of the fact that some colloquial expressions often appear in the speaker's voice, which causes the speech recognition text input into the machine for machine translation to contain colloquial words, an embodiment of the present invention provides a text regularization method and device, which respectively extract corresponding regularization features from the current speech data and its recognized text, and use the word vector of each word unit in the recognized text, the feature vector corresponding to the regularization feature, and a pre-built text regularization model to determine the label of each word unit, the label is used to identify whether the word unit should be retained or deleted, and then the recognized text is regularized according to the label to obtain the regularized speech recognition text.
[0053] In the embodiment of the present invention, the text regularization model adopts a neural network model, such as Figure 1 As shown, the neural network model includes: input layer, feature vector conversion layer, BLSTM (Bidirectional Long Short-term Memory) layer, classifier layer, and output layer. Among them:
[0054] The input of the input layer includes: word vector W, text level feature vector HCF (for the convenience of description, hereinafter referred to as text feature), speech level feature vector S (for the convenience of description, hereinafter referred to as speech feature), and further, image level feature vector.
[0055] The feature vector conversion layer is used to convert the word vectors and feature vectors corresponding to each feature input into a vector representation of a fixed length, and concatenate these vectors to obtain the input of the next layer.
[0056] The BLSTM layer is used to automatically extract the high-dimensional abstract features of each word unit and each feature in the text to be regularized from front to back and from back to front, and obtain the hidden layer output vector after vectorization and splicing. In the neural network model of this example, two BLSTM layers are set. Of course, one BLSTM layer can also be set, which is not limited in this embodiment of the present invention.
[0057] The classifier layer is the softmax layer, which is used to determine the score of each word unit corresponding to each label based on the vector output by the hidden layer.
[0058] The output layer is used to output the label corresponding to each word unit, namely Disflag1, Disflag2, ... Disflagn.
[0059] In the embodiment of the present invention, the categories of tags that can be set include at least: deletion, and optionally: retention.
[0060] The text regularization model can be pre-constructed. Specifically, a large amount of training data can be collected. The training data includes: voice data and recognition text corresponding to the voice data, and each word unit in the recognition text is marked, and information such as modal particles, repeated words, incomplete words, etc. are marked, and the labels of these words are set to be deleted.
[0061] The text features and speech features of the training data are extracted, and these features are used as regularization features. The word vectors corresponding to each word unit, the feature vectors corresponding to the regularization features, and the annotation information are used to train and obtain the parameters of the text regularization model.
[0062] The training process of the text regularization model is similar to that of a conventional neural network model and will not be described in detail here.
[0063] Since different words have different corresponding mouth shapes, in another embodiment of the method of the present invention, mouth shape information synchronized with the voice data can also be collected and used as auxiliary information to further improve the accuracy of voice recognition. For example, the mouth shape information can assist in determining whether there are repeated words in the voice recognition text. Extracting mouth shape features from the mouth shape information, and comprehensively utilizing the text features, voice features, and mouth shape features to train the text regularization model can make the model parameters obtained in the training more accurate, and thus the labels corresponding to each word unit in the recognized text obtained subsequently using the text regularization model are more accurate.
[0064] The text regularization method and device provided by the embodiment of the present invention utilize the above-mentioned text regularization model to determine the label type corresponding to each word unit in the recognized text, and remove the word unit with the label type of deletion from the recognized text.
[0065] It should be noted that the solution of the present invention can be applied not only to the regularization processing of Chinese speech recognition text, but also to the regularization processing of speech recognition text in other languages, such as English text obtained by English speech recognition.
[0066] like Figure 2 FIG. 1 is a flowchart of a text regularization method according to an embodiment of the present invention, comprising the following steps:
[0067] Step 201, obtaining voice data and recognition text corresponding to the voice data.
[0068] The recognized text can be obtained by performing voice recognition processing on the voice data. The embodiment of the present invention does not limit the specific voice recognition processing technology. No matter which voice recognition processing technology is used, it will not affect the technical effect of the solution of the present invention.
[0069] Step 202, obtaining word units in the recognized text and word vectors corresponding to the word units.
[0070] For a language based on words, such as English, each word in the recognized text is a word unit.
[0071] For a language that uses characters as units, such as Chinese, if the recognized text is a sequence of Chinese characters, it is also necessary to perform word segmentation processing on it. Of course, the word units in the recognized text can also be directly obtained based on the speech decoding network.
[0072] The word segmentation process is to divide a Chinese character sequence into individual words. For the convenience of description, in the embodiment of the present invention, the individual words obtained after the word segmentation process are called word units.
[0073] Specific word segmentation processing methods include: word segmentation method based on string matching, word segmentation method based on statistics, etc.
[0074] The word segmentation method based on string matching is to match the recognized text with entries in a pre-established dictionary according to a certain strategy to obtain word units.
[0075] The statistical-based word segmentation method is to use a statistical model to learn the rules of word segmentation given a large number of segmented texts, so as to achieve segmentation of unknown texts. The statistical model can be, for example: N-gram model, Hidden Markov Model (HMM), Maximum Entropy Model (ME), Conditional Random Field Model, etc.
[0076] The Glov model can be used to obtain the word vector of each word unit. The Glov model is a model used to realize the distributed vectorized expression of Chinese words. It combines the advantages of global statistical information and local context window of matrix decomposition (Latent Semantic Analysis, LSA) and incorporates global prior statistical information.
[0077] Step 203, extracting regular features, specifically includes extracting voice features from the voice data and extracting text features from the recognized text.
[0078] The text features may include but are not limited to any one or more of the following: part-of-speech features, Ngram (N-gram grammar model) repetition features, and adjacent word fuzzy matching scores.
[0079] Part-of-speech feature: The part-of-speech feature refers to the classification of word classes based on the characteristics of words, such as nouns, verbs, adverbs, etc. The extraction of part-of-speech features can use existing toolkits, such as the StanfordNLP or LTP tools, etc.
[0080] Ngram repetition feature: The Ngram includes unigrams and bigrams. The repetition feature refers to whether the current phrase is the same as the N phrases before and after.
[0081] For example, the recognized text is: Well, yesterday I arrived and I went to Tsinghua Science Park.
[0082] Taking "went to" as an example, it is a unigram. Taking the current phrase "went to" as the center, compare it with the 15 phrases on the left and the 15 phrases on the right. If they are the same, take 1; if they are different, take 0, forming a 30-dimensional feature vector for the current word.
[0083] Taking "I went to" as an example, it is a bigram. Taking it as the center, compare it with 4 bigrams before and after respectively. If they are the same, take 1; if they are different, take 0, forming an 8-dimensional vector for the current bigram.
[0084] Nearby word fuzzy matching score: The nearby word fuzzy matching score refers to twice the number of identical characters (such as Chinese characters) or the number of identical characters (such as English characters) between the current phrase and the nearby phrase on the left, divided by the sum of the two phrases.
[0085] The voice features may include, but are not limited to, any one or more of the following: fundamental frequency feature, pause feature, energy feature.
[0086] Fundamental frequency feature: The fundamental frequency feature refers to the continuous fundamental frequency values of the current phrase; when extracting the fundamental frequency feature, take the fundamental frequency values of the M frames of data before and after the center frame of the current word as the fundamental frequency vector of the current phrase, where M takes an appropriate value according to the experiment.
[0087] Pause feature: The pause feature refers to the time interval between the voice data corresponding to the current phrase and the voice data corresponding to the next phrase; when extracting the pause feature, directly obtain the time interval length between the end position of the current phrase and the start position of the next phrase, with the unit of s.
[0088] Energy feature: The energy feature refers to the energy change of the current phrase; the extraction of the energy feature is similar to the extraction of the fundamental frequency feature.
[0089] Step 204, input the word vector and the feature vector corresponding to the regularization feature into a pre-constructed text regularization model, and obtain the label corresponding to the word unit according to the output of the text regularization model. The label includes at least: deletion.
[0090] Step 205: regularize the recognized text according to the labels corresponding to the word units to obtain regularized speech recognition text.
[0091] Specifically, word units with a label of "delete" in the recognition text are deleted to obtain a regularized speech recognition text.
[0092] By using the text regularization method provided in the embodiment of the present invention, the spoken speech recognition text can be converted into a standardized written language expression, so that the regularized text has better readability and fluency.
[0093] For machine translation systems, standardized text input can produce better translation results than colloquial text input. Moreover, based on standardized text, more accurate punctuation can be performed on speech recognition text.
[0094] For example, the colloquial text is: Well, I arrived yesterday and went to Tsinghua Science Park;
[0095] Input the above spoken text into the machine translation system, and the machine translation result is: Well, I arrived yesterday, and I went to Tsinghua Science Park.
[0096] The above colloquial text is regularized by using the scheme of the present invention, and the regularized text is: I went to Tsinghua Science Park yesterday.
[0097] The above regularized text is input into the machine translation system, and the machine translation result is: Yesterday I went to Tsinghua science par.
[0098] Obviously, the latter translation result is significantly better than the former one.
[0099] For another example, the colloquial text is: you know I mean my health is very good;
[0100] Input the above colloquial text into the machine translation system, and the machine translation result is: You know what I mean is that my health is very good;
[0101] The above colloquial text is regularized by using the scheme of the present invention, and the regularized text is: my health is very good;
[0102] The above regularized text is input into the machine translation system, and the machine translation result is: I am in good health.
[0103] Obviously, the following translation results can enable users to get a more accurate understanding.
[0104] The text regularization method provided in the embodiment of the present invention can not only be applied to machine simultaneous interpretation systems to provide standardized text input for machine translation; it can also be applied to other speech-to-text scenarios. For example, in a conference scenario, the speech of the on-site speaker can be synchronously recorded and speech-to-text processed, the obtained speech recognition text can be regularized, and the regularized speech recognition text can be presented to the user, which can enhance the user's reading experience.
[0105] Usually, when people pronounce different words, the corresponding mouth shapes are different, but the mouth shapes of the same word are basically the same in most cases. Based on this feature, in another embodiment of the method of the present invention, the speaker's mouth shape information can be comprehensively used to train a text regularization model, and the text regularization model is used to determine the label of each word unit in the speech recognition text, so as to further improve the accuracy of the label.
[0106] like Figure 3 FIG. 2 is another flow chart of a text regularization method according to an embodiment of the present invention, comprising the following steps:
[0107] Step 301: Acquire voice data, recognition text corresponding to the voice data, and speaker mouth shape information synchronized with the voice data.
[0108] The recognized text can be obtained by performing voice recognition processing on the voice data. The embodiment of the present invention does not limit the specific voice recognition processing technology. No matter which voice recognition processing technology is used, it will not affect the technical effect of the solution of the present invention.
[0109] The speaker's mouth shape information may be obtained by recording the speaker to obtain a video image, and the speaker's mouth shape information may be extracted from the video image.
[0110] Step 302, obtaining word units in the recognized text and word vectors corresponding to the word units.
[0111] Step 303: extract voice features from the voice data, extract text features from the recognition text, and extract lip shape features from the lip shape information to obtain a voice feature vector, a text feature vector, and a lip shape feature vector.
[0112] Step 304, input the word vector, speech feature vector, text feature vector, and lip shape feature vector into a pre-built text regularization model, and obtain a label corresponding to the word unit according to the output of the text regularization model, and the label at least includes: delete.
[0113] It should be noted that when constructing the text regularization model described here, the regularization features not only use speech features and text features, but also comprehensively consider lip shape features.
[0114] Step 305: regularize the recognized text according to the labels corresponding to the word units to obtain regularized speech recognition text.
[0115] The text regularization method provided by the embodiment of the present invention extracts voice features, text features, and lip shape features from voice data, text data, and image data, respectively, and comprehensively utilizes these features to determine the label type corresponding to each word unit in the recognized text, thereby obtaining a more accurate label judgment result, making the regularized text more standardized, and improving the text regularization effect.
[0116] Using the solution of the present invention, the words that can be regularized mainly include: modal particles, repeated words, semantically incomplete words, etc. Of course, specific words that need to be regularized can also be selected according to application needs, and this embodiment of the present invention does not limit this.
[0117] In addition, some modal particles can be filtered out by simple rules, dictionary matching and other methods. Therefore, in the above embodiments, the modal particles can be filtered out before the recognized text is regularized.
[0118] Accordingly, an embodiment of the present invention further provides a text regularization device, such as Figure 4 Shown is a structural block diagram of the device.
[0119] In this embodiment, the text regularization device includes the following modules:
[0120] A voice receiving module 401 is used to obtain voice data, for example, the voice data can be obtained through a field recording device;
[0121] A recognition text acquisition module 402 is used to acquire the recognition text corresponding to the voice data. For example, the recognition text corresponding to the voice data can be acquired through a voice recognition module or a voice transcription device.
[0122] A word unit acquisition module 403 is used to acquire each word unit and its word vector in the recognized text;
[0123] The feature extraction module 404 is used to extract regular features; the feature extraction module 404 includes: a speech feature extraction unit 441 and a text feature extraction unit 442; the speech feature extraction unit 441 is used to extract speech features from the speech data; the text feature extraction unit 442 is used to extract text features from the recognition text;
[0124] The judgment module 405 is used to input the word vector and the feature vector corresponding to the regularization feature into a pre-built text regularization model, and obtain the label corresponding to the word unit according to the output of the text regularization model, and the label at least includes: delete, and optionally includes: retain.
[0125] The regularization module 406 is used to regularize the recognized text according to the labels corresponding to the word units to obtain regularized speech recognition text.
[0126] The word unit acquisition module 403 can use existing technologies to acquire each word unit and its word vector in the recognized text, and the acquisition method will be different for recognized texts in different languages, and can be adaptively adjusted according to the characteristics of each language.
[0127] In an embodiment of the present invention, the text features may include but are not limited to any one or more of the following: part-of-speech features, Ngram repetition features, and adjacent word fuzzy matching scores; the speech features may include but are not limited to any one or more of the following: fundamental frequency features, pause features, and energy features.
[0128] The text regularization model can adopt a neural network model, such as Figure 2 The model structure shown includes: input layer, feature vector conversion layer, BLSTM layer, classifier layer, and output layer.
[0129] It should be noted that the text regularization model can be pre-constructed by a corresponding model construction module (not shown), and the model training process is similar to the training process of a conventional neural network model, which will not be described in detail here. In addition, the model construction module can be used as a part of the text regularization device of the present invention, or it can be independent of the text regularization device of the present invention, which is not limited.
[0130] Using the text regularization model, the judgment module 405 inputs the word vector, the feature vector corresponding to the text feature, and the feature vector corresponding to the speech feature into the text regularization model, and obtains the label corresponding to the word unit according to the output of the text regularization model, and the label is used to identify whether the word unit needs to be deleted.
[0131] Correspondingly, the regularization module 406 deletes the word units with the label of deletion in the recognized text to obtain the regularized speech recognition text.
[0132] For example, the recognized text is: Um, I arrived yesterday and went to Tsinghua Science Park;
[0133] Using the text regularization model, the label corresponding to each word unit is obtained. For the sake of clarity, it is expressed in a table as follows:
[0134] Word Unit Forehead yesterday I arrive I Went Tsinghua University Science Park Label delete reserve delete delete reserve reserve reserve reserve
[0135] After regularization, the regularized text is: I went to Tsinghua Science Park yesterday.
[0136] The text regularization device provided in the embodiment of the present invention can be applied to various scenarios. For example, when applied to a machine simultaneous interpretation system, it can provide standardized text input for machine translation; when applied to a speech-to-text conversion scenario, such as a conference scenario, it can synchronously record the speech of the speaker on site, perform speech-to-text conversion processing, and regularize the obtained speech recognition text.
[0137] like Figure 5 The figure shows another structural block diagram of the text regularization device according to an embodiment of the present invention.
[0138] and Figure 4 Compared with the illustrated embodiment, in this embodiment, the text regularization device further includes: a lip shape information acquisition module 407, which is used to acquire the speaker's lip shape information synchronized with the voice data.
[0139] Accordingly, in this embodiment, the feature extraction module 404 further includes: a lip shape feature extraction unit 443, which is used to extract lip shape features from the lip shape information.
[0140] In the text regularization device provided by the embodiment of the present invention, the feature extraction module 404 extracts voice features, text features, and lip shape features from the voice data, text data, and image data respectively; the judgment module 405 comprehensively utilizes these features to determine the label type corresponding to each word unit in the recognized text, thereby obtaining a more accurate label judgment result, making the regularized voice recognition text more standardized, and improving the text regularization effect.
[0141] The words that can be regularized by the text regularization device of the present invention mainly include: modal particles, repeated words, semantically incomplete words, etc. Of course, specific words that need to be regularized can also be selected according to application needs, and this embodiment of the present invention does not limit this.
[0142] In addition, in the above Figure 4 and Figure 5 The illustrated embodiment may further include: a filtering module (not shown) for filtering out modal particles in the recognized text before the word unit acquisition module 403 acquires each word unit in the recognized text and its word vector.
[0143] Furthermore, in another embodiment of the text regularization device of the present invention, it may further include: a display module (not shown) for displaying the regularized speech recognition text on the screen. For example, in a conference scene, the speech of the speaker on site can be synchronously recorded, and speech-to-text processing can be performed, and the obtained speech recognition text can be regularized, and the regularized speech recognition text can be presented to the user, which can improve the user's reading experience.
[0144] Figure 6 8 is a block diagram of a device 800 for determining a search term weight according to an exemplary embodiment. For example, the device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.
[0145] Reference Figure 6 , the device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .
[0146] The processing component 802 generally controls the overall operation of the device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.
[0147] The memory 804 is configured to store various categories of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any category of volatile or non-volatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
[0148] The power component 806 provides power to the various components of the device 800. The power component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 800.
[0149] The multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
[0150] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the device 800 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.
[0151] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.
[0152] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the device 800, and the sensor assembly 814 can also detect the position change of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration / deceleration of the device 800, and the temperature change of the device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
[0153] The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and other devices. The device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
[0154] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.
[0155] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the instructions can be executed by the processor 820 of the device 800 to complete the above-mentioned key mis-touch error correction method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0156] The present invention also provides a non-temporary computer-readable storage medium. When the instructions in the storage medium are executed by a processor of a mobile terminal, the mobile terminal can execute all or part of the steps in the above-mentioned method embodiment of the present invention.
[0157] Figure 71 is a schematic diagram of the structure of the server in an embodiment of the present invention. The server 1900 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 1922 (for example, one or more processors) and memory 1932, and one or more storage media 1930 (for example, one or more mass storage devices) storing application programs 1942 or data 1944. Among them, the memory 1932 and the storage medium 1930 can be short-term storage or permanent storage. The program stored in the storage medium 1930 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 1922 can be configured to communicate with the storage medium 1930 and execute a series of instruction operations in the storage medium 1930 on the server 1900.
[0158] The server 1900 may also include one or more power supplies 1926, one or more wired or wireless network interfaces 1950, one or more input and output interfaces 1958, one or more keyboards 1956, and / or, one or more operating systems 1941, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
[0159] Those skilled in the art will readily appreciate other embodiments of the invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed in this disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.
[0160] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.
[0161] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A text regularization method, characterized in that: The method comprises: Acquire voice data and recognized text corresponding to the voice data; Obtain word units in the recognized text and word vectors corresponding to the word units; Extracting regular features, specifically including extracting voice features from the voice data and extracting text features from the recognized text; Input the word vector and the feature vector corresponding to the regularization feature into a pre-built text regularization model, and obtain a label corresponding to the word unit according to the output of the text regularization model, wherein the label at least includes: deletion; The recognized text is regularized according to the labels corresponding to the word units to obtain a regularized speech recognition text.
2. The method according to claim 1, characterized in that The text features include any one or more of the following: part-of-speech features, Ngram repetition features, and adjacent word fuzzy matching scores.
3. The method according to claim 1, characterized in that The speech features include any one or more of the following: fundamental frequency features, pause features, and energy features.
4. The method according to claim 1, characterized in that: The step of regularizing the recognized text according to the labels corresponding to the word units to obtain the regularized text comprises: The word units with the label of "delete" in the recognized text are deleted to obtain a regularized text.
5. The method according to claim 1, characterized in that Before obtaining each word unit in the recognized text and the word vector corresponding to the word unit, the method further includes: Modal particles in the recognized text are filtered out.
6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Acquiring speaker mouth shape information synchronized with the voice data; The extracting regular features also includes: Extracting lip shape features from the lip shape information.
7. A text regularization device, characterized in that: The device comprises: A voice receiving module is used to obtain voice data; A recognition text acquisition module, used to acquire the recognition text corresponding to the voice data; A word unit acquisition module, used to acquire each word unit in the recognized text and the word vector corresponding to the word unit; A feature extraction module is used to extract regular features; the feature extraction module includes: a speech feature extraction unit and a text feature extraction unit; the speech feature extraction unit is used to extract speech features from the speech data; the text feature extraction unit is used to extract text features from the recognition text; A judgment module is used to input the word vector and the feature vector corresponding to the regularization feature into a pre-built text regularization model, and obtain a label corresponding to the word unit according to the output of the text regularization model, wherein the label at least includes: deletion; The regularization module is used to regularize the recognition text according to the labels corresponding to the word units to obtain the regularized speech recognition text.
8. The device according to claim 7, characterized in that The device also includes: A lip shape information acquisition module, used to acquire the speaker's lip shape information synchronized with the voice data; The feature extraction module comprises: The lip shape feature extraction unit is used to extract the lip shape features from the lip shape information.
9. An electronic device, characterized in that: include: One or more processors, memory; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the method according to any one of claims 1 to 6.
10. A readable storage medium having instructions stored thereon, wherein the instructions are executed to implement the method according to any one of claims 1 to 6.
