Text recognition method and apparatus, electronic device, and readable storage medium

Abstract

The present application relates to the field of natural language processing, and discloses a text recognition method and apparatus, an electronic device, and a readable storage medium. The method comprises: inputting a first text to be recognized into a first text encoding model, wherein said first text comprises a first polyphone, and the first text encoding model comprises a convolution layer; extracting a first text feature sequence corresponding to said first text; performing segment-based convolution processing on the first text feature sequence by means of the convolution layer to obtain a second text feature sequence, wherein the second text feature sequence at least comprises global semantic information of said first text and local semantic information of a first text segment in said first text, and the first text segment comprises the first polyphone; and on the basis of the second text feature sequence, obtaining first code information corresponding to said first text, and obtaining a recognition result of said first text on the basis of the first code information.

Description

Text recognition method, device, electronic device and readable storage medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent application No. 202310873503.9 filed in China on July 14, 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present application relates to the field of natural language processing, and specifically relates to a text recognition method, device, electronic device, and readable storage medium. Background Art

[0004] Speech synthesis is a technology that converts text into speech. It provides a new way for users to interact with electronic devices. Currently, in Chinese speech synthesis announcements, due to the large number of polyphones in Chinese characters, speech synthesis may produce mispronunciations when processing polyphones.

[0005] In related technologies, in order for electronic devices to correctly read a text, a polyphonetic character model is typically used to predict the text's pronunciation. This polyphonetic character model relies on the results of text encoding. Text encoding involves converting natural language text into vectors for easier computer understanding and processing. Currently, the most commonly used character encoding method for text encoding is character-by-character encoding, considering only the content of the current character.

[0006] However, when using character encoding to process texts with complex language structures, pronunciation prediction errors of polyphonetic characters may occur, resulting in a low accuracy rate in pronunciation prediction of polyphonetic characters in the text.

[0007] Summary of the Invention

[0008] The purpose of the embodiments of the present application is to provide a text recognition method, device, electronic device and readable storage medium, which can better utilize local semantic information and global semantic information to analyze the pronunciation of long and difficult sentences and complex texts, thereby improving the accuracy of predicting the pronunciation of polyphonetic characters in the text.

[0009] In a first aspect, an embodiment of the present application provides a text recognition method, which includes: inputting a first text to be recognized into a first text encoding model, the first text to be recognized contains a first polyphonetic character, and the first text encoding model includes a convolutional layer; extracting a first text feature sequence corresponding to the first text to be recognized; using a convolutional layer to perform segmented convolution processing on the first text feature sequence to obtain a second text feature sequence, the second text feature sequence at least contains global semantic information of the first text to be recognized and local semantic information of a first text segment in the first text to be recognized, the first text segment contains a first polyphonetic character; based on the second text feature sequence, obtaining first encoding information corresponding to the first text to be recognized, and obtaining a recognition result of the first text to be recognized based on the above-mentioned first encoding information.

[0010] In the second aspect, an embodiment of the present application provides a text recognition device, which includes: a processing module and an extraction module, wherein: the processing module is used to input a first text to be recognized into a first text encoding model, the first text to be recognized contains a first polyphonetic character, and the first text encoding model includes a convolutional layer; the extraction module is used to extract a first text feature sequence corresponding to the first text to be recognized; the processing module is also used to use a convolutional layer to perform segmented convolution processing on the first text feature sequence to obtain a second text feature sequence, the second text feature sequence at least contains the global semantic information of the first text to be recognized and the local semantic information of the first text segment in the first text to be recognized, and the first text segment contains the first polyphonetic character; the processing module is also used to obtain the first encoding information corresponding to the first text to be recognized based on the second text feature sequence, and obtain the recognition result of the first text to be recognized based on the above-mentioned first encoding information.

[0011] In a third aspect, an embodiment of the present application provides an electronic device comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0012] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0013] In a fifth aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the method described in the first aspect.

[0014] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the method described in the first aspect.

[0015] In an embodiment of the present application, a text recognition device inputs a first text to be recognized into a first text encoding model, where the first text to be recognized contains a first polyphonetic character. The first text encoding model includes a convolutional layer. Then, a first text feature sequence corresponding to the first text to be recognized is extracted, and the convolutional layer is used to perform segmented convolution processing on the above-mentioned first text feature sequence to obtain a second text feature sequence. The above-mentioned second text feature sequence at least contains the global semantic information of the first text to be recognized and the local semantic information of the first text segment in the first text to be recognized, and the above-mentioned first text segment contains the first polyphonetic character. Finally, based on the above-mentioned second text feature sequence, the first encoding information corresponding to the first text to be recognized is obtained, and the recognition result of the first text to be recognized is obtained based on the above-mentioned first encoding information. Through this method, the text recognition device performs a convolution operation on the global text feature sequence of the text through the convolution layer in the first text encoding model to obtain a text feature sequence containing global semantic information and local semantic information, and obtains a more accurate recognition result based on the text feature sequence, thereby enhancing the ability to model the local information of the text sequence, and can make up for the shortcomings of the existing model in extracting local feature information, so as to more accurately analyze the pronunciation of polyphones in long and difficult sentences and complex texts, thereby improving the accuracy of pronunciation prediction of polyphones in the text. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] FIG1 is a flow chart of a text recognition method according to an embodiment of the present application;

[0017] FIG2 is a schematic diagram of the structure of the CNN layer in the improved DeBERTa model provided in an embodiment of the present application;

[0018] FIG3 is a schematic diagram of the structure of the improved DeBERTa model provided in an embodiment of the present application;

[0019] FIG4 is a flowchart of a text recognition method provided in an embodiment of the present application;

[0020] FIG5 is a schematic diagram of the structure of a text recognition device provided in an embodiment of the present application;

[0021] FIG6 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0022] FIG7 is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0024] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0025] Speech synthesis is a technology that converts text into speech. It provides a new way for users to interact with their mobile phones. Compared to visual information interaction, speech synthesis can convey information more conveniently and quickly in certain scenarios, such as driving or sports. Furthermore, speech synthesis helps users better understand information by simulating the intonation, tone, and emotion of language, providing a more realistic interactive experience.

[0026] In Chinese speech synthesis broadcast scenarios, because Chinese characters contain many polyphones, speech synthesis may mispronounce polyphones. This mispronunciation conveys incorrect information to users, leading to misunderstandings and reducing the reliability and accuracy of speech information transmission. Therefore, correcting mispronunciations is extremely important, especially for the recognition and correction of polyphonetic characters and other difficult words.

[0027] In the related art, in order for electronic devices to correctly read a text, they typically use dictionary rules and polyphonetic character models to predict the text's pronunciation. Polyphonetic character models predict pronunciation based on text encoding results. Text encoding involves converting natural language text into vectors for easier computer understanding and processing. Currently, there are two encoding methods: character encoding and sentence encoding. Character encoding involves encoding each character, considering only the content of the current character. While computationally efficient, it can lead to errors in polyphonetic character pronunciation prediction in complex and ambiguous scenarios due to contextual connections. Sentence encoding considers context and semantics, allowing for more accurate pronunciation prediction of polyphonetic characters. However, sentence encoding is computationally slow and requires a large corpus for training. Furthermore, existing sentence encoding methods capture contextual relationships within a sequence through a global self-attention mechanism. Each position in the input sequence is aware of information from all other positions in the sequence. However, this also makes it susceptible to influences from other positions when processing local information, making it difficult to extract fine-grained local features. Consequently, sentence encoding-based pronunciation prediction for polyphones in complex texts, such as long and difficult sentences, is inaccurate.

[0028] In response to the above problems, this application proposes a text recognition method by adding a convolution module, i.e., a convolution layer, to the DeBERTa (Decoding-enhanced BERT with disentangled attention) model, and extracting the local feature information of the text to be recognized based on the global text feature sequence of the text to be recognized through the convolution layer, thereby enhancing the ability to model the local information of the text sequence. The convolution network obtains local features by performing convolution operations on the sequence segments through local connections and convolution kernel weight sharing. Adding a new convolution module after the Transformer layer can make up for the shortcomings of the Transformer layer in local feature extraction, so that local information and global information can be better used to analyze the pronunciation of long and difficult sentences and complex texts, thereby improving the accuracy of pronunciation prediction for polyphones in complex texts such as long and difficult sentences.

[0029] It should be noted that DeBERTa, or BERT with De-Attention Decoding Enhancement, is an improved model based on the BERT (Bidirectional Encoder Representation from Transformers) model. BERT is the encoding of the bidirectional transformer.

[0030] It should be noted that the BERT model and the DeBERTa model are new, general-purpose language models that support various natural language processing (NLP) tasks. The DeBERTa model's encoder structure is a multi-layered Transformer stack. The Transformer is a model widely used in text and time series modeling tasks. It models the entire sequence and captures contextual relationships within the sequence through a global self-attention mechanism. Each position in the input sequence can pay attention to information from all other positions in the sequence.

[0031] The text recognition method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0032] FIG1 is a flow chart of a text recognition method provided in an embodiment of the present application. As shown in FIG1 , the text recognition method may include the following steps S201 to S204:

[0033] Step S201: The text recognition apparatus inputs a first text to be recognized into a first text encoding model.

[0034] The first text to be recognized includes a first polyphonetic character, and the first text encoding model includes a convolutional layer.

[0035] Optionally, in an embodiment of the present application, the above-mentioned first text encoding model may be a DeBERTa model including a CNN (Convolutional Neural Networks) layer.

[0036] It should be noted that the explanation of the DeBERTa model can be found in the above description and will not be repeated here.

[0037] Optionally, in an embodiment of the present application, the first text to be recognized may be text input by a user, or text already on a page, etc., and this embodiment of the present application does not limit this.

[0038] Optionally, in an embodiment of the present application, the first text to be recognized may include: at least one sentence or at least one paragraph.

[0039] Optionally, in an embodiment of the present application, the first text to be recognized may be a text that requires speech synthesis.

[0040] Optionally, in an embodiment of the present application, the first polyphonetic character may include one or more polyphonetic characters.

[0041] In the embodiment of the present application, the convolution layer includes a convolutional kernel.

[0042] Optionally, in an embodiment of the present application, the window length of the convolution window corresponding to the above-mentioned convolution kernel may be 3, and the moving step length of the convolution kernel may be 1.

[0043] It should be noted that the convolution window corresponding to the convolution kernel mentioned above refers to the window length when the convolution kernel performs the convolution operation based on the step size. For example, if the convolution kernel size is 3×1, the window length corresponding to each movement of the convolution kernel is 3.

[0044] Further optionally, the convolution kernel is used to extract local features of the first text to be recognized.

[0045] Further optionally, the above-mentioned convolution kernel may include at least one convolution kernel with a fixed size.

[0046] It should be noted that the above-mentioned convolution window, also known as convolution layer window, convolution window, etc., is used to extract feature information from input data.

[0047] Step S202: The text recognition device extracts a first text feature sequence corresponding to a first text to be recognized.

[0048] Optionally, in an embodiment of the present application, the text recognition device may input the first text to be recognized into the Transformer layer of the DeBERTa model, perform text feature extraction on the first text to be recognized, and obtain a first text feature sequence corresponding to the text to be recognized.

[0049] It should be noted that the above Transformer layer is a bidirectional long short-term memory network based on a multi-head self-attention mechanism, which includes two parts: encoding and decoding.

[0050] Optionally, in an embodiment of the present application, the above-mentioned first text feature sequence can be: an attention result obtained by the Transformer layer of the DeBERTa model based on the contextual semantic features of the first text to be recognized, and the attention result can be represented by H0.

[0051] Further optionally, when the input text is processed by the Transformer layer of the DeBERTa model, the content information and position information of the characters in the text are represented by two vectors respectively to calculate the attention weights between characters at different positions. Then, the weighted sum of each position in the character sequence is calculated using the attention weights, and different weights are assigned to different positions in the sequence to obtain the above-mentioned attention result.

[0052] Furthermore, each character in the text can be converted into a vector form of fixed dimension to represent the content information corresponding to each character in the text, and the position information of each character can be recorded using absolute position encoding.

[0053] It's important to note that in BERT, each character in the input text is converted into a fixed-dimensional vector to represent the content information corresponding to each character in the text, using absolute position encoding to record the position of each character. The DeBERTa model, on the other hand, considers the importance of the relative position of characters. For example, the words "depth" and "learning" are more closely related when they appear adjacent to each other in a sentence than when they appear separately at different positions. Therefore, the character content and position information are represented in two separate vectors to calculate attention weights.

[0054] The following explains the process of extracting text features from the first text to be recognized through the Transformer layer of the DeBERTa model through a specific embodiment.

[0055] For example, taking the processing of the i-th character in the text as an example, {H ij} represents the embedding vector of the i-th character, that is, converting the character into a vector of fixed dimension. i|j} represents the relative position of the i-th character and the j-th character, so the attention weight between the two characters can be expressed by the following formula:

[0056] From the above formula, we can see that the attention weight A between characters is i，j It can be decomposed into four parts: "content to content", "content to position", "position to content", and "position to position". Due to the use of relative position encoding, "position to position" itself has no meaning and is therefore omitted in subsequent calculations. Compared to the related art that only considers "content to content" and "content to position" when calculating attention weights through BERT, the embodiment of the present application calculates attention weights based on the DeBERTa model, taking into account the "position to content" information, and using position information to help the model better understand the semantic content in the sequence.

[0057] It should be noted that the above-mentioned "content to content" refers to the connection between the semantics of one character and another character, for example, the connection between the semantics of character A and the semantics of character B in the text to be recognized; the above-mentioned "content to position" refers to the connection between the content of one character and the position of another character, which can reflect the closeness of the connection between the characters; the above-mentioned "position to content" refers to the connection between the position of one character and the content of another character; the above-mentioned "position to position" refers to the relationship between the position of one character and the position of another character, for example, the position of character A in the text to be recognized and the distance between the position of character B in the text.

[0058] Combined with the above formula (1), the standard attention mechanism calculation formula is as follows:

[0059] Among them, H represents the input vector, Q, K and V are the content vectors obtained by mapping H, H0 is the output result of the self-attention mechanism, W q , W k , W v is a learnable mapping matrix, A is the attention weight matrix, N is the input vector length, and d is the dimension of the input vector.

[0060] It should be noted that H is a real matrix with dimension N×d, which can be expressed as H∈R N×d ;W v It is a real matrix with dimension d×d, which can be expressed as W v ∈R d×d ; A is a real matrix with dimension N×N, which can be expressed as A∈R N×N .

[0061] For the above formula (2), the method in formula (1) is used to further decompose it into formula (3). c ,K c ,V c Represented by the matrix W q,c ,W k,c ,W v,c The mapped content vector, Q r ,K r W q,r ,W k,r According to the mapped relative position vector, the formula is as follows:

[0062] in, is the attention matrix The elements in represent the attention weights between the i-th character and the j-th character. The calculation formula corresponding to the first three items in formula (1) is: Q c The i-th row vector of Q c The j-th row vector of , δ(i|j) represents the relative distance between character i and character j in the text, is the scale factor and d is the dimension of the input vector.

[0063] It should be noted that the attention weights can be used to calculate the weighted value of each position in the sequence, thereby helping the model better capture the correlation in the sequence and obtain an accurate text feature sequence.

[0064] For example, after the attention weight is calculated by the above formula (2), each element in the sequence is multiplied by the attention weight corresponding to the element to obtain the weighted value of each element in the sequence. That is to say, by assigning different weights to different positions in the sequence, the attention result H0 is obtained, thereby obtaining a more accurate text feature sequence.

[0065] Optionally, in an embodiment of the present application, the first text to be recognized may be a text obtained by concatenating multiple first texts. In other words, the multiple first texts may be text fragments in the first text to be recognized.

[0066] Optionally, in the embodiment of the present application, the first text feature sequence may include multiple text feature sequences, and one text feature sequence corresponds to a text segment in the first text to be recognized.

[0067] Further optionally, when the text obtained after the above splicing is processed by the Transformer layer of the DeBERTa model, the text recognition device inputs the first text to be recognized into the DeBERTa model, converts the first text to be recognized into a vector through the DeBERTa model, obtains the character embedding vector and type embedding vector corresponding to each text segment in the first text to be recognized, and uses the sum of the character embedding vector and the type embedding vector as the input of the Transformer layer to obtain the attention result H0.

[0068] Furthermore, the above character embedding vector is a vector expression of the character content; the type embedding vector is a vector obtained by manually marking each spliced ​​sentence and then converting the mark into a vector. The type embedding vector enables the model to distinguish different sentences.

[0069] It should be noted that the input vector is a concatenation of multiple sentences input into the model, and the different sentences in the concatenated sentences are marked by the type embedding vector, so that the model can distinguish different sentences.

[0070] Furthermore, the above H0 may include M text feature sequences, each sequence including up to N elements, that is, the length of each sequence may be N. Among them, one element may correspond to a character embedding vector, and the dimensions of each character embedding vector may be the same.

[0071] Furthermore, each text feature sequence includes a text feature sequence corresponding to a text segment in the first text to be recognized.

[0072] For example, the first text to be recognized is the text obtained by concatenating text 1, text 2 and text 3. After processing the text through the Transformer layer, the attention result H0 obtained includes the sequence corresponding to text 1, the sequence corresponding to text 2 and the sequence corresponding to text 3, where each sequence can include 8 elements, that is, the length of the sequence is 8.

[0073] It should be noted that the above values ​​of M and N are only examples and do not constitute a limitation to the present application. The specific values ​​of M and N can be determined according to actual needs.

[0074] Step S203: The text recognition device uses a convolution layer to perform segmented convolution processing on the first text feature sequence to obtain a second text feature sequence.

[0075] The second text feature sequence at least includes global semantic information of the first text to be recognized and local semantic information of a first text segment in the first text to be recognized, and the first text segment includes the first polyphonetic character.

[0076] Optionally, in an embodiment of the present application, the text recognition device may input the above-mentioned first text feature sequence into the CNN layer of the DeBERTa model, and perform segmented convolution processing on the first text feature sequence through the CNN layer to obtain a second text feature sequence.

[0077] For example, taking the first text feature sequence as the output vector H0 of the Transformer layer, when the text recognition device performs segmented convolution processing on the first text feature sequence through the CNN layer, it can use a fixed-size convolution kernel to perform segmented convolution processing on the sequence of the output vector H0. Specifically, the fixed-size convolution kernel is segmented and convolved with the output vector H0 to obtain local convolution results. After the calculation is completed, the convolution kernel is moved forward and convolved with the next segment of the sequence. The convolution operation is repeated throughout the entire sequence, and finally the overall convolution result is output.

[0078] The following is an example of the segmented convolution processing process in the embodiment of the present application with reference to the accompanying drawings.

[0079] Figure 2 is a schematic diagram of the structure of the CNN layer in the improved DeBERTa model provided in an embodiment of the present application. As shown in Figure 2, the above-mentioned DeBERTa model at least includes a CNN input layer, a convolution layer and an output layer, wherein the above-mentioned CNN input layer is used to input the above-mentioned first text feature sequence, that is, the output vector H0 of the Transformer layer, and a sequence of the output vector H0 includes eight elements of X1, X2, X3, X4, X5, X6, X7 and X8, each element represents a character embedding vector, and the dimension of each character embedding vector is 3. The above-mentioned convolution layer is used to perform segmented convolution on the above-mentioned first text feature sequence through a convolution kernel, and the above-mentioned output layer is used to output the convolution result.

[0080] It should be noted that each column in the input layer matrix in Figure 2 represents the dimension of the corresponding character embedding vector.

[0081] Referring to Figure 2 above, after the output vector H0 is input into the convolutional layer, a fixed-size convolution kernel is convolved with the output vector H0 in segments to obtain local convolution results. After the calculation is completed, the convolution kernel is moved forward and convolved with the next segment of the sequence. This convolution operation is repeated throughout the entire sequence, and the overall convolution result is finally output. For example, if the size of the convolution kernel is much smaller than the sequence length of the output vector H0, convolution can be performed segment by segment when using the convolution kernel with the vector.

[0082] For example, if the convolution kernel size is 3×1 and H0 is 42×1, then during the first convolution operation, the kernel is multiplied by the 1st, 2nd, and 3rd positions of H0 and the sum is calculated to obtain a convolution result. The kernel is then moved forward and multiplied by the 2nd, 3rd, and 4th positions of the sequence to obtain the corresponding convolution results. The final convolution result is then obtained by summing the convolution results. In this way, by controlling the kernel size in the convolution layer, the convolution operation can focus only on the content within a specified range, thereby obtaining accurate local semantic information.

[0083] Optionally, in an embodiment of the present application, the text recognition device may determine a text segment corresponding to the local semantic information based on the obtained local semantic information, thereby determining the semantic information of the text segment where the polyphone is located.

[0084] For example, taking the first text to be recognized as "Those who walk together with us are all fellow travelers" as an example, the global semantic information of the text obtained through the Transformer layer may be "we are walking", and the global semantic information of the text further obtained through the CNN layer may be "we are walking", and the local semantic information obtained may include "we are walking" and "are fellow travelers".

[0085] It can be understood that when predicting the pronunciation of polyphonic characters through the global semantic information of the text subsequently, the pronunciation of "行" in this text may be predicted as "xing2", resulting in poor accuracy. When predicting the pronunciation of polyphonic characters through local semantic information, the pronunciation of "行" at the first position can be predicted as "xing2" according to the semantic information of the text segment where "行" appears for the first time, i.e., "我们同行", and the pronunciation of "行" at the second position can be predicted as "hang2" according to the semantic information of the text segment where "行" appears at the second position.

[0086] Step S204: The text recognition device obtains the first encoding information corresponding to the first text to be recognized based on the above second text feature sequence.

[0087] Optionally, in the embodiment of the present application, the text recognition device may input the above second text feature sequence into the encoding module of the DeBERTa model to obtain the above first encoding information.

[0088] The encoding method provided by the embodiment of the present application will be explained below with reference to the accompanying drawings.

[0089] Figure 3 is a structural diagram of the improved DeBERTa model provided in an embodiment of the present application. As shown in Figure 3, the DeBERTa model at least includes: a Transformer layer, a CNN layer and an encoding module, namely, a Mask Decoder. As shown in Figure 3, the text recognition device splices a plurality of different texts and inputs them into the DeBERTa model as model input information, and obtains the character embedding vector and type embedding vector corresponding to the spliced ​​text, and then uses the sum of the character embedding vector and the type embedding vector as the input of the Transformer layer. After the text feature vector representing the global semantics corresponding to the spliced ​​text is obtained by combining the relative position embedding vector calculation of the Transformer layer, the text feature vector is input into the CNN layer for segmented convolution processing to obtain a text feature vector representing local semantics and global semantics, namely, a second text feature sequence. Finally, the text feature vector representing local semantics and global semantics is input into the encoding module to obtain the final sentence encoding result. In this way, by combining the Transformer with the CNN, the model not only obtains global semantic information from the Transformer, but also better models local sequence information through the CNN. This takes into account the possible strong local correlation of sequence semantic information, enabling the model to integrate global and local information when predicting polyphones. The output of the CNN output layer is passed through the Decoder to obtain the final sentence encoding result. In a test on an annotated validation set, the pronunciation prediction results based on character and sentence encoding were compared with the annotated pronunciation results. Statistical results show that the accuracy of polyphone prediction results based on sentence encoding generated by the improved DeBERTA is much higher than that of polyphone pronunciation results based on character encoding.

[0090] The text recognition method provided by the embodiment of the present application is as follows: the text recognition device inputs a first text to be recognized into a first text encoding model, the first text to be recognized contains a first polyphonetic character, and the first text encoding model includes a convolution layer. Then, the first text feature sequence corresponding to the first text to be recognized is extracted, and the convolution layer is used to perform segmented convolution processing on the above-mentioned first text feature sequence to obtain a second text feature sequence. The above-mentioned second text feature sequence at least contains the global semantic information of the first text to be recognized and the local semantic information of the first text segment in the first text to be recognized, and the above-mentioned first text segment contains the first polyphonetic character. Finally, based on the above-mentioned second text feature sequence, the first encoding information corresponding to the first text to be recognized is obtained. Through this method, the text recognition device performs a convolution operation on the global text feature sequence of the text through the convolution layer in the first text encoding model to obtain a text feature sequence containing global semantic information and local semantic information, and obtains a more accurate recognition result based on the text feature sequence, thereby enhancing the ability to model the local information of the text sequence, and can make up for the shortcomings of the existing model in extracting local feature information, so as to more accurately analyze the pronunciation of polyphones in long and difficult sentences and complex texts, thereby improving the accuracy of pronunciation prediction of polyphones in the text.

[0091] Optionally, in the embodiment of the present application, the above step S203 may include the following steps S203a and S203b:

[0092] Step S203a: The text recognition apparatus performs segmented convolution processing on the first text feature sequence through the convolution window corresponding to the convolution kernel in the convolution layer to obtain N sequence segments.

[0093] The text feature sequence corresponding to any sequence segment includes the local semantic information of the text segment corresponding to any sequence segment.

[0094] Optionally, in an embodiment of the present application, the second text feature sequence is composed of the N sequence segments mentioned above.

[0095] Optionally, in an embodiment of the present application, the first text feature sequence may include multiple text feature sequences, and each of the text feature sequences includes at least one text feature element arranged in sequence.

[0096] It should be noted that the explanation of the convolutional layer can be found above and will not be repeated here.

[0097] Optionally, in an embodiment of the present application, the text recognition device can select at least two feature elements in a first text feature sequence for convolution processing each time according to the window length of the convolution window, and move the convolution window according to the convolution step size to select feature elements for the next convolution processing.

[0098] Optionally, in an embodiment of the present application, the number of text feature elements contained in a sequence segment is determined based on the window length of the convolution window.

[0099] Exemplarily, in combination with the above embodiment, taking the text feature sequence including eight elements X1, X2, X3, X4, X5, X6, X7 and X8, the size of the convolution kernel is 3*1, and the convolution step is 1 as an example, when performing segmented convolution processing, the convolution kernel is multiplied and summed with the element X1 at the first position of the text feature sequence, the element X2 at the second position, and the element X3 at the third position to obtain a convolution result R1. Then, the convolution kernel is moved forward, and the convolution kernel is multiplied and summed with the elements at the second, third, and fourth positions of the sequence (i.e., X2, X3, X4) to obtain the corresponding convolution result R1. Similarly, the convolution kernel is multiplied and summed with the elements at the sixth, seventh, and eighth positions of the sequence (i.e., X6, X7, X8) to obtain the corresponding convolution result R6. Then, based on the sum of each convolution result, the final convolution result is obtained.

[0100] In this way, by controlling the size of the convolution kernel in the convolution layer, we can focus only on the content within the specified range when performing convolution operations, thereby obtaining accurate local semantic information.

[0101] Optionally, in the embodiment of the present application, before the above step S201, the text recognition method provided in the embodiment of the present application further includes the following steps C1 to C3:

[0102] Step C1: The text recognition device receives a first input from a user.

[0103] Step C2: The text recognition device stores the to-be-recognized text corresponding to the first input into a first file in response to the first input.

[0104] Step C3: When the number of the to-be-recognized texts stored in the first file exceeds a first threshold, the text recognition device synthesizes the to-be-recognized texts stored in the first file into the first to-be-recognized text.

[0105] Optionally, in the embodiment of the present application, the first input may be any feasible input such as a user's touch input, voice input, or gesture input, and the embodiment of the present application does not limit this.

[0106] Furthermore, the touch input may be a click input, a long press input, or a sliding input by the user.

[0107] Optionally, in an embodiment of the present application, the first threshold may be 50, 100, or 150, etc., which is not limited in the embodiment of the present application.

[0108] Optionally, in the embodiments of the present application, the text recognition device may splice the text to be recognized stored in the above first document to obtain the above first text to be recognized.

[0109] Exemplarily, taking the first input as the user's click input on the text to be recognized, after the user clicks on the text that needs to be synthesized by voice displayed on the interface of the electronic device, the text recognition device obtains the text and automatically writes the text into the fixed directory file D. When the number of texts in the directory file D reaches a preset number, the pronunciation correction system will be automatically started, and the accumulated texts will be spliced to obtain the above first text to be recognized, and the first text to be recognized will be input into the first text encoding model for processing.

[0110] In the text recognition method provided by the embodiments of the present application, when the user requests to read aloud the text on the mobile phone side, the encoding device can store the text requested to be read aloud each time into a file, and after the number of texts exceeds a certain threshold, input the text into the first text encoding model for processing, so that the first text encoding model can accurately perform semantic recognition and encoding on the text that the user usually needs to read aloud, obtain more accurate encoding information, and thus can more accurately predict the pronunciation of polyphonic characters in the text based on the encoding subsequently.

[0111] Optionally, in the embodiments of the present application, after the above step S204, the text recognition method provided by the embodiments of the present application further includes the following step S205:

[0112] Step S205: The text recognition device inputs the above first encoding information into the first polyphonic character model to predict the pronunciation of the first polyphonic character, and obtains the first pronunciation prediction information of the first polyphonic character.

[0113] Optionally, in the embodiments of the present application, the above first polyphonic character model may be obtained by training a bidirectional long short-term memory neural network and a linear neural network.

[0114] Optionally, in the embodiments of the present application, after the text recognition device inputs the above first encoding information into the first polyphonic character model, based on the inference of the polyphonic character model of sentence encoding, the corresponding polyphonic character pronunciation prediction result is obtained.

[0115] Taking an example, taking the first text to be recognized including "差点着了他的道" as an example, after obtaining the encoding information of the text based on the above first text encoding model, input the encoding information into the first polyphonic character model. Through the first polyphonic character model, the probability that the polyphonic character "着" in the text is pronounced as "zhao1" is 85%, which is greater than the probability of being pronounced as "zhe5", then the pronunciation prediction information "zhao1" of the text is output. It should be noted that in the pronunciation prediction information "zhao1", "zhao" represents the pinyin, and "1" represents the tone, that is, the first tone.

[0116] Further optionally, in the embodiment of the present application, after the above step S205, the text recognition method provided in the embodiment of the present application further includes the following step S206:

[0117] Step S206: The text recognition device takes the first text to be recognized as a preset text, and stores it in a text pronunciation database in association with the first pronunciation prediction information.

[0118] The text pronunciation library contains a plurality of preset texts and pronunciation prediction information corresponding to each preset text, and each preset text contains at least one polyphonic character.

[0119] For ease of understanding, the above-mentioned text pronunciation library can be called an error correction intervention library.

[0120] Optionally, in an embodiment of the present application, the data format of the data stored in the above-mentioned text pronunciation library is: <text>|<text with the pinyin of the target polyphonetic character>. For example, the data stored in the text pronunciation library includes: "He is dressed very ordinary|He is dressed (zhuo2) very ordinary", "He almost got into trouble|He almost got into trouble", "The house is on fire|The house is on fire".

[0121] In this way, by performing pronunciation prediction on the accumulated multiple texts to be recognized based on the sentence encoding results, a more accurate pronunciation prediction result can be obtained, and the more accurate pronunciation prediction information obtained is pre-stored in the text pronunciation library, so that when online speech synthesis is performed subsequently, the pronunciation prediction result obtained after performing pronunciation prediction based on the character encoding result can be corrected through the text pronunciation library, thereby ensuring the high accuracy of pronunciation prediction while ensuring the processing speed.

[0122] Further optionally, in an embodiment of the present application, the text recognition device obtains the first pronunciation prediction information of the first polyphonetic character, and takes the first text to be recognized as the preset text, and stores it in a text pronunciation library in association with the first pronunciation prediction information. Then, the incorrect pronunciation prediction information in the text pronunciation library can be obtained, and the incorrect pronunciation prediction information can be added to the training data set of the above-mentioned first polyphonetic character model to train the first polyphonetic character model through the incorrect pronunciation prediction information, thereby iteratively updating and optimizing the model.

[0123] Optionally, the text recognition device can check the pronunciation prediction information corresponding to the preset text stored in the text pronunciation library. When it is checked that the pronunciation prediction information corresponding to the preset text is erroneous information, the erroneous pronunciation prediction information is added to the training data set of the above-mentioned first polyphonetic character model to train the first polyphonetic character model through the erroneous pronunciation prediction information.

[0124] Exemplarily, the text recognition device adds erroneous pronunciation prediction information to the training set, iteratively trains the polyphone model through the accumulated data set, repairs examples of incorrect pronunciation in the original online reading system, enhances the generalization ability of the model, and thus periodically improves the pronunciation prediction accuracy of the polyphone model.

[0125] Optionally, in an embodiment of the present application, an embodiment of the present application provides a pronunciation prediction method, which may include the following steps A1 to A4:

[0126] Step A1: The text recognition device inputs the second text to be recognized into the second text encoding model, and extracts a third text feature sequence corresponding to the second text to be recognized.

[0127] The second text to be recognized includes a second polyphonetic character, and the second text encoding model is different from the first text encoding model.

[0128] Step A2: The text recognition device obtains second coding information corresponding to the second text to be recognized based on the third text feature sequence.

[0129] Step A3: The text recognition device inputs the second encoding information into a second polyphonetic character model, performs pronunciation prediction on the second polyphonetic character, and obtains third pronunciation prediction information of the second polyphonetic character.

[0130] Step A4: When the target preset text exists in the text pronunciation database, the text recognition device updates the third pronunciation prediction information to the fourth pronunciation prediction information corresponding to the target preset text.

[0131] The target preset text is the text in the text pronunciation library that matches the second text to be recognized.

[0132] Optionally, in an embodiment of the present application, the second text encoding model may be a character encoding model.

[0133] Optionally, in an embodiment of the present application, the polyphone model can be obtained by training a bidirectional long short-term memory neural network and a linear neural network.

[0134] For example, when a user needs to read aloud a text, after the user clicks on the text, the encoding device obtains the text request and sends the text request to the speech synthesis engine end in real time. The speech synthesis engine end encodes the text using a word encoding method, and the encoded result is input into the second polyphonetic character model.

[0135] Optionally, in an embodiment of the present application, when performing word encoding processing, the speech synthesis engine uses a Focal Loss function to mitigate the impact of data imbalance, thereby improving the pronunciation accuracy of polyphones. The formula is as follows:

[0136] Focal Loss = -α t (1-p t ) γ log (p t ) (4)

[0137] Among them, the imbalance of the number of positive and negative samples is suppressed by the weight factor α, and the value of the weight factor α can be {0,1}, that is, α∈[0,1]; (1-p t ) γ It is a modulation factor that helps distinguish between difficult and easy samples. The Focal Loss function improves the problem of sample class imbalance.

[0138] It should be noted that due to the different usage frequencies and scenarios of polyphonetic characters, it is difficult to maintain a balance in the sample data for each pronunciation in the training data. For example, the amount of data for the pronunciation "ba3" in the training set is ten times that of "ba4", which causes the model prediction results to tend to predict "ba3" and mispredict some "ba4" pronunciations.

[0139] Optionally, in an embodiment of the present application, the model structure and loss function of the second polyphonetic character model are the same as those of the first polyphonetic character model.

[0140] Optionally, in an embodiment of the present application, when the third pronunciation prediction information of the second text to be recognized is obtained based on the character encoding, the second text to be recognized is matched with the preset text in the text pronunciation library. When there is a target preset text matching the above-mentioned second text to be recognized in the text pronunciation library, the above-mentioned third pronunciation prediction information is updated with the fourth pronunciation prediction information corresponding to the target preset text in the text pronunciation library.

[0141] Further optionally, when there is a target preset text matching the above-mentioned second text to be recognized in the text pronunciation library, it can be determined whether the third pronunciation prediction information of the second text to be recognized is the same as the fourth pronunciation prediction information corresponding to the target preset text. If they are the same, the above-mentioned third pronunciation prediction information is output; if they are different, the third pronunciation prediction information is updated to the fourth pronunciation prediction information.

[0142] For example, taking the second text to be recognized as "His clothes are very ordinary" as an example, assuming that the pronunciation prediction result of the second text to be recognized based on the character encoding is "His clothes (zhe5) are very ordinary", the second text to be recognized is matched with the preset text in the text pronunciation library, and the pronunciation prediction result corresponding to the matching text is "His clothes (zhuo2) are very ordinary", then the pronunciation prediction result of the second text to be recognized is corrected to "His clothes (zhuo2) are very ordinary".

[0143] It should be noted that the "zhe" in the pronunciation "zhe5" is the pinyin, and the "5" is the tone, which means the light tone. Similarly, the "zhuo" in the pronunciation "zhuo2" is the pinyin, and the "2" is the tone, which means the second tone.

[0144] It should be noted that the above steps A1 to A4 may be performed before step S201 or after step S201.

[0145] The text recognition method provided in the embodiment of the present application combines the advantages of sentence encoding and character encoding. While ensuring low-latency voice reading service, it can correct incorrect pronunciations in a timely and rapid manner. According to the error correction data of the error correction system, the polyphone model is periodically iterated to improve the accuracy of polyphone pronunciation, greatly reducing the manpower and time costs required for manual error checking and correction.

[0146] Further optionally, in the embodiment of the present application, after the above step A3, the text recognition method provided in the embodiment of the present application further includes the following step B1:

[0147] Step B1: When the text recognition device detects that the third pronunciation prediction information is erroneous pronunciation prediction information, the text recognition device adds the third pronunciation prediction information to the training data set.

[0148] The training data set is used to train the second polyphonetic character model.

[0149] Optionally, in an embodiment of the present application, when the third pronunciation prediction information of the second text to be recognized is different from the fourth pronunciation prediction information matched from the text pronunciation library, the third pronunciation prediction information is added as erroneous pronunciation prediction information to the training set of the above-mentioned second polyphonetic character model to perform model iterative update and optimization of the second polyphonetic character model.

[0150] Exemplarily, the text recognition device adds the erroneous pronunciation prediction information obtained based on the character encoding to the training set of the polyphonetic character model that predicts the pronunciation based on the encoding result of the character encoding, and trains the polyphonetic character model based on the accumulated erroneous pronunciation prediction information to repair the examples of incorrect pronunciation in the original online reading system, enhance the generalization ability of the model, and thus periodically improve the pronunciation prediction accuracy of the polyphonetic character model.

[0151] The text recognition method provided in the embodiment of the present application is described below through specific examples in conjunction with a flowchart.

[0152] FIG4 is a flowchart of a text recognition method provided in an embodiment of the present application. As shown in FIG4 , the text recognition method may include the following steps 101 to 109:

[0153] Step 101: Encode the text requested for recognition by the user using a second text encoding model.

[0154] For example, when a user clicks on a text to read it aloud, the text request is sent to the speech synthesis engine in real time, the text is encoded, and the encoded result is input into the polyphone model of step 102.

[0155] Step 102: Obtain a pronunciation prediction result through a second polyphonetic character model.

[0156] It should be noted that the explanation of this step can be found in the above description and will not be repeated here.

[0157] Step 103: Match the query target text in the intervention database.

[0158] For example, if the text exists in several pre-libraries and the matching result is True, the result corresponding to the dry library is used and the process jumps to step 104 ; if the text does not exist in several pre-libraries, the matching result is False and the process jumps to step 105 .

[0159] Step 104: Pronunciation result intervention and repair.

[0160] For example, if the matching result in step 104 is True, the corresponding text pronunciation result in the intervention library is used, and the process jumps to step 105 to complete the correction of the incorrect pronunciation.

[0161] Step 105: Output the final pronunciation.

[0162] Step 106: Text accumulation.

[0163] Exemplarily, the pronunciation correction system operates in non-real time, and the text requested to be read aloud by the user on the mobile phone is automatically written into a fixed directory file D. When the number of requested texts reaches a preset number, the pronunciation correction system is automatically started and the accumulated text is sent to step 107.

[0164] Step 107: Encode the text sentence based on the large model DeBERTa.

[0165] Exemplarily, the text accumulated in step 106 is read, and the text sentence encoding results are output in batches using the pre-trained large model DeBERTa, which is the first text encoding model mentioned above.

[0166] Step 108: Obtain a pronunciation prediction result based on the second phonetic character model of the sentence code.

[0167] Step 109a: iteratively update the first polyphonetic character model.

[0168] Step 109b: iteratively update the second polyphonetic character model.

[0169] It should be noted that the explanation of this embodiment can be found above and will not be repeated here.

[0170] The text recognition method provided in the embodiment of the present application, on the one hand, can automatically correct the pronunciation of polyphonetic characters in online speech synthesis through the automatic error correction system for polyphonetic characters, and the sentence encoding generated by the improved DeBERTa pre-training model can better express and capture context and semantic information, ensuring a high accuracy rate of pronunciation. On the other hand, the polyphonetic character model trained by the pre-trained large model sentence encoding guides the iterative update of the online speech synthesis polyphonetic character model, thereby improving the pronunciation accuracy of the online polyphonetic character model itself. On the other hand, error correction through the automatic error correction system avoids the high time and labor costs required for manual error correction, while improving the error correction efficiency.

[0171] The text recognition method provided in the embodiment of the present application can be executed by a text recognition device. In the embodiment of the present application, the text recognition device provided in the embodiment of the present application is described by taking the text recognition method performed by the text recognition device as an example.

[0172] FIG5 is a schematic diagram of the structure of a text recognition device provided in an embodiment of the present application. As shown in FIG5 , the text recognition device 500 includes an execution module 501 and a processing module 502, wherein:

[0173] The execution module 501 is configured to input a first text to be recognized into a first text encoding model, wherein the first text to be recognized includes a first polyphonetic character, and the first text encoding model includes a convolutional layer;

[0174] The processing module 502 is used to extract a first text feature sequence corresponding to the first text to be recognized;

[0175] The processing module 502 is further configured to perform segmented convolution processing on the first text feature sequence using the convolution layer to obtain a second text feature sequence, wherein the second text feature sequence includes at least global semantic information of the first text to be recognized and local semantic information of a first text segment in the first text to be recognized, wherein the first text segment includes the first polyphonetic character;

[0176] The processing module 502 is further configured to obtain first encoding information corresponding to the first text to be recognized based on the second text feature sequence, and obtain a recognition result of the first text to be recognized based on the first encoding information.

[0177] Optionally, in an embodiment of the present application, the above-mentioned processing module is specifically used to perform segmented convolution processing on the above-mentioned first text feature sequence through the convolution window corresponding to the convolution kernel in the above-mentioned convolution layer to obtain text feature sequences corresponding to N sequence fragments, the number of text feature elements contained in the above-mentioned sequence fragment is determined based on the window length of the above-mentioned convolution window, and the text feature sequence corresponding to any sequence fragment contains the local semantic information of the text fragment corresponding to any of the above-mentioned sequence fragments; wherein, the above-mentioned second text feature sequence is composed of the above-mentioned N sequence fragments.

[0178] Optionally, in an embodiment of the present application, the processing module is specifically configured to input the first coding information into a first polyphonetic character model, perform pronunciation prediction on the first polyphonetic character, and obtain first pronunciation prediction information of the first polyphonetic character.

[0179] Optionally, in the embodiment of the present application, the above-mentioned device further includes: a storage module;

[0180] The storage module is configured to, after the processing module inputs the first coding information into a first polyphonetic character model, performs pronunciation prediction on the first polyphonetic character, and obtains first pronunciation prediction information of the first polyphonetic character, treat the first to-be-recognized text as a preset text, and store the information in a text pronunciation database in association with the first pronunciation prediction information;

[0181] The text pronunciation library includes a plurality of preset texts and pronunciation prediction information corresponding to each preset text, and each of the preset texts includes at least one polyphonic character.

[0182] Optionally, in an embodiment of the present application, the above-mentioned processing module is also used to input the second text to be recognized into a second text encoding model, extract a third text feature sequence corresponding to the above-mentioned second text to be recognized, the second text to be recognized contains a second polyphonetic character, and the above-mentioned second text encoding model is different from the above-mentioned first text encoding model; the above-mentioned processing module is also used to obtain the second encoding information corresponding to the above-mentioned second text to be recognized based on the above-mentioned third text feature sequence; the above-mentioned processing module is also used to input the above-mentioned second encoding information into a second polyphonetic character model, perform pronunciation prediction on the above-mentioned second polyphonetic character, and obtain the third pronunciation prediction information of the above-mentioned second polyphonetic character; the above-mentioned device also includes: an updating module; the above-mentioned updating module is used to update the above-mentioned third pronunciation prediction information to the fourth pronunciation prediction information corresponding to the above-mentioned target preset text when there is a target preset text in the above-mentioned text pronunciation library; wherein the above-mentioned target preset text is the text in the above-mentioned text pronunciation library that matches the above-mentioned second text to be recognized.

[0183] Optionally, in an embodiment of the present application, the above-mentioned processing module is also used to add the above-mentioned third pronunciation prediction information to a training data set when it is detected that the above-mentioned third pronunciation prediction information is erroneous pronunciation prediction information; wherein the above-mentioned training data set is used to train the above-mentioned second polyphonetic character model.

[0184] Optionally, in an embodiment of the present application, the above-mentioned device also includes: a receiving module and a storage module, wherein: the above-mentioned receiving module is used to receive the user's first input; the above-mentioned storage module is used to store the text to be recognized corresponding to the above-mentioned first input in the first file in response to the above-mentioned first input received by the above-mentioned receiving module; the above-mentioned processing module is also used to synthesize the text to be recognized stored in the above-mentioned first file into the above-mentioned first text to be input when the number of text to be recognized stored in the above-mentioned first file exceeds a first threshold.

[0185] The text recognition device provided in an embodiment of the present application inputs a first text to be recognized into a first text encoding model, where the first text to be recognized contains a first polyphonetic character. The first text encoding model includes a convolutional layer. Then, a first text feature sequence corresponding to the first text to be recognized is extracted, and the first text feature sequence is subjected to segmented convolution processing using the convolutional layer to obtain a second text feature sequence. The second text feature sequence at least contains global semantic information of the first text to be recognized and local semantic information of a first text segment in the first text to be recognized, where the first text segment contains a first polyphonetic character. Finally, based on the second text feature sequence, first encoding information corresponding to the first text to be recognized is obtained, and based on the first encoding information, a recognition result of the first text to be recognized is obtained. Through this method, the text recognition device performs a convolution operation on the global text feature sequence of the text through the convolution layer in the first text encoding model to obtain a text feature sequence containing global semantic information and local semantic information, and obtains a more accurate recognition result based on the text feature sequence, thereby enhancing the ability to model the local information of the text sequence, and can make up for the shortcomings of the existing model in extracting local feature information, so as to more accurately analyze the pronunciation of polyphones in long and difficult sentences and complex texts, thereby improving the accuracy of pronunciation prediction of polyphones in the text.

[0186] The text recognition device in the embodiment of the present application can be an electronic device or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices other than a terminal. For example, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a mobile Internet device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), etc. It can also be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., and the embodiment of the present application does not specifically limit it.

[0187] The text recognition device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0188] The text recognition device provided in the embodiment of the present application can implement each process implemented in the method embodiments of Figures 1 to 4. To avoid repetition, they will not be described here.

[0189] Optionally, as shown in Figure 6, an embodiment of the present application also provides an electronic device 600, including a processor 601 and a memory 602, and the memory 602 stores a program or instruction that can be run on the above-mentioned processor 601. When the program or instruction is executed by the processor 601, the various steps of the above-mentioned text recognition method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0190] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0191] FIG7 is a schematic diagram of the hardware structure of an electronic device implementing an embodiment of the present application.

[0192] The electronic device 1000 includes but is not limited to: a radio frequency unit 1001, a network module 1002, an audio output unit 1003, an input unit 100, a sensor 1005, a display unit 1006, a user input unit 1007, an interface unit 1008, a memory 1009, and a processor 1010.

[0193] Those skilled in the art will appreciate that electronic device 1000 may also include a power source (e.g., a battery) to power various components. This power source may be logically connected to processor 1010 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The electronic device structure shown in FIG7 does not limit the electronic device. The electronic device may include more or fewer components than shown, or may combine certain components or arrange the components differently, which will not be further described here.

[0194] The processor 1010 is configured to input a first text to be recognized into a first text encoding model, wherein the first text to be recognized includes a first polyphonetic character, and the first text encoding model includes a convolutional layer.

[0195] The processor 1010 is configured to extract a first text feature sequence corresponding to a first text to be recognized;

[0196] The processor 1010 is further configured to perform segmented convolution processing on the first text feature sequence using the convolution layer to obtain a second text feature sequence, wherein the second text feature sequence includes at least global semantic information of the first text to be recognized and local semantic information of a first text segment in the first text to be recognized, wherein the first text segment includes the first polyphonetic character;

[0197] The processor 1010 is further configured to obtain first encoding information corresponding to the first text to be recognized based on the second text feature sequence, and obtain a recognition result of the first text to be recognized based on the first encoding information.

[0198] Optionally, in an embodiment of the present application, the above-mentioned processor 1010 is specifically used to perform segmented convolution processing on the above-mentioned text feature sequence through the convolution window corresponding to the convolution kernel in the above-mentioned convolution layer to obtain text feature sequences corresponding to N sequence fragments, and the text feature sequence corresponding to any sequence fragment contains the local semantic information of the text fragment corresponding to any of the above-mentioned sequence fragments; wherein the above-mentioned second text feature sequence is composed of the above-mentioned N sequence fragments.

[0199] Optionally, in an embodiment of the present application, the processor 1010 is specifically configured to input the first coding information into a first polyphonetic character model, perform pronunciation prediction on the first polyphonetic character, and obtain first pronunciation prediction information of the first polyphonetic character.

[0200] Optionally, in an embodiment of the present application, the memory 1009 is configured to, after inputting the first encoding information into a first polyphonetic character model, perform pronunciation prediction on the first polyphonetic character, and obtain first pronunciation prediction information of the first polyphonetic character, store the first to-be-recognized text as a preset text in association with the first pronunciation prediction information in a text pronunciation database;

[0201] The text pronunciation library includes a plurality of preset texts and pronunciation prediction information corresponding to each preset text, and each of the preset texts includes at least one polyphonic character.

[0202] Optionally, in an embodiment of the present application, the processor 1010 is further used to input the second text to be recognized into a second text encoding model, extract a third text feature sequence corresponding to the second text to be recognized, the second text to be recognized includes a second polyphonetic character, and the second text encoding model is different from the first text encoding model; the processor 1010 is further used to obtain second encoding information corresponding to the second text to be recognized based on the third text feature sequence; the processor 1010 is further used to input the second encoding information into a second polyphonetic character model, perform pronunciation prediction on the second polyphonetic character, and obtain third pronunciation prediction information of the second polyphonetic character; the device further includes: an update module; the update module is used to update the third pronunciation prediction information to the fourth pronunciation prediction information corresponding to the target preset text when there is a target preset text in the text pronunciation library; wherein the target preset text is the text in the text pronunciation library that matches the second text to be recognized.

[0203] Optionally, in an embodiment of the present application, the processor 1010 is further configured to add the third pronunciation prediction information to a training data set when it is detected that the third pronunciation prediction information is erroneous pronunciation prediction information; wherein the training data set is used to train the second polyphonetic character model.

[0204] Optionally, in an embodiment of the present application, the above-mentioned user input unit 1007 is used to receive the user's first input; the above-mentioned memory 1009 is used to store the text to be recognized corresponding to the above-mentioned first input in the first file in response to the above-mentioned first input received by the above-mentioned user input unit 1007; the above-mentioned processor 1010 is also used to synthesize the text to be recognized stored in the above-mentioned first file into the above-mentioned first text to be input when the number of texts to be recognized stored in the above-mentioned first file exceeds a first threshold.

[0205] An electronic device provided by an embodiment of the present application inputs a first text to be recognized into a first text encoding model, wherein the first text to be recognized contains a first polyphonetic character, and the first text encoding model includes a convolutional layer. Then, a first text feature sequence corresponding to the first text to be recognized is extracted, and the first text feature sequence is subjected to segmented convolution processing using the convolutional layer to obtain a second text feature sequence. The second text feature sequence at least contains global semantic information of the first text to be recognized and local semantic information of a first text segment in the first text to be recognized, wherein the first text segment contains a first polyphonetic character. Finally, based on the second text feature sequence, first encoding information corresponding to the first text to be recognized is obtained, and based on the first encoding information, a recognition result of the first text to be recognized is obtained. Through this method, the text recognition device performs a convolution operation on the global text feature sequence of the text through the convolution layer in the first text encoding model to obtain a text feature sequence containing global semantic information and local semantic information, and obtains a more accurate recognition result based on the text feature sequence, thereby enhancing the ability to model the local information of the text sequence, and can make up for the shortcomings of the existing model in extracting local feature information, so as to more accurately analyze the pronunciation of polyphones in long and difficult sentences and complex texts, thereby improving the accuracy of pronunciation prediction of polyphones in the text.

[0206] It should be understood that in an embodiment of the present application, the input unit 1004 may include a graphics processing unit (GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 1006 may include a display panel 1061, and the display panel 1061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 1007 includes a touch panel 1071 and at least one of other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Other input devices 1072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and an operating stick, which will not be repeated here.

[0207] The memory 1009 can be used to store software programs and various data. The memory 1009 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, applications or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 1009 may include a volatile memory or a non-volatile memory, or the memory 1009 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM), and a direct memory bus random access memory (DRRAM). The memory 1009 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0208] Processor 1010 may include one or more processing units. Optionally, processor 1010 integrates an application processor and a modem processor. The application processor primarily handles operations related to the operating system, user interface, and application programs, while the modem processor primarily processes wireless communication signals, such as a baseband processor. It is understood that the modem processor may not be integrated into processor 1010.

[0209] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned text recognition method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0210] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0211] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned text recognition method embodiment and achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0212] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0213] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned text recognition method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0214] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0215] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0216] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. A text recognition method, the method comprising: Inputting a first text to be recognized into a first text encoding model, wherein the first text to be recognized includes a first polyphonetic character, and the first text encoding model includes a convolutional layer; Extracting a first text feature sequence corresponding to a first text to be recognized, wherein the first text feature sequence includes semantic information of the first text to be recognized; Using the convolution layer to perform segmented convolution processing on the first text feature sequence to obtain a second text feature sequence, wherein the second text feature sequence at least includes global semantic information of the first text to be recognized and local semantic information of a first text segment in the first text to be recognized, and the first text segment includes the first polyphonetic character; Based on the second text feature sequence, first encoding information corresponding to the first text to be recognized is obtained, and based on the first encoding information, a recognition result of the first text to be recognized is obtained.

2. The method according to claim 1, wherein: The step of using the convolution layer to perform convolution processing on the first text feature sequence to obtain a second text feature sequence includes: Performing segmented convolution processing on the first text feature sequence through a convolution window corresponding to a convolution kernel in the convolution layer to obtain N sequence segments, wherein the text feature sequence corresponding to any sequence segment contains local semantic information of the text segment corresponding to any sequence segment; The second text feature sequence is composed of the N sequence segments.

3. The method according to claim 1 or 2, wherein: The obtaining a recognition result of the first to-be-recognized text based on the first encoding information includes: The first coding information is input into a first polyphonetic character model, and pronunciation prediction is performed on the first polyphonetic character to obtain first pronunciation prediction information of the first polyphonetic character.

4. The method according to claim 3, wherein: After inputting the first coding information into a first polyphonetic character model, performing pronunciation prediction on the first polyphonetic character, and obtaining first pronunciation prediction information of the first polyphonetic character, the method further includes: The first text to be recognized is used as a preset text, and is associated with the first pronunciation prediction information and stored in a text pronunciation library; The text pronunciation library includes a plurality of preset texts and pronunciation prediction information corresponding to each preset text, and each preset text includes at least one polyphonic character.

5. The method according to claim 4, wherein: The method further comprises: Inputting a second text to be recognized into a second text encoding model, extracting a third text feature sequence corresponding to the second text to be recognized, wherein the second text to be recognized includes a second polyphonetic character, and the second text encoding model is different from the first text encoding model; Based on the third text feature sequence, obtaining second encoding information corresponding to the second text to be recognized; Inputting the second encoding information into a second polyphonetic character model, performing pronunciation prediction on the second polyphonetic character, and obtaining third pronunciation prediction information of the second polyphonetic character; When there is a target preset text in the text pronunciation library, the third pronunciation prediction information is updated to the fourth pronunciation prediction information corresponding to the target preset text; wherein the target The preset text is the text in the text pronunciation library that matches the second text to be recognized.

6. The method according to claim 5, wherein: After obtaining the third pronunciation prediction information of the second polyphonetic character based on the second encoding information, the method further includes: In the case where it is detected that the third pronunciation prediction information is erroneous pronunciation prediction information, adding the third pronunciation prediction information to the training data set; The training data set is used to train the second polyphonetic character model.

7. The method according to claim 1, wherein: Before inputting the first text to be recognized into the text encoding model, the method further includes: receiving a first input from a user; In response to the first input, storing the to-be-recognized text corresponding to the first input into a first file; When the number of the texts to be recognized stored in the first file exceeds a first threshold, the texts to be recognized stored in the first file are synthesized into the first text to be input.

8. A text recognition device, comprising: An execution module and a processing module, wherein: The execution module is used to input a first text to be recognized into a first text encoding model, wherein the first text to be recognized includes a first polyphonetic character, and the first text encoding model includes a convolutional layer; The processing module is used to extract a first text feature sequence corresponding to the first text to be recognized; The processing module is further used to perform segmented convolution processing on the first text feature sequence using the convolution layer to obtain a second text feature sequence, wherein the second text feature sequence at least includes global semantic information of the first text to be recognized and local semantic information of a first text segment in the first text to be recognized, and the first text segment includes the first polyphonetic character; The processing module is further configured to obtain first encoding information corresponding to the first text to be recognized based on the second text feature sequence, and obtain a recognition result of the first text to be recognized based on the first encoding information.

9. The device according to claim 8, wherein: The processing module is specifically used to perform segmented convolution processing on the first text feature sequence through a convolution window corresponding to a convolution kernel in the convolution layer to obtain N sequence segments, wherein the text feature sequence corresponding to any sequence segment contains local semantic information of the text segment corresponding to any sequence segment; The second text feature sequence is composed of the N sequence segments.

10. The device according to claim 8 or 9, wherein: The processing module is specifically used to input the first coding information into a first polyphonetic character model, perform pronunciation prediction on the first polyphonetic character, and obtain first pronunciation prediction information of the first polyphonetic character.

11. The device according to claim 10, wherein: The device also includes: a storage module; The storage module is used for, after the processing module inputs the first coding information into a first polyphone model, predicting the pronunciation of the first polyphone, and obtaining first pronunciation prediction information of the first polyphone, taking the first text to be recognized as a preset text, and storing it in a text pronunciation library in association with the first pronunciation prediction information; The text pronunciation library includes a plurality of preset texts and pronunciation prediction information corresponding to each preset text, and each preset text includes at least one polyphonic character.

12. The device according to claim 11, wherein The processing module is further used to input a second text to be recognized into a second text encoding model, extract a third text feature sequence corresponding to the second text to be recognized, the second text to be recognized includes a second polyphonetic character, and the second text encoding model is different from the first text encoding model; The processing module is further used to obtain second encoding information corresponding to the second text to be recognized based on the third text feature sequence; The processing module is further configured to input the second coding information into a second polyphonetic character model, perform pronunciation prediction on the second polyphonetic character, and obtain third pronunciation prediction information of the second polyphonetic character; The device further comprises: an updating module; The updating module is used to update the third pronunciation prediction information to the fourth pronunciation prediction information corresponding to the target preset text when there is a target preset text in the text pronunciation library; The target preset text is the text in the text pronunciation library that matches the second text to be recognized.

13. The device according to claim 12, wherein: The processing module is further configured to add the third pronunciation prediction information to the training data set when it is detected that the third pronunciation prediction information is erroneous pronunciation prediction information; The training data set is used to train the second polyphonetic character model.

14. The device according to claim 8, wherein: The device further comprises: a receiving module and a storage module, wherein: The receiving module is used to receive a first input from a user; The storage module is used for storing the to-be-recognized text corresponding to the first input into a first file in response to the first input received by the receiving module; The processing module is further configured to synthesize the text to be recognized stored in the first file into the first text to be input when the amount of the text to be recognized stored in the first file exceeds a first threshold.

15. An electronic device comprising a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the text recognition method according to any one of claims 1 to 7 are implemented.

16. A readable storage medium storing a program or instruction, wherein the program or instruction, when executed by a processor, implements the steps of the text recognition method according to any one of claims 1 to 7.

17. A computer program product, wherein the computer program product is stored in a storage medium and is executed by at least one processor to implement the text recognition method according to any one of claims 1 to 7.

18. A chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the text recognition method according to any one of claims 1 to 7.

Images