Text information recognition method, device, electronic device and storage medium

Abstract

The present application provides a kind of text information recognition method, comprising: by text information recognition model, the word meaning recognition processing of to-be-recognized text information is carried out, and the word meaning feature vector corresponding to to-be-recognized text information is obtained;By text information recognition model, the word pronunciation recognition processing of to-be-recognized text information is carried out, and the word pronunciation feature vector corresponding to to-be-recognized text information is obtained;By text information recognition model, the word form recognition processing of to-be-recognized text information is carried out, and the word form feature vector corresponding to to-be-recognized text information is obtained;To the word meaning feature vector, word pronunciation feature vector and word form feature vector, feature fusion processing is carried out, and the fusion feature vector is obtained;Based on fusion feature vector, the to-be-recognized text information is recognized.The present application also provides text information recognition device, electronic equipment and storage medium.The present application can realize automatically to to-be-recognized text information is recognized, and efficiently finds the error character in to-be-recognized text information.

Description

Technical Field

[0001] The present invention relates to information processing technology, and in particular to a text information recognition method, device, electronic device and storage medium. Background Art

[0002] There are many types of text information, such as news, papers, microblogs, and chats. Recognizing text information and displaying correct text information for users to read can save users' text browsing time. For incorrect text information, timely correction is required. Checking text information through artificial intelligence is an efficient solution. To this end, artificial intelligence technology (AI) provides an appropriate text processing process operation mechanism to support the above application solutions. Among them, artificial intelligence is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making. In the field of text processing, digital computers or machines controlled by digital computers can be used to realize the recognition of text information. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a text information recognition method, device, electronic device, and storage medium. The technical solution of the embodiments of the present invention is implemented as follows:

[0004] An embodiment of the present invention provides a text information recognition method, comprising:

[0005] Acquiring target text information and preprocessing the target text information to obtain text information to be recognized;

[0006] Performing word meaning recognition processing on the text information to be recognized by using a text information recognition model to obtain a word meaning feature vector corresponding to the text information to be recognized;

[0007] Performing phonetic recognition processing on the text information to be recognized by using a text information recognition model to obtain a phonetic feature vector corresponding to the text information to be recognized;

[0008] Performing glyph recognition processing on the text information to be recognized by using a text information recognition model to obtain a glyph feature vector corresponding to the text information to be recognized;

[0009] Performing feature fusion processing on the character meaning feature vector, the character pronunciation feature vector, and the character shape feature vector to obtain a fused feature vector;

[0010] The text information to be identified is identified based on the fused feature vector.

[0011] An embodiment of the present invention further provides a text information recognition device, the device comprising:

[0012] An information transmission module is used to obtain target text information and pre-process the target text information to obtain text information to be identified;

[0013] An information processing module is used to perform word meaning recognition processing on the text information to be recognized through a text information recognition model to obtain a word meaning feature vector corresponding to the text information to be recognized;

[0014] The information processing module is used to perform phonetic recognition processing on the text information to be recognized through a text information recognition model to obtain a phonetic feature vector corresponding to the text information to be recognized;

[0015] The information processing module is used to perform glyph recognition processing on the text information to be recognized through a text information recognition model to obtain a glyph feature vector corresponding to the text information to be recognized;

[0016] The information processing module is used to perform feature fusion processing on the character meaning feature vector, the character pronunciation feature vector and the character shape feature vector to obtain a fused feature vector;

[0017] The information processing module is used to identify the text information to be identified based on the fused feature vector.

[0018] In the above scheme,

[0019] The information processing module is used to extract text parameter information carried by the target text according to the recognition environment of the target text information and trigger the corresponding word segmentation library;

[0020] The information processing module is used to perform word segmentation processing on the target text through the triggered word segmentation library dictionary, extract Chinese character text, and form different word-level feature vectors;

[0021] The information processing module is used to perform noise removal on the different word-level feature vectors to form a set of word-level feature vectors corresponding to the target text.

[0022] In the above scheme,

[0023] The information processing module is used to determine a dynamic noise threshold that matches the recognition environment of the target text information;

[0024] The information processing module is configured to perform noise removal processing on the different word-level feature vectors according to the dynamic noise threshold, and trigger a dynamic word segmentation strategy that matches the dynamic noise threshold;

[0025] The information processing module is used to perform word segmentation processing on the target text according to a dynamic word segmentation strategy that matches the dynamic noise threshold, extract Chinese character text, and form a dynamic word-level feature vector set corresponding to the target text.

[0026] In the above scheme,

[0027] The information processing module is used to determine a fixed noise threshold corresponding to the recognition environment of the target text information;

[0028] The information processing module is configured to perform noise removal processing on the different word-level feature vectors according to the fixed noise threshold, and trigger a fixed word segmentation strategy that matches the fixed noise threshold;

[0029] The information processing module is used to perform word segmentation processing on the target text according to a fixed word segmentation strategy that matches the fixed noise threshold, extract Chinese character text, and a fixed word-level feature vector set corresponding to the target text.

[0030] In the above scheme,

[0031] The information processing module is used to process the text information to be recognized through the word meaning encoder network in the text information recognition model to determine the character feature vector corresponding to the text information to be recognized;

[0032] The information processing module is configured to determine, through the word meaning encoder network, at least one word-level latent variable corresponding to the text information to be recognized based on the character feature vector;

[0033] The information processing module is used to determine the word meaning feature vector corresponding to the text information to be recognized based on the at least one word-level latent variable and the character feature vector corresponding to the text information to be recognized through the word meaning encoder network.

[0034] In the above scheme,

[0035] The information processing module is configured to extract the pinyin corresponding to each character in the text information to be recognized and the intonation corresponding to each character in the text information to be recognized through the character-to-phonetic encoder network in the text information recognition model according to the recognition environment of the target text information;

[0036] The information processing module is used to determine a single character pronunciation feature vector at the level of each character in the text information to be recognized based on the pinyin corresponding to each character in the text information to be recognized and the intonation corresponding to each character in the text information to be recognized;

[0037] The information processing module is used to combine and process the single character pronunciation feature vector corresponding to each character in the text information to be recognized through the character-to-phonetic encoder network in the text information recognition model to form a sentence-level pronunciation feature vector;

[0038] The information processing module is used to determine the pronunciation feature vector corresponding to the text information to be recognized based on the pronunciation feature vector at the sentence level.

[0039] In the above scheme,

[0040] The information processing module is used to extract the character images corresponding to the text information to be recognized in a simplified manner through the glyph encoder network in the text information recognition model;

[0041] The information processing module is used to perform noise reduction processing on the simplified character image through the glyph encoder network;

[0042] The information processing module is configured to perform cross-downsampling processing on the image of the character that has undergone noise reduction processing through the glyph encoder network to obtain a downsampling result of the character image, and perform normalization processing on the downsampling result;

[0043] The information processing module is used to perform horizontal normalization processing on the downsampling result of the character image to obtain a horizontal normalization processing result;

[0044] The information processing module is used to determine the glyph feature vector corresponding to the text information to be recognized based on the horizontal normalization processing result.

[0045] In the above scheme,

[0046] The information processing module is used to select simplified Chinese bold character images, traditional Chinese bold character images and small seal Chinese character images according to the recognition environment of the target text information as character images corresponding to the text information to be recognized.

[0047] In the above scheme,

[0048] The information processing module is configured to perform feature fusion processing on the word meaning feature vector, the word pronunciation feature vector, and the word shape feature vector through a feature fusion network of a text information recognition model to obtain a fused feature vector at a single character level;

[0049] The information processing module is used to perform fusion processing on the fused feature vector at the single character level through the feature fusion network of the text information recognition model to obtain the fused feature vector at the sentence level.

[0050] In the above scheme,

[0051] The information processing module is configured to generate a probability of an erroneous character being selected corresponding to the recognized text information based on the fused feature vector through a decoder network of the text information recognition model;

[0052] The information processing module is used to compare the probability of the incorrect character being selected with the confidence threshold of the recognition environment of the target text information to obtain a confidence threshold comparison result;

[0053] The information processing module is configured to perform at least one of the following according to the confidence threshold comparison result:

[0054] Keeping the error characters corresponding to the identified text information;

[0055] The erroneous characters corresponding to the recognized text information are adjusted.

[0056] In the above scheme,

[0057] The information processing module is configured to determine a training sample set that matches the text information recognition model according to the recognition environment type of the target text information, wherein the training sample set includes at least one group of training samples;

[0058] The information processing module is used to extract a training sample set that matches the training sample through a noise threshold that matches the text information recognition model;

[0059] The information processing module is used to train the text information recognition model according to a training sample set that matches the training sample.

[0060] In the above scheme,

[0061] The information processing module is used to determine a multi-task loss function that matches the text information recognition model;

[0062] The information processing module is used to adjust the different encoder network parameters and decoder network parameters in the text information recognition model based on the multi-task loss function until the loss functions of different dimensions corresponding to the text information recognition model reach corresponding convergence conditions; so as to achieve the adaptation of the parameters of the text information recognition model to the recognition environment type of the target text information.

[0063] An embodiment of the present invention further provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium to implement the aforementioned text information recognition method.

[0064] An embodiment of the present invention further provides an electronic device, comprising:

[0065] a memory for storing executable instructions;

[0066] The processor is configured to implement the preceding text information recognition method when running the executable instructions stored in the memory.

[0067] An embodiment of the present invention further provides a computer-readable storage medium storing executable instructions, wherein the executable instructions, when executed by a processor, implement the aforementioned text information recognition method.

[0068] The embodiments of the present invention have the following beneficial effects:

[0069] The embodiment of the present invention obtains target text information and preprocesses the target text information to obtain text information to be recognized; performs word meaning recognition processing on the text information to be recognized through a text information recognition model to obtain a word meaning feature vector corresponding to the text information to be recognized; performs word pronunciation recognition processing on the text information to be recognized through a text information recognition model to obtain a word pronunciation feature vector corresponding to the text information to be recognized; performs word shape recognition processing on the text information to be recognized through a text information recognition model to obtain a word shape feature vector corresponding to the text information to be recognized; performs feature fusion processing on the word meaning feature vector, the word pronunciation feature vector and the word shape feature vector to obtain a fused feature vector; and recognizes the text information to be recognized based on the fused feature vector. Thus, the text information to be recognized can be automatically recognized through the fused feature vector, and erroneous characters appearing in the text information to be recognized can be efficiently discovered. This not only eliminates the tedious steps of manually marking confusion sets, but also can detect and recognize erroneous characters that are not in the confusion sets. Through the joint detection of word shape, word pronunciation and word meaning, the recognition accuracy is higher, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 A schematic diagram of a usage scenario of the text information recognition method provided by an embodiment of the present invention;

[0071] Figure 2 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention;

[0072] Figure 3 An optional flowchart of the text information recognition method provided by an embodiment of the present invention;

[0073] Figure 4 Schematic diagram of the model structure of the text information recognition model in an embodiment of the present invention;

[0074] Figure 5 This is an optional structural diagram of a text information recognition model in an embodiment of the present invention;

[0075] Figure 6 Schematic diagram of an optional recognition process of a text information recognition model in an embodiment of the present invention;

[0076] Figure 7 Schematic diagram of an optional structure of an encoder in a text information recognition model in an embodiment of the present invention;

[0077] Figure 8 Schematic diagram of vector splicing of an encoder in a text information recognition model according to an embodiment of the present invention;

[0078] Figure 9 Schematic diagram of the encoding process of the encoder in the text information recognition model according to an embodiment of the present invention;

[0079] Figure 10 Schematic diagram of the decoding process of the decoder in the text information recognition model according to an embodiment of the present invention;

[0080] Figure 11 Schematic diagram of the decoding process of the decoder in the text information recognition model according to an embodiment of the present invention;

[0081] Figure 12 Schematic diagram of the decoding process of the decoder in the text information recognition model according to an embodiment of the present invention;

[0082] Figure 13 An optional flowchart of the text information recognition method provided by an embodiment of the present invention;

[0083] Figure 14 Schematic diagram of the application environment of the text information recognition model in an embodiment of the present invention;

[0084] Figure 15 This is a schematic diagram of the working process of the text information recognition model provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0085] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting the present invention. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0086] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0087] Before further explaining the embodiments of the present invention in detail, the nouns and terms involved in the embodiments of the present invention are explained. The nouns and terms involved in the embodiments of the present invention are subject to the following interpretations.

[0088] 1) In response, it is used to indicate the conditions or states on which the executed operations depend. When the dependent conditions or states are met, one or more operations executed can be real-time or have a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations executed are executed.

[0089] 2) Word segmentation library: also known as word segmentation library, refers to a specific word segmentation method. Different word segmentation libraries have corresponding word dictionaries, and can perform word segmentation on the corresponding text information based on their corresponding word dictionaries.

[0090] 3) Model training: Multi-classification learning is performed on image datasets. This model can be built using deep learning frameworks such as TensorFlow and Torch, using multiple layers of neural network layers such as CNN to form a multi-classification model. The model input is a three-channel or raw channel matrix of images read using tools such as OpenCV. The model output is multi-class probabilities, and finally the webpage category is output using algorithms such as softmax. During training, the model approaches the correct trend using objective functions such as cross entropy.

[0091] 4) Neural Network (NN): Artificial Neural Network (ANN), also known as neural network or quasi-neural network, is a mathematical model or computational model that imitates the structure and function of biological neural networks (the central nervous system of animals, especially the brain) in the fields of machine learning and cognitive science. It is used to estimate or approximate functions.

[0092] 5) Encoder-Decoder Architecture: This is a common network structure used in machine learning. It consists of two parts: an encoder and a decoder. The encoder converts the input text into a series of context vectors that represent the characteristics of the input text. The decoder receives the encoder's output as input and outputs the corresponding text sequence in another language.

[0093] 6) Mini Programs are programs developed using a front-end language (e.g., JavaScript) that implement services within a Hypertext Markup Language (HTML) page. These programs are downloaded by a client (e.g., a browser or any client with an embedded browser core) via a network (e.g., the Internet) and interpreted and executed within the client's browser environment, eliminating the need for client-side installation. For example, by waking up a mini program on a terminal through voice commands, a social network client can download and run mini programs for various services, such as ticket purchases, task processing and creation, and data presentation.

[0094] 7) BERT: Bidirectional Encoder Representations from Transformers, a language model training method that leverages massive amounts of text. This method is widely used in various natural language processing tasks, such as text classification, text matching, and machine reading comprehension.

[0095] 8) Hidden representation: A continuous vector that represents the semantic information of the word or sentence it represents. Sentence-level / word-level: sentence-level and word-level.

[0096] Figure 1 Schematic diagram of the use scenario of the text information recognition method provided by the embodiment of the present invention, see Figure 1, the terminal (including terminal 10-1 and terminal 10-2) is provided with a corresponding client that can perform different functions. Among them, the client is that the terminal (including terminal 10-1 and terminal 10-2) obtains different text information (including but not limited to: news articles, professional papers, public account articles, novels) from the corresponding server 200 through the network 300 for browsing. The terminal is connected to the server 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two, and a wireless link is used to realize data transmission. Among them, the types of text information obtained by the terminal (including terminal 10-1 and terminal 10-2) from the corresponding server 200 through the network 300 are not the same. For example: the terminal (including terminal 10-1 and terminal 10-2) can obtain any type of text information from the corresponding server 200 through the network 300, or obtain browsing summary information from the corresponding server 200 through the network 300 and then select the required text information to be identified for browsing. The server 200 can store different machine learning processes to check the obtained text information through different text processing neural networks to find typos. Through the text information recognition method provided by this application, the target text information can be obtained by obtaining the target text information and preprocessing the target text information to obtain the text information to be recognized; the text information to be recognized is processed by the text information recognition model to perform word meaning recognition to obtain the word meaning feature vector corresponding to the text information to be recognized; the text information to be recognized is processed by the text information recognition model to perform word pronunciation recognition to obtain the word pronunciation feature vector corresponding to the text information to be recognized; the text information to be recognized is processed by the text information recognition model to perform word shape recognition to obtain the word shape feature vector corresponding to the text information to be recognized; the word meaning feature vector, the word pronunciation feature vector and the word shape feature vector are subjected to feature fusion processing to obtain a fused feature vector; based on the fused feature vector, the text information to be recognized is recognized. In some embodiments of the present invention, the different types of text information stored in the server 200 can be written in different software code environments using programming languages ​​in different text information recognition environments, and the code objects can be different types of code entities. For example, in C language software code, a code object can be a function. In Java language software code, a code object can be a class, and in iOS OC language, it can be a piece of object code. In C++ language software code, a code object can be a class or a function to execute text processing instructions from different terminals. In this application, the source of the text information to be processed by the text information recognition model is no longer distinguished.

[0097] In the process of the server 200 sending different types of text information to the terminal (terminal 10-1 and / or terminal 10-2) via the network 300, the server 200 needs to determine the text information selected by the user for detection. Therefore, as an example, the server 200 is used to obtain the target text in the text information recognition environment, recognize the text information to be recognized based on the fused feature vector, modify the erroneous character information, and finally present the matching correct text information or the modification process of the erroneous information in the display interface, so that the user can perform different subsequent operations.

[0098] The structure of the electronic device of the embodiment of the present invention is described in detail below. The electronic device can be implemented in various forms, such as a dedicated terminal with a text processing function, or an electronic device with a text processing function, such as the aforementioned Figure 1 Server 200 in. Figure 2 The schematic diagram of the structure of the electronic device provided in the embodiment of the present invention can be understood as follows: Figure 2 Only exemplary structures of electronic devices are shown, not all structures, and may be implemented as needed. Figure 2 Partial or complete structure shown.

[0099] The electronic device provided in the embodiment of the present invention includes: at least one processor 201, a memory 202, a user interface 203 and at least one network interface 204. The various components in the electronic device 20 are coupled together via a bus system 205. It can be understood that the bus system 205 is used to achieve connection and communication between these components. In addition to including a data bus, the bus system 205 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 205 is not described in detail. Figure 2 Various buses are labeled as bus system 205 .

[0100] The user interface 203 may include a display, a keyboard, a mouse, a trackball, a click wheel, keys, buttons, a touch pad or a touch screen.

[0101] It is understood that the memory 202 can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. The memory 202 in the embodiment of the present invention can store data to support the operation of the terminal (such as 10-1). Examples of such data include: any computer program used to operate on the terminal (such as 10-1), such as an operating system and an application program. Among them, the operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application program can include various application programs.

[0102] In some embodiments, the text information recognition device provided by the embodiments of the present invention can be implemented using a combination of software and hardware. As an example, the text information recognition device provided by the embodiments of the present invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the text information recognition method provided by the embodiments of the present invention. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0103] As an example of a text information recognition device provided by an embodiment of the present invention being implemented by a combination of software and hardware, the text information recognition device provided by an embodiment of the present invention can be directly embodied as a combination of software modules executed by the processor 201. The software module can be located in a storage medium, and the storage medium is located in the memory 202. The processor 201 reads the executable instructions included in the software module in the memory 202, and combines with the necessary hardware (for example, including the processor 201 and other components connected to the bus 205) to complete the text information recognition method provided by the embodiment of the present invention.

[0104] As an example, the processor 201 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0105] As an example of a hardware implementation of the text information recognition device provided in an embodiment of the present invention, the device provided in an embodiment of the present invention can be directly executed by a processor 201 in the form of a hardware decoding processor. For example, the text information recognition method provided in an embodiment of the present invention can be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0106] The memory 202 in the embodiment of the present invention is used to store various types of data to support the operation of the electronic device 20. Examples of such data include any executable instructions for operating on the electronic device 20, such as executable instructions. The program for implementing the method for identifying text information in the embodiment of the present invention may be included in the executable instructions.

[0107] In other embodiments, the text information recognition device provided by the embodiment of the present invention can be implemented in software. Figure 2 The text information recognition device 2020 stored in the memory 202 is shown. This device may be software in the form of a program or plug-in, and may include a series of modules. An example of a program stored in the memory 202 may include the text information recognition device 2020. The text information recognition device 2020 includes the following software modules: an information transmission module 2081 and an information processing module 2082. When the software modules in the text information recognition device 2020 are read into the RAM and executed by the processor 201, the text information recognition method provided in the embodiment of the present invention is implemented. The functions of each software module in the text information recognition device 2020 are described below:

[0108] The information transmission module 2081 is used to obtain target text information and pre-process the target text information to obtain text information to be recognized.

[0109] The information processing module 2082 is used to perform word meaning recognition processing on the text information to be recognized through a text information recognition model to obtain a word meaning feature vector corresponding to the text information to be recognized.

[0110] The information processing module 2082 is configured to perform phonetic recognition processing on the text information to be recognized using a text information recognition model to obtain a phonetic feature vector corresponding to the text information to be recognized.

[0111] The information processing module 2082 is configured to perform glyph recognition processing on the text information to be recognized using a text information recognition model to obtain a glyph feature vector corresponding to the text information to be recognized.

[0112] The information processing module 2082 is used to perform feature fusion processing on the character meaning feature vector, the character pronunciation feature vector and the character shape feature vector to obtain a fused feature vector.

[0113] The information processing module 2082 is used to identify the text information to be identified based on the fused feature vector.

[0114] according to Figure 2 In one aspect of the electronic device shown, the present application further provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of the computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the various embodiments and combinations of embodiments provided in the various optional implementations of the above-mentioned text information recognition method.

[0115] Combine Figure 2 The electronic device 20 shown illustrates the text information recognition method provided by an embodiment of the present invention, see Figure 3 , Figure 3 This is an optional flow chart of the text information recognition method provided by the embodiment of the present invention. It can be understood that: Figure 3 The steps shown can be performed by various electronic devices running text information recognition devices, such as a dedicated terminal with a search instruction check and processing function, an electronic device with a function of detecting input characters, or a cluster of electronic devices that check user uploaded text information. Figure 3 The steps shown are explained.

[0116] Step 301: The text information recognition device obtains target text information and preprocesses the target text information to obtain text information to be recognized.

[0117] In some embodiments of the present invention, preprocessing the target text information to obtain the text information to be recognized can be achieved in the following ways:

[0118] Based on the recognition environment of the target text information, the text parameter information carried by the target text is extracted and the corresponding word segmentation library is triggered. The target text is segmented using the word dictionary of the triggered word segmentation library to extract Chinese character text and form different word-level feature vectors. The different word-level feature vectors are then subjected to noise removal to form a set of word-level feature vectors corresponding to the target text. During the preprocessing process, Arabic numerals in the target text may not be converted to Chinese characters. Only conversions unrelated to the numbers, such as traditional Chinese to simplified Chinese, are performed, retaining the original form of Arabic numerals in the sentence. Furthermore, international unit abbreviations associated with numbers, such as g, kg, and cm, are not converted and remain in their original state. For Chinese text, word segmentation is required, as only words in Chinese can contain complete information. The Chinese word segmentation tool Jieba can be used to segment Chinese text accordingly. For example, "this incident happened in 2001" becomes "this incident happened in 2001" after word segmentation. Among them, the so-called segmentation has both verbal meaning and noun meaning; each segmentation is a word or phrase, that is, the smallest semantic unit with a definite meaning; for different users or different text information recognition models received, the smallest semantic units contained therein need to be divided differently, and adjustments need to be made in a timely manner. This process is called segmentation, that is, segmentation can refer to the process of dividing the smallest semantic units mentioned above; on the other hand, the smallest semantic unit obtained after division is often also called segmentation, that is, the word obtained after the segmentation operation is performed; sometimes in order to distinguish two meanings from each other, the smallest semantic unit referred to by the latter meaning is called segmentation object (Term); the term segmentation object is used in this application; the segmentation object corresponds to the keyword in the inverted list as the index basis. For Chinese, since the word as the smallest semantic unit is often composed of different numbers of characters, there is no natural distinguishing mark in phonetic characters such as blank partitions between words. Therefore, for Chinese, accurately performing segmentation to obtain a reasonable segmentation object is an important step.

[0119] In some embodiments of the present invention, performing denoising on the different word-level feature vectors to form a set of word-level feature vectors corresponding to the target text can be achieved by:

[0120] Determine a dynamic noise threshold that matches the recognition environment of the target text information; perform noise removal on the different word-level feature vectors according to the dynamic noise threshold, and trigger a dynamic word segmentation strategy that matches the dynamic noise threshold; perform word segmentation on the target text according to the dynamic word segmentation strategy that matches the dynamic noise threshold, extract Chinese character text, and form a set of dynamic word-level feature vectors corresponding to the target text. Alternatively, determine a fixed noise threshold that corresponds to the recognition environment of the target text information; perform noise removal on the different word-level feature vectors according to the fixed noise threshold, and trigger a fixed word segmentation strategy that matches the fixed noise threshold; perform word segmentation on the target text according to the fixed word segmentation strategy that matches the fixed noise threshold, extract Chinese character text, and form a set of fixed word-level feature vectors corresponding to the target text. Specifically, when performing text information recognition, since the text information displayed by the terminal in the academic recognition environment only includes the text information of academic papers, the dynamic noise threshold that matches the use environment of the text information recognition model needs to be smaller than the dynamic noise threshold in the reading environment of entertainment information text. On the contrary, when the text information recognition model is solidified in the corresponding hardware structure, such as in a smart learning desk lamp, and the usage environment is to check the Chinese homework uploaded by the user (or Chinese test paper or the user's composition), since the noise is relatively single, the fixed noise threshold corresponding to the fixed text information recognition model can be used to effectively improve the processing speed of the text information recognition model, reduce the user's waiting time, and improve the user experience.

[0121] Step 302: The text information recognition device performs word meaning recognition processing on the text information to be recognized through a text information recognition model to obtain a word meaning feature vector corresponding to the text information to be recognized.

[0122] Step 303: The text information recognition device performs phonetic recognition processing on the text information to be recognized through a text information recognition model to obtain a phonetic feature vector corresponding to the text information to be recognized.

[0123] In some embodiments of the present invention, reference Figure 4 , Figure 4 Schematic diagram of the model structure of the text information recognition model in an embodiment of the present invention, wherein obtaining the character meaning feature vector corresponding to the text information to be recognized can be achieved by the following methods:

[0124] The text information to be recognized is processed by the semantic encoder network in the text information recognition model to determine the character feature vector corresponding to the text information to be recognized; the semantic encoder network is used to determine at least one word-level latent variable corresponding to the text information to be recognized based on the character feature vector; the semantic encoder network is used to determine the semantic feature vector corresponding to the text information to be recognized based on the at least one word-level latent variable and the character feature vector corresponding to the text information to be recognized. Specifically, Figure 4 As shown, the input of the semantic encoder network is denoted as X = (x1, x2, ..., x n ), N is the length of the input sentence. In this application, a 12-layer pre-trained BERT model can be used to encode word meaning information. The BERT model consists of 12 layers of Transformers. The specific encoding of each layer refers to Formula 1:

[0125]

[0126] Where L is the number of layers of the semantic encoder, 12 is used here.

[0127] The encoded word meaning feature vector refers to Formula 2:

[0128]

[0129] Continue to refer Figure 5 , Figure 5 This figure illustrates an alternative structure of a text information recognition model in an embodiment of the present invention. The encoder consists of N = 6 identical layers, each containing two sub-layers. The first sub-layer is a multi-head attention layer, followed by a simple fully connected layer. Each sub-layer incorporates a residual connection and normalization.

[0130] The decoder consists of N = 6 identical layers, each distinct from the encoder. Each layer contains three sub-layers: a self-attention layer, an encoder-decoder attention layer, and finally a fully connected layer. The first two sub-layers are based on a multi-head attention layer. Specifically, the Nx on the left represents the structure of one encoder layer. This layer includes two sub-layers: the first is a multi-head attention layer, and the second is a feed-forward layer. The input and output of each sub-layer are correlated, with the output of the current sub-layer serving as the input for the next. Each sub-layer is followed by a normalization operation, which improves model convergence. The Nx on the right represents the structure of one decoder layer. This layer includes three sub-layers. The first is a multi-head attention layer controlled by a mask matrix, which is used to model the generated target sentence vector. During training, a mask matrix is ​​used to control the multi-head attention calculation, ensuring that only the first t-1 words are considered during each calculation. The second sublayer is the multi-head attention sublayer, which serves as the attention mechanism between the encoder and decoder. Specifically, it searches for relevant semantic information in the source text. This layer uses a dot product approach for computation. The third sublayer is the forward propagation sublayer, which uses the same computational approach as the forward propagation sublayer in the encoder. Each sublayer in the decoder is also interconnected, with the output of the current sublayer serving as the input for the next. Each sublayer in the decoder is also followed by a normalization operation to accelerate model convergence.

[0131] Continue to refer Figure 6 , Figure 6 This diagram illustrates an optional recognition process for a text information recognition model in an embodiment of the present invention. The encoder and decoder components each include six encoders and decoders. The input to the first encoder is combined with both an embedding and a positional embedding. After passing through the six encoders, the output is sent to each decoder in the decoder component. The input sentence to be recognized is the English sentence "I am a student." After processing the text information recognition model, the output recognition result is: "I am a student."

[0132] Continue to refer Figure 7 , Figure 7This is a schematic diagram of an optional structure of an encoder in a text information recognition model in an embodiment of the present invention, wherein its input consists of a query (Q) and key (K) of dimension d, and a value (V) of dimension d. All keys calculate the dot product of the query and apply the softmax function to obtain the weight of the value.

[0133] Continue to refer Figure 7 , Figure 7 This is a schematic diagram of the encoder vectors in the text information recognition model in an embodiment of the present invention. Q, K, and V are obtained by multiplying the encoder input vector x by W^Q, W^K, and W^V. The dimensions of W^Q, W^K, and W^V in the article are (512, 64). Assuming the dimensions of our inputs are (m, 512), where m represents the number of words, the dimensions of Q, K, and V obtained by multiplying the input vector by W^Q, W^K, and W^V are (m, 64).

[0134] Continue to refer Figure 8 , Figure 8 This is a schematic diagram of the encoder vector concatenation in the text information recognition model according to an embodiment of the present invention. Z0 through Z7 correspond to eight parallel heads (of dimension (m, 64)). These eight heads are then concat-ed to form an output matrix of dimension (m, 512). Finally, multiplication by W^O yields an output matrix of dimension (m, 512). This matrix maintains the same dimension as that fed into the next encoder.

[0135] Continue to refer Figure 9 , Figure 9 This is a schematic diagram of the encoding process of the encoder in the text information recognition model in an embodiment of the present invention. Here, x1 undergoes self-attention and reaches the state of z1. The self-attentioned tensor then undergoes residual network and later norm processing before entering the fully connected feedforward network. The feedforward network also undergoes the same residual processing and normalization. Finally, the output tensor can enter the next encoder. This operation is iterated six times, and the result of the iterative processing enters the decoder.

[0136] Continue to refer Figure 10 , Figure 10 This is a schematic diagram of the decoding process of the decoder in the text information recognition model according to an embodiment of the present invention, wherein the decoder input, output and decoding process are as follows:

[0137] Output: probability distribution of the output word corresponding to position i;

[0138] Input: encoder output & corresponding decoder output at position i-1. Therefore, the middle attention is not self-attention, its K and V come from the encoder, and Q comes from the output of the previous decoder.

[0139] Continue to refer Figure 11 and Figure 12 , Figure 11 The figure is a schematic diagram of the decoding process of the decoder in the text information recognition model according to an embodiment of the present invention, wherein the vector output by the last decoder in the decoder network passes through the linear layer and the softmax layer. Figure 12 This is a schematic diagram of the decoding process of the decoder in the text information recognition model in an embodiment of the present invention. The function of the Linear layer is to map the vector output by the decoder into a logits vector. The softmax layer then converts this logits vector into a probability value. Finally, the position of the maximum probability is found, completing the decoder output.

[0140] Step 304: The text information recognition device performs glyph recognition processing on the text information to be recognized using a text information recognition model to obtain a glyph feature vector corresponding to the text information to be recognized.

[0141] Combine Figure 2 The model structure shown in Figure 13 , Figure 13 This is an optional flow chart of the text information recognition method provided by the embodiment of the present invention. It can be understood that: Figure 13 The steps shown can be performed by various electronic devices running a text information recognition device to obtain the phonetic feature vector and the glyph feature vector corresponding to the text information to be recognized, and specifically include the following steps:

[0142] Step 1301: According to the recognition environment of the target text information, the pinyin corresponding to each character in the text information to be recognized and the intonation corresponding to each character in the text information to be recognized are extracted through the phonetic encoder network in the text information recognition model.

[0143] Step 1302: Determine a single character pronunciation feature vector at the level of each character in the text information to be recognized based on the pinyin corresponding to each character in the text information to be recognized and the intonation corresponding to each character in the text information to be recognized.

[0144] Step 1303: The single character pronunciation feature vector corresponding to each character in the text information to be recognized is combined and processed through the phonetic encoder network in the text information recognition model to form a sentence-level pronunciation feature vector.

[0145] First, the Chinese pinyin of each character can be extracted. The first, second, third, fourth tones and the light tone of the pinyin are represented by the five numbers 1, 2, 3, 4, and 0. Taking the character '轻' as an example, its pinyin is denoted as the letter and number sequence p=(q, i, n, g, 1) in this article.

[0146] To capture the information of Chinese character pronunciation at the single character level and the whole sentence level, character-level pronunciation encoding and sentence-level pronunciation encoding can be performed respectively. Among them, the character-level pronunciation encoding can use a unidirectional RNN model. Here, the processing process of the GRU model is referred to Formula 3:

[0147]

[0148] Among them, E(p i,j ) is the embedding of the pinyin, is the vector representation of the j-th pinyin or digital letter of the i-th character. The sentence-level pronunciation encoding adopts a 4-layer Transformer model, and the input is the output of the character-level pronunciation encoder. It should be noted that the Gated Recurrent Unit network (GRU) is a model with fewer parameters than LSTM that can handle sequence information well. Next, the fused features will be input into the feedforward neural network, aiming to process the effective information of other features. Regarding the recognition of wrong characters as a problem of predicting the occurrence probability, the sigmoid function (logical function) is used as the output layer, and the loss function is the standard cross-entropy loss. Refer to the following:

[0149]

[0150] Among them, the GRU layer is for deep feature extraction. The GRU layer can also be omitted and replaced by multiple concatenated feedforward neural network layers, which can also effectively process and fuse features.

[0151] The encoded pronunciation feature vector can be expressed as Formula 4:

[0152]

[0153] The embodiments of the present invention may be implemented in conjunction with cloud technology or blockchain network technology. Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network or local area network to achieve data computing, storage, processing, and sharing. It can also be understood as a general term for network technology, information technology, integration technology, management platform technology, and application technology based on cloud computing business models. The backend services of technical network systems require a large amount of computing and storage resources, such as video websites, image websites, and more portal websites. Therefore, cloud technology needs to be supported by cloud computing.

[0154] It should be noted that cloud computing is a computing model that distributes computing tasks across a resource pool consisting of a large number of computers, enabling various application systems to obtain computing power, storage space, and information services as needed. The network that provides these resources is called the "cloud." To users, the resources in the "cloud" appear to be infinitely scalable and can be accessed at any time, used on demand, and expanded at any time, with a pay-per-use fee. As a provider of cloud computing's basic capabilities, a cloud computing resource pool platform, often referred to as Infrastructure as a Service (IaaS), is established. Various types of virtual resources are deployed within the resource pool for external customers to choose from. The cloud computing resource pool primarily includes computing devices (which can be virtualized machines, including operating systems), storage devices, and network devices.

[0155] In some embodiments of the present invention, the TTS server in the cloud can use the wake-up text to generate N different wake-up word voices (pronunciations) to form feature vectors of different frame lengths. For example, the user can arbitrarily modify the text to be recognized according to different usage scenarios. The TTS server converts each character contained in the text to be recognized into a syllable identifier according to the pronunciation dictionary to extract the corresponding wake-up word feature.

[0156] In some embodiments of the present invention, the text-to-speech conversion server can also convert each character contained in the text to be recognized into a syllable identifier based on a pronunciation dictionary; and construct a set of mapping relationships between the syllable identifiers and the characters contained in the text to be recognized, forming different syllable combination sequences as elements in the wake-up speech feature set. Specifically, when the text to be recognized is Chinese, each Chinese character is a character, and the pronunciation of each character corresponds to a syllable identifier. For example, the text information to be recognized is the word "difficult", and its pronunciation can be the second tone or the fourth tone. Each pronunciation is assigned an identification ID (Identifier) ​​for use as a syllable identifier, and then a mapping relationship set between the syllable identifier and the characters contained in the text to be recognized is constructed to form different syllable combination sequences as elements in the wake-up voice feature set. Furthermore, when the wake-up word text information is the word "thin", its pronunciation can be the second tone "bao" or the second tone "bo", and each pronunciation is assigned an identification ID (Identifier) ​​for use as a syllable identifier, and then a mapping relationship set between the syllable identifier and the characters contained in the text to be recognized is constructed to form different syllable combination sequences as elements in the wake-up voice feature set.

[0157] Step 1304: Based on the sentence-level pronunciation feature vector, determine the character pronunciation feature vector corresponding to the text information to be recognized.

[0158] Step 1305: The character image corresponding to the text information to be recognized is simplified and extracted through the glyph encoder network in the text information recognition model, and the simplified character image is subjected to noise reduction processing through the glyph encoder network.

[0159] Step 1306: cross-downsampling the image of the character that has undergone noise reduction processing is performed through the glyph encoder network to obtain a downsampling result of the character image, and normalizing the downsampling result.

[0160] In some embodiments of the present invention, simplified Chinese boldface character images, traditional Chinese boldface character images, and small seal Chinese character images can be selected as character images corresponding to the text information to be recognized based on the recognition environment of the target text information. Among them, a 5-layer ResNet network can be used as the glyph encoder, and the processing process is referred to Formula 5:

[0161]

[0162] Step 1307: performing horizontal normalization processing on the downsampling result of the character image to obtain a horizontal normalization processing result.

[0163] Horizontal normalization considers all dimensions of input for a layer, calculates the average input value and input variance for that layer, and then applies the same normalization operation to transform the input across all dimensions. Horizontal normalization (LayerNormalization) is performed on a single training sample and is independent of other data. Therefore, it avoids the issue of mini-batch data distribution affecting batch normalization (BN). It can be used in small mini-batch scenarios, dynamic network scenarios, and RNNs, particularly in natural language processing. Furthermore, LNs do not need to store the mean and variance of the mini-batch, saving additional storage space.

[0164] Step 1308: Determine the glyph feature vector corresponding to the text information to be recognized based on the horizontal normalization processing result.

[0165] Step 305: The text information recognition device performs feature fusion processing on the character meaning feature vector, the character pronunciation feature vector, and the character shape feature vector to obtain a fused feature vector.

[0166] In some embodiments of the present invention, the feature fusion network of the text information recognition model can be used to perform feature fusion processing on the word meaning feature vector, the word pronunciation feature vector, and the word shape feature vector to obtain a fused feature vector at the single character level; and the feature fusion network of the text information recognition model can be used to perform fusion processing on the single character level fused feature vector to obtain a fused feature vector at the sentence level. For single word level fusion, refer to Formula 6:

[0167]

[0168] Then a three-layer Transformer is used to perform sentence-level information fusion, refer to Formula 7:

[0169]

[0170] Where L' is 3. Finally, predict the correct Chinese character corresponding to the i-th character and its probability, refer to Formula 8.

[0171]

[0172] Step 306: The text information recognition device recognizes the text information to be recognized based on the fused feature vector.

[0173] In some embodiments of the present invention, when determining whether an erroneous character occurs, the decoder network of the text information recognition model can generate a probability of the erroneous character corresponding to the recognized text information being selected based on the fused feature vector; the probability of the erroneous character being selected is compared with the confidence threshold of the recognition environment of the target text information to obtain a confidence threshold comparison result; and based on the confidence threshold comparison result, perform at least one of the following: maintain the erroneous character corresponding to the recognized text information; or, adjust the erroneous character corresponding to the recognized text information.

[0174] Of course, before deploying the text information recognition model in the server, the text information recognition model needs to be trained to determine the model parameters. Specifically, a training sample set that matches the text information recognition model can be determined based on the recognition environment type of the target text information, wherein the training sample set includes at least one group of training samples; a training sample set that matches the training samples is extracted through a noise threshold that matches the text information recognition model; and the text information recognition model is trained based on the training sample set that matches the training samples. During the training process, a multi-task loss function that matches the text information recognition model can be determined; based on the multi-task loss function, the different encoder network parameters and decoder network parameters in the text information recognition model are adjusted until the loss functions of different dimensions corresponding to the text information recognition model reach the corresponding convergence conditions; so as to achieve the adaptation of the parameters of the text information recognition model to the recognition environment type of the target text information.

[0175] The text information recognition model provided by the embodiment of the present invention is described below using the text information recognition model encapsulated in the corresponding APP, wherein: Figure 14 FIG. 1 is a schematic diagram of an application environment of a text information recognition model according to an embodiment of the present invention. Figure 14 As shown in the figure, with the development of human-computer interaction technology, more and more intelligent products based on human-computer interaction technology have emerged, such as automatic reply robots in instant messaging clients. These intelligent products can chat with users and generate corresponding answer information based on users' questions. During this process, typos in the information input by users need to be detected to ensure the correctness of the reply.

[0176] Figure 15 The working process of the text information recognition model provided in the embodiment of the present invention is schematically shown, which specifically includes the following steps:

[0177] Step 1501: Obtain the text information to be processed and the associated text information, and trigger the text information recognition model in the instant messaging APP.

[0178] Step 1502: Determine the usage context type based on the conversation information in the instant messaging APP.

[0179] Step 1503: Acquire target text information, and pre-process the target text information to obtain text information to be recognized.

[0180] Step 1504: Perform word meaning recognition processing on the text information to be recognized through a text information recognition model to obtain a word meaning feature vector corresponding to the text information to be recognized.

[0181] Step 1505: Perform phonetic recognition processing on the text information to be recognized through a text information recognition model to obtain a phonetic feature vector corresponding to the text information to be recognized.

[0182] Step 1506: Performing glyph recognition processing on the text information to be recognized through a text information recognition model to obtain a glyph feature vector corresponding to the text information to be recognized.

[0183] Step 1507: performing feature fusion processing on the character meaning feature vector, the character pronunciation feature vector, and the character shape feature vector to obtain a fused feature vector.

[0184] Step 1508: Compare with the threshold, perform different processing, and determine whether it is higher than the threshold. If so, execute step 1509; otherwise, execute step 1510.

[0185] Step 1509: Return the corrected character.

[0186] Step 1510: Return to the original character.

[0187] Beneficial technical effects:

[0188] The embodiment of the present invention obtains target text information and preprocesses the target text information to obtain text information to be recognized; performs word meaning recognition processing on the text information to be recognized through a text information recognition model to obtain a word meaning feature vector corresponding to the text information to be recognized; performs word pronunciation recognition processing on the text information to be recognized through a text information recognition model to obtain a word pronunciation feature vector corresponding to the text information to be recognized; performs word shape recognition processing on the text information to be recognized through a text information recognition model to obtain a word shape feature vector corresponding to the text information to be recognized; performs feature fusion processing on the word meaning feature vector, the word pronunciation feature vector and the word shape feature vector to obtain a fused feature vector; and recognizes the text information to be recognized based on the fused feature vector. Thus, the text information to be recognized can be automatically recognized through the fused feature vector, and erroneous characters appearing in the text information to be recognized can be efficiently discovered. This not only eliminates the tedious steps of manually marking confusion sets, but also can detect and recognize erroneous characters that are not in the confusion sets. Through the joint detection of word shape, word pronunciation and word meaning, the recognition accuracy is higher, thereby improving the user experience.

[0189] The above description is merely an embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A text information recognition method, characterized in that: The method comprises: Acquiring target text information and preprocessing the target text information to obtain text information to be recognized; Performing word meaning recognition processing on the text information to be recognized by using a text information recognition model to obtain a word meaning feature vector corresponding to the text information to be recognized; Extracting the pinyin corresponding to each character in the text information to be recognized and the intonation corresponding to each character in the text information to be recognized through the phonetic encoder network in the text information recognition model according to the recognition environment of the target text information; Determining a single character pronunciation feature vector at a character level in the text information to be recognized according to the pinyin corresponding to each character in the text information to be recognized and the intonation corresponding to each character in the text information to be recognized; Combining the single character pronunciation feature vectors corresponding to each character in the text information to be recognized through the character-to-phonetic encoder network to form a sentence-level pronunciation feature vector; Determining a pronunciation feature vector corresponding to the text information to be recognized based on the sentence-level pronunciation feature vector; Performing glyph recognition processing on the text information to be recognized by using the text information recognition model to obtain a glyph feature vector corresponding to the text information to be recognized; Performing feature fusion processing on the character meaning feature vector, the character pronunciation feature vector, and the character shape feature vector to obtain a fused feature vector; Based on the fused feature vector, the text information to be recognized is recognized to obtain a probability of an erroneous character corresponding to the text information to be recognized being selected.

2. The method according to claim 1, characterized in that The step of obtaining target text information and preprocessing the target text information to obtain text information to be recognized includes: According to the recognition environment of the target text information, extract the text parameter information carried by the target text and trigger the corresponding word segmentation library; Performing word segmentation processing on the target text through the triggered word segmentation library dictionary, extracting Chinese character text, and forming different word-level feature vectors; The different word-level feature vectors are subjected to noise removal processing to form a word-level feature vector set corresponding to the target text.

3. The method according to claim 2, characterized in that The performing denoising on the different word-level feature vectors to form a word-level feature vector set corresponding to the target text includes: Determining a dynamic noise threshold that matches a recognition environment of the target text information; Performing noise removal on the different word-level feature vectors according to the dynamic noise threshold, and triggering a dynamic word segmentation strategy that matches the dynamic noise threshold; According to the dynamic word segmentation strategy that matches the dynamic noise threshold, the target text is segmented, Chinese character text is extracted, and a dynamic word-level feature vector set corresponding to the target text is formed.

4. The method according to claim 2, characterized in that The performing denoising on the different word-level feature vectors to form a word-level feature vector set corresponding to the target text includes: determining a fixed noise threshold corresponding to a recognition environment of the target text information; Performing denoising on the different word-level feature vectors according to the fixed noise threshold, and triggering a fixed word segmentation strategy that matches the fixed noise threshold; According to a fixed word segmentation strategy that matches the fixed noise threshold, the target text is segmented to extract Chinese character text and a fixed word-level feature vector set corresponding to the target text.

5. The method according to claim 1, wherein The step of performing word meaning recognition processing on the text information to be recognized by using a text information recognition model to obtain a word meaning feature vector corresponding to the text information to be recognized includes: Processing the text information to be recognized through the word meaning encoder network in the text information recognition model to determine the character feature vector corresponding to the text information to be recognized; Determining, by the word meaning encoder network, at least one word-level latent variable corresponding to the text information to be recognized according to the character feature vector; The word meaning encoder network determines the word meaning feature vector corresponding to the text information to be recognized based on the at least one word-level latent variable and the character feature vector corresponding to the text information to be recognized.

6. The method according to claim 1, characterized in that The step of performing glyph recognition processing on the text information to be recognized by using a text information recognition model to obtain a glyph feature vector corresponding to the text information to be recognized includes: Simplifying and extracting the character images corresponding to the text information to be recognized through the glyph encoder network in the text information recognition model; Performing noise reduction on the simplified character image through the glyph encoder network; Performing cross-downsampling processing on the image of the character that has undergone noise reduction processing by the glyph encoder network to obtain a downsampling result of the character image, and normalizing the downsampling result; Performing horizontal normalization processing on the downsampling result of the character image to obtain a horizontal normalization processing result; According to the horizontal normalization processing result, a glyph feature vector corresponding to the text information to be recognized is determined.

7. The method according to claim 6, characterized in that The method further comprises: According to the recognition environment of the target text information, simplified Chinese bold character images, traditional Chinese bold character images and small seal Chinese character images are selected as character images corresponding to the text information to be recognized.

8. The method according to claim 1, characterized in that Performing feature fusion processing on the character meaning feature vector, the character pronunciation feature vector, and the character shape feature vector to obtain a fused feature vector, including: Performing feature fusion processing on the word meaning feature vector, the word pronunciation feature vector, and the word shape feature vector through a feature fusion network of a text information recognition model to obtain a fused feature vector at a single character level; The fused feature vectors at the single character level are fused through the feature fusion network of the text information recognition model to obtain the fused feature vectors at the sentence level.

9. The method according to claim 1, characterized in that The step of identifying the text information to be identified based on the fused feature vector to obtain a probability of selecting an erroneous character corresponding to the text information to be identified includes: Generate, by means of a decoder network of the text information recognition model, a probability of selecting an erroneous character corresponding to the text information to be recognized based on the fused feature vector; The method further comprises: Comparing the probability of the erroneous character being selected with a confidence threshold of the recognition environment of the target text information to obtain a confidence threshold comparison result; Based on the confidence threshold comparison result, perform at least one of the following: Keeping the error characters corresponding to the text information to be recognized; Adjust the erroneous characters corresponding to the text information to be recognized.

10. The method according to claim 1, characterized in that The method further comprises: Determining a training sample set that matches the text information recognition model according to the recognition environment type of the target text information, wherein the training sample set includes at least one group of training samples; Extracting a training sample set that matches the training sample through a noise threshold that matches the text information recognition model; The text information recognition model is trained according to a training sample set that matches the training sample.

11. The method according to claim 10, characterized in that The training of the text information recognition model according to the training sample set matching the training sample includes: Determining a multi-task loss function that matches the text information recognition model; Based on the multi-task loss function, the different encoder network parameters and decoder network parameters in the text information recognition model are adjusted until the loss functions of different dimensions corresponding to the text information recognition model reach the corresponding convergence conditions; so as to achieve the adaptation of the parameters of the text information recognition model to the recognition environment type of the target text information.

12. A text information recognition device, characterized in that: The device comprises: An information transmission module is used to obtain target text information and pre-process the target text information to obtain text information to be identified; An information processing module is used to perform word meaning recognition processing on the text information to be recognized through a text information recognition model to obtain a word meaning feature vector corresponding to the text information to be recognized; The information processing module is configured to extract, based on the recognition environment of the target text information and through the word-phonetic encoder network in the text information recognition model, the pinyin corresponding to each character in the text information to be recognized and the intonation corresponding to each character in the text information to be recognized; determine, based on the pinyin corresponding to each character in the text information to be recognized and the intonation corresponding to each character in the text information to be recognized, a single-character pronunciation feature vector at the level of each character in the text information to be recognized; combine the single-character pronunciation feature vectors corresponding to each character in the text information to be recognized through the word-phonetic encoder network to form a sentence-level pronunciation feature vector; and determine, based on the sentence-level pronunciation feature vector, a word-phonetic feature vector corresponding to the text information to be recognized; The information processing module is used to perform glyph recognition processing on the text information to be recognized through a text information recognition model to obtain a glyph feature vector corresponding to the text information to be recognized; The information processing module is used to perform feature fusion processing on the character meaning feature vector, the character pronunciation feature vector and the character shape feature vector to obtain a fused feature vector; The information processing module is used to identify the text information to be identified based on the fused feature vector, and obtain the probability of selecting an erroneous character corresponding to the text information to be identified.

13. The device according to claim 12, characterized in that The information processing module is used to process the text information to be recognized through the meaning encoder network in the text information recognition model to determine the character feature vector corresponding to the text information to be recognized; determine at least one word-level latent variable corresponding to the text information to be recognized based on the character feature vector through the meaning encoder network; and determine the meaning feature vector corresponding to the text information to be recognized through the meaning encoder network based on the at least one word-level latent variable and the character feature vector corresponding to the text information to be recognized.

14. The device according to claim 12, characterized in that The information processing module is used to extract the character images corresponding to the text information to be recognized by using the glyph encoder network in the text information recognition model; and perform noise reduction processing on the simplified character images by using the glyph encoder network; Performing cross-downsampling processing on the image of the character that has undergone noise reduction processing by the glyph encoder network to obtain a downsampling result of the character image, and normalizing the downsampling result; Performing horizontal normalization processing on the downsampling result of the character image to obtain a horizontal normalization processing result; and determining a glyph feature vector corresponding to the text information to be recognized based on the horizontal normalization processing result.

15. An electronic device, characterized in that: The electronic device comprises: a memory for storing executable instructions; The processor is configured to implement the text information recognition method according to any one of claims 1 to 11 when running the executable instructions stored in the memory.

16. A computer-readable storage medium storing executable instructions, characterized in that: When the executable instructions are executed by a processor, the text information recognition method according to any one of claims 1 to 11 is implemented.

17. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the text information recognition method according to any one of claims 1 to 11 is implemented.

Images