A character recognition method, device, equipment, medium and product

Abstract

The present application provides a character recognition method, apparatus, device and medium, the method comprising: an electronic device acquires an image to be recognized, recognizes the image through a character recognition model including an encoder and a decoder, and obtains a recognition result sequence. The loss function of the decoder includes an autoregressive decoding loss function and a generative adversarial loss function, the generative adversarial loss function is obtained according to a discriminator, and the discriminator is used to discriminate the output results in the training of the character recognition model to improve the recognition accuracy of the character recognition model and avoid the exposure of the training data.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence (AI) technology, and in particular to a character recognition method, device, equipment, computer-readable storage medium, and computer program product. Background Art

[0002] At present, many business scenarios require the recognition of text information in images. For example, online banking services often require character recognition of ID images and bill images uploaded by users to input relevant information. The industry usually uses optical character recognition (OCR) to recognize text information in images.

[0003] OCR refers to the process of analyzing and identifying images to obtain text information. OCR usually includes text detection and text recognition. Text detection refers to detecting text areas in an image, and text recognition refers to recognizing text areas to obtain text information.

[0004] As deep learning has made great progress in many tasks, more and more researchers are introducing deep learning into text recognition. Text recognition based on deep learning is mainly divided into connectionist temporal classification (CTC) methods represented by convolutional recurrent neural network (CRNN) and attention methods represented by transformer.

[0005] Both the CTC method and the attention method use an autoregressive structure, specifically using the generated characters to predict the next character. The recognition models in the above methods usually use a larger model architecture and are trained on massive data sets, which improves the expressiveness of the recognition model. However, a highly expressive recognition model can recognize blank areas in an image as sentences in the training data based on the autoregressive property, which exposes the privacy of the training data and reduces the recognition accuracy. Summary of the invention

[0006] The purpose of the present disclosure is to provide a character recognition method, apparatus, device, computer-readable storage medium and computer program product, which can improve the recognition accuracy of characters and avoid the exposure of training data.

[0007] In a first aspect, the present disclosure provides a character recognition method, the method comprising:

[0008] Obtain an image to be recognized;

[0009] The image is recognized by a character recognition model to obtain a recognition result sequence; wherein the character recognition model includes an encoder and a decoder, and the loss function of the decoder includes an autoregressive decoding loss function and a generative adversarial loss function.

[0010] The recognition result sequence is discriminated by using a discriminator, and when the discrimination is passed, the recognition result sequence is output.

[0011] In a second aspect, the present disclosure provides a character recognition device, characterized in that the device comprises:

[0012] A communication module, used for acquiring an image to be identified;

[0013] A recognition module, used to recognize the image through a character recognition model to obtain a recognition result sequence; wherein the character recognition model includes an encoder and a decoder, and the loss function of the decoder includes an autoregressive decoding loss function and a generative adversarial loss function;

[0014] The discriminant module is used to discriminate the recognition result sequence using a discriminator, and output the recognition result sequence when the discrimination passes.

[0015] In a third aspect, the present disclosure provides an electronic device, including:

[0016] a storage device having a computer program stored thereon;

[0017] A processing device is used to execute the computer program in the storage device to implement the steps of the method described in any one of the first aspect or the second aspect of the present disclosure.

[0018] In a fourth aspect, the present disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the method described in any one of the first aspect or the second aspect of the present disclosure.

[0019] In a fifth aspect, the present disclosure provides a computer program product comprising instructions, which, when executed on a device, enables the device to execute the method described in any one of the implementations of the first or second aspect.

[0020] It can be seen from the above technical solutions that the present disclosure has at least the following advantages:

[0021] In the above technical solution, the electronic device obtains an image to be recognized, recognizes the image through a character recognition model including an encoder and a decoder, and obtains a recognition result sequence. The loss function of the decoder includes an autoregressive decoding loss function and a generative adversarial loss function, and the generative adversarial loss function is obtained according to the discriminator. The discriminator is used to discriminate the output results in the training of the character recognition model to improve the recognition accuracy of the character recognition model and avoid the exposure of the training data.

[0022] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical method of the embodiments of the present application, the drawings required for use in the embodiments are briefly introduced below.

[0024] Figure 1 A flowchart of a character recognition method provided in an embodiment of the present application;

[0025] Figure 2 A schematic diagram of an encoding-decoding model provided in an embodiment of the present application;

[0026] Figure 3 A schematic diagram of another encoding-decoding model provided in an embodiment of the present application;

[0027] Figure 4 A diagram of an encoding-decoding model with an additional discriminator provided in an embodiment of the present application;

[0028] Figure 5 A schematic diagram of an image to be recognized provided in an embodiment of the present application;

[0029] Figure 6 A structural schematic diagram of a character recognition device provided by an embodiment of the present disclosure;

[0030] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] The terms "first" and "second" in the embodiments of the present application are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0032] First, some technical terms involved in the embodiments of the present application are introduced.

[0033] In the process of recognizing text information in an image based on optical character recognition (OCR), text recognition based on deep learning is usually used, such as the connectionist temporal classification (CTC) method and the attention method. Both the CTC method and the attention method use an autoregressive structure, which means predicting the character at the next position based on the character predicted at the previous position. Therefore, if there is a problem with the character prediction at any position, a series of errors may be caused. For example, when there is a blank area, the blank area may be predicted as a certain character due to a prediction error, and continuous errors may occur due to the characteristics of the autoregressive structure, resulting in the "sentence making" phenomenon. "Sentence making" means creating a sentence that does not exist in the image to be recognized based on the training data, which affects the recognition accuracy. In general, the character that is mistakenly predicted as a blank area is a common character in the training data, and the content generated by "sentence making" is a common string in the training data. Therefore, this method may lead to the exposure of training data, affect the security of training data, and have low recognition accuracy.

[0034] In view of this, the present application provides a character recognition method, which is applied to electronic devices. An electronic device refers to a device with data processing capabilities, such as a server or a terminal. Among them, terminals include but are not limited to smart phones, tablet computers, laptops, personal digital assistants (PDAs) or smart wearable devices. The server can be a cloud server, such as a central server in a central cloud computing cluster, or an edge server in an edge cloud computing cluster. Of course, the server can also be a server in a local data center. A local data center refers to a data center directly controlled by a user.

[0035] Specifically, the electronic device obtains an image to be recognized, recognizes the image through a character recognition model including an encoder and a decoder, and obtains a recognition result sequence. The loss function of the decoder includes an autoregressive decoding loss function and a generative adversarial loss function, and the generative adversarial loss function is obtained according to the discriminator. The discriminator is used to discriminate the output results in the training of the character recognition model to improve the recognition accuracy of the character recognition model and avoid the exposure of the training data.

[0036] On the one hand, the loss function of the decoder includes an autoregressive decoding loss function and a generative adversarial loss function. The autoregressive decoding loss value generated by the autoregressive decoding loss function and the generative adversarial loss value generated by the generative adversarial loss function can update the parameters in the character recognition model, thereby improving the recognition accuracy of the model.

[0037] On the other hand, the loss function of the decoder includes a generative adversarial loss function, which is obtained based on the discriminator. During the training process of the character recognition model, the discriminator can discriminate the recognition result sequence output by the character recognition model, so the trained character recognition model can avoid exposure of the training data.

[0038] In order to make the technical solution of the present disclosure clearer and easier to understand, the character recognition method provided by the embodiment of the present disclosure is introduced below by taking the electronic device as a terminal as an example.

[0039] S102: The terminal obtains an image to be recognized.

[0040] The image to be recognized refers to an image including a text line. The text in the text line can be text in multiple languages, such as Chinese, English, Japanese, Russian, etc. The image of the text line can be an image in various forms, such as a screenshot of an electronic text, or a photo of a printed text, or a photo of a handwritten character. OCR can recognize the image of the text line.

[0041] OCR refers to the process by which electronic devices convert character shapes into computer text and output them. Usually, OCR includes text detection and text recognition. Text detection is used to find and segment text areas in images, and text recognition is used to convert text characters into computer text. The judgment of whether an image is an image to be recognized, that is, whether an image is an image to be recognized, can be made by the user, by the terminal, or by both the user and the terminal.

[0042] Specifically, the user can input the image to be identified in the page corresponding to the terminal, or the user can turn on the camera connected to the terminal, and the terminal determines whether there is an image to be identified within the capture range of the camera. In the scenario where the user and the terminal jointly make a judgment, the terminal can judge whether the image input by the user is the image to be identified, and the user can also judge the image captured by the terminal through the camera, and after confirmation, use the image as the image to be identified.

[0043] The terminal can obtain the image to be recognized in a variety of ways. For example, the terminal can obtain the image to be recognized by calling the corresponding camera, or calling the image to be recognized in the picture stored in the terminal, or calling the image to be recognized in the current page of the terminal. For example, when the user needs to perform character recognition, the terminal can turn on the camera according to the user's choice and obtain the image to be recognized through the information captured by the camera. Among them, the user can capture a certain frame of image as the image to be recognized by clicking the "shoot" control in the terminal, and the terminal can also directly use the image with clear character part in the multiple frames of images captured by the camera as the image to be recognized. Furthermore, when the image automatically determined by the terminal includes an image with clear character part, the terminal can provide the image to the user by displaying it. When the user confirms the image on the terminal, the terminal obtains the image to be recognized.

[0044] S104: The terminal recognizes the image through a character recognition model to obtain a recognition result sequence.

[0045] The character recognition model can be Figure 2 As shown in the figure, it is an encoder-decoder model, which includes an encoder and a decoder. The encoder is used to convert real-world problems into mathematical problems, and the decoder is used to solve mathematical problems and convert the solution results into solutions for the real world. Specifically, Figure 3 As shown, the encoder is used to output the input text, picture, audio and other data as a vector, and the decoder is used to generate corresponding text, etc. from the vector output by the encoder. For example, in this embodiment, the input of the encoder is an image to be recognized, the output of the encoder is a vector, the input of the decoder is a vector, and the output is an initial recognition result sequence. In the encoding-decoding model, regardless of the length of the input and output, the length of the vector output by the encoder to the decoder input is fixed, so the problem of information loss may occur. Between the encoder and the decoder, information is transmitted through vectors, so the encoder needs to compress the information of the entire sequence into a vector of fixed length. On the one hand, the vector cannot fully represent the information of the entire sequence. On the other hand, the information carried by the content input first will be diluted by the information carried by the content input later. Therefore, the decoder cannot obtain enough information of the input sequence when decoding according to the vector, which affects the accuracy of the decoding result.

[0046] The loss function is used to measure the degree of inconsistency between the model's predicted value and the true value. The loss function can be used to measure the quality of the model's prediction model, and can also be used to supplement the model to make the model's prediction results closer to the true value. For example, in the training of the character recognition model, the autoregressive decoding loss value can be determined based on the initial recognition result sequence, the label sequence of the training data, and the autoregressive decoding loss function, and the parameters in the character recognition model are updated using the obtained autoregressive decoding loss value, so that the prediction results of the character recognition model are closer to the true value. The autoregressive decoding loss function can be used to compensate for the errors caused by the limitation of the vector length in the encoding-decoding model, but the autoregressive decoding loss function cannot completely reduce the error. Moreover, even if the loss function of the decoder in the character recognition model includes the autoregressive decoding loss function, there may still be an output result based on the autoregressive property of the encoding-decoding model, identifying the blank area in the image as a sentence in the training data, exposing the privacy of the training data, and reducing the recognition accuracy.

[0047] In this embodiment, the terminal may add a discriminator during the training of the character recognition model to judge the recognition result output by the character recognition model. Specifically, the terminal may use the discriminator in the generative adversarial network (GAN) to generate an adversarial loss function. Specifically, Figure 4 As shown, the discriminator can output the discrimination result based on the encoding features of the recognition result sequence and label sequence output by the decoder for the training data and the encoding features in the encoder, and then determine the generative adversarial loss value based on the recognition result sequence, the discrimination result and the generative adversarial loss function, thereby updating the parameters of the character recognition model according to the loss value of the generative adversarial loss function.

[0048] The "real" / "fake" results output by the discriminator can be used to train the decoder. Generative adversarial network is a deep learning model that includes a generator and a discriminator. Taking the generation of pictures as an example, the generator is used to receive a random noise z and generate pictures through the noise, and the discriminator is used to determine whether the picture is real. During the training process, the goal of the generator is to generate real pictures as much as possible to deceive the discriminator, and the goal of the discriminator is to try to distinguish the pictures generated by the generator from the real pictures, so that the generator can be used to generate pictures.

[0049] Specifically, in GAN, the generation adversarial loss function can be:

[0050]

[0051] Among them, G represents the generator, D represents the discriminator, x represents the real data, pdata represents the probability density distribution of the real data, and z represents the randomly input data.

[0052] From formula (1), we can see that from the perspective of the discriminator D, the discriminator D needs to distinguish the real sample x from the fake sample G(z) as much as possible, so D(x) needs to be as large as possible, and D(G(z)) needs to be as small as possible, that is, V(D, G) needs to be as large as possible. From the perspective of the generator G, the generator hopes that the data G(z) it generates can deceive the discriminator as much as possible, that is, it hopes that D(G(z)) is as large as possible, that is, V(D, G) is as small as possible. Therefore, the two modules of GAN are trained in mutual confrontation to achieve the global optimum.

[0053] In this embodiment, the generator in the GAN is a decoder, which is used to decode according to the vector output by the encoder, and the discriminator is used to judge the decoding result (output). When the discriminator judges the output result of the decoder, the label sequence (groundtruth) can be used as a reference. When the output result of the decoder is the same as the reference label sequence, the output is true (real), and when it is different, the output is false (fake).

[0054] In some possible implementations, the encoder and decoder encode and decode the text line characters in the image to be recognized character by character, and the discriminator judges the output result of the decoder sentence by sentence. When the output result of the decoder (output sequence) is the same as the reference label sequence (ground truth sequence), the output is true (real), and when they are different, the output is false (fake).

[0055] Since the generative adversarial loss function is generated based on the discriminator and the decoder, and the role of the discriminator is to judge the true / false output of the decoder, a binary classification method can be used to generate the adversarial loss function. Specifically, the terminal can use the crossentropy loss function and the hinge loss function to obtain the generative adversarial loss function.

[0056] The cross entropy loss function is an algorithm commonly used in classification problems. It can be applied to binary classification problems or multi-classification problems. Cross entropy is mainly used to measure the difference between two probability distributions. It can measure the difference between two different probability distributions in the same random variable. In machine learning, it is expressed as the difference between the true probability distribution and the predicted probability distribution. The smaller the cross entropy value, the better the model prediction effect.

[0057] Hinge loss function is an algorithm specifically for binary classification problems. In the hinge loss function, the label value is -1 or 1, and the predicted value y^∈R. Hinge loss function is usually used as the objective function of support vector machine (SVM). SVM is a generalized linear classifier that performs binary classification of data in a supervised learning manner. SVM uses hinge loss function to calculate empirical risk and adds regularization terms to the solution system to optimize structural risk. SVM is a classifier with sparsity and robustness.

[0058] Therefore, the loss function of the decoder includes not only the autoregressive decoding loss function, but also the generative adversarial loss function. In the process of training the character recognition model by the terminal, the autoregressive decoding loss value can be determined according to the recognition result sequence, the label sequence and the autoregressive decoding loss function. The terminal determines the generative adversarial loss value according to the recognition result sequence, the discrimination result and the generative adversarial loss function, and updates the parameters of the character recognition model according to the autoregressive decoding loss value and the generative adversarial loss value, so as to obtain a character recognition model that does not expose the privacy of the training data and has a high recognition accuracy.

[0059] In some possible implementations, since the character recognition model includes a decoder, and the loss function of the decoder includes a generative adversarial loss function, the decoder is updated after being trained together with the discriminator, and the recognition result sequence generated by the character recognition model can avoid exposing the privacy of training data and improve recognition accuracy. Furthermore, the terminal can also discriminate the output recognition result sequence through the discriminator updated by co-training with the decoder, and output the recognition result sequence when the discrimination passes.

[0060] S106: The terminal uses a discriminator to discriminate the recognition result sequence, and outputs the recognition result sequence when the discrimination passes.

[0061] In GAN, the generator (decoder) and the discriminator are trained together, and the decoder and the discriminator are trained alternately during the training process. Specifically, the initial decoder is G0, and the initial discriminator is D0. First, D0 can be fixed, and G0 can be trained to obtain G1. Then G1 can be fixed and D0 can be trained to obtain D1. Then D1 can be fixed and G1 can be trained to obtain G2. Then G2 can be fixed and D1 can be trained to obtain D2, and so on. The decoder and the discriminator are both optimized during the alternating training process.

[0062] The training of the decoder and the discriminator can be done by using the gradient descent method. The gradient descent method is based on the following observation: if the real-valued function F(x) is differentiable and defined at point a, then the function F(x) at point a follows the direction opposite to the gradient. The biggest drop.

[0063] Next, when the user needs to recognize the characters on the paper, Figure 5 Taking the scenario shown as an example, the character recognition method provided by this embodiment is introduced.

[0064] S102: The terminal obtains an image to be recognized.

[0065] The terminal obtains the image to be identified, wherein the terminal can obtain the image to be identified by calling the camera to shoot the content in the paper, or by calling the camera to scan and obtain the image to be identified in the paper. Figure 5 As shown, the picture includes two parts: part A and part B, wherein both part A and part B include text line characters, but part B also includes blank sentences.

[0066] S104: The terminal recognizes the image through a character recognition model to obtain a recognition result sequence.

[0067] When the decoder in the character recognition model has not been trained by the discriminator, the recognition result sequence output for part B may expose the training data. For example, when the training data is a Chinese test, the content of the output part B may be: "The moon shines on the bed, and fill in the blanks with ancient poems." etc. Based on the autoregressive property, the character recognition model recognizes the blank area "" as a sentence in the training data, such as the "fill in the blanks with ancient poems" that appears frequently in Chinese test questions. In this way, the output recognition result sequence will expose the privacy of the training data. For example, in this environment, it can be inferred that the training data includes "fill in the blanks with ancient poems", which leads to the exposure of the training data of the character recognition model.

[0068] When the decoder in the character recognition model is trained by the discriminator, it can output accurate output results that do not expose the training data. When the training data is as shown in the figure, the label sequence of part B is: "1. The moonlight shines on the bed." Therefore, the recognition result sequence output by the decoder is: "The moonlight shines on the bed, fill in the blanks of ancient poems." The discriminator obtains the judgment result: false based on the recognition result sequence of the decoder, the encoding features of the label sequence in the training data, and the encoding features of the encoder in the character recognition model. The character recognition model re-recognizes to avoid exposing the training data.

[0069] Thus, in the application scenario shown in the figure, the character recognition model in this embodiment can obtain an accurate recognition result sequence by recognizing the image to be recognized. When the image to be recognized includes a blank area, an accurate recognition result sequence can be output, thereby avoiding the exposure of training data and protecting data security.

[0070] S106: The terminal uses a discriminator to discriminate the recognition result sequence, and outputs the recognition result sequence when the discrimination passes.

[0071] Furthermore, the discriminator trained together with the decoder can also be used to discriminate the recognition result sequence to ensure the security of the training data. When the discriminator indicates that the discrimination result is passed, the recognition result sequence is output. The recognition result sequence output in this way can further ensure the security of the training data.

[0072] Based on the description of the above content, an embodiment of the present disclosure provides a character recognition method. The terminal acquires an image to be recognized, recognizes the acquired image through a character recognition model, and obtains a recognition result sequence, wherein the character recognition model includes an encoder and a decoder, and the loss function of the decoder includes an autoregressive decoding loss function and a generative adversarial loss function. In the process of training the decoder through a generative adversarial network, the discriminator can discriminate the recognition result sequence output by the character recognition model, thereby improving the recognition accuracy of the character recognition model and preventing the exposure of the training data. Therefore, the character recognition method provided in this embodiment can effectively improve the accuracy of character recognition and avoid the exposure of the privacy of training data.

[0073] Figure 6 is a schematic diagram of a character recognition device according to an exemplary disclosed embodiment. Figure 6 As shown, the character recognition device 600 includes:

[0074] The communication module 602 is used to obtain the image to be identified;

[0075] The recognition module 604 is used to recognize the image through a character recognition model to obtain a recognition result sequence; wherein the character recognition model includes an encoder and a decoder, and the loss function of the decoder includes an autoregressive decoding loss function and a generative adversarial loss function.

[0076] The discriminant module 606 is used to discriminate the recognition result sequence using a discriminator, and output the recognition result sequence when the discrimination passes.

[0077] Optionally, the character recognition model is trained in the following manner:

[0078] Inputting training data into the character recognition model to obtain a recognition result sequence of training images in the training data;

[0079] Inputting the recognition result sequence and the encoding features of the label sequence in the training data and the encoding features of the encoder in the character recognition model into a discriminator to obtain a discrimination result;

[0080] Determine an autoregressive decoding loss value according to the recognition result sequence, the label sequence and the autoregressive decoding loss function, and determine a generative adversarial loss value according to the recognition result sequence, the discrimination result and the generative adversarial loss function;

[0081] Update the parameters of the character recognition model according to the autoregressive decoding loss value and the generative adversarial loss value.

[0082] Optionally, the decoder performs decoding in character units, and the discriminator performs discrimination in sentence units.

[0083] Optionally, the generative adversarial loss function is a cross entropy loss function.

[0084] The functions of the above modules have been described in detail in the method steps in the previous embodiment and will not be repeated here.

[0085] Reference below Figure 7 , which shows a schematic diagram of the structure of an electronic device 700 suitable for implementing the embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 1 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0086] like Figure 7 As shown, the electronic device 700 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 are also stored. The processing device 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0087] Typically, the following devices may be connected to the I / O interface 705: an input device 706 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 708 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 709. The communication device 709 may allow the electronic device 700 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 7 The electronic device 700 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0088] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 709, or installed from a storage device 708, or installed from a ROM 702. When the computer program is executed by the processing device 701, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0089] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0090] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0091] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0092] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains an image to be recognized; recognizes the image through a character recognition model to obtain a recognition result sequence; wherein the character recognition model includes an encoder and a decoder, and the loss function of the decoder includes an autoregressive decoding loss function and a generative adversarial loss function; the recognition result sequence is discriminated by a discriminator, and when the discrimination is passed, the recognition result sequence is output. The computer program code for performing the operation of the present disclosure can be written in one or more programming languages ​​or a combination thereof, and the programming languages ​​include but are not limited to object-oriented programming languages ​​such as Java, Smalltalk, C++, and also conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code can be executed completely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0093] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0094] The modules involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a module does not, in some cases, limit the module itself.

[0095] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0096] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0097] According to one or more embodiments of the present disclosure, Example 1 provides a character recognition method, which includes: acquiring an image to be recognized; recognizing the image through a character recognition model to obtain a recognition result sequence; wherein the character recognition model includes an encoder and a decoder, and the loss function of the decoder includes an autoregressive decoding loss function and a generative adversarial loss function; using a discriminator to discriminate the recognition result sequence, and when the discrimination passes, outputting the recognition result sequence.

[0098] According to one or more embodiments of the present disclosure, Example 2 provides the method of Example 1, and the character recognition model is trained in the following manner: inputting training data into the character recognition model to obtain a recognition result sequence of training images in the training data; inputting the encoding features of the recognition result sequence and the label sequence in the training data and the encoding features of the encoder in the character recognition model into the discriminator to obtain a discrimination result; determining an autoregressive decoding loss value based on the recognition result sequence, the label sequence and the autoregressive decoding loss function, and determining a generative adversarial loss value based on the recognition result sequence, the discriminant result and the generative adversarial loss function; and updating the parameters of the character recognition model based on the autoregressive decoding loss value and the generative adversarial loss value.

[0099] According to one or more embodiments of the present disclosure, Example 3 provides the method of Example 2, wherein the decoder performs decoding in units of characters, and the discriminator performs discrimination in units of sentences.

[0100] According to one or more embodiments of the present disclosure, Example 4 provides the method of any one of Examples 1 to 3, wherein the generative adversarial loss function is a cross entropy loss function.

[0101] According to one or more embodiments of the present disclosure, Example 5 provides a character recognition device, which includes: a communication module, used to acquire an image to be recognized; a recognition module, used to recognize the image through a character recognition model to obtain a recognition result sequence; wherein the character recognition model includes an encoder and a decoder, and the loss function of the decoder includes an autoregressive decoding loss function and a generative adversarial loss function; a discriminant module, used to discriminate the recognition result sequence using a discriminator, and when the discrimination is passed, output the recognition result sequence.

[0102] According to one or more embodiments of the present disclosure, Example 6 provides the device of Example 5, wherein the character recognition model is trained in the following manner: inputting training data into the character recognition model to obtain a recognition result sequence of training images in the training data; inputting the encoding features of the recognition result sequence and the label sequence in the training data and the encoding features of the encoder in the character recognition model into a discriminator to obtain a discrimination result; determining an autoregressive decoding loss value based on the recognition result sequence, the label sequence and the autoregressive decoding loss function, and determining a generative adversarial loss value based on the recognition result sequence, the discrimination result and the generative adversarial loss function; and updating the parameters of the character recognition model based on the autoregressive decoding loss value and the generative adversarial loss value.

[0103] According to one or more embodiments of the present disclosure, Example 7 provides the apparatus of Example 6, wherein the decoder performs decoding in units of characters, and the discriminator performs discrimination in units of sentences.

[0104] According to one or more embodiments of the present disclosure, Example 8 provides the apparatus of Examples 5 to 7, wherein the generative adversarial loss function is a cross entropy loss function.

[0105] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0106] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0107] Although the subject matter has been described in language specific to structural features and / or method logic actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims. Regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Claims

1. A character recognition method, characterized in that: The method comprises: Obtain an image to be recognized; Recognize the image through a character recognition model to obtain a recognition result sequence; wherein the character recognition model includes an encoder and a decoder, and the loss function of the decoder includes an autoregressive decoding loss function and a generative adversarial loss function; Using a discriminator to discriminate the recognition result sequence, and when the discrimination passes, outputting the recognition result sequence; The character recognition model is trained in the following way: Inputting training data into the character recognition model to obtain a recognition result sequence of training images in the training data; Inputting the recognition result sequence and the encoding features of the label sequence in the training data and the encoding features of the encoder in the character recognition model into a discriminator to obtain a discrimination result; Determine an autoregressive decoding loss value according to the recognition result sequence, the label sequence and the autoregressive decoding loss function, and determine a generative adversarial loss value according to the recognition result sequence, the discrimination result and the generative adversarial loss function; Update the parameters of the character recognition model according to the autoregressive decoding loss value and the generative adversarial loss value.

2. The method according to claim 1, characterized in that: The decoder performs decoding in character units, and the discriminator performs discrimination in sentence units.

3. The method according to claim 1 or 2, characterized in that: The generative adversarial loss function is a cross entropy loss function.

4. A character recognition device, characterized in that: The device comprises: A communication module, used for acquiring an image to be identified; A recognition module, used to recognize the image through a character recognition model to obtain a recognition result sequence; wherein the character recognition model includes an encoder and a decoder, and the loss function of the decoder includes an autoregressive decoding loss function and a generative adversarial loss function; A discriminant module, used for discriminating the recognition result sequence using a discriminator, and outputting the recognition result sequence when the discrimination passes; The character recognition model is trained in the following way: Inputting training data into the character recognition model to obtain a recognition result sequence of training images in the training data; Inputting the recognition result sequence and the encoding features of the label sequence in the training data and the encoding features of the encoder in the character recognition model into a discriminator to obtain a discrimination result; Determine an autoregressive decoding loss value according to the recognition result sequence, the label sequence and the autoregressive decoding loss function, and determine a generative adversarial loss value according to the recognition result sequence, the discrimination result and the generative adversarial loss function; Update the parameters of the character recognition model according to the autoregressive decoding loss value and the generative adversarial loss value.

5. The device according to claim 4, characterized in that The decoder performs decoding in character units, and the discriminator performs discrimination in sentence units.

6. The device according to claim 4 or 5, characterized in that The generative adversarial loss function is a cross entropy loss function.

7. An electronic device, characterized in that: include: a storage device having a computer program stored thereon; A processing device, configured to execute the computer program in the storage device to implement the steps of the method according to any one of claims 1 to 3.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processing device, the steps of the method described in any one of claims 1 to 3 are implemented.

9. A computer program product, characterized in that When the computer program product is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 3.

Images