A character recognition method, apparatus, device and medium

Abstract

This application discloses a character recognition method. Character recognition is achieved using a pre-trained character recognition model, which is generated by training on a feature vector set corresponding to a sample image and a processed feature vector set. The processed feature vector set refers to the feature vector set after masking a portion of the feature vectors in the original feature vector set. In other words, this application trains the character recognition network model without manually annotating the sample images; instead, it trains the model by aligning the original and processed feature vector sets, reducing annotation costs and improving training efficiency. In practical use, the text image to be processed is input into the character recognition network model, enabling the model to completely extract the features of the characters to be recognized from the text image and perform recognition based on the extracted features to obtain the recognition result.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to a character recognition method, apparatus, device, and medium. Background Technology

[0002] Optical Character Recognition (OCR) is a technology that analyzes and recognizes text in image files containing text data to obtain the text. It is an important aspect of automatic recognition technology research and application.

[0003] Typically, OCR recognition network models are generated through supervised training methods. This training process requires collecting manually labeled sample data, which is then used for training. To improve the recognition accuracy of the OCR network model, collecting a large amount of sample data necessitates significant manual labeling, increasing training costs. Summary of the Invention

[0004] In view of this, embodiments of this application provide a character recognition method, apparatus, device, and medium to enable training of a model using unlabeled sample data, reduce training costs, and then use the trained character recognition model for text recognition.

[0005] To achieve the above objectives, the technical solutions provided in this application are as follows:

[0006] In a first aspect of this application, a character recognition method is provided, the method comprising:

[0007] Acquire a text image to be processed, wherein the text image to be processed includes text to be recognized;

[0008] The text image to be processed is input into a character recognition network model to obtain a recognition result, the recognition result including the text to be recognized;

[0009] The character recognition network model is generated by training on sample images. During training, the model is trained using the feature vector set corresponding to the sample images and the processed feature vector set. The processed feature vector set refers to the feature vector set after masking a portion of the feature vectors in the feature vector set. The sample images include text information.

[0010] In a second aspect of this application, a character recognition device is provided, the device comprising:

[0011] The first acquisition unit is used to acquire a text image to be processed, wherein the text image to be processed includes text to be recognized.

[0012] The second acquisition unit is used to input the text image to be processed into the character recognition network model to obtain the recognition result, wherein the recognition result includes the text to be recognized.

[0013] The character recognition network model is generated by training on sample images. During training, the model is trained using the feature vector set corresponding to the sample images and the processed feature vector set. The processed feature vector set refers to the feature vector set after masking a portion of the feature vectors in the feature vector set. The sample images include text information.

[0014] In a third aspect of this application, an electronic device is provided, the device comprising: a processor and a memory;

[0015] The memory is used to store instructions or computer programs;

[0016] The processor is configured to execute the instructions or computer program in the memory to cause the electronic device to perform the character recognition method described in the first aspect of this application.

[0017] In a fourth aspect of this application, a computer-readable storage medium is provided, wherein instructions are stored therein, which, when executed on a device, cause the device to perform the character recognition method described in the first aspect of this application.

[0018] In a fifth aspect of this application, a computer program product is provided that, when the computer program product is run on a computer, causes the computer to perform the character recognition method described in the first aspect.

[0019] Therefore, the embodiments of this application have the following beneficial effects:

[0020] In this embodiment, character recognition is achieved using a pre-trained character recognition model. This model is generated through training using the feature vector set corresponding to the sample image and the processed feature vector set. The processed feature vector set refers to the feature vector set after masking a portion of the feature vectors. In other words, this application does not require manual annotation of the sample images during character recognition model training. Instead, it trains the character recognition network model by aligning the original and processed feature vector sets, reducing annotation costs and improving training efficiency. In practical use, the text image to be processed is input into the character recognition network model, enabling the model to completely extract the features of the characters to be recognized from the text image and perform recognition based on the extracted features to obtain the recognition result. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart of a character recognition network model training method provided in this application embodiment;

[0023] Figure 2 This is a schematic flowchart of a character recognition method provided in an embodiment of this application;

[0024] Figure 3 A schematic diagram of a character recognition device provided in an embodiment of this application;

[0025] Figure 4 This is a schematic diagram of an electronic device structure provided in an embodiment of this application. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0027] To facilitate understanding of the technical solutions provided in the embodiments of this application, the technical terms involved in this application will be explained below.

[0028] The OCR recognition network model mainly consists of two steps in its processing flow: text detection and text recognition. Text detection is used to locate text regions in an image, that is, the image is input into a text detection algorithm to obtain text boxes; text recognition is used to identify the text in the text regions, and each text box is fed into a text recognition algorithm to obtain the recognition result. The accuracy of text region localization directly affects the subsequent recognition results. Typically, the structure of an OCR recognition network model includes Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), and uses the Connectionist Temporal Classification (CTC) algorithm to achieve text recognition.

[0029] In practical applications, training an OCR recognition network model requires a large amount of data, typically 1,000 times the size of the character set. For example, to train a model that can recognize 5,000 characters, at least 5 million data points are needed. Labeling each data point would consume a lot of time, resulting in high training costs.

[0030] Based on this, the present application provides a character recognition method. The character recognition network model used in this method is trained and generated using the feature vector set corresponding to the sample image and the feature vector set after masking. By using self-supervised learning between the feature vector sets before and after processing, the character recognition network model is trained and generated, eliminating the need to label the sample images and reducing training costs.

[0031] To facilitate understanding of the training method provided in this application, the following description will be provided in conjunction with the accompanying drawings.

[0032] See Figure 1 This figure is a flowchart of a character recognition network model training method provided in an embodiment of this application. This training method can be executed by a character recognition device. The character recognition device can be an electronic device, or other devices, without limitation. The electronic device can include mobile phones, tablets, desktop computers, laptops, vehicle terminals, wearable electronic devices, all-in-one machines, smart home devices, and other devices with communication functions, or devices simulated by virtual machines or simulators. Figure 1 As shown, the method may include the following steps:

[0033] S101: Obtain the sample image.

[0034] In this embodiment, to train a character recognition network model with high recognition accuracy, a large number of sample images, including text information, can be acquired. Specifically, when training for different application scenarios, sample images corresponding to that scenario are acquired, ensuring that the trained character recognition network model can recognize the text information in the images collected for that scenario. The text information can include Chinese characters, English letters, numbers, and other characters.

[0035] S102: Input the sample image into the encoder of the initial network model to obtain the feature vector set.

[0036] After acquiring the sample image, it is input into the encoder of the initial network model to obtain a feature vector set through the encoder's encoding operation. The feature vectors in this feature vector set are the feature vectors of the text information. Specifically, the sample image is input into the img2vec module to perform downsampling processing on the image, obtaining a sequence of a preset length. This preset length sequence is then input into the encoder to obtain the feature vector set.

[0037] S103: Decode the feature vector set to obtain a first decoding result.

[0038] After obtaining the feature vector set, perform decoding processing on the feature vector set to obtain a first decoding result corresponding to the feature vector set. The first decoding result includes text information. The text information includes various types of characters included in the sample image, such as Chinese characters, English words, numbers, etc. That is, through the initial decoding of the feature vector set, the recognition result is obtained.

[0039] Optionally, a pre-configured decoding algorithm can be used to decode the feature vector set to obtain a first decoding result. Among them, the pre-configured decoding algorithm can be the CTC algorithm, the attention algorithm (Attention), and the ACE algorithm.

[0040] S104: Perform fine-grained partitioning on the first decoding result to obtain a partitioning result, and determine a masked object in the partitioning result.

[0041] After obtaining the first decoding result, perform fine-grained partitioning on the first decoding result to obtain a partitioning result, and determine a masked object in the partitioning result. Among them, performing fine-grained partitioning on the first decoding result means partitioning the first decoding result according to different partitioning granularities to obtain a partitioning result at that partitioning granularity.

[0042] Among them, the partitioning granularities include word granularity, letter granularity, and entity granularity. Word granularity means partitioning the text information in the first decoding result according to the granularity of individual words. For example, if the first decoding result is "This year's flag is to become a teacher...", the partitioning result according to word granularity is "This", "year", "s", "flag", "is", etc. Letter granularity is partitioning according to the granularity of individual letters, mainly referring to the partitioning of English words, and for Chinese characters, it is still partitioning according to the granularity of individual characters. For example, the result according to letter granularity is "This", "year", "s", "f", "l", "a", "g", "is", etc. Entity granularity is partitioning according to a pre-configured entity name, and common recognitions include company names, professional names, etc.

[0043] Optionally, the first decoding result is divided based on different partitioning granularities to obtain partitioning results at each granularity; a target partitioning granularity is determined, and the partitioning result corresponding to the target granularity is obtained. That is, the first decoding result is divided into different partitioning granularities to obtain a partitioning result for each granularity, and then one of these granularities is selected as the target partitioning granularity, with the partitioning result corresponding to the target granularity being determined as the final partitioning result. For example, the first decoding result is divided into word-level, letter-level, and entity-level partitioning results, and if word-level is randomly selected as the target partitioning granularity, then the partitioning result corresponding to the word-level partitioning granularity is taken as the final partitioning result.

[0044] Optionally, the first decoding result can be divided based on the selected target partitioning granularity to obtain the partitioning result. That is, firstly, the target partitioning granularity is determined from multiple partitioning granularities, and then the first decoding result is divided according to the target partitioning granularity to directly obtain the partitioning result. For example, if the entity granularity is determined as the target partitioning granularity from three partitioning granularities—word granularity, letter granularity, and entity granularity—then the first decoding result is divided according to the entity granularity to obtain the partitioning result.

[0045] After determining the partitioning result, mask objects are identified within this result. These mask objects are the objects that require masking. Specifically, mask objects can be determined from the partitioning result based on a preset masking ratio, typically between 5% and 25%. Further, mask objects are determined according to the preset masking ratio and a random selection function. For example, if the partitioning result includes 100 objects and the preset masking ratio is 5%, then the number of mask objects is 5. Therefore, 5 objects are randomly selected from the partitioning result as mask objects.

[0046] S105: Determine the feature vector corresponding to the masked object in the feature vector set, and perform masking processing on the feature vector to obtain the processed feature vector set.

[0047] After determining the mask object from the partitioning results, the eigenvector corresponding to the mask object in the eigenvector set is determined, and the eigenvector is masked to obtain the processed eigenvector set. The masking process may include setting the eigenvector parameter corresponding to the mask object to 0.

[0048] Optionally, determining the feature vector corresponding to the mask object in the feature vector set includes: determining the feature position of the mask object in the feature vector set based on the position of the mask object in the text information; and determining the feature vector located at the feature position as the feature vector corresponding to the mask object in the feature vector set. Here, there is a mapping relationship between the position of the mask object in the text information and the feature position of the mask object in the feature vector set. After determining the position information of the mask object in the text information, the feature position of the mask object in the feature vector set can be determined according to the above mapping relationship, thereby determining the feature vector corresponding to the mask object in the feature vector set.

[0049] S106: Input the processed feature vector set into the decoder of the initial network model to obtain the second decoding result.

[0050] S107: Adjust the parameters of the initial network model based on the first and second decoding results, re-execute the encoder of the initial network model that inputs the sample image and subsequent operations until the preset conditions are met, and generate a character recognition model.

[0051] After obtaining the processed feature vector set, this set is input into the decoder of the initial network model to obtain the second decoding result. That is, the decoder of the initial network model uses the masked feature vector set for decoding and recognition to obtain the second decoding result. After obtaining the second decoding result, since it is determined based on the masked feature vector set, while the first decoding result is determined based on the complete feature vector set, there is a difference between the two. To improve the predictive ability of the decoder of the initial network model, the parameters of the initial network model (including the parameters of the encoder and decoder) are adjusted based on the first and second decoding results. After adjusting the parameters of the initial network model, the training operations S102-S107 above are continued until the loss between the first and second decoding results is less than a preset threshold, thus satisfying the preset condition and obtaining the character recognition model.

[0052] As can be seen, the character recognition network model in this embodiment is trained and generated using the feature vector set corresponding to the sample image and the feature vector set after masking. By using self-supervised learning between the feature vector sets before and after processing, the character recognition network model is trained and generated without the need to label the sample images, thus reducing the training cost.

[0053] Based on the above method embodiments, this application provides a character recognition method, which will be described below with reference to the accompanying drawings.

[0054] See Figure 2This figure illustrates a character recognition method provided in an embodiment of this application. This method can be executed by a character recognition device. The character recognition device can be an electronic device, or other devices, without limitation. The electronic device can include mobile phones, tablets, desktop computers, laptops, vehicle terminals, wearable electronic devices, all-in-one machines, smart home devices, and other devices with communication functions, or devices simulated by virtual machines or simulators. Figure 2 As shown, the method may include the following steps:

[0055] S201: Obtain the text image to be processed, which includes the text to be recognized.

[0056] S202: Input the text image to be processed into the character recognition network model to obtain the recognition result, which includes the text to be recognized.

[0057] In this embodiment, through Figure 1 After the character recognition network model is trained and generated in the embodiment described above, it can be directly used to recognize the text in the text image to be processed.

[0058] The character recognition network model is trained using sample images. Training utilizes both the feature vector set corresponding to the sample images and the processed feature vector set. The processed feature vector set refers to the feature vector set obtained by masking a portion of the feature vectors. The sample images include text information. For a detailed explanation of the character recognition network model's training process, please refer to [link to relevant documentation]. Figure 1 The embodiments shown are not described in detail here.

[0059] As described above, character recognition is achieved using a pre-trained character recognition model. This model is generated through training using the feature vector set corresponding to the sample image and the processed feature vector set. The processed feature vector set refers to the feature vector set obtained by masking a portion of the feature vectors in the original feature vector set. In other words, this application eliminates the need for manual annotation of sample images when training the character recognition model. Instead, it trains the character recognition network model by aligning the original and processed feature vector sets, reducing annotation costs and improving training efficiency. In practical use, the text image to be processed is input into the character recognition network model, enabling the model to fully extract the features of the text to be recognized in the image and perform recognition based on the extracted features to obtain the recognition result.

[0060] Based on the above method embodiments, this application provides a character recognition device and an electronic device, which will be described below with reference to the accompanying drawings.

[0061] See Figure 3 This figure is a structural diagram of a character recognition device provided in an embodiment of this application, such as... Figure 3 As shown, the device may include: a first acquisition unit 301 and a second acquisition unit 302.

[0062] The first acquisition unit 301 is used to acquire a text image to be processed, wherein the text image to be processed includes text to be recognized.

[0063] The second acquisition unit 302 is used to input the text image to be processed into a character recognition network model to obtain a recognition result, wherein the recognition result includes the text to be recognized.

[0064] The character recognition network model is generated by training on sample images. During training, the model is trained using the feature vector set corresponding to the sample images and the processed feature vector set. The processed feature vector set refers to the feature vector set after masking a portion of the feature vectors in the feature vector set. The sample images include text information.

[0065] In one specific implementation, the device further includes: a training unit;

[0066] The training unit is specifically configured to: input the sample image into the encoder of the initial network model to obtain a feature vector set, wherein the feature vectors in the feature vector set are feature vectors of the text information; decode the feature vector set to obtain a first decoding result, wherein the first decoding result includes the text information; perform fine-grained segmentation on the first decoding result to obtain a segmentation result, and determine a mask object in the segmentation result, wherein the mask object is one or more objects in the segmentation result; determine the feature vector corresponding to the mask object in the feature vector set, and perform masking processing on the feature vector to obtain a processed feature vector set; input the processed feature vector set into the decoder of the initial network model to obtain a second decoding result; adjust the parameters of the initial network model according to the first decoding result and the second decoding result, and re-execute the input of the sample image into the encoder of the initial network model and subsequent operations until the preset conditions are met to generate the character recognition model.

[0067] In one specific implementation, the training unit is specifically used to divide the first decoding result based on a selected target partitioning granularity to obtain a partitioning result, wherein the target partitioning granularity is one of word granularity, letter granularity, and entity granularity; and / or to divide the first decoding result based on different partitioning granularities to obtain partitioning results under each partitioning granularity, wherein the partitioning granularity includes word granularity, letter granularity, and entity granularity; to determine the target partitioning granularity and obtain the partitioning result corresponding to the target partitioning granularity.

[0068] In one specific implementation, the training unit is specifically used to determine the mask object in the partitioning result according to a preset mask ratio, wherein the preset mask ratio is greater than or equal to 5% and less than or equal to 25%.

[0069] In one specific implementation, the training unit is specifically used to determine the feature position of the mask object in the feature vector set based on the position of the mask object in the text information; and to determine the feature vector located at the feature position as the feature vector corresponding to the mask object in the feature vector set.

[0070] In one specific implementation, the training unit is specifically used to decode the feature vector set using a preset decoding algorithm to obtain a first decoding result.

[0071] In one specific implementation, the preset decoding algorithm includes the Connectionist Temporal Classification (CTC) decoding algorithm.

[0072] It should be noted that the implementation of each unit in this embodiment can be found in the relevant descriptions in the above method embodiments.

[0073] See Figure 4 This document illustrates a structural schematic diagram of an electronic device 400 suitable for implementing embodiments of this application. The terminal devices in these embodiments may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Android devices), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs (television sets), desktop computers, etc. Figure 4 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0074] like Figure 4 As shown, electronic device 400 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 402 or a program loaded from storage device 408 into random access memory (RAM) 403. RAM 403 also stores various programs and data required for the operation of electronic device 400. The processing device 401, ROM 402, and RAM 403 are interconnected via bus 404. Input / output (I / O) interface 405 is also connected to bus 404.

[0075] Typically, the following devices can be connected to I / O interface 405: input devices 406 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 408 including, for example, magnetic tapes, hard disks, etc.; and communication devices 409. Communication device 409 allows electronic device 400 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4 An electronic device 400 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0076] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 409, or installed from storage device 408, or installed from ROM 402. When the computer program is executed by processing device 401, it performs the functions defined in the methods of embodiments of this application.

[0077] The electronic device provided in this application embodiment belongs to the same inventive concept as the method provided in the above embodiments. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0078] This application provides a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any of the above embodiments.

[0079] It should be noted that the computer-readable medium described above in this application can be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0080] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0081] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0082] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the aforementioned character recognition method.

[0083] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0084] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0085] The units described in the embodiments of this application can be implemented in software or in hardware. The name of the unit / module does not necessarily limit the unit itself; for example, a voice data acquisition module can also be described as a "data acquisition module".

[0086] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0087] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0088] According to one or more embodiments of this application, a character recognition method is provided, which may include:

[0089] Acquire a text image to be processed, wherein the text image to be processed includes text to be recognized;

[0090] The text image to be processed is input into a character recognition network model to obtain a recognition result, the recognition result including the text to be recognized;

[0091] The character recognition network model is generated by training on sample images. During training, the model is trained using the feature vector set corresponding to the sample images and the processed feature vector set. The processed feature vector set refers to the feature vector set after masking a portion of the feature vectors in the feature vector set. The sample images include text information.

[0092] According to one or more embodiments of this application, the training process of the character recognition network model includes:

[0093] The sample image is input into the encoder of the initial network model to obtain a feature vector set, and the feature vectors in the feature vector set are the feature vectors of the text information.

[0094] The feature vector set is decoded to obtain a first decoding result, the first decoding result including the text information;

[0095] The first decoding result is divided into fine-grained parts to obtain the division result, and a mask object is determined in the division result, wherein the mask object is one or more objects in the division result;

[0096] Determine the feature vector corresponding to the masked object in the feature vector set, and perform masking processing on the feature vector to obtain the processed feature vector set;

[0097] The processed feature vector set is input into the decoder of the initial network model to obtain the second decoding result;

[0098] Based on the first decoding result and the second decoding result, the parameters of the initial network model are adjusted, and the encoder of the initial network model that inputs the sample image is re-executed and subsequent operations are performed until the preset conditions are met, thereby generating the character recognition model.

[0099] According to one or more embodiments of this application, dividing the first decoding result to obtain a division result includes:

[0100] The first decoding result is divided based on the selected target partitioning granularity to obtain a partitioning result, wherein the target partitioning granularity is one of word granularity, letter granularity, and entity granularity; and / or,

[0101] The first decoding result is divided based on different partitioning granularities to obtain the partitioning result under each partitioning granularity. The partitioning granularity includes word granularity, letter granularity, and entity granularity.

[0102] Determine the target partitioning granularity and obtain the partitioning result corresponding to the target partitioning granularity.

[0103] According to one or more embodiments of this application, determining the mask object in the partitioning result includes:

[0104] The masked object is determined in the partitioning result according to a preset mask ratio, wherein the preset mask ratio is greater than or equal to 5% and less than or equal to 25%.

[0105] According to one or more embodiments of this application, determining the feature vector corresponding to the mask object in the feature vector includes:

[0106] The feature position of the mask object in the feature vector set is determined based on the position of the mask object in the text information;

[0107] The feature vector located at the feature position is determined as the feature vector corresponding to the mask object in the feature vector set.

[0108] According to one or more embodiments of this application, decoding the feature vector set to obtain a first decoding result includes:

[0109] The feature vector set is decoded using a preset decoding algorithm to obtain a first decoding result.

[0110] According to one or more embodiments of this application, the preset decoding algorithm includes the Connectionist Temporal Classification (CTC) decoding algorithm.

[0111] According to one or more embodiments of this application, a character recognition device is provided, the device comprising:

[0112] The first acquisition unit is used to acquire a text image to be processed, wherein the text image to be processed includes text to be recognized.

[0113] The second acquisition unit is used to input the text image to be processed into the character recognition network model to obtain the recognition result, wherein the recognition result includes the text to be recognized.

[0114] The character recognition network model is generated by training on sample images. During training, the model is trained using the feature vector set corresponding to the sample images and the processed feature vector set. The processed feature vector set refers to the feature vector set after masking a portion of the feature vectors in the feature vector set. The sample images include text information.

[0115] According to one or more embodiments of this application, the apparatus further includes: a training unit;

[0116] The training unit is specifically configured to: input the sample image into the encoder of the initial network model to obtain a feature vector set, wherein the feature vectors in the feature vector set are feature vectors of the text information; decode the feature vector set to obtain a first decoding result, wherein the first decoding result includes the text information; perform fine-grained segmentation on the first decoding result to obtain a segmentation result, and determine a mask object in the segmentation result, wherein the mask object is one or more objects in the segmentation result; determine the feature vector corresponding to the mask object in the feature vector set, and perform masking processing on the feature vector to obtain a processed feature vector set; input the processed feature vector set into the decoder of the initial network model to obtain a second decoding result; adjust the parameters of the initial network model according to the first decoding result and the second decoding result, and re-execute the input of the sample image into the encoder of the initial network model and subsequent operations until the preset conditions are met to generate the character recognition model.

[0117] According to one or more embodiments of this application, the training unit is specifically configured to divide the first decoding result based on a selected target partitioning granularity to obtain a partitioning result, wherein the target partitioning granularity is one of word granularity, letter granularity, and entity granularity; and / or to divide the first decoding result based on different partitioning granularities to obtain partitioning results under each partitioning granularity, wherein the partitioning granularity includes word granularity, letter granularity, and entity granularity; to determine a target partitioning granularity and obtain the partitioning result corresponding to the target partitioning granularity.

[0118] According to one or more embodiments of this application, the training unit is specifically used to determine a mask object in the partitioning result according to a preset mask ratio, wherein the preset mask ratio is greater than or equal to 5% and less than or equal to 25%.

[0119] According to one or more embodiments of this application, the training unit is specifically configured to determine the feature position of the mask object in the feature vector set based on the position of the mask object in the text information; and to determine the feature vector located at the feature position as the feature vector corresponding to the mask object in the feature vector set.

[0120] According to one or more embodiments of this application, the training unit is specifically used to decode the feature vector set using a preset decoding algorithm to obtain a first decoding result.

[0121] According to one or more embodiments of this application, the preset decoding algorithm includes the Connectionist Temporal Classification (CTC) decoding algorithm.

[0122] According to one or more embodiments of this application, an electronic device is provided, the device comprising: a processor and a memory;

[0123] The memory is used to store instructions or computer programs;

[0124] The processor is configured to execute the instructions or computer program in the memory to cause the electronic device to perform the character recognition method described above.

[0125] According to one or more embodiments of this application, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, and when the instructions are executed on a device, the device performs the character recognition method described above.

[0126] According to one or more embodiments of this application, a computer program product is provided that, when the computer program product is run on a computer, causes the computer to perform the character recognition method described above.

[0127] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems or apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and relevant parts can be referred to the method section.

[0128] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0129] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0130] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0131] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A character recognition method characterized by, The method comprises: acquiring a to-be-processed text image, the to-be-processed text image comprising to-be-recognized text; inputting the to-be-processed text image into a character recognition network model to obtain a recognition result, the recognition result comprising the to-be-recognized text; wherein the character recognition network model is generated by training using sample images, and the character recognition network model is generated by training through aligning a feature vector set corresponding to the sample images and a processed feature vector set during training, the processed feature vector set being a feature vector set obtained by performing mask processing on part of the feature vectors in the feature vector set, and the sample images comprising text information; wherein the character recognition network model is trained through the following training process: inputting the sample images into an encoder of an initial network model to obtain a feature vector set, the feature vectors in the feature vector set being feature vectors of the text information; decoding the feature vector set to obtain a first decoding result, the first decoding result comprising the text information; performing fine-grained division on the first decoding result to obtain a division result, and determining a mask object in the division result, the mask object being one or more objects in the division result; determining the feature vector corresponding to the mask object in the feature vector set, and performing mask processing on the feature vector to obtain a processed feature vector set; inputting the processed feature vector set into a decoder of the initial network model to obtain a second decoding result; adjusting parameters of the initial network model according to the first decoding result and the second decoding result, re-executing the operation of inputting the sample images into the encoder of the initial network model and subsequent operations until a preset condition is met, and generating the character recognition network model.

2. The method of claim 1, wherein, The division of the first decoding result to obtain a division result comprises: dividing the first decoding result based on a selected target division granularity to obtain a division result, the target division granularity being one of a word granularity, a letter granularity, and an entity granularity; and / or dividing the first decoding result based on different division granularities to obtain division results under each division granularity, the division granularities comprising a word granularity, a letter granularity, and an entity granularity; determining a target division granularity and obtaining a division result corresponding to the target division granularity.

3. The method according to claim 1 or 2, characterized in that, The determination of a mask object in the division result comprises: determining a mask object in the division result according to a preset mask proportion, the preset mask proportion being greater than or equal to 5% and less than or equal to 25%.

4. The method of claim 1, wherein, The determination of the feature vector corresponding to the mask object in the feature vector set comprises: determining a feature position of the mask object in the feature vector set according to a position of the mask object in the text information; determining the feature vector at the feature position as the feature vector corresponding to the mask object in the feature vector set.

5. The method of claim 1, wherein, The decoding of the feature vector set to obtain a first decoding result comprises: decoding the feature vector set using a preset decoding algorithm to obtain a first decoding result.

6. The method of claim 5, wherein, The preset decoding algorithm includes the Connectionist Temporal Classification (CTC) decoding algorithm.

7. A character recognition apparatus characterized by comprising: The device includes: The first acquisition unit is used to acquire a text image to be processed, wherein the text image to be processed includes text to be recognized. The second acquisition unit is used to input the text image to be processed into the character recognition network model to obtain the recognition result, wherein the recognition result includes the text to be recognized. The character recognition network model is generated by training on sample images. During training, the character recognition network model is generated by aligning the feature vector set corresponding to the sample image and the processed feature vector set. The processed feature vector set refers to the feature vector set after masking a portion of the feature vectors in the feature vector set. The sample image includes text information. The character recognition network model is trained through the following training process: The sample image is input into the encoder of the initial network model to obtain a feature vector set, and the feature vectors in the feature vector set are the feature vectors of the text information. The feature vector set is decoded to obtain a first decoding result, the first decoding result including the text information; The first decoding result is divided into fine-grained parts to obtain the division result, and a mask object is determined in the division result, wherein the mask object is one or more objects in the division result; Determine the feature vector corresponding to the masked object in the feature vector set, and perform masking processing on the feature vector to obtain the processed feature vector set; The processed feature vector set is input into the decoder of the initial network model to obtain the second decoding result; Based on the first decoding result and the second decoding result, the parameters of the initial network model are adjusted, and the encoder of the initial network model that inputs the sample image is re-executed and subsequent operations are performed until the preset conditions are met, thereby generating the character recognition network model.

8. An electronic device, comprising: The device includes: a processor and a memory; The memory is used to store instructions or computer programs; The processor is configured to execute the instructions or computer program in the memory to cause the electronic device to perform the character recognition method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on the device, cause the device to perform the character recognition method according to any one of claims 1-6.

Images