A method and apparatus for training a character recognition model

Abstract

The application provides a training method of a character recognition model, comprising the following steps: obtaining real character information in a to-be-recognized object, recognizing the characters in the to-be-recognized object through a prediction module of an initial character recognition model to obtain prediction result information of the characters in the to-be-recognized object, generating feature relationships through a feature relationship module of the initial character recognition model according to the prediction result information of the characters in the to-be-recognized object and the real character information of the characters in the to-be-recognized object, processing the feature relationships through an attention module of the initial character recognition model to obtain semantic similar information of the characters in the to-be-recognized object, inputting an image containing scene characters in a to-be-recognized scene and the semantic similar information into the initial character recognition model, training the initial character recognition model to obtain a target character recognition model. The model training does not fully utilize semantic information and a large amount of scene characters are required, and the effect of reducing the calculation complexity and improving the recognition accuracy is achieved.

Description

Technical Field

[0001] This invention relates to the field of data processing, and more specifically, to a method and apparatus for training a character recognition model. Background Technology

[0002] Currently, banks receive a large number of manually filled-out documents from customers in their daily operations, such as checks, transfer checks, and deposit slips. The ability to accurately recognize these documents is crucial for the digitalization and online transformation of banking services. Inaccurate recognition can lead to serious incidents and damage the bank's reputation. The accuracy of existing Optical Character Recognition (OCR) technology is insufficient to meet the requirements of banking operations.

[0003] Problems with existing technologies: 1. Semantic information is not fully utilized; 2. Incremental training requires a large amount of scene text data, and the data owned by non-professional scene text R&D institutions is difficult to meet the training requirements.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This invention provides a method for training a text recognition model, which at least addresses the problems in related technologies in the financial field where text recognition model training does not fully utilize semantic information and incremental training requires a large amount of scene text data, while non-specialized scene text research institutions often lack the data to meet training requirements.

[0006] According to an embodiment of the present invention, a method for training a character recognition model is provided, comprising: acquiring real text information in an object to be recognized, and recognizing the text in the object to be recognized through a prediction module of an initial character recognition model to obtain prediction result information of the text in the object to be recognized; generating a feature relationship through a feature relationship module of the initial character recognition model based on the prediction result information and the real text information of the text in the object to be recognized, wherein the feature relationship represents the dependency relationship between the prediction result information and the real text information; processing the feature relationship through an attention module of the initial character recognition model to obtain semantic similarity information of the text in the object to be recognized, wherein the semantic similarity information represents the dependency relationship between the semantics of the text in the scene to be recognized and similar characters in the scene to be recognized; inputting an image containing the text in the scene to be recognized and the semantic similarity information into the initial character recognition model to train the initial character recognition model to obtain a target character recognition model.

[0007] Optionally, the step of obtaining the real text information in the object to be identified and recognizing the text in the object to be identified through the prediction module of the initial text recognition model to obtain the prediction result information of the text in the object to be identified includes: recognizing text features from the image of the text in the scene to be identified through the optical character recognition layer of the prediction module; inputting the text features into the first feedforward neural network of the prediction module, inputting the label information corresponding to the real text information in the object to be identified into the first feedforward neural network, and obtaining the key-value pair output by the first feedforward neural network, which consists of the prediction result information of the text in the object to be identified and the real text information in the object to be identified.

[0008] Optionally, the step of generating a feature relationship through the feature relationship module of the initial character recognition model based on the predicted result information of the characters in the object to be recognized and the actual text information of the characters in the object to be recognized, wherein the feature relationship characterizes the dependency relationship between the predicted result information and the actual text information, includes: inputting the key-value pair into the second feedforward neural network of the feature relationship module to obtain the correspondence between the predicted result information of the characters in the object to be recognized and the actual text information of the object to be recognized output by the second feedforward neural network.

[0009] Optionally, the step of processing the feature relationship through the attention module of the initial character recognition model to obtain the semantic similarity information of the characters in the object to be recognized includes: inputting the feature relationship into the similarity-guided attention layer of the attention module to obtain the similarity of similar characters in the scene to be recognized; inputting the feature relationship into the semantically guided attention layer of the attention module to obtain the semantic similarity of the characters in the scene to be recognized; and adding the similarity of similar characters in the scene to be recognized and the semantic similarity of the characters in the scene to be recognized to obtain the semantic similarity information of the characters in the scene to be recognized.

[0010] Optionally, the step of inputting the feature relationship into the form-guided attention layer of the attention module to obtain the form-similar character similarity of the text in the scene to be identified includes: dividing the feature relationship into multiple non-overlapping windows; independently performing spatial prefix-guided attention in each of the non-overlapping windows to obtain the form-similar character similarity of the text in the scene to be identified; wherein, the spatial prefix is ​​the text prediction result information of the text in the scene to be identified or the glyph information of the actual text content of the text in the scene to be identified.

[0011] Optionally, the step of inputting the feature relationship into the semantically guided attention layer of the attention module to obtain the semantic similarity of the scene text to be identified includes: performing standard attention-guided attention on the relationship features to obtain the semantic similarity of the scene text to be identified.

[0012] According to another embodiment of the present invention, a training apparatus for a character recognition model is also provided, comprising: an acquisition unit, configured to acquire real text information in an object to be recognized, and to recognize the text in the object to be recognized through a prediction module of an initial character recognition model to obtain prediction result information of the text in the object to be recognized; a feature relationship unit, configured to generate a feature relationship through the feature relationship module of the initial character recognition model based on the prediction result information of the text in the object to be recognized and the real text information of the text in the object to be recognized, wherein the feature relationship represents the dependency relationship between the prediction result information and the real text information; a semantic similarity unit, configured to process the feature relationship through the attention module of the initial character recognition model to obtain semantic similarity information of the text in the object to be recognized, wherein the semantic similarity information represents the dependency relationship between the semantics of the text in the scene to be recognized and the similar-looking characters of the text in the scene to be recognized; and a training unit, configured to input an image containing the text in the scene to be recognized and the semantic similarity information into the initial character recognition model, train the initial character recognition model, and obtain a target character recognition model.

[0013] Optionally, the semantic similarity unit includes: a similarity unit, used to input the feature relationship into the similarity-guided attention layer of the attention module to obtain the similarity of similar characters in the scene to be identified; a semantic unit, used to input the feature relationship into the semantically guided attention layer of the attention module to obtain the semantic similarity of the scene to be identified; and a calculation unit, used to add the similarity of similar characters in the scene to be identified and the semantic similarity of the scene to be identified to obtain the semantic similarity information of the scene to be identified.

[0014] According to yet another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, wherein the computer program is executed by a processor to perform the steps in any of the above method embodiments.

[0015] According to yet another embodiment of the present invention, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0016] This invention addresses the problems in related technologies in the financial field where text recognition model training does not fully utilize semantic information and requires a large amount of scene text data for incremental training. It also addresses the difficulty for non-professional scene text research institutions to have sufficient data to meet training requirements. The invention aims to reduce the computational complexity of text recognition models, improve the training speed and recognition accuracy of text recognition models, and achieve a pluggable and flexible application. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0018] Figure 1 This is a hardware structure block diagram of a mobile terminal for training a text recognition model according to an embodiment of the present invention.

[0019] Figure 2 This is a flowchart of a method for training a text recognition model according to an embodiment of the present invention;

[0020] Figure 3 This is a flowchart of the inference stage of the training method for a character recognition model according to an embodiment of the present invention;

[0021] Figure 4 This is a flowchart of the training phase of a text recognition model training method according to an embodiment of the present invention;

[0022] Figure 5 This is a flowchart of a method for training a text recognition model according to another embodiment of the present invention;

[0023] Figure 6 This is a block diagram of a training method apparatus for a character recognition model according to an embodiment of the present invention. Detailed Implementation

[0024] The present invention will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the present application can be combined with each other.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0026] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Taking its operation on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal for the business data analysis method according to an embodiment of the present invention, such as... Figure 1 As shown, a mobile terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. Optionally, the mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0027] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the business data analysis method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0028] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0029] This embodiment provides a training method for a text recognition model running on the aforementioned mobile terminal or network architecture. Figure 2 This is a flowchart of a text recognition model training method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps:

[0030] Step S202: Obtain the real text information in the object to be identified, and use the prediction module of the initial text recognition model to identify the text in the object to be identified, so as to obtain the prediction result information of the text in the object to be identified.

[0031] Step S204: Based on the predicted result information of the text in the object to be identified and the real text information of the text in the object to be identified, a feature relationship is generated through the feature relationship module of the initial text recognition model, wherein the feature relationship represents the dependency relationship between the predicted result information and the real text information;

[0032] Step S206: The feature relationship is processed by the attention module of the initial character recognition model to obtain the semantic similarity information of the characters in the object to be recognized, wherein the semantic similarity information represents the dependency relationship between the semantics of the characters in the scene to be recognized and the similar characters in the scene to be recognized.

[0033] Step S208: Input the image containing the scene text to be recognized and the semantic similarity information into the initial character recognition model, train the initial character recognition model, and obtain the target character recognition model.

[0034] In one specific embodiment, Figure 3 This is a flowchart of the inference stage of the training method for the character recognition model according to an embodiment of the present invention, as follows: Figure 3 As shown. The prediction module of the initial character recognition model corresponds to Figure 3 The basic module (Handwriting Base Module) in the initial text recognition model corresponds to the attention module. Figure 3 The semantic similarity character mining module first inputs the image of the scene text to be recognized into the Handwriting Base Module to obtain the predicted text information of the object to be recognized. Then, based on the predicted information, a feature relationship representing the dependency between the predicted information and the real text information is obtained. This feature relationship is then input into the semantic similarity character mining module to obtain the semantic similarity information of the scene text to be recognized. Finally, the predicted information of the text in the object to be recognized is calibrated based on this semantic similarity information.

[0035] Through the above steps S202 to S208, the problems of insufficient utilization of semantic information and the need for a large amount of scene text data for incremental training when training text recognition models in the financial field can be solved. Non-professional scene text research institutions have their own data that is difficult to meet the training requirements. The results can reduce the computational complexity of text recognition models, improve the training speed and recognition accuracy of text recognition models, and achieve the effect of pluggable and flexible use.

[0036] In an optional embodiment, the step of obtaining the real text information in the object to be identified and recognizing the text in the object to be identified through the prediction module of the initial text recognition model to obtain the prediction result information of the text in the object to be identified includes: recognizing text features from the image of the text in the scene to be identified through the optical character recognition layer of the prediction module; inputting the text features into the first feedforward neural network of the prediction module, inputting the label information corresponding to the real text information in the object to be identified into the first feedforward neural network, and obtaining a key-value pair output by the first feedforward neural network consisting of the prediction result information of the text in the object to be identified and the real text information in the object to be identified.

[0037] In one specific embodiment, Figure 4 This is a flowchart illustrating the training phase of the text recognition model training method according to an embodiment of the present invention, as follows: Figure 4 As shown. The prediction module of the initial character recognition model corresponds to Figure 4 The HandwritingBase Module in the framework can be used with any optical character recognition model. Figure 4 The CRNN and CTC in the model extract text features from the image of the text in the scene to be recognized. These text features are then input into the feedforward neural network (FFN) to label the real text information in the object to be recognized. The real text information in the object to be recognized is then input into the FFN in the form of labels, resulting in key-value pairs composed of the predicted text information and the real text information in the object to be recognized. The calculation formula is as follows:

[0038]

[0039]

[0040] Where t takes values ​​from 0 to N, e represents the input text features and label information, and W... key and W value These are the transformation matrices corresponding to the predicted result information and the actual text information, respectively, b key and b value These are the bias matrices corresponding to the predicted results and the actual text information, respectively. and This refers to the key-value pairs of the predicted information and the actual text information output by the feedforward neural network (FFN).

[0041] In an optional embodiment, the step of generating a feature relationship through the feature relationship module of the initial character recognition model based on the predicted result information of the characters in the object to be recognized and the actual text information of the characters in the object to be recognized, wherein the feature relationship characterizes the dependency relationship between the predicted result information and the actual text information, includes: inputting the key-value pair into the second feedforward neural network of the feature relationship module to obtain the correspondence between the predicted result information of the characters in the object to be recognized and the actual text information of the object to be recognized output by the second feedforward neural network.

[0042] In one specific embodiment, such as Figure 4 As shown, the feature relationship module of the initial character recognition model corresponds to Figure 4 The relation feature generation module (RFG Module) in the middle will... Figure 4 The key-value pairs output from the Handwriting Base Module are input to the Relationship Feature Generation Module (RFG Module), which includes a classifier and an FFN (Focused Feature Filter). Through the FFN in the RFG Module, the correspondence between the predicted text information and the actual text information in the object to be identified is obtained. The calculation formula is as follows:

[0043]

[0044] R (t) =W r l (t) +b r

[0045] Wherein, W1 is the above and Relationship matrix, For calculating the alignment between the predicted results and the actual text information, W2 is the bias matrix, l (t) This is an intermediate result showing the correspondence between predicted information and actual textual information. r W is the bias matrix. r Let R be the transformation matrix. (t) The output result of the previous step l (t) Perform further relationship calculations.

[0046] In another optional embodiment, the step of processing the feature relationship through the attention module of the initial character recognition model to obtain the semantic similarity information of the characters in the object to be recognized includes: inputting the feature relationship into the similarity-guided attention layer of the attention module to obtain the similarity of similar characters in the scene to be recognized; inputting the feature relationship into the semantically guided attention layer of the attention module to obtain the semantic similarity of the characters in the scene to be recognized; and adding the similarity of similar characters in the scene to be recognized and the semantic similarity of the characters in the scene to be recognized to obtain the semantic similarity information of the characters in the scene to be recognized.

[0047] In an optional embodiment, the step of inputting the feature relationship into the form-guided attention layer of the attention module to obtain the form-similar character similarity of the text to be identified includes: segmenting the feature relationship into multiple non-overlapping windows; independently performing spatial prefix-guided attention in each of the non-overlapping windows to obtain the form-similar character similarity of the text to be identified; wherein, the spatial prefix is ​​the text prediction result information of the text to be identified or the glyph information of the actual text content of the text to be identified.

[0048] In an optional embodiment, the step of inputting the feature relationship into the semantically guided attention layer of the attention module to obtain the semantic similarity of the scene text to be identified includes: performing standard attention-guided attention on the relationship features to obtain the semantic similarity of the scene text to be identified.

[0049] In one specific embodiment, such as Figure 4 As shown, the attention module of the initial character recognition model corresponds to Figure 4 The semantic similar-form character mining module consists of a proximity-guided correction layer and a semantic-guided detection layer. The module progressively mines semantic similar-form information using a semantic-proximity approach.

[0050] The proximity-guided correction layer first segments the feature relationships into multiple non-overlapping windows, and then independently performs spatial prefix-guided local self-attention within each window. In the construction of the proximity-guided correction layer, the text prediction results of the text to be recognized or the glyph information of the actual text content of the text to be recognized are used as spatial prefixes to obtain the proximity similarity of the text to be recognized. For example, the proximity similarity can be obtained by calculating the cosine distance between proximity characters; this application does not impose any limitations on this.

[0051] The semantic-guided detection layer performs a standard attention-guided semantic-guided attention layer on the input relational features to obtain the semantic similarity of the text in the scene to be identified.

[0052] The attention calculation method guided by semantic similarity is as follows:

[0053]

[0054] in, It is standard attention. It is an attention layer guided by similar shapes. of Guided by semantics, In S k ,S v Guided by the glyph information of the text prediction results or the actual text content, λ(R) (t) ) is the weight.

[0055] As shown in the formula above, the calculation results of the attention layer guided by similar shapes and the attention layer guided by semantics are added together, thus capturing the dependency relationship between the semantics of the text in the scene to be recognized and the similar-shaped characters in the scene to be recognized. The above semantic similarity-guided attention calculation method can be simplified to the following calculation formula:

[0056] Loss = w·loss p +(1-w)·loss s

[0057] Where w is the weight, and loss p For semantic similarity generated by semantic guidance, loss s The similarity score of similar-looking characters is determined by normalization based on different weights, and then the two scores are output.

[0058] This application obtains the real text information of the object to be identified, and uses the prediction module of an initial text recognition model to identify the text in the object to be identified, obtaining the prediction result information of the text in the object to be identified. Based on the prediction result information and the real text information of the text in the object to be identified, the feature relationship module of the initial text recognition model generates a feature relationship, and the attention module of the initial text recognition model processes the feature relationship to obtain the semantic similarity information of the text in the object to be identified. The image containing the text of the scene to be identified and the semantic similarity information are input into the initial text recognition model to train the initial text recognition model and obtain the target text recognition model. This can solve the problems in related technologies in the financial field where the training of text recognition models does not fully utilize semantic information and incremental training requires a large amount of scene text data, and non-professional scene text research institutions have difficulty meeting the training requirements with their own data. It achieves the effect of reducing the computational complexity of text recognition models, improving the training speed and recognition accuracy of text recognition models, and realizing the effect of pluggable and flexible use.

[0059] Figure 5 This is a flowchart of training a character recognition model according to another embodiment of the present invention, such as... Figure 5 As shown, the process includes the following steps:

[0060] Step S501: Extract text features from the image of the scene text to be recognized through the optical character recognition layer of the prediction module of the initial text recognition model;

[0061] Step S502: Label the real text information in the object to be identified, and input the label information corresponding to the real text information in the object to be identified into the first feedforward neural network of the prediction module.

[0062] Step S503: Input the text features into the first feedforward neural network;

[0063] Step S504: The first feedforward neural network outputs a key-value pair consisting of the predicted text information in the object to be identified and the actual text information in the object to be identified.

[0064] Step S505: Input the key-value pairs into the second feedforward neural network of the feature relationship module of the initial character recognition model to obtain the correspondence between the predicted text information in the object to be recognized and the real text information in the object to be recognized output by the second feedforward neural network.

[0065] Step S506: Segment the feature relationships into multiple non-overlapping windows;

[0066] Step S507: Spatial prefix-guided attention is performed independently in each non-overlapping window to obtain the similarity of similar characters in the scene to be identified;

[0067] Step S508: Perform standard attention-guided attention on relational features to obtain the semantic similarity of the text in the scene to be identified;

[0068] Step S509: Add the similarity scores of the similar-looking characters in the scene to be identified and the semantic similarity scores of the scene to be identified to obtain the semantic similarity information of the scene to be identified;

[0069] Step S510: Input the image containing the text of the scene to be recognized and the semantic similarity information into the initial text recognition model, train the initial text recognition model, and obtain the target text recognition model.

[0070] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0071] According to another embodiment of the present invention, a training apparatus for a character recognition model is also provided. Figure 6 This is a block diagram of a training apparatus for a character recognition model according to an embodiment of the present invention, such as... Figure 6 As shown, the training device for the character recognition model includes: an acquisition unit 602, a feature relationship unit 604, a semantic similarity unit 606, and a training unit 608.

[0072] In one embodiment, the acquisition unit 602 is used to acquire real text information in the object to be identified, and to identify the text in the object to be identified through the prediction module of the initial text recognition model to obtain prediction result information of the text in the object to be identified; the feature relationship unit 604 is used to generate feature relationship through the feature relationship module of the initial text recognition model based on the prediction result information and the real text information of the text in the object to be identified, wherein the feature relationship represents the dependency relationship between the prediction result information and the real text information; the semantic similarity unit 606 is used to process the feature relationship through the attention module of the initial text recognition model to obtain semantic similarity information of the text in the object to be identified, wherein the semantic similarity information represents the dependency relationship between the semantics of the text in the scene to be identified and the similar characters of the text in the scene to be identified; the training unit 608 is used to input the image containing the text in the scene to be identified and the semantic similarity information into the initial text recognition model to train the initial text recognition model to obtain the target text recognition model.

[0073] In an optional embodiment, the acquisition unit includes: an extraction unit, configured to identify text features from the image of the text in the scene to be identified through the optical character recognition layer of the prediction module; and a key-value pair unit, configured to input the text features into the first feedforward neural network of the prediction module, input the label information corresponding to the real text information in the object to be identified into the first feedforward neural network, and obtain a key-value pair output by the first feedforward neural network consisting of the prediction result information of the text in the object to be identified and the real text information in the object to be identified.

[0074] In an optional embodiment, the feature relationship unit includes: a correspondence relationship unit, used to input the key-value pair into the second feedforward neural network of the feature relationship module to obtain the correspondence between the predicted result information of the text in the object to be identified output by the second feedforward neural network and the actual text information in the object to be identified.

[0075] In an optional embodiment, the semantic similarity unit includes: a similarity unit, used to input the feature relationship into the similarity-guided attention layer of the attention module to obtain the similarity of similar characters in the scene text to be identified; a semantic unit, used to input the feature relationship into the semantically guided attention layer of the attention module to obtain the semantic similarity of the scene text to be identified; and a calculation unit, used to add the similarity of similar characters in the scene text to be identified and the semantic similarity of the scene text to be identified to obtain the semantic similarity information of the scene text to be identified.

[0076] In an optional embodiment, the similarity unit includes: a segmentation unit for segmenting the feature relationship into multiple non-overlapping windows; and a similarity attention module for independently performing spatial prefix-guided attention in each non-overlapping window to obtain the similarity of similar characters in the scene text to be identified; wherein the spatial prefix is ​​the character prediction result of the scene text to be identified or the glyph information of the actual text content of the scene text information to be identified.

[0077] In an optional embodiment, the semantic unit includes a semantic attention module for performing standard attention-guided attention on the relational features to obtain the semantic similarity of the scene text to be identified.

[0078] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0079] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to perform the steps in any of the above method embodiments.

[0080] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0081] S1, obtain the real text information in the object to be identified, and use the prediction module of the initial text recognition model to identify the text in the object to be identified, and obtain the prediction result information of the text in the object to be identified;

[0082] S2, based on the predicted result information of the text in the object to be identified and the real text information of the text in the object to be identified, a feature relationship is generated through the feature relationship module of the initial text recognition model, wherein the feature relationship represents the dependency relationship between the predicted result information and the real text information;

[0083] S3, the feature relationship is processed by the attention module of the initial character recognition model to obtain the semantic similarity information of the characters in the object to be recognized, wherein the semantic similarity information represents the dependency relationship between the semantics of the characters in the scene to be recognized and the similar characters in the scene to be recognized.

[0084] S4, input the image containing the text of the scene to be recognized and the semantic similarity information into the initial text recognition model, train the initial text recognition model, and obtain the target text recognition model.

[0085] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0086] Embodiments of the present invention also provide an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.

[0087] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0088] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0089] S1, obtain the real text information in the object to be identified, and use the prediction module of the initial text recognition model to identify the text in the object to be identified, and obtain the prediction result information of the text in the object to be identified;

[0090] S2, based on the predicted result information of the text in the object to be identified and the real text information of the text in the object to be identified, a feature relationship is generated through the feature relationship module of the initial text recognition model, wherein the feature relationship represents the dependency relationship between the predicted result information and the real text information;

[0091] S3, the feature relationship is processed by the attention module of the initial character recognition model to obtain the semantic similarity information of the characters in the object to be recognized, wherein the semantic similarity information represents the dependency relationship between the semantics of the characters in the scene to be recognized and the similar characters in the scene to be recognized.

[0092] S4, input the image containing the text of the scene to be recognized and the semantic similarity information into the initial text recognition model, train the initial text recognition model, and obtain the target text recognition model.

[0093] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0094] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0095] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for training a character recognition model, characterized in that, include: The real text information in the object to be identified is obtained, and the text in the object to be identified is identified by the prediction module of the initial text recognition model to obtain the prediction result information of the text in the object to be identified. Based on the predicted text information and the actual text information in the object to be identified, a feature relationship is generated through the feature relationship module of the initial text recognition model, wherein the feature relationship represents the dependency relationship between the predicted text information and the actual text information. The feature relationship is divided into multiple non-overlapping windows; Spatial prefix-guided attention is performed independently in each non-overlapping window to obtain the similarity of similar characters in the scene to be identified, wherein the spatial prefix is ​​the character prediction result information of the scene to be identified or the glyph information of the actual text content of the scene to be identified; The feature relationships are input into the semantically guided attention layer of the attention module to obtain the semantic similarity of the text in the scene to be identified; The similarity between the similar-looking characters and the semantic similarity of the characters in the scene to be identified are added together to obtain the semantic similarity information of the characters in the scene to be identified. The semantic similarity information represents the dependency relationship between the semantics of the characters in the scene to be identified and the similar-looking characters of the characters in the scene to be identified. The image containing the text of the scene to be recognized and the semantic similarity information are input into the initial text recognition model, and the initial text recognition model is trained to obtain the target text recognition model.

2. The method according to claim 1, characterized in that, The process of obtaining the real text information in the object to be identified, and recognizing the text in the object through the prediction module of the initial text recognition model to obtain the prediction result information of the text in the object to be identified, includes: The optical character recognition layer of the prediction module identifies text features from the image of the scene text to be recognized; The text features are input into the first feedforward neural network of the prediction module, and the label information corresponding to the real text information in the object to be identified is input into the first feedforward neural network to obtain the key-value pair output by the first feedforward neural network, which consists of the prediction result information of the text in the object to be identified and the real text information in the object to be identified.

3. The method according to claim 2, characterized in that, The step involves generating a feature relationship through the feature relationship module of the initial character recognition model based on the predicted text information and the actual text information in the object to be recognized. The feature relationship characterizes the dependency between the predicted text information and the actual text information, including: The key-value pairs are input into the second feedforward neural network of the feature relationship module to obtain the correspondence between the predicted text information of the object to be identified and the actual text information of the object to be identified, as output by the second feedforward neural network.

4. The method according to claim 1, characterized in that, The step of inputting the feature relationship into the semantically guided attention layer of the attention module to obtain the semantic similarity of the text in the scene to be identified includes: Standard attention-guided attention is performed on the feature relationships to obtain the semantic similarity of the text in the scene to be identified.

5. A training device for a character recognition model, characterized in that, include: The acquisition unit is used to acquire the real text information in the object to be identified, and to identify the text in the object to be identified through the prediction module of the initial text recognition model to obtain the prediction result information of the text in the object to be identified. The feature relationship unit is used to generate feature relationships through the feature relationship module of the initial character recognition model based on the predicted result information of the characters in the object to be recognized and the actual text information of the characters in the object to be recognized. The feature relationship represents the dependency relationship between the predicted result information and the actual text information. The semantic similarity unit is used to process the feature relationship through the attention module of the initial character recognition model to obtain the semantic similarity information of the characters in the object to be recognized, wherein the semantic similarity information represents the dependency relationship between the semantics of the characters in the scene to be recognized and the similar characters in the scene to be recognized. The training unit is used to input the image containing the text of the scene to be recognized and the semantic similarity information into the initial text recognition model, train the initial text recognition model, and obtain the target text recognition model. The semantic similarity unit includes: The similarity unit is used to input the feature relationship into the similarity-guided attention layer of the attention module to obtain the similarity of similar characters in the scene to be recognized; A semantic unit is used to input the feature relationship into the semantically guided attention layer of the attention module to obtain the semantic similarity of the text in the scene to be identified; The calculation unit is used to add the similarity of the characters in the scene to be identified and the semantic similarity of the characters in the scene to be identified to obtain the semantic similarity information of the characters in the scene to be identified; The similar-shaped units include: A segmentation unit is used to segment the feature relationship into multiple non-overlapping windows; The similarity attention module is used to independently perform spatial prefix-guided attention in each of the non-overlapping windows to obtain the similarity of similar characters in the scene text to be identified, wherein the spatial prefix is ​​the character prediction result of the scene text to be identified or the glyph information of the actual text content of the scene text information to be identified.

6. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, wherein the computer program is executed by a processor to perform the method described in any one of claims 1 to 4.

7. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the method of any one of claims 1 to 4.

Images