Text recognition model training, text recognition method and device, equipment and medium

Abstract

The application relates to a character recognition model training method, a character recognition method and device, equipment and a medium, and is applied to the field of artificial intelligence technology. The model training method comprises the following steps: connecting a character background segmentation model and a feature extraction network of a first character recognition model to obtain a second character recognition model; keeping the network parameter values of the character background segmentation model unchanged, training the feature extraction network and a sequence modeling network by using a first sample image and a character recognition result, and obtaining a third character recognition model. A second sample image and a character recognition result and a character background segmentation result are obtained; each second sample image is input into the third character recognition model to obtain a character recognition prediction result and a character background segmentation prediction result. The weighted sum of a character recognition loss value and a character background segmentation loss value is taken as a whole loss value, and the network parameter values of each part of the third character recognition model are adjusted to obtain a fourth character recognition model. The accuracy of recognition can be improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a text recognition model training, text recognition method, device, or medium. Background Technology

[0002] Deep learning-based character recognition is one of the key technologies for visual perception in the fields of intelligent document processing and information extraction. Many character recognition models trained based on deep learning have emerged and achieved good results.

[0003] Currently, for scenarios where complex backgrounds (such as background text or patterns that do not need to be recognized) are intertwined with the text content to be recognized, data augmentation can be used to improve the model's adaptability to different background interferences, thereby improving the text recognition performance in complex backgrounds. However, the accuracy of this method is still relatively low. Summary of the Invention

[0004] To address the aforementioned technical problems, this application provides a character recognition model training method, device, and medium for character recognition.

[0005] According to a first aspect of this application, a method for training a character recognition model is provided, comprising:

[0006] Obtain a pre-trained text-background segmentation model, which is used to separate the text and background in an image.

[0007] The text background segmentation model is connected to multiple layers of different scales in the feature extraction network of the pre-constructed first text recognition model to obtain the second text recognition model; the first text recognition model includes: a feature extraction network and a sequence modeling network.

[0008] Obtain multiple first sample images and the text recognition results of each first sample image;

[0009] Keeping the network parameter values ​​of the text background segmentation model part of the second character recognition model unchanged, the feature extraction network part and the sequence modeling network part of the second character recognition model are trained using the first sample image and the character recognition results in the first sample image to obtain the third character recognition model.

[0010] Obtain multiple second sample images and the text recognition results and text background segmentation results for each second sample image;

[0011] Each second sample image is input into the third character recognition model to obtain the character recognition prediction result and the character background segmentation prediction result of the second sample image;

[0012] Based on the text recognition results and text recognition prediction results of the second sample image, the text recognition loss value is obtained;

[0013] Based on the text background segmentation result and text background segmentation prediction result of the second sample image, the text background segmentation loss value is obtained;

[0014] The weighted sum of the text recognition loss value and the text background segmentation loss value is taken as the overall loss value;

[0015] Based on the overall loss value, the network parameter values ​​of the text background segmentation model, feature extraction network, and sequence modeling network in the third character recognition model are adjusted to obtain the fourth character recognition model.

[0016] Optionally, the step of training the feature extraction network and sequence modeling network in the second character recognition model using the first sample image and the character recognition results in the first sample image includes:

[0017] Obtain a set of similar characters that are easily mistaken, the set of similar characters including: similar characters that can be distinguished by the human eye and similar characters that cannot be distinguished by the human eye;

[0018] The first sample image is input into the second character recognition model to obtain the character recognition prediction result of the first sample image;

[0019] Based on the character recognition results and character recognition prediction results of the first sample image, the character loss value of each erroneous character in the first sample image is obtained;

[0020] A corresponding weight is assigned to each erroneous character in the first sample image; wherein, the weight decreases sequentially when the erroneous character belongs to similar characters that can be distinguished by the human eye, dissimilar characters, and similar characters that cannot be distinguished by the human eye.

[0021] The weighted sum of the character loss values ​​of each erroneous character in the first sample image is used as the character recognition loss value of the first sample image.

[0022] Based on the character recognition loss value of the first sample image, the network parameter values ​​of the feature extraction network part and the sequence modeling network part in the second character recognition model are adjusted.

[0023] Optionally, setting a corresponding weight for each erroneous character in the first sample image includes:

[0024] Get the total number of training epochs and the current training epoch;

[0025] Set the weight W when the erroneous character belongs to a dissimilar character. Normal W Normal =1;

[0026] According to the following formula: The weight W when the erroneous character belongs to a similar character that can be distinguished by the human eye. Dis ;

[0027] According to the following formula: The weight W when the erroneous character belongs to a similar character that is indistinguishable to the human eye. Indis ;

[0028] in, Indicates rounding up. Interval represents the interval between training rounds where the weights are changed, where M is an integer greater than 1.

[0029] According to a second aspect of this application, a character recognition method is provided, the method comprising:

[0030] Acquire the image to be recognized;

[0031] The image to be recognized is input into a pre-trained image recognition model to obtain the text recognition result of the image to be recognized; wherein the image recognition model is trained based on the method described in the first aspect.

[0032] According to a third aspect of this application, a character recognition model training apparatus is provided, comprising:

[0033] The text background segmentation model acquisition module is used to acquire a pre-trained text background segmentation model, which is used to separate the text and background in the image.

[0034] The text recognition model construction module is used to connect multiple layers of different scales in the text background segmentation model to multiple layers in the feature extraction network part of the pre-constructed first text recognition model to obtain a second text recognition model; the first text recognition model includes: a feature extraction network part and a sequence modeling network part;

[0035] The first sample data acquisition module is used to acquire multiple first sample images and the text recognition results in each first sample image;

[0036] The text recognition model training module is used to keep the network parameter values ​​of the text background segmentation model part of the second text recognition model unchanged, and use the first sample image and the text recognition results in the first sample image to train the feature extraction network part and the sequence modeling network part of the second text recognition model to obtain the third text recognition model.

[0037] The second sample data acquisition module is used to acquire multiple second sample images and the text recognition results and text background segmentation results of each second sample image;

[0038] The prediction result determination module is used to input each second sample image into the third character recognition model to obtain the character recognition prediction result and the character background segmentation prediction result of the second sample image;

[0039] The text recognition loss value determination module is used to obtain the text recognition loss value based on the text recognition result and text recognition prediction result of the second sample image;

[0040] The text background segmentation loss value determination module is used to obtain the text background segmentation loss value based on the text background segmentation result and the text background segmentation prediction result of the second sample image;

[0041] The overall loss value determination module is used to take the weighted sum of the text recognition loss value and the text background segmentation loss value as the overall loss value;

[0042] The network parameter value adjustment module is used to adjust the network parameter values ​​of the text background segmentation model, feature extraction network, and sequence modeling network in the third character recognition model based on the overall loss value, so as to obtain the fourth character recognition model.

[0043] Optionally, the character recognition model training module is specifically used to train the feature extraction network and sequence modeling network parts of the second character recognition model using the first sample image and the character recognition results in the first sample image through the following steps:

[0044] Obtain a set of similar characters that are easily mistaken, the set of similar characters including: similar characters that can be distinguished by the human eye and similar characters that cannot be distinguished by the human eye;

[0045] The first sample image is input into the second character recognition model to obtain the character recognition prediction result of the first sample image;

[0046] Based on the character recognition results and character recognition prediction results of the first sample image, the character loss value of each erroneous character in the first sample image is obtained;

[0047] A corresponding weight is assigned to each erroneous character in the first sample image; wherein, the weight decreases sequentially when the erroneous character belongs to similar characters that can be distinguished by the human eye, dissimilar characters, and similar characters that cannot be distinguished by the human eye.

[0048] The weighted sum of the character loss values ​​of each erroneous character in the first sample image is used as the character recognition loss value of the first sample image.

[0049] Based on the character recognition loss value of the first sample image, the network parameter values ​​of the feature extraction network part and the sequence modeling network part in the second character recognition model are adjusted.

[0050] Optionally, the character recognition model training module is specifically used to assign a corresponding weight to each erroneous character in the first sample image through the following steps:

[0051] Get the total number of training epochs and the current training epoch;

[0052] Set the weight W when the erroneous character belongs to a dissimilar character. Normal W Normal =1;

[0053] According to the following formula: The weight W when the erroneous character belongs to a similar character that can be distinguished by the human eye. Dis ;

[0054] According to the following formula: The weight W when the erroneous character belongs to a similar character that is indistinguishable to the human eye. Indis ;

[0055] in, Indicates rounding up. Interval represents the interval between training rounds where the weights are changed, where M is an integer greater than 1.

[0056] According to a fourth aspect of this application, a character recognition device is provided, the device comprising:

[0057] The image acquisition module is used to acquire the image to be recognized.

[0058] An image recognition module is used to input the image to be recognized into a pre-trained image recognition model to obtain the text recognition result of the image to be recognized; wherein the image recognition model is trained based on the method described in the first aspect.

[0059] According to a fifth aspect of this application, an electronic device is provided, comprising: a processor configured to execute a computer program stored in a memory, wherein the computer program, when executed by the processor, implements the method described in the first or second aspect.

[0060] According to a sixth aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method described in the first or second aspect.

[0061] According to a seventh aspect of this application, a computer program product is provided that, when the computer program product is run on a computer, causes the computer to perform the method described in the first or second aspect.

[0062] The technical solution provided in this application has the following advantages compared with the prior art:

[0063] Multiple layers of different scales in the text background segmentation model are connected to multiple layers in the feature extraction network of a pre-built first text recognition model, thus adding a text background segmentation structure to the text recognition model. First, keeping the network parameter values ​​of the text background segmentation structure in the text recognition model unchanged, the feature extraction network and sequence modeling network of the text recognition model are trained using first sample images and the text recognition results of each first sample image. Then, by acquiring multiple second sample images and the text recognition and text background segmentation results of each second sample image, the parameter values ​​in each network part of the text recognition model are adjusted to improve the accuracy of the text recognition model. In this way, when applying the text recognition model to recognize the image to be recognized, the addition of the text background segmentation structure reduces the interference of complex backgrounds on text features during the feature extraction stage, thereby improving the performance of the text recognition model in recognizing text under complex background interference. Attached Figure Description

[0064] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0065] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0066] Figure 1 This is a schematic diagram of a text recognition model training method in an embodiment of this application;

[0067] Figure 2A This is a flowchart of a text recognition model training method in an embodiment of this application;

[0068] Figure 2B This is a flowchart illustrating one method for training a second character recognition model in an embodiment of this application.

[0069] Figure 3 This is a flowchart of a character recognition method in an embodiment of this application;

[0070] Figure 4 This is a schematic diagram of a character recognition model training device in an embodiment of this application;

[0071] Figure 5 This is a schematic diagram of the structure of a character recognition device in an embodiment of this application;

[0072] Figure 6 This is a schematic diagram of the structure of an electronic device in an embodiment of this application. Detailed Implementation

[0073] To better understand the above-mentioned objectives, features, and advantages of this application, the solution of this application will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0074] Many specific details are set forth in the following description in order to provide a full understanding of this application, but this application may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some embodiments of this application, and not all embodiments.

[0075] Figure 1 This is a schematic diagram of a text recognition model training method in an embodiment of this application. For example... Figure 1 As shown, a pre-trained text background segmentation model and a pre-built first text recognition model are connected to form a second text recognition model. The network parameter values ​​of the text background segmentation part of the second text recognition model are frozen, i.e., the network parameter values ​​of the text background segmentation part of the second text recognition model are kept unchanged, while the network parameter values ​​of the other parts of the second text recognition model are trained to obtain a third text recognition model. Then, the network parameter values ​​of the text background segmentation part of the third text recognition model are unfrozen, and all network parameter values ​​in the third text recognition model are trained. Furthermore, multi-loss training is performed during this process, i.e., the losses of the text background segmentation part and the text recognition part are comprehensively considered, ultimately training to generate a fourth text recognition model. In this way, the text recognition model including the text background segmentation structure can reduce background feature interference in the feature extraction stage, obtain higher quality text features, and achieve the goal of improving text recognition performance in complex backgrounds.

[0076] See Figure 2A , Figure 2A This is a flowchart of a text recognition model training method in an embodiment of this application, which may include the following steps:

[0077] Step S202: Obtain the pre-trained text-background segmentation model, which is used to separate the text and background in the image.

[0078] The text-background segmentation model takes an image as input and outputs the segmentation results of both text and background. The model can contain multiple layers of different scales, and can use network structures such as UET, DBNet, or DeepLabV3, employing loss functions such as cross-entropy loss.

[0079] Step S204: Connect multiple layers of different scales in the text background segmentation model to multiple layers in the feature extraction network part of the pre-constructed first text recognition model to obtain the second text recognition model.

[0080] In this embodiment, the first character recognition model includes a feature extraction network and a sequence modeling network. The feature extraction network is the backbone network, which can be a CNN (Convolutional Neural Networks) backbone network, such as ResNet (Deep Residual Network). The sequence modeling network can be a Transformer, etc.

[0081] Multiple layers of different scales in the text background segmentation model are connected to multiple layers in the feature extraction network of the first character recognition model, thereby merging the segmented features from each layer of the text background segmentation model into the backbone network. For example, if the text background segmentation model includes three layers of different scales and the feature extraction network of the first character recognition model contains six layers, the three layers of different scales in the text background segmentation model can be connected to layers 2, 4, and 6 of the feature extraction network, respectively.

[0082] Step S206: Obtain multiple first sample images and the text recognition results in each first sample image.

[0083] Step S208: Keep the network parameter values ​​of the text background segmentation model part in the second character recognition model unchanged, and use the first sample image and the character recognition results in the first sample image to train the feature extraction network part and the sequence modeling network part in the second character recognition model to obtain the third character recognition model.

[0084] The training method for the second character recognition model can be the same as the existing training method. The difference is that during the training process, the network parameter values ​​of the text background segmentation model part of the second character recognition model are kept unchanged, and only the network parameter values ​​of the feature extraction network part and the sequence modeling network part are updated.

[0085] In some embodiments, the second text recognition model may also be trained in the following manner, during which the network parameter values of the text-background segmentation model part in the second text recognition model remain unchanged. See Figure 2B , which may include the following steps:

[0086] Step S2081: Obtain a similar character set that is prone to misrecognition.

[0087] The similar character set prone to misrecognition includes: similar characters distinguishable by the human eye (such as "gan" and "yu") and similar characters indistinguishable by the human eye (such as 0 and O). The similar character set may be obtained by manual sorting.

[0088] Step S2082: Input the first sample image into the second text recognition model, and obtain a text recognition prediction result of the first sample image.

[0089] Step S2083: Obtain a character loss value for each erroneous character in the first sample image according to the text recognition result and the text recognition prediction result of the first sample image.

[0090] The text recognition prediction result usually contains characters with recognition errors, i.e., erroneous characters, and the text recognition loss value of the first sample image can be obtained according to the loss values of all the erroneous characters.

[0091] Step S2084: Set a corresponding weight for each erroneous character in the first sample image; wherein, when the erroneous character respectively belongs to the similar characters distinguishable by the human eye, non-similar characters, and similar characters indistinguishable by the human eye, the corresponding weights decrease in sequence.

[0092] In the embodiments of the present application, a similar character weight may be added to the loss function, to specifically increase the loss weight brought by the recognition error of characters distinguishable by the human eye, and reduce the loss weight brought by the recognition error of characters indistinguishable by the human eye, so as to strengthen the training of similar characters distinguishable by the human eye, reduce the interference of similar characters indistinguishable by the human eye, and further improve the accuracy of similar character recognition.

[0093] In some embodiments, the weight corresponding to the erroneous character may be directly set as a constant, for example, if the erroneous character belongs to a similar character distinguishable by the human eye, the corresponding weight is a; if the erroneous character belongs to a non-similar character, the corresponding weight is b; if the erroneous character belongs to a similar character indistinguishable by the human eye, the corresponding weight is c, where a is greater than b and b is greater than c. For example, a is 1.3, b is 1, c is 0.8, etc.

[0094] In some embodiments, the weight corresponding to the erroneous character may also be determined according to the total number of training rounds Epochs and the current training round epoch. Specifically: the total number of training rounds Epochs and the current training round epoch may be acquired.

[0095] Set the weight W when the erroneous character belongs to a dissimilar character. Normal W Normal =1.

[0096] According to the following formula: The weight W when the erroneous character belongs to a similar character that can be distinguished by the human eye. Dis ;

[0097] According to the following formula: The weight W when the erroneous character belongs to a similar character that is indistinguishable to the human eye. Indis ;

[0098] in, Indicates rounding up. Interval represents the training epoch interval for changing the weights. M is an integer greater than 1; a larger value of M indicates that the weight reinforcement is more concentrated in the final stage of training, such as 10. By setting the weights corresponding to erroneous characters in this way, the stage at which the weight reinforcement occurs during the training process can be adjusted.

[0099] Step S2085: The weighted sum of the character loss values ​​of each erroneous character in the first sample image is used as the character recognition loss value of the first sample image.

[0100] After setting the weights for all error characters, you can use the following formula:

[0101] Loss Recog =W Dis ×Loss EC∈CDis +W Indis ×Loss EC∈CIndis +W Normal ×Loss EC∈Other ;

[0102] Determine the text recognition loss value (Loss) for the first sample image. Recog ;

[0103] Where EC represents an error character; Loss EC∈CDis This represents the loss value when the erroneous character belongs to a similar character that can be distinguished by the human eye; Loss EC∈CIndis This represents the loss value when the erroneous character belongs to a group of similar characters that are indistinguishable to the human eye; Loss EC∈Other This represents the loss value when the erroneous character belongs to a dissimilar character.

[0104] Step S2086: Based on the character recognition loss value of the first sample image, adjust the network parameter values ​​of the feature extraction network part and the sequence modeling network part in the second character recognition model.

[0105] After training the third character recognition model, the network parameter values ​​of the text background segmentation part of the third character recognition model are unfrozen. Multiple loss functions are used for training, and the character recognition loss value and the text background segmentation loss value are used to further optimize the recognition performance of the entire model.

[0106] Step S210: Obtain multiple second sample images and the text recognition result and text background segmentation result of each second sample image.

[0107] Step S212: Input each second sample image into the third character recognition model to obtain the character recognition prediction result and the character background segmentation prediction result of the second sample image.

[0108] Step S214: Based on the text recognition results and text recognition prediction results of the second sample image, obtain the text recognition loss value.

[0109] Step S216: Based on the text background segmentation result and text background segmentation prediction result of the second sample image, obtain the text background segmentation loss value.

[0110] Step S218: The weighted sum of the text recognition loss value and the text background segmentation loss value is used as the overall loss value.

[0111] If the text recognition loss value of the second sample image is Loss Recog The text-to-background segmentation loss value for the second sample image is Loss. Seg Then, it can be determined according to the following formula:

[0112] Loss Total =λLoss Recog +(1-λ)Loss Seg ;

[0113] Obtain the overall loss value. Total Where λ represents Loss Recog The weights, 1-λ represent the loss. Seg The weight λ can be 0.7, 0.8, etc.

[0114] Step S220: Based on the overall loss value, adjust the network parameter values ​​of the text background segmentation model, feature extraction network, and sequence modeling network in the third character recognition model to obtain the fourth character recognition model.

[0115] The text recognition model training method in this application incorporates a text background segmentation structure during training. This reduces interference from complex backgrounds on text features during feature extraction, improving the model's performance under complex background interference. Furthermore, the training process specifically increases the loss weight for errors in recognizing characters indistinguishable to the human eye and decreases the loss weight for errors in recognizing similar characters indistinguishable to the human eye. This strengthens the training of similar characters that can be distinguished by the human eye and reduces interference from similar characters that cannot be distinguished by the human eye, further improving the accuracy of similar character recognition.

[0116] This application also provides a text recognition method, see [link to relevant documentation]. Figure 3 This may include the following steps:

[0117] Step S302: Obtain the image to be recognized.

[0118] Step S304: Input the image to be recognized into the pre-trained image recognition model to obtain the text recognition result of the image to be recognized; wherein, the image recognition model is trained based on the method described in the embodiment of Figure 2.

[0119] The specific training method for the image recognition model can be found in the description of the embodiment in Figure 2, and will not be repeated here.

[0120] It should be noted that although the steps of the method in this application are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0121] This application provides a text recognition model training device, see [link to relevant documentation]. Figure 4 The text recognition model training device 400 includes:

[0122] The text background segmentation model acquisition module 402 is used to acquire a pre-trained text background segmentation model, which is used to separate the text and background in the image.

[0123] The character recognition model construction module 404 is used to connect multiple layers of different scales in the character background segmentation model to multiple layers in the feature extraction network part of the pre-built first character recognition model to obtain the second character recognition model; the first character recognition model includes: a feature extraction network part and a sequence modeling network part;

[0124] The first sample data acquisition module 406 is used to acquire multiple first sample images and the text recognition results in each first sample image;

[0125] The text recognition model training module 408 is used to keep the network parameter values ​​of the text background segmentation model part in the second text recognition model unchanged, and to train the feature extraction network part and sequence modeling network part in the second text recognition model using the first sample image and the text recognition results in the first sample image to obtain the third text recognition model.

[0126] The second sample data acquisition module 410 is used to acquire multiple second sample images and the text recognition results and text background segmentation results of each second sample image;

[0127] The prediction result determination module 412 is used to input each second sample image into the third character recognition model to obtain the character recognition prediction result and the character background segmentation prediction result of the second sample image;

[0128] The character recognition loss value determination module 414 is used to obtain the character recognition loss value based on the character recognition result and the character recognition prediction result of the second sample image;

[0129] The text background segmentation loss value determination module 416 is used to obtain the text background segmentation loss value based on the text background segmentation result and the text background segmentation prediction result of the second sample image.

[0130] The overall loss value determination module 418 is used to take the weighted sum of the text recognition loss value and the text background segmentation loss value as the overall loss value;

[0131] The network parameter value adjustment module 420 is used to adjust the network parameter values ​​of the text background segmentation model, feature extraction network, and sequence modeling network in the third character recognition model based on the overall loss value, so as to obtain the fourth character recognition model.

[0132] Optionally, the character recognition model training module 408 is specifically used to train the feature extraction network and sequence modeling network parts of the second character recognition model using the first sample image and the character recognition results in the first sample image through the following steps:

[0133] Obtain a set of similar characters that are easily mistaken, including: similar characters that can be distinguished by the human eye and similar characters that cannot be distinguished by the human eye;

[0134] The first sample image is input into the second character recognition model to obtain the character recognition prediction result of the first sample image;

[0135] Based on the character recognition results and character recognition prediction results of the first sample image, the character loss value of each erroneous character in the first sample image is obtained;

[0136] A corresponding weight is assigned to each erroneous character in the first sample image; the weight decreases sequentially when the erroneous character belongs to the category of similar characters that can be distinguished by the human eye, dissimilar characters, and similar characters that cannot be distinguished by the human eye.

[0137] The weighted sum of the character loss values ​​of each erroneous character in the first sample image is used as the character recognition loss value of the first sample image.

[0138] Based on the character recognition loss value of the first sample image, the network parameter values ​​of the feature extraction network and sequence modeling network in the second character recognition model are adjusted.

[0139] Optionally, the character recognition model training module 408 is specifically used to set corresponding weights for each erroneous character in the first sample image through the following steps:

[0140] Get the total number of training epochs and the current training epoch;

[0141] Set the weight W when the erroneous character belongs to a dissimilar character. Normal W Normal =1;

[0142] According to the following formula: The weight W when the erroneous character belongs to a similar character that can be distinguished by the human eye. Dis ;

[0143] According to the following formula: The weight W when the erroneous character belongs to a similar character that is indistinguishable to the human eye. Indis ;

[0144] in, Indicates rounding up. Interval represents the interval between training rounds where the weights are changed, where M is an integer greater than 1.

[0145] This application also provides a character recognition device, see [link to relevant documentation]. Figure 5 The character recognition device 500 includes:

[0146] Image acquisition module 502 is used to acquire the image to be recognized;

[0147] The image recognition module 504 is used to input the image to be recognized into a pre-trained image recognition model to obtain the text recognition result of the image to be recognized; wherein, the image recognition model is trained based on the method described in the embodiment of Figure 2.

[0148] The specific details of each module or unit in the above-mentioned device have been described in detail in the corresponding methods, so they will not be repeated here.

[0149] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0150] This application also provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the character recognition model training method or character recognition method described in this example embodiment.

[0151] Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. It should be noted that... Figure 6 The electronic device 600 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0152] like Figure 6 As shown, the electronic device 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage section 608 into a random access memory (RAM) 603. The RAM 603 also stores various programs and data required for system operation. The CPU 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0153] The following components are connected to I / O interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a local area network (LAN) card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to I / O interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 610 as needed so that computer programs read from it can be installed into storage section 608 as needed.

[0154] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611. When the computer program is executed by central processing unit 601, it performs the various functions defined in the apparatus of this application.

[0155] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described text recognition model training method or text recognition method.

[0156] It should be noted that the computer-readable storage medium shown in this application can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, radio frequency, etc., or any suitable combination thereof.

[0157] In this embodiment of the application, a computer program product is also provided, which, when run on a computer, causes the computer to execute the above-described character recognition model training method or character recognition method.

[0158] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0159] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for training a character recognition model, characterized in that, The method includes: Obtain a pre-trained text-background segmentation model, which is used to separate the text and background in an image. The text background segmentation model is connected to multiple layers of different scales in the feature extraction network of the pre-constructed first text recognition model to obtain the second text recognition model; the first text recognition model includes: a feature extraction network and a sequence modeling network. Obtain multiple first sample images and the text recognition results of each first sample image; Keeping the network parameter values ​​of the text background segmentation model part of the second character recognition model unchanged, the feature extraction network part and the sequence modeling network part of the second character recognition model are trained using the first sample image and the character recognition results in the first sample image to obtain the third character recognition model. Obtain multiple second sample images and the text recognition results and text background segmentation results for each second sample image; Each second sample image is input into the third character recognition model to obtain the character recognition prediction result and the character background segmentation prediction result of the second sample image; Based on the text recognition results and text recognition prediction results of the second sample image, the text recognition loss value is obtained; Based on the text background segmentation result and text background segmentation prediction result of the second sample image, the text background segmentation loss value is obtained; The weighted sum of the text recognition loss value and the text background segmentation loss value is taken as the overall loss value; Based on the overall loss value, the network parameter values ​​of the text background segmentation model, feature extraction network, and sequence modeling network in the third character recognition model are adjusted to obtain the fourth character recognition model.

2. The method according to claim 1, characterized in that, The step of training the feature extraction network and sequence modeling network in the second character recognition model using the first sample image and the character recognition results in the first sample image includes: Obtain a set of similar characters that are easily mistaken, the set of similar characters including: similar characters that can be distinguished by the human eye and similar characters that cannot be distinguished by the human eye; The first sample image is input into the second character recognition model to obtain the character recognition prediction result of the first sample image; Based on the character recognition results and character recognition prediction results of the first sample image, the character loss value of each erroneous character in the first sample image is obtained; A corresponding weight is assigned to each erroneous character in the first sample image; wherein, the weight decreases sequentially when the erroneous character belongs to similar characters that can be distinguished by the human eye, dissimilar characters, and similar characters that cannot be distinguished by the human eye. The weighted sum of the character loss values ​​of each erroneous character in the first sample image is used as the character recognition loss value of the first sample image. Based on the character recognition loss value of the first sample image, the network parameter values ​​of the feature extraction network part and the sequence modeling network part in the second character recognition model are adjusted.

3. The method according to claim 2, characterized in that, The step of assigning a corresponding weight to each erroneous character in the first sample image includes: Get the total number of training epochs and the current training epoch; Set the weight W when the erroneous character belongs to a dissimilar character. Normal W Normal =1; According to the following formula: The weight W when the erroneous character belongs to a similar character that can be distinguished by the human eye. Dis ; According to the following formula: The weight W when the erroneous character belongs to a similar character that is indistinguishable to the human eye. Indis ; in, Indicates rounding up. Interval represents the interval between training rounds where the weights are changed, where M is an integer greater than 1.

4. A character recognition method, characterized in that, The method includes: Acquire the image to be recognized; The image to be recognized is input into a pre-trained image recognition model to obtain the text recognition result of the image to be recognized; wherein the image recognition model is trained based on the method described in any one of claims 1 to 3.

5. A character recognition model training device, characterized in that, The device includes: The text background segmentation model acquisition module is used to acquire a pre-trained text background segmentation model, which is used to separate the text and background in the image. The text recognition model construction module is used to connect multiple layers of different scales in the text background segmentation model to multiple layers in the feature extraction network part of the pre-constructed first text recognition model to obtain a second text recognition model; the first text recognition model includes: a feature extraction network part and a sequence modeling network part; The first sample data acquisition module is used to acquire multiple first sample images and the text recognition results in each first sample image; The text recognition model training module is used to keep the network parameter values ​​of the text background segmentation model part of the second text recognition model unchanged, and use the first sample image and the text recognition results in the first sample image to train the feature extraction network part and the sequence modeling network part of the second text recognition model to obtain the third text recognition model. The second sample data acquisition module is used to acquire multiple second sample images and the text recognition results and text background segmentation results of each second sample image; The prediction result determination module is used to input each second sample image into the third character recognition model to obtain the character recognition prediction result and the character background segmentation prediction result of the second sample image; The text recognition loss value determination module is used to obtain the text recognition loss value based on the text recognition result and text recognition prediction result of the second sample image; The text background segmentation loss value determination module is used to obtain the text background segmentation loss value based on the text background segmentation result and the text background segmentation prediction result of the second sample image; The overall loss value determination module is used to take the weighted sum of the text recognition loss value and the text background segmentation loss value as the overall loss value; The network parameter value adjustment module is used to adjust the network parameter values ​​of the text background segmentation model, feature extraction network, and sequence modeling network in the third character recognition model based on the overall loss value, so as to obtain the fourth character recognition model.

6. The apparatus according to claim 5, characterized in that, The character recognition model training module is specifically used to train the feature extraction network and sequence modeling network of the second character recognition model by utilizing the first sample image and the character recognition results in the first sample image through the following steps: Obtain a set of similar characters that are easily mistaken, the set of similar characters including: similar characters that can be distinguished by the human eye and similar characters that cannot be distinguished by the human eye; The first sample image is input into the second character recognition model to obtain the character recognition prediction result of the first sample image; Based on the character recognition results and character recognition prediction results of the first sample image, the character loss value of each erroneous character in the first sample image is obtained; A corresponding weight is assigned to each erroneous character in the first sample image; wherein, the weight decreases sequentially when the erroneous character belongs to similar characters that can be distinguished by the human eye, dissimilar characters, and similar characters that cannot be distinguished by the human eye. The weighted sum of the character loss values ​​of each erroneous character in the first sample image is used as the character recognition loss value of the first sample image. Based on the character recognition loss value of the first sample image, the network parameter values ​​of the feature extraction network part and the sequence modeling network part in the second character recognition model are adjusted.

7. The apparatus according to claim 6, characterized in that, The character recognition model training module is specifically used to assign a corresponding weight to each erroneous character in the first sample image through the following steps: Get the total number of training epochs and the current training epoch; Set the weight W when the erroneous character belongs to a dissimilar character. Normal W Normal =1; According to the following formula: The weight W when the erroneous character belongs to a similar character that can be distinguished by the human eye. Dis ; According to the following formula: The weight W when the erroneous character belongs to a similar character that is indistinguishable to the human eye. Indis ; in, Indicates rounding up. Interval represents the interval between training rounds where the weights are changed, where M is an integer greater than 1.

8. A character recognition device, characterized in that, The device includes: The image acquisition module is used to acquire the image to be recognized. An image recognition module is used to input the image to be recognized into a pre-trained image recognition model to obtain the text recognition result of the image to be recognized; wherein the image recognition model is trained based on the method described in any one of claims 1 to 3.

9. An electronic device, characterized in that, include: A processor for executing a computer program stored in a memory, wherein the computer program, when executed by the processor, implements the method of any one of claims 1-3, or the method of claim 4.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1-3, or the method of claim 4.

Images