Text recognition model training method, text recognition method, device and apparatus

Abstract

The application discloses a text recognition model training method, a text recognition method, a device and equipment. The text recognition model training method comprises the following steps: acquiring a first text recognition model; training the first text recognition model; in the case that the training process of the first text recognition model reaches a first training condition, reducing the network layer in a decoding module to a target number to obtain a second text recognition model; training the second text recognition model; in the case that the training process of the second text recognition model reaches a second training condition, replacing the encoding module in the second text recognition model based on a light-weight coding module to obtain a third text recognition model; training the third text recognition model; and in the case that the training process of the third text recognition model reaches a third training condition, obtaining a trained text recognition model. The application adjusts and trains the text recognition model multiple times, considers the model accuracy and size, and improves the application range of the model.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and more particularly to training methods for text recognition models, text recognition methods, apparatus, and devices. Background Technology

[0002] In recent years, deep learning technology has achieved great success in many areas, especially in the field of computer vision.

[0003] As one of the core problems in the field of computer vision, OCR text recognition has a huge application base in the real world. However, due to the high complexity and large number of parameters of existing deep neural network models, their deployment in some scenarios and on some devices is limited, making it impossible to ensure model accuracy while taking into account model size. Summary of the Invention

[0004] This invention provides a training method, a text recognition method, an apparatus, and a device for text recognition models, in order to solve the problem that existing text recognition models cannot balance model accuracy with model size.

[0005] According to one aspect of the present invention, a method for training a text recognition model is provided, characterized in that it includes:

[0006] Obtain a first text recognition model, wherein the first text recognition model includes an encoding module and a decoding module;

[0007] The first text recognition model is trained, and when the training process of the first text recognition model reaches the first training condition, the number of network layers in the decoding module is reduced to the target number to obtain the second text recognition model.

[0008] The second text recognition model is trained. When the training process of the second text recognition model reaches the second training condition, the encoding module in the second text recognition model is replaced by a lightweight encoding module to obtain a third text recognition model. The number of network layers of the lightweight encoding module is less than the number of network layers of the encoding module in the second text recognition model.

[0009] The third text recognition model is trained, and when the training process of the third text recognition model reaches the third training condition, a trained text recognition model is obtained.

[0010] According to another aspect of the present invention, a text recognition method is provided, characterized in that it includes:

[0011] Receive the image to be recognized;

[0012] The image to be recognized is input into a pre-trained text recognition model to obtain the text information in the image to be recognized. The text recognition model is trained based on the text recognition model training method described in any embodiment of the present invention.

[0013] According to another aspect of the present invention, a training apparatus for a text recognition model is provided, characterized in that it comprises:

[0014] The first text recognition model acquisition module is used to acquire a first text recognition model, wherein the first text recognition model includes an encoding module and a decoding module;

[0015] The second text recognition model acquisition module is used to train the first text recognition model. When the training process of the first text recognition model reaches the first training condition, the network layers in the decoding module are reduced to the target number to obtain the second text recognition model.

[0016] The third text recognition model acquisition module is used to train the second text recognition model. When the training process of the second text recognition model reaches the second training condition, the encoding module in the second text recognition model is replaced by the lightweight encoding module to obtain the third text recognition model. The number of network layers of the lightweight encoding module is less than the number of network layers of the encoding module in the second text recognition model.

[0017] The text recognition model acquisition module is used to train the third text recognition model. When the training process of the third text recognition model reaches the third training condition, the trained text recognition model is obtained.

[0018] According to another aspect of the present invention, a text recognition device is provided, characterized in that it comprises:

[0019] The image receiving module is used to receive the image to be recognized;

[0020] The text recognition module is used to input the image to be recognized into a pre-trained text recognition model to obtain the text information in the image to be recognized. The text recognition model is trained based on the text recognition model training method described in any embodiment of the present invention.

[0021] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0022] At least one processor; and

[0023] A memory communicatively connected to the at least one processor; wherein,

[0024] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the text recognition model training method according to any embodiment of the present invention.

[0025] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the text recognition model training method according to any embodiment of the present invention.

[0026] The technical solution of this invention, through multi-stage adjustment and training of the text recognition model, makes the model smaller while ensuring model accuracy. This solves the problem that existing text recognition models cannot balance model accuracy and model size, reduces the limitations of deploying text recognition models in some scenarios and on some devices, and improves the applicability of text recognition models.

[0027] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a flowchart of a text recognition model training method provided in Embodiment 1 of the present invention;

[0030] Figure 2 This is a schematic diagram of the structure of a text recognition model provided in Embodiment 1 of the present invention;

[0031] Figure 3 This is a schematic diagram of the structure of a lightweight text recognition model provided in Embodiment 1 of the present invention;

[0032] Figure 4 This is a flowchart of a text recognition method provided in Embodiment 2 of the present invention;

[0033] Figure 5 This is a schematic diagram of the structure of a training device for a text recognition model provided in Embodiment 3 of the present invention;

[0034] Figure 6This is a schematic diagram of the structure of a text recognition device provided in Embodiment 4 of the present invention;

[0035] Figure 7 This is a schematic diagram of the structure of an electronic device provided in Embodiment 5 of the present invention. Detailed Implementation

[0036] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0037] It should be noted that the terms "first text recognition model," "second text recognition model," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0038] Example 1

[0039] Figure 1 This is a flowchart of a text recognition model training method provided in Embodiment 1 of the present invention. This embodiment is applicable to the case of lightweight text recognition models. The method can be executed by a text recognition model training device, which can be implemented in hardware and / or software. This text recognition model training device can be configured in the electronic device provided in the embodiment of the present invention. Figure 1 As shown, the method includes:

[0040] S110. Obtain a first text recognition model, wherein the first text recognition model includes an encoding module and a decoding module.

[0041] The first text recognition model can be a pre-built text recognition model. Optionally, the first text recognition model includes an encoding module and a decoding module. The encoding module can be, for example, an encoder, used to encode the input image to obtain an encoded feature sequence that the decoding module can recognize. For example, the encoding module can be a network structure such as, but not limited to, a ResNet18 network, and is not limited thereto. The decoding module can be, for example, a decoder, used to decode the encoded features output by the encoding module, thereby realizing the recognition of text information in the input image. For example, the encoding module can be, but not limited to, a Transformer model. Optionally, the number of network layers in the encoding module is greater than a first preset value, and the number of network layers in the decoding module is greater than a second preset value. The first and second preset values ​​can be set according to user needs. In this embodiment, the pre-built first text recognition model is obtained and used to perform lightweight processing of the network layers based on the first text recognition model to achieve lightweighting of the text recognition model.

[0042] S120. Train the first text recognition model. When the training process of the first text recognition model reaches the first training condition, reduce the number of network layers in the decoding module to the target number to obtain the second text recognition model.

[0043] The first training condition can be a preset number of training iterations for the first text recognition model. For example, the preset number of training iterations for the first text recognition model can be 3000 epochs (i.e., one forward propagation and one backward propagation using all training samples in the training set in the neural network), set by those skilled in the art based on experience or training requirements, and is not limited here. In this embodiment, the first text recognition model is trained, and when the preset number of training iterations for the first recognition model is reached, the number of network layers in the decoding module is reduced to the target number to obtain the second text recognition model. Optionally, the first training condition can also be a preset recognition accuracy rate for the first text recognition model, where the preset recognition accuracy rate is set by those skilled in the art based on task requirements, and is not limited here. In this embodiment, the first text recognition model is trained, and when the accuracy rate of the first recognition model reaches the preset accuracy rate during training, the number of network layers in the decoding module is reduced to the target number to obtain the second text recognition model. Here, the target number refers to the number of network layers in the decoding module after reduction, or the number of network layers retained in the decoding module. Optionally, the target number can be set by those skilled in the art based on the needs of the model size. For example, the target number can be set to 2 layers, but this is not limited. In this embodiment, by reducing the number of network layers in the decoding module, the complexity and number of parameters of the model are reduced, thereby compressing the size of the text recognition model and expanding the applicability of the text recognition model.

[0044] Based on the above embodiments, optionally, before training the first text recognition model, the method further includes: obtaining an initial training sample set, wherein the initial training samples include training data formed by sample images and text labels, and the sample images are images including text information; performing enhancement processing on the sample images in the initial training sample set to obtain enhanced images of each sample image; adding the enhanced images and the text labels of the sample images to the initial training sample set to obtain an enhanced training sample set, wherein the enhanced training sample set is used to train the first text recognition model.

[0045] In this embodiment, the initial training sample set refers to the sample set used to train the text recognition model. The initial training sample set includes training data formed by sample images and text labels. The sample images are images containing text information, i.e., images carrying character sequences. The training data formed by text labels refers to characters in the sample images that have been labeled with text tags, such as Chinese characters, English characters, italicized Chinese characters, and characters with underlines. In practical applications, sample images and training data formed by text labels shared online can be directly obtained, such as open-source natural scene Chinese images; alternatively, sample images can be collected independently and processed uniformly (such as scaling the collected sample images to a specified size) to obtain a sample image sample set.

[0046] Enhanced images are obtained by performing enhancement processing on sample images in the initial training sample set. These enhanced images, along with the text labels of the sample images, form training data and are added to the initial training sample set to obtain an updated training sample set. This updated training sample set is then used as the enhanced training sample set. The enhancement processing can include image enhancement operations such as color transformation, noise reduction, rotation, and cropping, and is not limited thereto. In this embodiment, the enhanced training sample set is divided into an enhanced training set and an enhanced validation set according to a first preset ratio. The enhanced training set is used to train the first text recognition model, and the enhanced validation set is used to validate the first text recognition model. The first preset ratio is set according to the actual training environment and is not limited here. For example, 90% of the training samples in the enhanced training sample set are used as the enhanced training set, and the remaining 10% are used as the enhanced validation set.

[0047] Based on the above embodiments, optionally, the number of network layers in the decoding module is reduced to the target number to obtain a second text recognition model, including: randomly determining the target number of network layers in the decoding module, and maintaining the network parameters of the target number of network layers to form a new decoding module.

[0048] The new decoding module refers to the decoding module formed by reducing the number of network layers in the decoding module of the first text recognition model to the target number. In this embodiment, the target number of network layers is randomly determined in the decoding module of the first text recognition model, and the network parameters in each determined network layer are kept unchanged. The new decoding module is formed by the target number of network layers, and the second text recognition model is obtained by replacing the decoding module in the first text recognition model with the new decoding module.

[0049] Optionally, the number of network layers in the decoding module is reduced to a target number to obtain a second text recognition model. In this embodiment, the following steps are taken: determining the importance of each network layer based on the network parameters corresponding to each network layer in the decoding module, and determining a target number of network layers from each network layer in the decoding module based on the importance of each network layer to form a new decoding module.

[0050] In this embodiment, the importance of each network layer is determined based on its network parameters. Optionally, the weights of each network layer can be obtained, and the importance of each network layer is determined based on its weights. A target number of network layers are then selected from the network layers in the decoding module based on their importance. The importance of each network layer can be ranked based on its weights, and the target number of network layers with the highest importance can be selected as the network layers for a new decoding module. This target number of network layers forms a new decoding module, which replaces the decoding module in the first text recognition model to obtain the second text recognition model. In this embodiment, determining the target number of network layers based on their importance and then forming a new decoding module reduces the number of network layers in the decoding module, thereby reducing the model's complexity and parameter count, compressing the text recognition model's size, and expanding the applicability of the text recognition model while maintaining accuracy to a certain extent.

[0051] S130. The second text recognition model is trained. When the training process of the second text recognition model reaches the second training condition, the encoding module in the second text recognition model is replaced by a lightweight encoding module to obtain a third text recognition model. The number of network layers of the lightweight encoding module is less than the number of network layers of the encoding module in the second text recognition model.

[0052] The second training condition can be a preset number of training iterations for the second text recognition model. For example, the preset number of training iterations for the second text recognition model can be 1000 epochs, which can be set by those skilled in the art based on experience or training requirements, and is not limited here. In this embodiment, the second text recognition model is trained based on an enhanced training sample set, which is the same as the enhanced training sample set used in the first text recognition model described above, and will not be repeated here. In this embodiment, the second text recognition model includes a trained encoding module and a decoding module to be trained. During the training process of the second text recognition model, the trained encoding module can obtain accurate encoded features, which facilitates the adjustment of the decoding module to be trained.

[0053] Once the second text recognition model has been trained a preset number of times, a lightweight encoding module is used to replace the encoding module in the second text recognition model, resulting in a third text recognition module. This lightweight encoding module can be an autoencoder based on lightweight networks such as MobileNetV3 or SqueezeNet; there are no restrictions on its suitability. By replacing the encoding module in the second text recognition model with a lightweight encoding module, the number of network layers in the encoding module can be reduced, further decreasing the model's complexity and parameter count, and compressing the size of the text recognition model.

[0054] Optionally, the second training condition can also be a preset recognition accuracy rate of the second text recognition model. This preset recognition accuracy rate is set by those skilled in the art based on task requirements and is not limited here. In this embodiment, the second text recognition model is trained based on an initial training sample set. When the recognition accuracy rate of the second text recognition model reaches its preset accuracy rate, a lightweight encoding module replaces the encoding module in the second text recognition model to obtain a third text recognition module. In this embodiment, by training the second text recognition model, the model's recognition accuracy is maintained while reducing the number of network layers in the decoding module.

[0055] S140. The third text recognition model is trained, and when the training process of the third text recognition model reaches the third training condition, a trained text recognition model is obtained.

[0056] The third text recognition model includes a lightweight encoding module to be trained and a lightweight decoding module that has already been trained. The third training condition can be the number of training iterations of the third text recognition model in the second stage; for example, the third training condition can be 500 epochs, which can be set by those skilled in the art based on experience or training requirements, and is not limited here. Optionally, the third training condition can also be the accuracy of the model recognition during the second stage of training. In this embodiment, the third text recognition model is trained in the first stage based on an enhanced training sample set, wherein the enhanced training sample set is the same as the enhanced training sample sets used in the first and second text recognition models, and will not be repeated here. When the training process of the third text recognition model reaches the training condition of the first stage, the optimization algorithm of the third text recognition model is replaced with an optimization algorithm whose initial learning rate is lower than the learning rate of the third text recognition model. The third text recognition model is then trained in the second stage based on the initial training sample set and the replaced optimization algorithm. When the training process of the third text recognition model corresponding to the new optimization algorithm reaches the third training condition, the trained text recognition model is obtained. The training conditions in the first stage can be the number of training cycles for the third text recognition model in the first stage. For example, the number of training cycles in the first stage can be 500 epochs, which can be set by those skilled in the art based on experience or training requirements, and is not limited here. In this embodiment, the initial training sample set is divided into an initial training set and an initial validation set according to a second preset ratio. The initial training set is used to train the third text recognition model in the second stage, and the initial validation set is used to validate the third text recognition model. The second preset ratio is set according to the actual training environment and is not limited here. For example, 90% of the training samples in the initial training sample set are used as the initial training set, and the remaining 10% are used as the initial validation set.

[0057] Based on the above embodiments, optionally, the first text recognition model and the second text recognition model are trained based on a first learning rate, and the third text recognition model is trained at different stages based on the first learning rate and the second learning rate, wherein the second learning rate is less than the first learning rate.

[0058] In this embodiment, optimization training is performed on different stages of the first text recognition model, the second text recognition model, and the third text recognition model based on an optimization algorithm. Optionally, the optimization algorithm can be Adadelta, Adam, RMSProp, etc., and is not limited thereto. Specifically, the first stage of the first, second, and third text recognition models is optimized and trained using a first learning rate as the fixed learning rate of the optimization algorithm. The second stage of the third text recognition model is optimized and trained using a second learning rate as the initial learning rate of the optimization algorithm, wherein the second learning rate is less than the first learning rate. For example, Adadelta is used as the optimization algorithm with a fixed learning rate of 0.001 for the first stage of the first, second, and third text recognition models; Adam is used as the optimization algorithm with an initial learning rate of 0.0001 for the second stage of the third text recognition model.

[0059] Based on the above embodiments, optionally, when the training process of the third text recognition model reaches the third training condition, the method further includes: pruning the convolutional channels of each network layer in the trained third text recognition model.

[0060] In this embodiment, the absolute values ​​of the weights of each node in the convolutional channels of each network layer in the trained third text recognition model are compared with a preset pruning threshold. If the absolute value of a node's weight is less than the preset pruning threshold, the node is pruned, and the weights of the pruned third text recognition model are fine-tuned to restore the third text recognition model. The preset pruning threshold is a pre-set threshold for determining when to prune nodes in the convolutional channels; it can be set according to the model size requirements and is not limited here. In this embodiment, pruning further compresses the size of the text recognition model.

[0061] Based on the above embodiments, optionally, the text recognition model further includes a recurrent network module and a linear network module, wherein the recurrent network module is disposed between the encoding module and the decoding module, and the linear network module is disposed after the decoding module.

[0062] In this embodiment, as Figure 2As shown, in the text recognition model, the initial training sample set or the augmented training sample set is input to the encoding module. The output of the encoding module is used as the input to the recurrent network, the output of the recurrent network module is used as the input to the decoding module, the output of the decoding module is used as the input to the linear network module, and finally, the output of the linear network module is used as the output of the text recognition model. The recurrent network module is used to obtain the feature vector label distribution based on the feature sequence output by the encoding module. For example, the recurrent network module can be a bidirectional long short-term memory (Bi-LSTM) model. The linear network module is used to obtain the recognition result of the text recognition model based on the output of the decoding module. For example, Figure 3 This is a schematic diagram of the structure of a lightweight text recognition model provided in Embodiment 1 of the present invention. Figure 3 As shown, in this lightweight text recognition model, the image to be recognized is input into a mobile-v3 network for processing and the processing result is output. The processing result of the mobile-v3 network is input into a Bi-LSTM model for further processing and the processing result of the Bi-LSTM model is output. The processing result of the Bi-LSTM model is input into a Transformer model with two decoding blocks for decoding and the decoding result is output. Then, the decoding result is input into a two-layer linear network for further processing and the processing result of the linear network is output. Finally, the processing result of the linear network is input into a transcription layer, and the text information of the image to be recognized is obtained through the transcription layer.

[0063] The technical solution of this embodiment, by adjusting and training the encoding and decoding modules in the text recognition model step by step, compresses the model size while ensuring the model's recognition accuracy. This solves the problem that existing text recognition models cannot balance model size while ensuring model accuracy, reduces the limitations of deploying text recognition models in some scenarios and on some devices, and expands the applicability of text recognition models.

[0064] Example 2

[0065] Figure 4 This is a flowchart of a text recognition method provided in Embodiment 2 of the present invention. This embodiment performs text recognition based on a text recognition model trained using the training method provided in any embodiment of the present invention. Figure 4 As shown, the method includes:

[0066] S210, Receive the image to be identified.

[0067] S220. Input the image to be recognized into a pre-trained text recognition model to obtain the text information in the image to be recognized, wherein the text recognition model is trained based on the text recognition model training method described in any embodiment of the present invention.

[0068] The text information of the image to be recognized refers to the character information in the image for text recognition, such as a single Chinese character, a single English character, or a Chinese string. In this embodiment, the received image to be recognized is input into a text recognition model pre-trained based on the text recognition model training method provided in any embodiment of the present invention to obtain the text information of the image to be recognized. It should be noted that the training method of the text recognition model in this embodiment refers to the above embodiments and will not be repeated here.

[0069] The technical solution of this embodiment recognizes images by using a pre-trained lightweight text recognition model, which reduces the limitations of the text recognition model in terms of scene and device, and improves the adaptability of the text recognition model.

[0070] Example 3

[0071] Figure 5 This is a schematic diagram of the structure of a training device for a text recognition model provided in Embodiment 3 of the present invention. Figure 5 As shown, the device includes:

[0072] The first text recognition model acquisition module 310 is used to acquire a first text recognition model, wherein the first text recognition model includes an encoding module and a decoding module;

[0073] The second text recognition model acquisition module 320 is used to train the first text recognition model. When the training process of the first text recognition model reaches the first training condition, the network layers in the decoding module are reduced to the target number to obtain the second text recognition model.

[0074] The third text recognition model acquisition module 330 is used to train the second text recognition model. When the training process of the second text recognition model reaches the second training condition, the encoding module in the second text recognition model is replaced by the lightweight encoding module to obtain the third text recognition model. The number of network layers of the lightweight encoding module is less than the number of network layers of the encoding module in the second text recognition model.

[0075] The text recognition model acquisition module 340 is used to train the third text recognition model, and when the training process of the third text recognition model reaches the third training condition, the trained text recognition model is obtained.

[0076] Based on the above embodiments, optionally, before training the first text recognition model, the apparatus further includes:

[0077] The initial training sample set acquisition module is used to acquire the initial training sample set, wherein the initial training samples include training data formed by sample images and text labels, and the sample images are images that include text information;

[0078] The enhanced training sample set acquisition module is used to enhance the sample images in the initial training sample set to obtain enhanced images of each sample image. The enhanced images and the text labels of the sample images are combined to form training data, which is added to the initial training sample set to obtain the enhanced training sample set. The enhanced training sample set is used to train the first text recognition model and the second text recognition model, and the initial training sample set is used to train the third text recognition model.

[0079] Based on the above embodiments, optionally, the second text recognition model acquisition module 320 is used to randomly determine a target number of network layers in the decoding module, and maintain the network parameters of the target number of network layers to form a new decoding module.

[0080] Optionally, the second text recognition model acquisition module 320 is used to determine the importance of each network layer based on the network parameters corresponding to each network layer in the decoding module, and to determine a target number of network layers from each network layer in the decoding module according to the importance of each network layer, so as to form a new decoding module.

[0081] Based on the above embodiments, optionally, the first text recognition model and the second text recognition model are trained based on a first learning rate, and the third text recognition model is trained at different stages based on the first learning rate and the second learning rate, wherein the second learning rate is less than the first learning rate.

[0082] Based on the above embodiments, optionally, when the training process of the third text recognition model reaches the third training condition, the text recognition model acquisition module 340 is further used to prune the convolutional channels of each network layer in the trained third text recognition model.

[0083] Based on the above embodiments, optionally, the text recognition model further includes a recurrent network module and a linear network module, wherein the recurrent network module is disposed between the encoding module and the decoding module, and the linear network module is disposed after the decoding module.

[0084] The text recognition model training device provided in the embodiments of the present invention can execute the text recognition model training method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0085] Example 4

[0086] Figure 6 This is a schematic diagram of the structure of a text recognition device provided in Embodiment 4 of the present invention. Figure 6 As shown, the device includes:

[0087] Image receiving module 410 is used to receive the image to be recognized;

[0088] The text recognition module 420 is used to input the image to be recognized into a pre-trained text recognition model to obtain the text information in the image to be recognized. The text recognition model is trained based on the text recognition model training method described in any embodiment of the present invention.

[0089] The text recognition device provided in the embodiments of the present invention can execute the text recognition method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0090] Example 5

[0091] Figure 7 This is a schematic diagram of the structure of an electronic device provided in Embodiment 5 of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0092] like Figure 7As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0093] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0094] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as training methods for text recognition models and / or text recognition methods.

[0095] In some embodiments, the text recognition model training method and / or text recognition method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the text recognition model training method and / or text recognition method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the text recognition model training method and / or text recognition method by any other suitable means (e.g., by means of firmware).

[0096] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0097] The computer program for training the text recognition model and / or implementing the text recognition method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer program causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer program can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0098] Example 6

[0099] Embodiment 6 of the present invention also provides a computer-readable storage medium storing computer instructions for causing a processor to execute a text recognition model training method, the method comprising:

[0100] A first text recognition model is obtained, comprising an encoding module and a decoding module. The first text recognition model is trained, and when the training process of the first text recognition model meets a first training condition, the number of network layers in the decoding module is reduced to a target number to obtain a second text recognition model. The second text recognition model is trained, and when the training process of the second text recognition model meets a second training condition, the encoding module in the second text recognition model is replaced with a lightweight encoding module to obtain a third text recognition model, wherein the number of network layers in the lightweight encoding module is less than the number of network layers in the encoding module of the second text recognition model. The third text recognition model is trained, and when the training process of the third text recognition model meets a third training condition, a trained text recognition model is obtained.

[0101] And / or, perform the text recognition method provided in the embodiments of the present invention, the method comprising:

[0102] Receive an image to be recognized; input the image to be recognized into a pre-trained text recognition model to obtain the text information in the image to be recognized, wherein the text recognition model is trained based on the text recognition model training method described in any embodiment of the present invention.

[0103] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0104] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0105] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0106] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0107] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0108] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for training a text recognition model, characterized in that, include: Obtain a first text recognition model, wherein the first text recognition model includes an encoding module and a decoding module; The first text recognition model is trained, and when the training process of the first text recognition model reaches the first training condition, the number of network layers in the decoding module is reduced to the target number to obtain the second text recognition model. The second text recognition model is trained. When the training process of the second text recognition model reaches the second training condition, the encoding module in the second text recognition model is replaced by a lightweight encoding module to obtain a third text recognition model. The number of network layers of the lightweight encoding module is less than the number of network layers of the encoding module in the second text recognition model. The third text recognition model is trained, and when the training process of the third text recognition model reaches the third training condition, a trained text recognition model is obtained. The step of training the third text recognition model, and obtaining a trained text recognition model when the training process of the third text recognition model meets the third training condition, includes: The third text recognition model is trained in the first stage based on the enhanced training sample set. When the training process of the third text recognition model meets the training conditions of the first stage, the optimization algorithm of the third text recognition model is replaced with an optimization algorithm whose initial learning rate is lower than the learning rate of the third text recognition model. The third text recognition model is then trained in the second stage based on the initial training sample set and the replaced optimization algorithm. When the training process of the third text recognition model meets the third training conditions, the trained text recognition model is obtained. The third training conditions include the number of training cycles of the third text recognition model in the second stage or the accuracy of model recognition during the training process of the third text recognition model in the second stage.

2. The method according to claim 1, characterized in that, Before training the first text recognition model, the method further includes: Obtain an initial training sample set, wherein the initial training samples include training data formed by sample images and text labels, and the sample images are images that include text information; The sample images in the initial training sample set are augmented to obtain augmented images of each sample image. The augmented images and the text labels of the sample images are combined to form training data, which is added to the initial training sample set to obtain an augmented training sample set. The augmented training sample set is used to train the first text recognition model and the second text recognition model, and the initial training sample set is used to train the third text recognition model.

3. The method according to claim 1, characterized in that, The number of network layers in the decoding module is reduced to the target number to obtain the second text recognition model, which includes: In the decoding module, a target number of network layers are randomly determined, and the network parameters of the target number of network layers are maintained to form a new decoding module.

4. The method according to claim 1, characterized in that, The number of network layers in the decoding module is reduced to the target number to obtain the second text recognition model, which includes: Based on the network parameters corresponding to each network layer in the decoding module, the importance of each network layer is determined, and a target number of network layers are determined from each network layer in the decoding module according to the importance of each network layer to form a new decoding module.

5. The method according to claim 1, characterized in that, The first text recognition model and the second text recognition model are trained based on a first learning rate, and the third text recognition model is trained at different stages based on the first learning rate and the second learning rate, wherein the second learning rate is less than the first learning rate.

6. The method according to claim 1, characterized in that, When the training process of the third text recognition model reaches the third training condition, the method further includes: Pruning is performed on the convolutional channels of each network layer in the trained third text recognition model.

7. The method according to claim 1, characterized in that, The text recognition model further includes a recurrent network module and a linear network module, wherein the recurrent network module is located between the encoding module and the decoding module, and the linear network module is located after the decoding module.

8. A text recognition method, characterized in that, include: Receive the image to be recognized; The image to be recognized is input into a pre-trained text recognition model to obtain the text information in the image to be recognized, wherein the text recognition model is trained based on the text recognition model training method of any one of claims 1-7.

9. A training device for a text recognition model, characterized in that, include: The first text recognition model acquisition module is used to acquire a first text recognition model, wherein the first text recognition model includes an encoding module and a decoding module; The second text recognition model acquisition module is used to train the first text recognition model. When the training process of the first text recognition model reaches the first training condition, the network layers in the decoding module are reduced to the target number to obtain the second text recognition model. The third text recognition model acquisition module is used to train the second text recognition model. When the training process of the second text recognition model reaches the second training condition, the encoding module in the second text recognition model is replaced by the lightweight encoding module to obtain the third text recognition model. The number of network layers of the lightweight encoding module is less than the number of network layers of the encoding module in the second text recognition model. The text recognition model acquisition module is used to train the third text recognition model. When the training process of the third text recognition model reaches the third training condition, the trained text recognition model is obtained. The step of training the third text recognition model, and obtaining a trained text recognition model when the training process of the third text recognition model meets the third training condition, includes: The third text recognition model is trained in the first stage based on the enhanced training sample set. When the training process of the third text recognition model meets the training conditions of the first stage, the optimization algorithm of the third text recognition model is replaced with an optimization algorithm whose initial learning rate is lower than the learning rate of the third text recognition model. The third text recognition model is then trained in the second stage based on the initial training sample set and the replaced optimization algorithm. When the training process of the third text recognition model meets the third training conditions, the trained text recognition model is obtained. The third training conditions include the number of training cycles of the third text recognition model in the second stage or the accuracy of model recognition during the training process of the third text recognition model in the second stage.

10. A text recognition device, characterized in that, include: The image receiving module is used to receive the image to be recognized; The text recognition module is used to input the image to be recognized into a pre-trained text recognition model to obtain the text information in the image to be recognized, wherein the text recognition model is trained based on the text recognition model training method according to any one of claims 1-7.

11. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the training method of the text recognition model according to any one of claims 1-7 and / or the text recognition method according to claim 8.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute and implement the training method of the text recognition model according to any one of claims 1-7 and / or the text recognition method according to claim 8.

Images