A text recognition model compression method and system based on recoding

Abstract

The present invention proposes a text recognition model compression method and system based on recoding, which includes the following steps: S1, obtaining and organizing original samples, and using the original samples to train a first text recognition model; S2, performing knowledge distillation on the trained first text recognition model to obtain a compressed second text recognition model; S3, retaining the backbone network of the second text recognition model, rebuilding the classification layer of the second text recognition model, and obtaining a third text recognition model; and S4, re-labeling the original samples to obtain test samples, using the test samples to train the third text recognition model, and performing network performance verification on the trained third text recognition model. This method can fully compress large character library models, avoid significantly affecting model recognition accuracy, improve model inference speed, reduce model size, and enable the model to fully meet the needs of actual business scenarios.

Description

Technical Field

[0001] The present application relates to the field of model compression technology, and in particular to a text recognition model compression method and system based on recoding. Background Art

[0002] Scene text recognition is often considered a specialized and more challenging optical character recognition task. The rich and precise semantic information carried by text plays a crucial role in a variety of vision-based application scenarios. Scene text recognition tasks can be further subdivided into text detection, text recognition, and other auxiliary tasks. With the emergence of algorithms such as CTC Loss, single-word recognition, which often requires high annotation costs, has been gradually replaced by text recognition. Most text recognition models are based on an encoder-decoder architecture. Based on the characteristics of text sequences, decoders based on connection-based temporal classification or attention mechanisms are used to decode the text. The decoded features are then fed into the classification layer for classification.

[0003] Due to the increasing demand for scene text recognition on mobile devices, overly large models are difficult to implement industrially due to storage space and computing power constraints. This has led to the concept of model compression. Mainstream model compression methods include network pruning, knowledge distillation, parameter quantization, and architecture design. Most models can achieve model compression by utilizing one or more of these methods. Compression techniques used in text recognition models primarily focus on optimizing the backbone network, or the encoding and decoding network structure. Generally, since the backbone network consumes the majority of computing power, significant compression and optimization of the backbone network can achieve ideal model performance. However, unlike other languages ​​like English, Chinese has a vast character library, with commonly used characters alone reaching three to four thousand. This makes the classification layer of models designed for text recognition in languages ​​with large character libraries like Chinese much larger than those in languages ​​with smaller character libraries like English. As a result, even after compression and optimization, the classification layer remains large, with the majority of computing power concentrated in this layer, slowing down model inference speed. Although the optimization of the classification layer can use full convolution instead of the fully connected layer, and use singular value decomposition technology to compress the model, it is often impossible to achieve the ideal model compression effect while ensuring the performance of the model.

[0004] To address the above problems, this paper proposes a text recognition model compression method and system based on re-encoding. This method addresses the compression difficulties of large-character text recognition models by re-encoding the character library, redesigning the model classification layer, and using transfer learning technology to adjust network parameters. This method can significantly improve the compression rate of large-character text recognition models, and greatly optimize the model inference speed and model size while ensuring the model recognition rate. Summary of the Invention

[0005] In order to solve the problems such as the difficulty in compressing large-character text recognition models while ensuring model performance, the present application provides a text recognition model compression method and system based on re-encoding to solve the above technical defects.

[0006] According to one aspect of the present invention, a text recognition model compression method based on recoding is proposed, comprising the following steps:

[0007] S1. Obtain and organize original samples, and use the original samples to train a first text recognition model;

[0008] S2. Perform knowledge distillation on the trained first text recognition model to obtain a compressed second text recognition model;

[0009] S3, retaining the backbone network of the second text recognition model, rebuilding the classification layer of the second text recognition model, and obtaining a third text recognition model; and

[0010] S4. Re-label the original samples to obtain samples to be tested, use the samples to be tested to train the third text recognition model, and verify the network performance of the trained third text recognition model.

[0011] In a specific embodiment, in step S1, the first text recognition model uses a feature extractor based on the transformer architecture as the backbone network and a classification layer based on a fully connected layer as the model output head. For the original sample input of a single image, the first text recognition model outputs a posterior probability matrix of T×nclass, where T is the feature map width and nclass is the font length.

[0012] In a specific embodiment, in step S1, the first text recognition model uses CTCLoss as the network loss function:

[0013]

[0014]

[0015] Among them, L(S) is the loss function expression, p(z|x) is the probability of output label result, B -1It is the mapping function of the set of all paths of the target sequence z.

[0016] In a specific embodiment, in step S2, knowledge distillation is performed on the trained first text recognition model to obtain a second text recognition model with a smaller number of parameters and a smaller amount of computation than the first text recognition model. The loss function of the knowledge distillation is:

[0017] L=L ctc +αCE(q,p)

[0018] Where, L ctc is the loss function of CTC, CE is the cross entropy, q is the output result of the first text recognition model, p is the output result of the second text recognition model, and α is an adjustable parameter.

[0019] In a specific embodiment, in step S4, the original sample is re-labeled, including: performing hexadecimal Unicode encoding on all labeled characters of the original sample.

[0020] In a specific embodiment, in step S3, the backbone network of the second text recognition model is retained, the classification layer of the second text recognition model is rebuilt, and a third text recognition model is obtained. The third text recognition model uses four parallel classification layers to predict the four coding bits of the hexadecimal Unicode encoding respectively.

[0021] In a specific embodiment, in step S4, the third text recognition model is trained using the sample to be tested. During the training process, the backbone network parameters are fixed, and only four parallel classification layers are trained. The four parallel classification layers all use CTCLoss as the loss function. After the loss of the third text recognition model converges, the overall network parameters are fine-tuned.

[0022] In a specific embodiment, in step S4, performing network performance verification on the trained third text recognition model includes:

[0023] S41, inputting the text image to be verified into the trained third text recognition model; and

[0024] S42. Use the argmax function to obtain the prediction results of the four parallel classification layers, combine them according to the coding bits, and convert them into strings to verify the model performance.

[0025] In a second aspect, the present application provides a text recognition model compression system based on recoding, the system comprising:

[0026] an acquisition module configured to acquire and organize original samples and use the original samples to train the first text recognition model; and

[0027] a compression module configured to perform knowledge distillation on the trained first text recognition model to obtain a compressed second text recognition model; and

[0028] a reconstruction module configured to retain the backbone network of the second text recognition model, rebuild the classification layer of the second text recognition model, and obtain a third text recognition model; and

[0029] The training module is configured to re-label the original samples to obtain the samples to be tested, use the samples to be tested to train the third text recognition model, and verify the network performance of the trained third text recognition model.

[0030] In a third aspect, the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, any of the above methods is implemented.

[0031] Compared with the prior art, the present invention has the following advantages:

[0032] For most neural network models designed for visual image tasks, the network only needs to perform one inference on a single image to complete the inference task. However, there are often multiple texts on an image, and the text recognition model needs to perform one inference on each text in order to obtain all the text information on a single image. This makes the inference speed of the text recognition model higher in actual business scenarios. The present invention provides a text recognition model compression method based on recoding, which can fully compress large character library models, avoid obvious impact on model recognition accuracy, improve model inference speed, reduce model scale, and make the model fully meet the needs of actual business scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Other features, objects and advantages of the present application will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0034] Figure 1 is a flowchart of a text recognition model compression method based on recoding according to the present application;

[0035] Figure 2 is a structural diagram of a first text recognition model according to the present application;

[0036] Figure 3 is a structural diagram of a third text recognition model according to the present application;

[0037] Figure 4 is a structural diagram of a text recognition model compression system based on recoding according to the present application;

[0038] Figure 5It is a structural diagram of a computer system suitable for implementing the electronic device of the embodiment of the present application. DETAILED DESCRIPTION

[0039] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the relevant invention are shown in the accompanying drawings.

[0040] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0041] Figure 1 The flowchart of the text recognition model compression method based on recoding of this application is shown in FIG. Figure 1 , the method comprises the following steps:

[0042] S1. Obtain and organize original samples, and use the original samples to train a first text recognition model.

[0043] In this embodiment, collecting and organizing original samples refers to collecting text recognition related data sets and dividing them into training sets and validation sets according to a certain ratio. The ratio used in this application defaults to but is not limited to 50:1.

[0044] Figure 2 The structure diagram of the first text recognition model of this application is shown as Figure 2 As shown, this application needs to first build a first text recognition model based on a neural network. The first text recognition model consists of two parts: a backbone network and an output head. Among them, a feature extractor based on a transformer architecture is used as the backbone network, and a classification layer based on a fully connected layer is used as the model output head. For the original sample input of a single image, the first text recognition model outputs a T×nclass posterior probability matrix, where T is the feature map width, which is related to the width of the input image, and nclass is the length of the font library, that is, each element of the column vector represents the corresponding character prediction probability.

[0045] Then, the original samples are used to train the first text recognition model, and CTCLoss is used as the network loss function:

[0046]

[0047]

[0048] Where p(z|x) is the probability of outputting the label result, which is obtained by accumulating the probability of all the corresponding output sequences, that is, the paths mapped to the label results.-1 It is the mapping function of the set of all paths of the target sequence z, and the first text recognition model is trained to convergence.

[0049] S2. Perform knowledge distillation on the trained first text recognition model to obtain a compressed second text recognition model.

[0050] In this embodiment, the knowledge distillation technology is used to perform the first model compression on the trained first text recognition model to obtain a second text recognition model with a smaller number of parameters and a smaller amount of computation than the first text recognition model. The loss function of the knowledge distillation is:

[0051] L=L ctc +αCE(q,p)

[0052] Where CE is cross entropy, q is the output result of the first text recognition model, p is the output result of the second text recognition model, and α is an adjustable parameter.

[0053] S3. Retain the backbone network of the second text recognition model, rebuild the classification layer of the second text recognition model, and obtain the third text recognition model.

[0054] In this embodiment, the network architecture of the second text recognition model in step 2 is reconstructed, the backbone network of the second text recognition model is retained, and the classification layer of the second text recognition model is rebuilt to obtain the third text recognition model. Figure 3 The structure diagram of the third text recognition model of this application is shown in FIG. Figure 3 The third text recognition model uses four parallel classification layers to predict the four code bits of the hexadecimal Unicode encoding respectively.

[0055] S4. Re-label the original samples to obtain samples to be tested, use the samples to be tested to train the third text recognition model, and verify the network performance of the trained third text recognition model.

[0056] In this embodiment, the original sample is uniformly processed, and all the marked characters of the original sample are encoded to obtain the sample to be tested. This application adopts Unicode encoding and uses hexadecimal character encoding.

[0057] The third text recognition model is trained using the test samples. During the training process, the backbone network parameters are fixed, and only four parallel classification layers are trained. The four parallel classification layers all use CTCLoss as the loss function. After the loss of the third text recognition model converges, the overall network parameters are fine-tuned.

[0058] Verify the network performance of the trained third text recognition model, which includes the following sub-steps:

[0059] S41, inputting the text image to be verified into the trained third text recognition model; and

[0060] S42. Use the argmax function to obtain the prediction results of the four parallel classification layers, combine them according to the coding bits, and convert them into strings to verify the model performance.

[0061] For most neural network models designed for visual image tasks, the network only needs to perform one inference on a single image to complete the inference task. However, there are often multiple texts on an image, and the text recognition model needs to perform one inference on each text in order to obtain all the text information on a single image. This makes the inference speed of the text recognition model higher in actual business scenarios. The present invention provides a text recognition model compression method based on recoding, which can fully compress large character library models, avoid obvious impact on model recognition accuracy, improve model inference speed, reduce model scale, and make the model fully meet the needs of actual business scenarios.

[0062] Further references Figure 4 As an implementation of the above method, the present application provides an embodiment of a text recognition model compression system based on recoding. Figure 1 Corresponding to the method embodiment shown, the system can be specifically applied to various electronic devices. The system 400 includes the following modules:

[0063] An acquisition module 410 is configured to acquire and organize original samples and use the original samples to train a first text recognition model;

[0064] A compression module 420 is configured to perform knowledge distillation on the trained first text recognition model to obtain a compressed second text recognition model;

[0065] A reconstruction module 430 is configured to retain the backbone network of the second text recognition model, rebuild the classification layer of the second text recognition model, and obtain a third text recognition model; and

[0066] The training module 440 is configured to re-label the original samples to obtain samples to be tested, train the third text recognition model using the samples to be tested, and perform network performance verification on the trained third text recognition model.

[0067] Reference below Figure 5 , which shows a structural diagram of a computer system 500 suitable for implementing an electronic device of an embodiment of the present application. Figure 5 The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0068] like Figure 5 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage unit 508 into a random access memory (RAM) 503. Various programs and data required for the operation of the system 500 are also stored in the RAM 503. The CPU 501, ROM 502, and RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0069] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, and the like; an output section 507 including a liquid crystal display (LCD) and speakers; a storage section 508 including a hard disk; and a communication section 509 including a network interface card such as a LAN card or a modem. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 510 as needed, so that computer programs read therefrom can be installed into the storage section 508 as needed.

[0070] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 509, and / or installed from a removable medium 511. When the computer program is executed by the central processing unit (CPU) 501, the above-mentioned functions defined in the method of the present application are performed.

[0071] It should be noted that the computer-readable storage medium described in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code contained on a computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical cable, RF, or any suitable combination thereof.

[0072] Computer program code for performing the operations of the present application can be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0073] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0074] The units involved in the embodiments described in the present application can be implemented by software or by hardware. The described units can also be set in a processor. For example, it can be described as: a processor includes a first determination unit, a second determination unit, a generation unit, a first extraction unit and a first storage unit. Among them, the names of these units do not constitute a limitation on the units themselves under certain circumstances. For example, the first determination unit can also be described as "a unit for determining whether there is new event information in the preset event information list."

[0075] As another aspect, the present application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment; or it may exist independently without being assembled into the electronic device. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed by the electronic device, the electronic device: determines whether there is newly added event information in a preset event information list, wherein each event information in the event information list includes event description information; in response to determining the existence, determines the newly added event information as target event information; identifies the event description information of the target event information and generates a label for the target event information; extracts a set of element information from the target event information; and stores the target event information, the set of element information, and the label in a preset event information library.

[0076] The above description is merely a preferred embodiment of the present application and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned inventive concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A text recognition model compression method based on recoding, characterized in that: The following steps are involved: S1. Obtain and organize original samples, and use the original samples to train a first text recognition model; S2. Perform knowledge distillation on the trained first text recognition model to obtain a compressed second text recognition model; S3, retaining the backbone network of the second text recognition model, rebuilding the classification layer of the second text recognition model, and obtaining a third text recognition model; as well as S4. Re-labeling the original sample, specifically including: performing hexadecimal Unicode encoding on all labeled characters of the original sample to obtain a test sample, using the test sample to train the third text recognition model, and performing network performance verification on the trained third text recognition model, wherein the third text recognition model uses four parallel classification layers to predict the four coding bits of the hexadecimal Unicode encoding respectively.

2. The text recognition model compression method based on recoding according to claim 1, characterized in that In step S1, the first text recognition model uses a feature extractor based on a transformer architecture as a backbone network and a classification layer based on a fully connected layer as a model output head. For the original sample input of a single image, the first text recognition model outputs T The posterior probability matrix of nclass, where T is the width of the feature map and nclass is the length of the font.

3. The text recognition model compression method based on recoding according to claim 1, characterized in that In step S1, the first text recognition model uses CTCLoss as the network loss function: ， Among them, L(S) is the loss function expression, p(z|x) is the probability of the output label result, is the mapping function of the set of all paths of the target sequence z.

4. The text recognition model compression method based on recoding according to claim 1, characterized in that In step S2, knowledge distillation is performed on the trained first text recognition model to obtain a second text recognition model with a smaller number of parameters and a smaller amount of computation than the first text recognition model. The loss function of the knowledge distillation is: ， Where, is the loss function of CTC, CE is the cross entropy, q is the output result of the first text recognition model, and p is the output result of the second text recognition model. It is an adjustable parameter.

5. The text recognition model compression method based on recoding according to claim 1, characterized in that In step S4, the third text recognition model is trained using the test samples. During the training process, the backbone network parameters are fixed, and only four parallel classification layers are trained. The four parallel classification layers all use CTCLoss as the loss function. After the loss of the third text recognition model converges, the overall network parameters are fine-tuned.

6. The text recognition model compression method based on recoding according to claim 1, characterized in that In step S4, performing network performance verification on the trained third text recognition model includes: S41, inputting the text image to be verified into the trained third text recognition model; and S42. Use the argmax function to obtain the prediction results of the four parallel classification layers, combine them according to the coding bits, and convert them into strings to verify the model performance.

7. A text recognition model compression system based on recoding, characterized in that: The system comprises: an acquisition module configured to acquire and organize original samples, and use the original samples to train a first text recognition model; and a compression module configured to perform knowledge distillation on the trained first text recognition model to obtain a compressed second text recognition model; and a reconstruction module configured to retain the backbone network of the second text recognition model, rebuild the classification layer of the second text recognition model, and obtain a third text recognition model; and The training module is configured to re-label the original sample, specifically including: performing hexadecimal Unicode encoding on all labeled characters of the original sample to obtain a sample to be tested, using the sample to be tested to train the third text recognition model, and performing network performance verification on the trained third text recognition model, wherein the third text recognition model uses four parallel classification layers to predict the four coding bits of the hexadecimal Unicode encoding respectively.

8. A computer-readable storage medium, wherein a computer program is stored in the medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images