A method, device and storage medium for optimizing a character recognition model

By pruning nodes and optimizing layers in the text recognition model, and migrating the decoding layer to the GPU, the problem of slow text decoding speed in live images was solved, improving the model's decoding speed and the system's processing efficiency.

CN121170522BActive Publication Date: 2026-11-17GUANGZHOU HUYA TECH CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202511335958.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-09-18
Publication Date
2026-11-17
Estimated Expiration
2045-09-18

AI Technical Summary

Technical Problem

The current live image text decoding speed is relatively slow, mainly due to the redundancy of the dual decoder architecture and the limitation of CPU performance on GPU decoding speed.

Method used

By pruning nodes in the text recognition model, optimizing the encoding layer and bidirectional long short-term memory network layer, and generating a GPU plugin, the decoding layer is migrated to the GPU for execution, eliminating CPU performance limitations.

Benefits of technology

The text recognition model was optimized, which improved decoding speed, reduced computational load, and increased system throughput.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121170522B_ABST
    Figure CN121170522B_ABST
Patent Text Reader

Abstract

This invention proposes a method, device, and storage medium for optimizing a text recognition model, relating to the field of computer technology. The method includes: obtaining inference nodes of the text recognition model from its configuration file; pruning nodes in the text recognition model based on the inference nodes to obtain an inference model; fusing and optimizing the encoding layer based on the encoding layer structure information of the inference model to obtain an optimized encoding layer; optimizing the input interface of the bidirectional long short-term memory (BSSM) network layer of the inference model to obtain an optimized interface; connecting the BSSM network layer and the optimized encoding layer through the optimized interface; obtaining functional information of the decoding layer of the inference model; generating a GPU plugin and a GPU plugin interface based on the functional information; and connecting the BSSM network layer and the GPU plugin through the GPU plugin interface to obtain the optimized recognition model. This invention can reduce the computational load of the text recognition model, eliminate CPU performance limitations, and improve decoding speed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically, to a method, device, and storage medium for optimizing a character recognition model. Background Technology

[0002] As live streaming scenarios and content become increasingly complex, the demands on the speed of image-text processing in live streams are also rising. Live streaming image-text processing primarily involves converting text in live images into editable text sequences to generate real-time subtitles or perform real-time detection of sensitive content. Existing live streaming text recognition models typically use a dual-decoder architecture, combining an Attention decoder and a CTC decoder, to decode image text into text sequences. While dual decoders offer high accuracy, their redundant architecture and high computational cost result in slow decoding speeds. Furthermore, CTC decoding often involves the CPU processing massive amounts of text lines one by one before the GPU performs the actual decoding, causing the GPU's decoding speed to be limited by the CPU's performance. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a text recognition model optimization method, device and storage medium to solve the problem of slow text decoding speed in live images in the prior art.

[0004] To achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows: In a first aspect, embodiments of the present invention provide a method for optimizing a text recognition model, including: The inference nodes of the character recognition model are obtained from the configuration file of the character recognition model, and the inference model is obtained by pruning the nodes of the character recognition model according to the inference nodes. Obtain the structural information of the coding layer of the inference model, and perform fusion optimization on the coding layer based on the structural information to obtain an optimized coding layer; An optimized interface is obtained by optimizing the input interface of the bidirectional long short-term memory network layer of the inference model, and the bidirectional long short-term memory network layer is connected to the optimized coding layer through the optimized interface; The functional information of the decoding layer of the inference model is obtained, and a GPU plugin and a GPU plugin interface are generated based on the functional information. The bidirectional long short-term memory network layer is connected to the GPU plugin through the GPU plugin interface to obtain an optimized recognition model.

[0005] In an optional implementation, the step of obtaining the functional information of the decoding layer of the inference model and generating the GPU plugin and GPU plugin interface based on the functional information includes: Generate a GPU initial plugin and obtain the functional information of the decoding layer on the CPU. Configure the GPU initial plugin according to the functional information to obtain the GPU plugin. Based on the interaction information characteristics between the GPU plugin and the bidirectional long short-term memory network layer, configure the GPU plugin interface for the GPU plugin.

[0006] In an optional implementation, the step of configuring the GPU initial plugin according to the functional information to obtain the GPU plugin includes: Configure the GPU initial plugin with a maximum value index lookup function and shared memory, so that the GPU plugin can determine the maximum value index corresponding to the received information sequence and store the maximum value index in the shared memory; The GPU plugin is obtained by configuring parallel deduplication and parallel blank removal functions in the shared memory.

[0007] In an optional implementation, the step of configuring parallel deduplication and parallel blank removal functions in the shared memory includes: Configure adjacent character index comparison and atomic addition functions in the shared memory so that the GPU plugin can perform parallel deduplication of the information sequence based on the maximum value index; A parallel whitespace removal function is configured in the shared memory so that the GPU plugin can remove whitespace from the parallel deduplicated information sequence to obtain the output result corresponding to the information sequence.

[0008] In an optional implementation, the step of optimizing the input interface of the bidirectional long short-term memory network layer of the inference model to obtain an optimized interface includes: The input interface of the bidirectional long short-term memory network layer is replaced with a variable-length input interface. The optimized interface is obtained by configuring dynamic input function for the variable-length input interface, so that the time dimension of the input information of the variable-length input interface becomes dynamic.

[0009] In an optional implementation, the step of obtaining the structural information of the coding layer of the inference model and performing fusion optimization on the coding layer of the inference model based on the structural information to obtain an optimized coding layer includes: Obtain the structural and operational information corresponding to the depthwise and pointwise convolutional layers of the coding layer; The structural information and the operational information are analyzed to obtain the analysis results, and the depthwise convolutional layer and the pointwise convolutional layer are fused together based on the analysis results to obtain the optimized coding layer.

[0010] In an optional implementation, the step of pruning the text recognition model based on the inference nodes to obtain the inference model includes: Obtain the starting node, the ending node, and the nodes to be pruned between the starting node and the ending node from the inference nodes; Starting from the final node, perform a reverse traversal of all nodes, and keep the reachable nodes of the final node as the node retention set. Starting from the node to be pruned, traverse forward and take the reachable nodes of the node to be pruned as the node to be pruned set; The node pruning set is determined based on the node retention set and the node pruning set, and the inference model is obtained by pruning the nodes of the text recognition model based on the node pruning set.

[0011] Secondly, embodiments of the present invention provide a text recognition model optimization device, comprising: The node pruning module is used to obtain the inference nodes of the text recognition model from the configuration file of the text recognition model, and to prune the text recognition model according to the inference nodes to obtain the inference model. The fusion optimization module is used to obtain the structural information of the coding layer of the inference model, and to perform fusion optimization on the coding layer based on the structural information to obtain an optimized coding layer; An interface optimization module is used to optimize the input interface of the bidirectional long short-term memory network layer of the inference model to obtain an optimized interface, and to connect the bidirectional long short-term memory network layer with the optimized coding layer through the optimized interface. The plugin configuration module is used to obtain the functional information of the decoding layer of the inference model, generate a GPU plugin and a GPU plugin interface based on the functional information, and connect the bidirectional long short-term memory network layer with the GPU plugin through the GPU plugin interface to obtain an optimized recognition model.

[0012] Thirdly, embodiments of the present invention provide a character recognition model optimization device, including a processor and a memory, wherein the memory stores machine-executable instructions that can be executed by the processor, and the processor can execute the machine-executable instructions to implement the character recognition model optimization method described in the first aspect.

[0013] Fourthly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the character recognition model optimization method described in the first aspect.

[0014] This invention provides a text recognition model optimization method, device, and storage medium. By pruning nodes in a complex text recognition model, a simplified inference model is obtained, reducing redundant structure and computational load. Then, the encoding layer and bidirectional long short-term memory network layer of the inference model are optimized, and the decoding layer is optimized into a GPU plugin, so that the encoding to decoding process is executed entirely on the GPU, eliminating CPU performance limitations and achieving optimization of the text recognition model.

[0015] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 The diagram shows a block diagram of a text recognition model optimization device provided in an embodiment of the present invention; Figure 2 A flowchart illustrating a text recognition model optimization method provided by an embodiment of the present invention is shown; Figure 3 This diagram illustrates the optimization principle of a text recognition model according to an embodiment of the present invention. Figure 4 The diagram shows a functional module schematic of a text recognition model optimization device provided in an embodiment of the present invention.

[0018] Icons: 100 - Text recognition model optimization device; 110 - Memory; 120 - Processor; 130 - Communication module; 200 - Node pruning module; 300 - Fusion optimization module; 400 - Interface optimization module; 500 - Plugin configuration module. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0020] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0021] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0022] Please refer to Figure 1 , Figure 1 This is a block diagram of the text recognition model optimization device 100 provided in this embodiment. The text recognition model optimization device 100 includes a memory 110, a processor 120, and a communication module 130. The memory 110, processor 120, and communication module 130 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0023] The memory 110 is used to store programs or data. The memory 110 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0024] The processor 120 is used to read / write data or programs stored in the memory 110 and to perform corresponding functions.

[0025] The communication module 130 is used to establish a communication connection between the text recognition model optimization device 100 and other communication terminals through the network, and to send and receive data through the network.

[0026] It should be understood that, Figure 1 The structure shown is only a schematic diagram of the text recognition model optimization device 100. The text recognition model optimization device 100 may also include a ratio Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown. Figure 1 The components shown can be implemented using hardware, software, or a combination thereof.

[0027] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating a text recognition model optimization method provided in this embodiment. The method includes: S201. Obtain the inference nodes of the character recognition model from the configuration file of the character recognition model, and perform node pruning on the character recognition model according to the inference nodes to obtain the inference model.

[0028] Optical Character Recognition (OCR) systems are end-to-end systems for processing text in live images. An OCR system can consist of multiple models. The text recognition model primarily converts the text detected by the OCR system into editable and searchable text. The decoder of the text recognition model can employ a dual-decoder approach, combining an Attention decoder and a CTC decoder. The Attention decoder mainly handles long alignment issues, while the CTC decoder primarily handles monotonic alignment issues.

[0029] Inference nodes in a text recognition model can be viewed as all the computational steps involved in the process from receiving image text to outputting editable text, with each computational step corresponding to an inference node. However, text recognition models often contain some redundant inference nodes. For example, the Attention decoder itself has a relatively complex structure with many irrelevant computational branches. Therefore, node pruning can be performed on the text recognition model based on the inference nodes, retaining only one complete inference path to obtain an optimized inference model.

[0030] S202. Obtain the structural information of the coding layer of the inference model, and perform fusion optimization on the coding layer based on the structural information to obtain an optimized coding layer.

[0031] A text recognition model typically consists of three parts: an encoding layer, a bidirectional long short-term memory (BiLSTM) network layer, and a decoding layer. To optimize the text recognition model, these three parts can be optimized sequentially to reduce structural redundancy and improve efficiency.

[0032] The coding layer generally consists of two components: depthwise convolutional layers and pointwise convolutional layers. When optimizing the coding layer, an inference acceleration engine can be used to fuse the two components of the coding layer, thereby obtaining a faster and smaller coding computation unit that can run on a GPU.

[0033] S203. Optimize the input interface of the bidirectional long short-term memory network layer of the inference model to obtain an optimized interface, and connect the bidirectional long short-term memory network layer and the optimized coding layer through the optimized interface.

[0034] The bidirectional long short-term memory network layer is formed by superimposing two long short-term memory (LSTM) networks, which enables the model to take into account the context information when processing image text sequences. It encodes the text sequence generated by the encoder into a hidden state with context information, and then provides it to the decoder to generate an editable text sequence.

[0035] Since the text in images during live streaming is usually of varying lengths, i.e., variable-length text, the BiLSTM layer needs to fill in the content before processing the text, which results in a slow processing speed. Therefore, the interface between the BiLSTM layer and the encoding layer can be optimized to process the text in advance through the interface, thereby improving the processing speed of the BiLSTM layer.

[0036] S204. Obtain the functional information of the decoding layer of the inference model, generate a GPU plugin and a GPU plugin interface based on the functional information, and connect the bidirectional long short-term memory network layer with the GPU plugin through the GPU plugin interface to obtain an optimized recognition model.

[0037] To improve the decoding speed of the decoding layer, image and text processing tasks that were originally executed on the CPU can be copied to the GPU. First, the function information corresponding to the image and text processing functions on the CPU is copied to the GPU. Then, a GPU plugin is generated based on the function information. Finally, an interface between the decoding layer and the GPU plugin is generated, allowing the GPU to process and decode the image and text encoding information sent by the BiLSTM layer in parallel using threads.

[0038] This embodiment obtains a simplified inference model by pruning nodes in a complex text recognition model, reducing redundant structure and computational load. Then, the encoding layer and bidirectional long short-term memory network layer of the inference model are optimized, and the decoding layer is optimized into a GPU plugin, so that the encoding to decoding process is executed entirely on the GPU, eliminating the performance limitations of the CPU and achieving optimization of the text recognition model.

[0039] Please refer to Figure 3 , Figure 3 This is a schematic diagram illustrating the optimization principle of a text recognition model provided in this embodiment.

[0040] The text recognition model optimization process in this embodiment mainly includes three parts: first, static pruning and inference state transformation; second, core module optimization; and third, full-process GPU decoding.

[0041] In one implementation, the step of pruning the text recognition model based on the inference nodes to obtain the inference model includes: Obtain the starting node, the ending node, and the nodes to be pruned between the starting node and the ending node from the inference nodes; Starting from the final node, perform a reverse traversal of all nodes, and keep the reachable nodes of the final node as the node retention set. Starting from the node to be pruned, traverse forward and take the reachable nodes of the node to be pruned as the node to be pruned set; The node pruning set is determined based on the node retention set and the node pruning set, and the inference model is obtained by pruning the nodes of the text recognition model based on the node pruning set.

[0042] The computation graph of a text recognition model can be understood as a data flow diagram generated by all mathematical formulas in the text recognition model. In the data graph, nodes represent operations, and edges represent the flow of tensor data. The data graph can describe all the computational steps of the text recognition model from input to output and from loss to gradient.

[0043] To avoid unnecessary calculation steps, i.e., to remove unnecessary nodes, we can first determine all the nodes in the computation graph and divide all nodes into the starting node, the ending node, and the nodes to be pruned between the starting node and the ending node.

[0044] Then, an automated script is invoked to traverse the nodes. The script first traverses backward from the final node, taking all reachable nodes of the final node as the node retain set. Then, it traverses forward from the node to be pruned, taking all reachable nodes of the node to be pruned as the node pruning set. Then, nodes that belong to the node pruning set but not to the node retain set are marked as the node pruning set. Finally, all nodes in the node pruning set and their associated weight initializers are removed, resulting in the optimized inference model. The inference model removes all unnecessary nodes and retains only a streamlined inference-specific model containing an encoder layer, a BiLSTM layer, and a CTC decoder layer.

[0045] This embodiment determines the computation nodes based on the computation graph of the text recognition model, and then performs node pruning operations on the computation nodes. This statically transforms the original complex model containing a dual decoder path into a streamlined inference-specific model containing only an encoder layer, a BiLSTM layer, and a CTC decoder layer, greatly reducing the complexity of the text recognition model and the computational load of the model.

[0046] After pruning the nodes of the model, the core modules of the text recognition model can be optimized. The core modules mainly include the encoder layer and the BiLSTM layer.

[0047] The optimization of the encoder layer mainly involves optimizing and fusing the depthwise convolutional layers and the pointwise convolutional layers. Specifically, this can be achieved by leveraging the native support of the inference acceleration engine and log analysis capabilities to merge the depthwise and pointwise convolutional layers into a single computational unit, thereby obtaining an optimized encoder layer.

[0048] In one implementation, the step of optimizing the input interface of the bidirectional long short-term memory network layer of the inference model to obtain an optimized interface includes: The input interface of the bidirectional long short-term memory network layer is replaced with a variable-length input interface. The optimized interface is obtained by configuring dynamic input function for the variable-length input interface, so that the time dimension of the input information of the variable-length input interface becomes dynamic.

[0049] The original interface of the BiLSTM layer can be replaced by the IRNNv2Layer interface of TensorRT. The IRNNv2Layer interface is a recurrent neural network interface that can support variable-length sequence input. That is, it can directly receive variable-length sequences from the encoding layer. Then, by using TensorRT's optimization profiles, the time dimension of the input information of the BiLSTM layer is set to be dynamic, so that the BiLSTM layer can directly process variable-length sequences.

[0050] This embodiment replaces the original interface of the BiLSTM layer with the IRNNv2Layer interface, enabling the BiLSTM layer to support input of variable-length sequences without the need for content padding. This avoids the BiLSTM layer performing invalid calculations on the padding part. Furthermore, by optimizing the configuration file to make the time dimension of the input information of the BiLSTM layer dynamic, the BiLSTM layer does not need a fixed sequence length during inference and can directly process variable-length sequences, thus avoiding the waste of computing power of the BiLSTM layer.

[0051] In one implementation, the step of obtaining the functional information of the decoding layer of the inference model and generating a GPU plugin and a GPU plugin interface based on the functional information includes: Generate a GPU initial plugin and obtain the functional information of the decoding layer on the CPU. Configure the GPU initial plugin according to the functional information to obtain the GPU plugin. Based on the interaction information characteristics between the GPU plugin and the bidirectional long short-term memory network layer, configure the GPU plugin interface for the GPU plugin.

[0052] The CTC decoder's decoding process includes three steps: calculating ArgMax, deduplication, and whitespace removal. Calculating ArgMax is used to calculate the index of the maximum value in the received input sequence, while deduplication and whitespace removal are data compaction operations performed on the variable-length sequence.

[0053] First, a GPU initial plugin can be generated. The GPU initial plugin can be a CUDA Kernel, the smallest execution unit running on the GPU. Then, a plugin interface can be configured for the CUDA Kernel. The plugin interface can be CTCDecoderPlugin. CTCDecoderPlugin is a plugin layer provided by TensorRT that can move the CTC decoding process to the GPU for execution. CTCDecoderPlugin can receive the logits tensor [B, T, C] and sequence length tensor [B] output by the BiLSTM layer and directly output the final decoding result on the GPU.

[0054] Then, the functionality of the CTC decoder is encapsulated in the CUDA Kernel. First, the CUDA Kernel is configured with a maximum index lookup function and shared memory. Each thread block of the CUDA Kernel processes a variable-length sequence of inputs to a BiLSTM layer, then computes ArgMax in parallel through the maximum index lookup function, and writes ArgMax to shared memory.

[0055] Finally, deduplication and parallel whitespace removal are configured in shared memory. The deduplication function performs adjacent character indexing and atomic addition operations on the variable-length sequence in shared memory to achieve parallel deduplication of the variable-length sequence. Then, parallel whitespace removal is performed on the deduplicated result, and the obtained non-whitespace characters are written to the final global memory output buffer through atomic addition operations. The final output of the CUDA Kernel is a compact, variable-length result tensor.

[0056] This embodiment encapsulates the complete CTC decoding algorithm into a GPU plugin, enabling the entire CTC decoding process to be executed on the GPU. This effectively eliminates the CPU performance bottleneck, significantly reduces the latency of end-to-end large-volume text sequence processing, and significantly improves the system throughput.

[0057] To perform the corresponding steps in the above embodiments and various possible methods, an implementation of a text recognition model optimization device is given below. Please refer to [link / reference needed]. Figure 4 , Figure 4 This is a functional block diagram of a text recognition model optimization device provided in an embodiment of the present invention. It should be noted that the basic principle and technical effects of the text recognition model optimization device provided in this embodiment are the same as those in the above embodiments. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the above embodiments. The text recognition model optimization device 100 includes: The node pruning module 200 is used to obtain the inference nodes of the text recognition model from the configuration file of the text recognition model, and perform node pruning on the text recognition model according to the inference nodes to obtain the inference model. The fusion optimization module 300 is used to obtain the structural information of the coding layer of the inference model, and to perform fusion optimization on the coding layer based on the structural information to obtain an optimized coding layer; The interface optimization module 400 is used to optimize the input interface of the bidirectional long short-term memory network layer of the inference model to obtain an optimized interface, and connect the bidirectional long short-term memory network layer with the optimized coding layer through the optimized interface. The plugin configuration module 500 is used to obtain the functional information of the decoding layer of the inference model, generate a GPU plugin and a GPU plugin interface based on the functional information, and connect the bidirectional long short-term memory network layer with the GPU plugin through the GPU plugin interface to obtain an optimized recognition model.

[0058] Optionally, the above modules can be stored in the form of software or firmware. Figure 1 The memory shown may be stored in or embedded in the operating system (OS) of the character recognition model optimization device, and may be generated by... Figure 1The processor executes the commands. Meanwhile, the data and program code required to execute these modules can be stored in memory.

[0059] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0060] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0061] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0062] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for optimizing a text recognition model, characterized in that, include: The inference nodes of the character recognition model are obtained from the configuration file of the character recognition model, and the inference model is obtained by pruning the nodes of the character recognition model according to the inference nodes. Obtain the structural information of the coding layer of the inference model, and perform fusion optimization on the coding layer based on the structural information to obtain an optimized coding layer; The input interface of the bidirectional long short-term memory network layer of the inference model is optimized to obtain an optimized interface, including: replacing the input interface of the bidirectional long short-term memory network layer with a variable-length input interface; configuring dynamic input function for the variable-length input interface to obtain the optimized interface, so that the time dimension of the input information of the variable-length input interface becomes dynamic; and connecting the bidirectional long short-term memory network layer and the optimized coding layer through the optimized interface. The functional information of the decoding layer of the inference model is obtained, and a GPU plugin and a GPU plugin interface are generated based on the functional information. The bidirectional long short-term memory network layer is connected to the GPU plugin through the GPU plugin interface to obtain an optimized recognition model. 2.The method of claim 1, wherein, The step of obtaining the functional information of the decoding layer of the inference model and generating the GPU plugin and GPU plugin interface based on the functional information includes: Generate a GPU initial plugin and obtain the functional information of the decoding layer on the CPU. Configure the GPU initial plugin according to the functional information to obtain the GPU plugin. Based on the interaction information characteristics between the GPU plugin and the bidirectional long short-term memory network layer, configure the GPU plugin interface for the GPU plugin. 3.The method of claim 2, wherein, The step of configuring the GPU initial plugin according to the functional information to obtain the GPU plugin includes: Configure the GPU initial plugin with a maximum value index lookup function and shared memory, so that the GPU plugin can determine the maximum value index corresponding to the received information sequence and store the maximum value index in the shared memory; The GPU plugin is obtained by configuring parallel deduplication and parallel blank removal functions in the shared memory.

4. The method of claim 3, wherein, The steps of configuring parallel deduplication and parallel blank removal functions in the shared memory include: Configure adjacent character index comparison and atomic addition functions in the shared memory so that the GPU plugin can perform parallel deduplication of the information sequence based on the maximum value index; A parallel whitespace removal function is configured in the shared memory so that the GPU plugin can remove whitespace from the parallel deduplication information sequence to obtain the output result corresponding to the information sequence.

5. The text recognition model optimization method according to claim 1, characterized in that, The step of obtaining the structural information of the encoding layer of the inference model, and fusing and optimizing the encoding layer of the inference model based on the structural information to obtain an optimized encoding layer includes: Obtain the structural and operational information corresponding to the depthwise and pointwise convolutional layers of the coding layer; The structural information and the operational information are analyzed to obtain the analysis results, and the depthwise convolutional layer and the pointwise convolutional layer are fused together based on the analysis results to obtain the optimized coding layer.

6. The text recognition model optimization method according to any one of claims 1-5, characterized in that, The step of pruning the text recognition model based on the inference nodes to obtain the inference model includes: Obtain the starting node, the ending node, and the nodes to be pruned between the starting node and the ending node from the inference nodes; Starting from the final node, perform a reverse traversal of all nodes, and keep the nodes reachable from the final node as a node retention set. Starting from the node to be pruned, traverse forward and take the reachable nodes of the node to be pruned as the node to be pruned set; The node pruning set is determined based on the node retention set and the node pruning set, and the inference model is obtained by pruning the nodes of the text recognition model based on the node pruning set.

7. A text recognition model optimization device, characterized in that, include: The node pruning module is used to obtain the inference nodes of the text recognition model from the configuration file of the text recognition model, and to prune the text recognition model according to the inference nodes to obtain the inference model. The fusion optimization module is used to obtain the structural information of the coding layer of the inference model, and to perform fusion optimization on the coding layer based on the structural information to obtain an optimized coding layer; An interface optimization module is used to optimize the input interface of the bidirectional long short-term memory network layer of the inference model to obtain an optimized interface, including: replacing the input interface of the bidirectional long short-term memory network layer with a variable-length input interface; configuring a dynamic input function for the variable-length input interface to obtain the optimized interface, so that the time dimension of the input information of the variable-length input interface becomes dynamic; and connecting the bidirectional long short-term memory network layer and the optimized coding layer through the optimized interface. The plugin configuration module is used to obtain the functional information of the decoding layer of the inference model, generate a GPU plugin and a GPU plugin interface based on the functional information, and connect the bidirectional long short-term memory network layer with the GPU plugin through the GPU plugin interface to obtain an optimized recognition model.

8. A text recognition model optimization device, characterized in that, It includes a processor and a memory, the memory storing machine-executable instructions that can be executed by the processor, the processor executing the machine-executable instructions to implement the character recognition model optimization method according to any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the character recognition model optimization method as described in any one of claims 1-6.

Citation Information

Patent Citations

  • Text recognition method and system

    CN113283336A