Offline OCR (Optical Character Recognition) method and device, computer equipment and storage medium

Abstract

The invention relates to an offline OCR (Optical Character Recognition) method and device, computer equipment and a storage medium. The method comprises the following steps: when an instruction of starting off-line OCR identification is received, calling an identification engine to judge whether the mobile terminal has the authority of starting off-line OCR identification, and obtaining a to-be-identified image related to registration authentication in response to the authority of starting off-line OCR identification; calling an off-line OCR recognition model loaded to the mobile terminal, and performing image recognition and character recognition on the to-be-recognized image; wherein the offline OCR model is a model which is obtained by compressing and quantifying a trained OCR model which is trained on a machine learning platform in advance and then integrating the OCR model to the mobile terminal; and in response to successful image recognition and character recognition, performing local format processing on the image recognition and character recognition results to obtain an offline OCR recognition result. By adopting the method, the OCR recognition efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of optical character recognition, and in particular to an offline OCR recognition method, device, computer equipment and storage medium. Background Art

[0002] With the development of optical character recognition technology, OCR (Optical character recognition) recognition technology has emerged, that is, technology that can convert text in images or scans into editable text.

[0003] For example, in the application scenario of online car-hailing, users are usually required to upload relevant qualification information such as ID card, vehicle registration certificate, and driver's license during the registration and authentication process of online car-hailing. Since there are too many materials to fill in and the process is too cumbersome, if users are required to fill in the information manually, first of all, the user experience is not friendly; secondly, there are probabilistic errors in manual filling, and users may enter wrong information, such as spelling errors, number errors, etc., which will cause problems in subsequent processes; thirdly, users may miss filling in, which may cause subsequent business processes to fail to proceed normally or be delayed. Therefore, using OCR technology for automatic recognition and verification can improve the accuracy and efficiency of information entry.

[0004] However, in the traditional method, online ride-hailing uses online OCR recognition technology, which needs to communicate with the server and is therefore highly dependent on a stable network connection. In the case of an unstable network or no network connection, OCR recognition cannot be performed, affecting the user experience. Online OCR recognition technology requires that images uploaded by users be sent to the server for processing, which may involve the user's privacy information and pose a risk of data leakage or abuse. In addition, online OCR technology services will set quotas or restrictions for each user, such as the maximum number of images recognized per day, the size of recognized images, etc. Exceeding these limits may require additional fees or may not be recognized, all of which will lead to reduced OCR recognition efficiency. Summary of the invention

[0005] Based on this, it is necessary to provide an offline OCR recognition method, device, computer equipment and storage medium that can improve OCR recognition efficiency in response to the above technical problems.

[0006] In a first aspect, an offline OCR recognition method is provided, the method comprising:

[0007] Upon receiving an instruction to enable offline OCR recognition, calling a recognition engine to determine whether the mobile terminal has permission to start offline OCR recognition, and acquiring an image to be recognized related to registration authentication in response to having permission to start offline OCR recognition;

[0008] Calling the offline OCR recognition model loaded into the mobile terminal to perform image recognition and text recognition on the image to be recognized; wherein the offline OCR recognition model is a model integrated into the mobile terminal after compressing and quantizing the trained OCR recognition model pre-trained on the machine learning platform;

[0009] In response to the success of the image recognition and the text recognition, the results of the image recognition and the text recognition are processed in a local format to obtain an offline OCR recognition result.

[0010] In some embodiments, the training method of the trained OCR recognition model includes:

[0011] Use a lightweight backbone network and combine it with the RNN model to build an OCR recognition model to be trained;

[0012] Obtaining sample data, the sample data includes historical image samples related to registration and authentication, and the historical image samples include text information;

[0013] The sample data is divided into a training set and a validation set according to different partitioning strategies. The OCR recognition model to be trained is trained multiple times according to different partitioning strategies of the training set and the validation set to obtain a trained OCR recognition model.

[0014] In some embodiments, the method further comprises:

[0015] During the training of the OCR recognition model to be trained, the output result of the OCR recognition model to be trained based on the validation set is monitored, and the learning rate of the OCR recognition model to be trained is dynamically adjusted according to the output result.

[0016] In some embodiments, the method further includes: during the training of the OCR recognition model to be trained, monitoring the loss value of the OCR recognition model to be trained based on the validation set, and in response to the loss value of the validation set no longer improving within multiple epochs, stopping the training of the OCR recognition model to be trained to obtain a trained OCR recognition model.

[0017] In some embodiments, a method for compressing a trained OCR recognition model includes:

[0018] Use the trained OCR recognition model as the teacher model to define the corresponding student model for it;

[0019] Define a distillation loss function for the student model and calculate the difference between the output of the teacher model and the output of the student model according to the distillation loss function;

[0020] The structure and capacity of the student model and the distillation temperature parameter of the distillation loss function are adjusted until a distilled student model with a difference within a preset range and a capacity within an expected range is obtained, and the distilled learning model is used as an offline OCR recognition model.

[0021] In some embodiments, a method for quantizing a trained OCR recognition model includes:

[0022] The distilled student model is subjected to Slim quantization.

[0023] In some embodiments, the distilled student model is subjected to Slim quantization, including:

[0024] Call the pruning function, configure the pruning weight threshold of the pruning function through the amount parameter, and perform weight-based pruning on the distilled student model according to the pruning weight threshold;

[0025] The quantization parameters are adjusted according to the dynamic range of the input data to dynamically quantize the distilled student model.

[0026] In a second aspect, an offline OCR recognition device is provided, the device comprising:

[0027] The recognition enabling module is used to, upon receiving an instruction to enable offline OCR recognition, call the recognition engine to determine whether the mobile terminal has the authority to start offline OCR recognition, and obtain the image to be recognized related to the registration authentication in response to having the authority to start offline OCR recognition;

[0028] An offline recognition module, used to call an offline OCR recognition model loaded into the mobile terminal to perform image recognition and text recognition on the image to be recognized; wherein the offline OCR recognition model is a model integrated into the mobile terminal after compressing and quantizing a trained OCR recognition model pre-trained on a machine learning platform;

[0029] The local processing module is used for processing the results of the image recognition and the text recognition in a local format in response to the success of the image recognition and the text recognition to obtain an offline OCR recognition result.

[0030] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method of the first aspect when executing the computer program.

[0031] According to a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method according to the first aspect are implemented.

[0032] The above-mentioned offline OCR recognition method, device, computer equipment and storage medium compress and quantize the trained OCR recognition model and then load it into the system of the mobile terminal in an integrated manner. After starting the offline OCR recognition, the offline OCR recognition model loaded into the mobile terminal is called to recognize the image to be recognized and localized format processing is performed, so as to obtain information adapted for registration and authentication. Since there is no need for network communication with the server, first, the dependence of registration and authentication on a stable network connection can be reduced; second, in the registration and authentication process, such as online car-hailing, the relevant documents involve user privacy data. However, the recognition of this solution is performed offline locally on the mobile terminal, so there is no need to upload the privacy data to the server, avoiding the risk of user privacy data being leaked or abused; third, the pressure on the registration and authentication server is reduced, and the registration and authentication server will not set a quota for the user or limit the number of recognitions, which will lead to the occurrence of unrecognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A diagram showing an application environment of an offline OCR recognition method in some embodiments;

[0034] Figure 2 Schematic diagram of the process of offline OCR recognition method in some embodiments;

[0035] Figure 3 A flowchart of the steps of a training method for a trained OCR recognition model in some embodiments;

[0036] Figure 4 A flowchart diagram of the steps of a method for compressing the trained OCR recognition model in some embodiments;

[0037] Figure 5 A schematic diagram of the architecture of a mobile terminal and a machine learning platform on which the offline OCR recognition method of the present application relies in some application examples;

[0038] Figure 6 It is a structural block diagram of an offline OCR recognition device in some embodiments;

[0039] Figure 7 1 is a diagram of the internal structure of a computer device in some embodiments. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0041] The offline OCR recognition method provided in this application can be applied to Figure 1 In the application environment shown, the mobile terminal 102 communicates with the server 104 of the machine learning platform through the network. The mobile terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, portable wearable devices, and vehicle-mounted terminals, and the server 104 can be implemented as an independent server or a server cluster consisting of multiple servers.

[0042] In some embodiments, Figure 2 As shown, an offline OCR recognition method is provided, which is applied to Figure 1 The mobile terminal in the example is used as an example to illustrate, including the following steps:

[0043] Step S202, when receiving an instruction to enable offline OCR recognition, calling a recognition engine to determine whether the mobile terminal has the authority to start offline OCR recognition, and in response to having the authority to start offline OCR recognition, obtaining an image to be recognized related to registration authentication.

[0044] In this step, based on the client in the mobile terminal, when receiving the instruction to enable offline OCR recognition, the recognition engine can be first called to determine whether the user has enabled the offline OCR recognition permission in the mobile terminal. If so, the offline OCR recognition can be started, and the picture used for online car-hailing registration and authentication can be obtained as the image to be recognized by calling the camera or accessing the album. If not, the user can be prompted to enable the OCR recognition permission.

[0045] Exemplarily, the image to be recognized may include but is not limited to an ID card picture, a driver's license picture, a vehicle registration certificate picture, etc.

[0046] Step 204, calling the offline OCR recognition model loaded into the mobile terminal to perform image recognition and text recognition on the image to be recognized; wherein the offline OCR recognition model is a model integrated into the mobile terminal after compressing and quantizing the trained OCR recognition model pre-trained on the machine learning platform.

[0047] In this step, the mobile terminal can call the offline OCR recognition model loaded into its memory, and perform image recognition and text recognition on the image to be recognized through the offline OCR recognition model. During the recognition process, there is no need to communicate with the server, and there is no need to request the server or send the image to be recognized to the server. Even in the absence of network communication, offline localized image and text recognition can be achieved. Among them, the offline OCR recognition model is a trained OCR recognition model that has been compressed and quantized, and is deployed in the system of the mobile terminal in an integrated manner. After the offline OCR recognition is started, it can be called from the memory of the mobile terminal.

[0048] Step 206 , in response to the image recognition and the text recognition being successful, the results of the image recognition and the text recognition are processed in a local format to obtain an offline OCR recognition result.

[0049] In this step, if image recognition and text recognition can be performed synchronously or asynchronously, for example, image recognition is performed first, and if image recognition is successful, text recognition is performed; or text recognition is performed first, and if text recognition is successful, image recognition is performed, and so on.

[0050] After both the image recognition and the text recognition are successful, the recognition results can be processed in a local format so that the recognition results meet the business requirements. Furthermore, if either the image recognition or the text recognition is unsuccessful, a prompt message can be generated to warn the user of the recognition failure.

[0051] The above-mentioned offline OCR recognition method compresses and quantizes the trained OCR recognition model and then loads it into the system of the mobile terminal in an integrated manner. After starting the offline OCR recognition, the image to be recognized is recognized by calling the offline OCR recognition model loaded into the mobile terminal, and localized format processing is performed, so as to obtain information adapted for registration and authentication. Since there is no need for network communication with the server, firstly, the dependence of registration and authentication on a stable network connection can be reduced; secondly, in the registration and authentication process, for example, of online car-hailing, the relevant documents involve user privacy data. However, the recognition of this solution is performed offline locally on the mobile terminal, so there is no need to upload the privacy data to the server, avoiding the risk of leakage or abuse of user privacy data; thirdly, the pressure on the registration and authentication server is reduced, and the registration and authentication server will not set a quota for the user or limit the number of recognitions, which will lead to the occurrence of unrecognition.

[0052] In some embodiments, the training method of the trained OCR recognition model includes:

[0053] Step S302: Use a lightweight backbone network and combine it with the RNN model to build an OCR recognition model to be trained.

[0054] Specifically, a lightweight backbone network suitable for running on mobile terminal devices can be selected through comparative testing, such as the Mobile Net architecture, and the RNN (Recurrent Neural Network) model can be combined to build the OCR model to be trained. Furthermore, the model building techniques of deep learning can be used to add batch normalization after each convolutional layer to help accelerate training and improve model stability, and the residual network (Res Net) structure can be used to help train deeper networks and avoid gradient disappearance.

[0055] Step S304: Acquire sample data, where the sample data includes historical image samples related to registration and authentication, and the historical image samples include text information.

[0056] In some embodiments, after step S304, the following steps may be further included: preprocessing the acquired sample data. Specifically, image enhancement technology may be used to randomly rotate, translate, scale, shear, add noise, adjust brightness and contrast, etc., to the data set composed of the sample data to increase the diversity of the training sample data. In the data preprocessing stage, filters may also be used to reduce background noise, improve the clarity of characters in the image, unify image size and color channels, and ensure the consistency of input data.

[0057] Step S306: Divide the sample data into a training set and a validation set according to different division strategies. The OCR recognition model to be trained is trained multiple times according to different division strategies of the training set and the validation set to obtain a trained OCR recognition model.

[0058] Specifically, multiple trainings can be performed on different training set and validation set partitioning strategies to optimize and adjust the model training strategy. Based on the results of each training, the model performance can be comprehensively evaluated to reduce the risk of overfitting.

[0059] In some embodiments, during the training of the OCR recognition model to be trained, a GPU hardware accelerator can also be used to accelerate the model training and reasoning process. Hardware devices such as GPU hardware accelerators can greatly improve the computing speed of deep learning models.

[0060] In some embodiments, during the training of the OCR recognition model to be trained, the output result of the OCR recognition model to be trained based on the validation set is monitored, and the learning rate of the OCR recognition model to be trained is dynamically adjusted according to the output result.

[0061] Specifically, you can adjust the model's hyperparameters, such as using a learning rate scheduler (such as CyclicLR) to dynamically adjust the learning rate based on the performance of the validation set. You can gradually increase the learning rate at the beginning of training to prevent training instability. You can also try different batch sizes to observe the impact on model performance, and use L2 regularization to prevent overfitting and improve generalization.

[0062] In some embodiments, during the training of the OCR recognition model to be trained, the loss value of the OCR recognition model to be trained based on the validation set is monitored. In response to the loss value of the validation set no longer improving within multiple epochs, the training of the OCR recognition model to be trained is stopped to obtain a trained OCR recognition model.

[0063] Specifically, an early stopping strategy can be formulated to monitor the loss on the validation set. When the loss of the validation set no longer improves within multiple epochs, training can be stopped in time to prevent overfitting, and oversampling and undersampling strategies can be used to balance the category distribution.

[0064] In some embodiments, during the training of the OCR recognition model to be trained, a beam search algorithm can be used in the output stage of the model to enable the model to select multiple optimal character sequences as the result for output, rather than just the most likely sequence, thereby improving decoding accuracy. A dictionary language model can also be used to correct the recognition results to ensure that the generated text conforms to grammar and context.

[0065] In some embodiments, after the training of the OCR recognition model to be trained is completed, the model can be evaluated and iterated. By generating a classification report, the accuracy of each character or category output by the model can be analyzed to find out the weaknesses of the model. The processing of error samples by the visualization model can be realized to help understand the limitations of the model. According to the results of the error analysis, the data set, model architecture or training strategy of the sample data can be adjusted to perform iterative model training, thereby improving the accuracy of model recognition.

[0066] refer to Figure 4 As shown, in some embodiments, a method for compressing a trained OCR recognition model includes:

[0067] Step S402: Using the trained OCR recognition model as a teacher model to define a corresponding student model for it.

[0068] Specifically, the size of the trained OCR recognition model is relatively large, for example, it can reach 92M, which is very large for mobile terminals and is not suitable for deployment in mobile terminals. Therefore, the model needs to be compressed without significantly reducing performance.

[0069] For example, the trained OCR recognition model can be compressed based on the model distillation technology, which achieves the purpose of model compression by transferring the knowledge of a large teacher model to a small student model.

[0070] The trained OCR recognition model is used as the teacher model, and a smaller student model is defined for it. The student model can remove unnecessary parameters and has a simpler structural skeleton for faster reasoning.

[0071] Step S404: define a distillation loss function for the student model, and calculate the difference between the output of the teacher model and the output of the student model according to the distillation loss function.

[0072] Specifically, after defining the student model, a distillation loss function is defined for it. The distillation loss function is mainly used to calculate the difference between the output of the student model and the output of the teacher model. For example, the loss functions that can be used include but are not limited to the KL divergence (Kullback-Leibler Divergence) loss function. It is found through experiments that the optimal distillation temperature of the distillation loss function can be set to 3.

[0073] Step S406: Adjust the structure and capacity of the student model and the distillation temperature parameter of the distillation loss function until a distilled student model is obtained whose difference is within a preset range and whose capacity is within an expected range, and use the distilled learning model as an offline OCR recognition model.

[0074] Specifically, after multiple training of student models with different structures, that is, multiple adjustments to the structure and capacity of the student model, we finally find the student model that best balances the compression effect and output performance. Finally, we transfer the knowledge of the teacher model to the student model to obtain a smaller and better-performing distilled model, which can be used as an offline OCR recognition model. The size of the distilled model is reduced to 4.9M.

[0075] In some embodiments, in order to further reduce the size of the distilled model, the distilled student model can be Slim quantized, thereby further reducing the model size and improving the inference speed.

[0076] In some embodiments, the distilled student model is subjected to Slim quantization, including:

[0077] Step S408: calling the pruning function, configuring the pruning weight threshold of the pruning function through the amount parameter, and performing weight-based pruning processing on the distilled student model according to the pruning weight threshold.

[0078] Specifically, sparsity (pruning) can be defined, that is, sparsity can be introduced through pruning. For example, using the prune tool provided by PyTorch, the pruning weight threshold is set to 0.3 through the amount parameter, that is, 30% of the weight is pruned. After pruning, the parameters of the model can also be fine-tuned to restore possible performance losses.

[0079] Step S410: adjusting the quantization parameter according to the dynamic range of the input data to perform dynamic quantization processing on the distilled student model.

[0080] Specifically, the quantization tool provided by PyTorch can be used to perform dynamic quantization processing, and finally the quantized model can be exported. The final model size can be reduced to 1.1M. Through testing, it is found that the efficiency is significantly improved while the volume is greatly reduced, and the accuracy and performance are not significantly reduced. It is very suitable for deployment as an offline OCR recognition model on mobile terminals.

[0081] Below, you can refer to Figure 5 As shown, Figure 5 The figure shows a schematic diagram of the architecture of the mobile terminal and the machine learning platform on which the offline OCR recognition method of the present application relies in some application examples.

[0082] It should be understood that although Figures 2 to 4 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figures 2 to 4 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0083] In some embodiments, Figure 6 As shown, an offline OCR recognition device is provided, including: a recognition enabling module 610, an offline recognition module 620 and a local processing module 630, wherein:

[0084] The recognition enabling module 610 is used to call the recognition engine to determine whether the mobile terminal has the authority to start the offline OCR recognition when receiving the instruction to start the offline OCR recognition, and obtain the image to be recognized related to the registration authentication in response to having the authority to start the offline OCR recognition;

[0085] The offline recognition module 620 is used to call the offline OCR recognition model loaded into the mobile terminal to perform image recognition and text recognition on the image to be recognized; wherein the offline OCR recognition model is a model integrated into the mobile terminal after compressing and quantizing the trained OCR recognition model pre-trained on the machine learning platform;

[0086] The local processing module 630 is used to process the results of the image recognition and the text recognition in a local format in response to the success of the image recognition and the text recognition to obtain an offline OCR recognition result.

[0087] In some embodiments, the offline recognition module 620 is also used to use a lightweight backbone network and combine with the RNN model to construct an OCR recognition model to be trained; obtain sample data, the sample data includes historical image samples related to registration and authentication, and the historical image samples include text information; divide the sample data into training sets and verification sets according to different division strategies, and use different training set and verification set division strategies to train the OCR recognition model to be trained multiple times to obtain a trained OCR recognition model.

[0088] In some embodiments, the offline recognition module 620 is further used to monitor the output result of the OCR recognition model to be trained based on the validation set during the training of the OCR recognition model to be trained, and dynamically adjust the learning rate of the OCR recognition model to be trained according to the output result.

[0089] In some embodiments, the offline recognition module 620 is also used to monitor the loss value of the OCR recognition model to be trained based on the verification set during the training of the OCR recognition model to be trained. In response to the loss value of the verification set no longer improving within multiple epochs, the training of the OCR recognition model to be trained is stopped to obtain a trained OCR recognition model.

[0090] In some embodiments, the offline recognition module 620 is also used to define a corresponding student model for the trained OCR recognition model as a teacher model; define a distillation loss function for the student model, and calculate the difference between the output of the teacher model and the output of the student model according to the distillation loss function; adjust the structure and capacity of the student model and the distillation temperature parameter of the distillation loss function until a distilled student model with a difference within a preset range and a capacity within an expected range is obtained, and use the distilled learning model as an offline OCR recognition model.

[0091] In some embodiments, the offline recognition module 620 is further used to perform Slim quantization on the distilled student model.

[0092] In some embodiments, the offline recognition module 620 is also used to call a pruning function, configure a pruning weight threshold of the pruning function through an amount parameter, and perform weight-based pruning on the distilled student model according to the pruning weight threshold; and adjust the quantization parameter according to the dynamic range of the input data to perform dynamic quantization on the distilled student model.

[0093] For the specific definition of the offline OCR recognition device, please refer to the definition of the offline OCR recognition method above, which will not be repeated here. Each module in the above-mentioned offline OCR recognition device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0094] In some embodiments, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an offline OCR recognition method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0095] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0096] In some embodiments, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the following steps are implemented when the processor executes the computer program: upon receiving an instruction to enable offline OCR recognition, calling a recognition engine to determine whether a mobile terminal has permission to start offline OCR recognition, and obtaining an image to be recognized related to registration authentication in response to having permission to start offline OCR recognition; calling an offline OCR recognition model loaded into the mobile terminal to perform image recognition and text recognition on the image to be recognized; wherein the offline OCR recognition model is a model that is integrated into the mobile terminal after compressing and quantizing a trained OCR recognition model pre-trained on a machine learning platform; in response to successful image recognition and text recognition, processing the results of the image recognition and text recognition in a local format to obtain an offline OCR recognition result.

[0097] In some embodiments, when the processor executes the computer program, the following steps are also implemented: using a lightweight backbone network and combining with an RNN model to construct an OCR recognition model to be trained; obtaining sample data, the sample data including historical image samples related to registration and authentication, the historical image samples including text information; dividing the sample data into training sets and verification sets according to different division strategies, and training the OCR recognition model to be trained multiple times according to different training set and verification set division strategies to obtain a trained OCR recognition model.

[0098] In some embodiments, when the processor executes the computer program, the following steps are also implemented: during the training of the OCR recognition model to be trained, the output results of the OCR recognition model to be trained based on the verification set are monitored, and the learning rate of the OCR recognition model to be trained is dynamically adjusted according to the output results.

[0099] In some embodiments, when the processor executes the computer program, the following steps are also implemented: during the training of the OCR recognition model to be trained, the loss value of the OCR recognition model to be trained based on the verification set is monitored, and in response to the loss value of the verification set no longer improving within multiple epochs, the training of the OCR recognition model to be trained is stopped to obtain a trained OCR recognition model.

[0100] In some embodiments, the processor further implements the following steps when executing the computer program: using the trained OCR recognition model as a teacher model to define a corresponding student model for it; defining a distillation loss function for the student model, and calculating the difference between the output of the teacher model and the output of the student model according to the distillation loss function; adjusting the structure and capacity of the student model and the distillation temperature parameter of the distillation loss function until a distilled student model is obtained whose difference is within a preset range and whose capacity is within an expected range, and using the distilled learning model as an offline OCR recognition model.

[0101] In some embodiments, when the processor executes the computer program, the processor further implements the following steps: performing Slim quantization on the distilled student model.

[0102] In some embodiments, when the processor executes the computer program, it also implements the following steps: calling the pruning function, configuring the pruning weight threshold of the pruning function through the amount parameter, and performing weight-based pruning on the distilled student model according to the pruning weight threshold; adjusting the quantization parameter according to the dynamic range of the input data to dynamically quantize the distilled student model.

[0103] In some embodiments, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented: when an instruction to enable offline OCR recognition is received, a recognition engine is called to determine whether the mobile terminal has the authority to start offline OCR recognition, and in response to having the authority to start offline OCR recognition, an image to be recognized related to registration and authentication is obtained; an offline OCR recognition model loaded into the mobile terminal is called to perform image recognition and text recognition on the image to be recognized; wherein the offline OCR recognition model is a model that is integrated into the mobile terminal after compressing and quantizing a trained OCR recognition model pre-trained on a machine learning platform; in response to successful image recognition and text recognition, the results of the image recognition and text recognition are processed in a local format to obtain an offline OCR recognition result.

[0104] In some embodiments, when the computer program is executed by the processor, the following steps are also implemented: using a lightweight backbone network and combining with an RNN model to construct an OCR recognition model to be trained; obtaining sample data, the sample data including historical image samples related to registration and authentication, the historical image samples including text information; dividing the sample data into training sets and verification sets according to different division strategies, and training the OCR recognition model to be trained multiple times according to different training set and verification set division strategies to obtain a trained OCR recognition model.

[0105] In some embodiments, when the computer program is executed by the processor, the following steps are also implemented: during the training of the OCR recognition model to be trained, the output results of the OCR recognition model to be trained based on the validation set are monitored, and the learning rate of the OCR recognition model to be trained is dynamically adjusted according to the output results.

[0106] In some embodiments, when the computer program is executed by the processor, the following steps are also implemented: during the training of the OCR recognition model to be trained, the loss value of the OCR recognition model to be trained based on the verification set is monitored, and in response to the loss value of the verification set no longer improving within multiple epochs, the training of the OCR recognition model to be trained is stopped to obtain a trained OCR recognition model.

[0107] In some embodiments, when the computer program is executed by the processor, the following steps are also implemented: using the trained OCR recognition model as a teacher model to define a corresponding student model for it; defining a distillation loss function for the student model, and calculating the difference between the output of the teacher model and the output of the student model according to the distillation loss function; adjusting the structure and capacity of the student model and the distillation temperature parameter of the distillation loss function until a distilled student model is obtained whose difference is within a preset range and whose capacity is within an expected range, and using the distilled learning model as an offline OCR recognition model.

[0108] In some embodiments, when the computer program is executed by a processor, the following steps are also implemented: performing Slim quantization on the distilled student model.

[0109] In some embodiments, when the computer program is executed by the processor, the following steps are also implemented: calling the pruning function, configuring the pruning weight threshold of the pruning function through the amount parameter, and performing weight-based pruning on the distilled student model according to the pruning weight threshold; adjusting the quantization parameter according to the dynamic range of the input data to dynamically quantize the distilled student model.

[0110] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0111] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0112] In addition, the term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the characters in this article generally indicate that the associated objects before and after are in an "or" relationship.

[0113] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

[0114] It should be noted that in the embodiments of the present application, data related to user information or user data (for example, images to be identified) must be obtained and processed with the authorization and consent of the user. When the embodiments of the present application are applied to specific products or technologies, the user's permission or consent must be obtained, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

Claims

1. An offline OCR recognition method, applied to a mobile terminal, comprising: Upon receiving an instruction to enable offline OCR recognition, calling a recognition engine to determine whether the mobile terminal has permission to enable offline OCR recognition, and acquiring an image to be recognized related to registration authentication in response to having permission to enable offline OCR recognition; Calling an offline OCR recognition model loaded into the mobile terminal to perform image recognition and text recognition on the image to be recognized; wherein the offline OCR recognition model is a model integrated into the mobile terminal after compressing and quantizing a trained OCR recognition model pre-trained on a machine learning platform; In response to the image recognition and the text recognition being successful, the results of the image recognition and the text recognition are processed in a local format to obtain an offline OCR recognition result.

2. The method according to claim 1, characterized in that The training method of the trained OCR recognition model includes: Use a lightweight backbone network and combine it with the RNN model to build an OCR recognition model to be trained; Acquire sample data, wherein the sample data includes historical image samples related to registration and authentication, and the historical image samples include text information; The sample data is divided into a training set and a validation set according to different division strategies. The OCR recognition model to be trained is trained multiple times according to different division strategies of the training set and the validation set to obtain the trained OCR recognition model.

3. The method according to claim 2, characterized in that The method further comprises: During the training of the OCR recognition model to be trained, the output result of the OCR recognition model to be trained based on the verification set is monitored, and the learning rate of the OCR recognition model to be trained is dynamically adjusted according to the output result.

4. The method according to claim 2, characterized in that: The method also includes: during the training of the OCR recognition model to be trained, monitoring the loss value of the OCR recognition model to be trained based on the verification set, and in response to the loss value of the verification set no longer improving within multiple epochs, stopping the training of the OCR recognition model to be trained to obtain the trained OCR recognition model.

5. The method according to claim 1, characterized in that The method for compressing the trained OCR recognition model comprises: Using the trained OCR recognition model as a teacher model, defining a corresponding student model for it; defining a distillation loss function for the student model, and calculating the difference between the output of the teacher model and the output of the student model according to the distillation loss function; The structure and capacity of the student model and the distillation temperature parameter of the distillation loss function are adjusted until a distilled student model is obtained in which the difference is within a preset range and the capacity is within an expected range, and the distilled learning model is used as the offline OCR recognition model.

6. The method according to claim 5, characterized in that The method for quantizing the trained OCR recognition model includes: The distilled student model is subjected to Slim quantization.

7. The method according to claim 6, characterized in that The step of performing Slim quantization on the distilled student model comprises: Calling a pruning function, configuring a pruning weight threshold of the pruning function through an amount parameter, and performing weight-based pruning processing on the distilled student model according to the pruning weight threshold; A quantization parameter is adjusted according to a dynamic range of input data to dynamically quantize the distilled student model.

8. An offline OCR recognition device, characterized in that: The device comprises: A recognition enabling module, configured to, upon receiving an instruction to enable offline OCR recognition, call a recognition engine to determine whether the mobile terminal has permission to start offline OCR recognition, and obtain an image to be recognized related to registration authentication in response to having permission to start offline OCR recognition; An offline recognition module, used to call an offline OCR recognition model loaded into the mobile terminal to perform image recognition and text recognition on the image to be recognized; wherein the offline OCR recognition model is a model integrated into the mobile terminal after compressing and quantizing a trained OCR recognition model pre-trained on a machine learning platform; The local processing module is used for processing the results of the image recognition and the text recognition in a local format in response to the success of the image recognition and the text recognition to obtain an offline OCR recognition result.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images