Model training method, text detection method, terminal equipment and storage medium
Through the model training method based on text detection model, the model performance is gradually improved by using pseudo-labeled labels and real-labeled data, solving the problem of text detection algorithm dependence on labeled data, and achieving efficient model training and performance improvement.
Patent Information
- Application Number
- CN202310945255.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-07-28
- Publication Date
- 2025-02-14
AI Technical Summary
Text detection algorithms based on deep learning require a large amount of labeling data for training, but the data for labeling text detection requires higher labeling costs and consumes a lot of time and manpower.
A model training method based on text detection model is proposed. By obtaining the initial text detection model, using the training image to generate pseudo-notation labels, and training the model through pseudo-notation labels, generating medium-quality and high-quality training data, gradually improving the model performance.
By generating large batches of text detection pseudo-label data, used for pre-training iteration of the model, the detection performance of the model is significantly improved and the dependence on real labeled data is reduced.
Smart Images
Figure CN119445594A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of text analysis, and in particular to a model training method based on a text detection model, a text detection method, a terminal device, and a computer storage medium. Background Art
[0002] With the development of deep learning technology, text detection based on deep learning models has been widely used in industry and academia, such as instant translation of tourism, digitization of paper documents, sign recognition, image text review, etc.
[0003] The text detection algorithm based on deep learning requires a large amount of labeled data for training. However, compared with the image classification task, the labeled data for text detection requires a higher labeling cost, which consumes a lot of time and manpower. Summary of the invention
[0004] In order to solve the above technical problems, the present application proposes a model training method, a text detection method, a terminal device and a computer storage medium based on a text detection model.
[0005] In order to solve the above technical problems, the present application proposes a model training method based on a text detection model, and the model training method includes:
[0006] Obtaining an initialization text detection model, wherein the initialization text detection model is obtained by training with real annotation labels;
[0007] Inputting the training image into the initialized text detection model to obtain the pseudo-annotated label output by the initialized text detection model;
[0008] The initialized text detection model is trained using the pseudo-annotated labels to obtain the text detection model.
[0009] After obtaining the pseudo-annotation label output by the initialization text detection model, the model training method further includes:
[0010] Performing text recognition on the text image of the pseudo-labeled label to obtain a text recognition result;
[0011] Based on the relationship between the number of text characters and the text size of the text recognition result, the pseudo-annotation labels that do not meet the text filtering conditions are deleted.
[0012] Wherein, the text size is the ratio of the length to the width of the text line;
[0013] The pseudo-annotation labels that do not meet the text filtering conditions are deleted based on the relationship between the number of text characters and the text size of the text recognition result;
[0014] Setting a text filtering upper limit and a text filtering lower limit based on the text line length-width ratio;
[0015] The pseudo-annotation labels whose number of text characters is lower than the text filtering lower limit or higher than the text filtering upper limit are deleted.
[0016] After obtaining the pseudo-annotation label output by the initialization text detection model, the model training method further includes:
[0017] Obtaining the detection confidence of the pseudo-annotated label;
[0018] The pseudo-annotated labels whose detection confidence is lower than a preset threshold are deleted.
[0019] The step of training the initialized text detection model using the pseudo-annotated labels to obtain the text detection model includes:
[0020] Using the pseudo-labeled labels, generating medium-quality training data;
[0021] Training the initialization text detection model based on the medium-quality training data to obtain a first intermediate text detection model;
[0022] Using the real annotation labels, generating high-quality training data;
[0023] The first intermediate text detection model is trained using the high-quality training data to obtain the text detection model.
[0024] Before generating medium-quality training data using the pseudo-labeled labels, the model training method further includes:
[0025] Generate low-quality training data using unlabeled tags;
[0026] The initialization text detection model is trained based on the low-quality training data to obtain a second intermediate text detection model.
[0027] After obtaining the pseudo-annotation label output by the initialization text detection model, the model training method further includes:
[0028] Passing the pseudo-annotated labels through a pre-trained image classification model to classify the pseudo-annotated labels into high-quality training data, medium-quality training data and / or low-quality training data;
[0029] The step of training the initialized text detection model using the pseudo-annotated labels to obtain the text detection model includes:
[0030] The initialization text detection model is trained in sequence from low to high quality using the high-quality training data, the medium-quality training data and / or the low-quality training data to obtain the text detection model.
[0031] Wherein, the model training method further includes:
[0032] The image classification model is trained using the real annotated labels, the pseudo annotated labels, and the unannotated labels.
[0033] In order to solve the above technical problems, the present application also proposes a text detection method, which includes:
[0034] Get the text image to be detected;
[0035] Inputting the text image to be detected into a pre-trained text detection model;
[0036] Based on the output of the text detection model, obtaining text information in the text image to be detected;
[0037] Wherein, the text detection model is trained by the above-mentioned model training method.
[0038] In order to solve the above-mentioned technical problems, the present application also proposes a terminal device, which includes a memory and a processor coupled to the memory; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the model training method based on the text detection model as mentioned above, and / or the text detection method.
[0039] In order to solve the above technical problems, the present application also proposes a computer storage medium, which is used to store program data. When the program data is executed by a computer, it is used to implement the above-mentioned model training method based on text detection model, and / or text detection method.
[0040] Compared with the prior art, the beneficial effects of the present application are: the terminal device obtains an initialization text detection model, wherein the initialization text detection model is obtained by training with real annotation labels; the training image is input into the initialization text detection model to obtain the pseudo annotation labels output by the initialization text detection model; the initialization text detection model is trained with the pseudo annotation labels to obtain the text detection model. Through the above-mentioned model training method, a large amount of pseudo-label data for text detection can be generated for the pre-training iteration of the model, which can greatly improve the detection performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0042] in:
[0043] Figure 1 It is a flowchart of an embodiment of a model training method based on a text detection model provided by the present application;
[0044] Figure 2 It is a schematic diagram of the overall process of the model training method based on the text detection model provided in this application;
[0045] Figure 3 It is a flow chart of an embodiment of a pseudo-label filtering method provided by the present application;
[0046] Figure 4 It is a schematic diagram of the workflow of the pseudo-label filtering module provided by this application;
[0047] Figure 5 yes Figure 1 The specific flow chart of step S13 of the model training method based on the text detection model is shown;
[0048] Figure 6 It is a schematic diagram of the workflow of the pseudo-label quality classification module provided in this application;
[0049] Figure 7 is a schematic diagram of a high-quality pseudo-label image provided by this application;
[0050] Figure 8 is a schematic diagram of a medium-quality pseudo-label image provided by this application;
[0051] Fig. 9 is a schematic diagram of a low-quality pseudo-label image provided by this application;
[0052] Fig.10 It is a flowchart of an embodiment of a text detection method provided by the present application;
[0053] Fig.11 It is a structural schematic diagram of an embodiment of a terminal device provided by the present application;
[0054] Fig.12 It is a structural diagram of an embodiment of a computer storage medium provided by the present application. DETAILED DESCRIPTION
[0055] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0056] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0057] This application mainly divides the video frame image into multiple blocks for processing, and increases the difference in the average brightness value of the previous and next frames of the video in the judgment dimension. This is mainly done to determine whether the brightness value of the pixel in the image block has changed. At the same time, a judgment dimension binary directional response value is defined, and its calculation method is based on the Kirsch edge response value calculation method, so that the changes in edge information in various directions in the image can be taken into account. Then, the difference in the directional response value of the previous and next frames of the video is judged. When the difference value reaches a certain threshold, it is determined that the block is in switching. Finally, the proportion of the blocks that are determined to be changing in the total blocks is used to determine whether this frame of the video has a scene change.
[0058] Please refer to Figure 1 and Figure 2 , Figure 1 is a flowchart of an embodiment of a model training method based on a text detection model provided in this application, Figure 2 It is a schematic diagram of the overall process of the model training method based on the text detection model provided in this application.
[0059] The model training method of the present application obtains the model through training with an initial batch of manually annotated data, and then uses the model to generate pseudo labels, and then uses the pseudo labels to pre-train the model, and finally uses artificial real data for further training. In this cycle, better and better pseudo-label data and model results can be obtained.
[0060] The model training method based on the text detection model of the present application is applied to a terminal device, wherein the terminal device of the present application can be a server, a local terminal, or a system in which a server and a local terminal cooperate with each other. Accordingly, the various parts included in the terminal device, such as various units, subunits, modules, and submodules, can all be set in the server, can all be set in the local terminal, or can be set in the server and the local terminal respectively.
[0061] Furthermore, the above-mentioned server can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or it can be implemented as a single server. When the server is software, it can be implemented as multiple software or software modules, such as software or software modules used to provide distributed servers, or it can be implemented as a single software or software module, which is not specifically limited here.
[0062] like Figure 1 As shown, the specific steps are as follows:
[0063] Step S11: Obtain an initialized text detection model, wherein the initialized text detection model is obtained by training with real annotated labels.
[0064] In the embodiments of the present application, Figure 2 As shown, the terminal device obtains the initial manually annotated data, and uses the initial manually annotated data to train the text detection model to obtain the initialized text detection model. Among them, real manually annotated data is indispensable, and a batch of starting, real manually annotated data, such as 1,000 pieces, is required.
[0065] Specifically, after obtaining the real annotated data, the terminal device uses a text detection model such as a PAN model for iteration, and after training to convergence, an initialized text detection model can be obtained.
[0066] Step S12: input the training image into the initialized text detection model to obtain the pseudo-annotated label output by the initialized text detection model.
[0067] In the embodiments of the present application, Figure 2 As shown, the terminal device obtains several training images for training the model by analyzing large-scale business data, inputs these training images into the initialized text detection model, and generates a large number of predicted labels as pseudo-annotation labels.
[0068] It should be noted that the pseudo-label data of the present application includes pseudo-labeled labels and corresponding labeled images.
[0069] At present, the traditional pseudo-label generation technology mainly uses the probability of model prediction for screening. This method is effective to a certain extent, but the solution of simply using threshold filtering will still bring a lot of problematic label data.
[0070] The simple threshold solution will still bring in a lot of noisy data, and although the dynamic threshold solution can be applied to more data, it still does not actually solve the error situation of high-confidence results. For example, the supervised algorithm will confidently give some results, such as 99% confidence, but in fact it is wrong.
[0071] This application combines the characteristics of text detection tasks and proposes a new pseudo-label filtering solution to eliminate problematic data to the greatest extent. Figure 3 and Figure 4 , Figure 3 is a flow chart of an embodiment of a pseudo-label filtering method provided by the present application. Figure 4 It is a schematic diagram of the workflow of the pseudo-label filtering module provided in this application.
[0072] like Figure 3 As shown, the specific steps are as follows:
[0073] Step S21: performing text recognition on the text image with the pseudo-labeled label to obtain a text recognition result.
[0074] In the embodiments of the present application, Figure 4 As shown, the terminal device obtains the confidence of each pseudo-labeled label and filters out pseudo-labeled labels with a confidence lower than 0.95 or other preset thresholds.
[0075] Then, the terminal device uses the text recognition model to perform text recognition on the text images of the remaining pseudo-annotated labels, and obtains the text recognition result corresponding to each pseudo-annotated label.
[0076] Step S22: based on the relationship between the number of text characters and the text size in the text recognition result, the pseudo-annotation labels that do not meet the text filtering conditions are deleted.
[0077] In an embodiment of the present application, after obtaining text recognition, the terminal device will filter the text box based on the relationship between the number of text characters (Num Char) and the length-to-width ratio (Width / Height) of the text line.
[0078] Specifically, first, the terminal device deletes the pseudo-annotation tags whose number of text characters (Num Char) is 0. Then, the terminal device sets the text filtering upper limit 5*(Width / Height) and the text filtering lower limit (Width / Height) / 5 according to the text line length-width ratio (Width / Height). Finally, the terminal device deletes the pseudo-annotation tags whose number of text characters (Num Char) is lower than the text filtering lower limit (Width / Height) / 5, or higher than the text filtering upper limit 5*(Width / Height).
[0079] In other implementations, the parameters for setting the text filtering upper limit and the text filtering lower limit may also adopt other values, and the parameters for setting the text filtering upper limit and the parameters for setting the text filtering lower limit may be the same or different.
[0080] Step S13: Use the pseudo-annotated labels to train the initialized text detection model to obtain a text detection model.
[0081] In an embodiment of the present application, the terminal device uses pseudo-annotated labels as real annotation labels to train the initialized text detection model, and then alternately uses manually annotated labels and pseudo-annotated labels predicted by the text detection model at each stage to train the text detection model until a preset number of iterations is reached or the training loss value reaches the training conditions.
[0082] Furthermore, the present application also provides a three-stage pre-training scheme based on training data of different qualities, which can avoid the impact of erroneous pseudo-labels on the final model to the greatest extent.
[0083] Please refer to Figure 5 , Figure 5 yes Figure 1 The specific flow chart of step S13 of the model training method based on the text detection model is shown.
[0084] like Figure 5 As shown, the specific steps are as follows:
[0085] Step S131: Generate medium-quality training data using pseudo-labeled tags.
[0086] Step S132: Train the initialization text detection model based on medium-quality training data to obtain a first intermediate text detection model.
[0087] In an embodiment of the present application, before training the text detection model using medium-quality training data, the terminal device can also use unlabeled labels to generate low-quality training data, and train the initialized text detection model based on the low-quality training data to obtain a second intermediate text detection model.
[0088] Step S133: Generate high-quality training data using real annotation labels.
[0089] Step S134: Train the first intermediate text detection model using high-quality training data to obtain a text detection model.
[0090] Among them, the training data of the high-quality category is 1000 manually annotated images, the training data of the medium-quality category is the pseudo-label data generated in step 12 (1000 randomly selected), and the low-quality data is unlabeled data (1000 randomly selected from the original business data).
[0091] based on Figure 5 The three-stage pre-training scheme shown, this application proposes a BoxMask pseudo-label image quality classification module for three-stage model pre-training, which can maximize the effect of pseudo-label data.
[0092] Please refer to Figure 6 , Figure 6 It is a schematic diagram of the workflow of the pseudo-label quality classification module provided in this application.
[0093] like Figure 6 As shown, the present application proposes a quality judgment module and an image classifier based on a CNN image classification model, which is used to define pseudo-annotation labels into three categories: high-quality data, medium-quality data, and low-quality data.
[0094] It should be noted that the input image classifier is used for classification, that is, the high-quality and medium-quality training data are the images after using the corresponding Box Mask, rather than the original images. Figures 7 to 9 As shown, Figure 7 is a schematic diagram of a high-quality pseudo-label image provided by this application, Figure 8 is a schematic diagram of a medium-quality pseudo-label image provided by this application, Fig. 9 Schematic diagram of a low-quality pseudo-label image provided in this application.
[0095] The terminal device can also use the real labeled labels, pseudo-labeled labels and unlabeled labels to train the image classification model, so that the image classification model classifies all pseudo-labeled data through a three-classification method.
[0096] After obtaining the category information of all pseudo-label data using the pseudo-label quality classification module, the terminal device divides all pseudo-label data into three categories and performs pre-training at different stages for different category data. The specific process is as follows:
[0097] Low-quality pseudo-label data: The initial model uses low-quality pseudo-label data for rapid iteration and ends after training N epochs.
[0098] Medium-quality pseudo-label data: The model trained on low-quality pseudo-labels continues to use medium-quality pseudo-label data for the second pre-training, and the training ends after M epochs.
[0099] High-quality pseudo-label data: After obtaining the model trained with medium-quality pseudo-labels, use high-quality pseudo-label data for final pre-training, which ends after K epochs.
[0100] In an embodiment of the present application, a terminal device obtains an initialization text detection model, wherein the initialization text detection model is obtained by training with real annotation labels; a training image is input into the initialization text detection model to obtain a pseudo annotation label output by the initialization text detection model; and the initialization text detection model is trained with the pseudo annotation label to obtain the text detection model. Through the above-mentioned model training method, a large amount of pseudo-label data for text detection can be generated for the pre-training iteration of the model, which can greatly improve the detection performance of the model.
[0101] This application proposes a high-quality pseudo-label generation method for text detection. This method can generate a large amount of text detection pseudo-label data for model pre-training iteration, which can greatly improve the detection performance of the model. At the same time, this application proposes a three-stage pre-training scheme based on a pseudo-label quality judgment module, which can avoid the impact of erroneous pseudo-labels on the final model to the greatest extent.
[0102] Please continue reading Fig.10 , Fig.10 It is a flowchart of an embodiment of a text detection method provided by the present application.
[0103] like Fig.10 As shown, the specific steps are as follows:
[0104] Step S31: Acquire the text image to be detected.
[0105] Step S32: input the text image to be detected into a pre-trained text detection model.
[0106] In an embodiment of the present application, the text detection model can be pre-trained through the above-mentioned model training method embodiment, and its training process will not be repeated here.
[0107] Step S33: Based on the output of the text detection model, obtain the text information in the text image to be detected.
[0108] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.
[0109] In order to implement the above-mentioned model training method based on text detection model and / or text detection method, the present application also proposes a terminal device, for details, please refer to Fig.11 , Fig.11 It is a structural diagram of an embodiment of a terminal device provided by the present application.
[0110] The terminal device 400 of this embodiment includes a processor 41 , a memory 42 , an input / output device 43 , and a bus 44 .
[0111] The processor 41, memory 42, and input / output device 43 are respectively connected to a bus 44. The memory 42 stores program data. The processor 41 is used to execute the program data to implement the model training method based on the text detection model and / or the text detection method described in the above embodiment.
[0112] In the embodiment of the present application, the processor 41 may also be referred to as a CPU (Central Processing Unit). The processor 41 may be an integrated circuit chip having the ability to process signals. The processor 41 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or the processor 41 may also be any conventional processor, etc.
[0113] This application also provides a computer storage medium, please continue to refer to Fig.12 , Fig.12 It is a structural diagram of an embodiment of a computer storage medium provided in the present application. The computer storage medium 600 stores a computer program 61. When the computer program 61 is executed by the processor, it is used to implement the model training method based on the text detection model and / or the text detection method of the above-mentioned embodiment.
[0114] When the embodiments of the present application are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.
[0115] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.
Claims
1. A model training method based on a text detection model, characterized in that: The model training method comprises: Obtaining an initialization text detection model, wherein the initialization text detection model is obtained by training with real annotation labels; Inputting the training image into the initialized text detection model to obtain the pseudo-annotated label output by the initialized text detection model; The initialized text detection model is trained using the pseudo-annotated labels to obtain the text detection model.
2. The model training method according to claim 1, characterized in that: After obtaining the pseudo-annotation label output by the initialized text detection model, the model training method further includes: Performing text recognition on the text image of the pseudo-labeled label to obtain a text recognition result; Based on the relationship between the number of text characters and the text size of the text recognition result, the pseudo-annotation labels that do not meet the text filtering conditions are deleted.
3. The model training method according to claim 2, characterized in that: The text size is the ratio of the length to the width of the text line; The pseudo-annotation labels that do not meet the text filtering conditions are deleted based on the relationship between the number of text characters and the text size of the text recognition result; Setting a text filtering upper limit and a text filtering lower limit based on the text line length-width ratio; The pseudo-annotation labels whose number of text characters is lower than the text filtering lower limit or higher than the text filtering upper limit are deleted.
4. The model training method according to any one of claims 1 to 3, characterized in that: After obtaining the pseudo-annotation label output by the initialized text detection model, the model training method further includes: Obtaining the detection confidence of the pseudo-annotated label; The pseudo-annotated labels whose detection confidence is lower than a preset threshold are deleted.
5. The model training method according to claim 1, characterized in that: The step of training the initialized text detection model using the pseudo-annotated labels to obtain the text detection model includes: Using the pseudo-labeled labels, generating medium-quality training data; Training the initialization text detection model based on the medium-quality training data to obtain a first intermediate text detection model; Using the real annotation labels, generating high-quality training data; The first intermediate text detection model is trained using the high-quality training data to obtain the text detection model.
6. The model training method according to claim 5, characterized in that: Before generating medium-quality training data using the pseudo-labeled labels, the model training method further includes: Generate low-quality training data using unlabeled tags; The initialization text detection model is trained based on the low-quality training data to obtain a second intermediate text detection model.
7. The model training method according to claim 1, characterized in that: After obtaining the pseudo-annotation label output by the initialized text detection model, the model training method further includes: Passing the pseudo-annotated labels through a pre-trained image classification model to classify the pseudo-annotated labels into high-quality training data, medium-quality training data and / or low-quality training data; The step of training the initialized text detection model using the pseudo-annotated labels to obtain the text detection model includes: The initialization text detection model is trained in sequence from low to high quality using the high-quality training data, the medium-quality training data and / or the low-quality training data to obtain the text detection model.
8. The model training method according to claim 7, characterized in that: The model training method further includes: The image classification model is trained using the real annotated labels, the pseudo annotated labels, and the unannotated labels.
9. A text detection method, characterized in that: The text detection method comprises: Get the text image to be detected; Inputting the text image to be detected into a pre-trained text detection model; Based on the output of the text detection model, obtaining text information in the text image to be detected; Wherein, the text detection model is trained by the model training method described in any one of claims 1 to 8.
10. A terminal device, characterized in that: The terminal device includes a memory and a processor coupled to the memory; The memory is used to store program data, and the processor is used to execute the program data to implement the model training method as described in any one of claims 1 to 8, and / or the text detection method as described in claim 9.
11. A computer storage medium, characterized in that: The computer storage medium is used to store program data, and when the program data is executed by a computer, it is used to implement the model training method according to any one of claims 1 to 8, and / or the text detection method according to claim 9.