Text recognition model training method and device, electronic equipment and storage medium

Abstract

The invention discloses a text recognition model training method and device, electronic equipment and a storage medium. The method comprises the steps of performing text prediction on a first training sample set of a current training round based on a first text recognition model of the current training round to determine a first prediction result and a first confidence coefficient of the first prediction result; selecting a first reference sample from each first training sample based on each first confidence coefficient; adjusting parameters of the first text recognition model based on a first loss value determined by the first prediction result to obtain a second text recognition model of the next training round; and determining a second reference sample based on the first recognition model and the first reference sample, adding the first reference sample and the second reference sample into the first training sample set to obtain a second training sample set of the next training round, and training a second text recognition model based on the second training sample set. According to the method, the problems of low text recognition accuracy and insufficient generalization ability of a text recognition model in a specific scene are solved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a text recognition model training method, apparatus, electronic device, and storage medium. Background Technology

[0002] OCR is widely used in document recognition, license plate recognition, and ticket recognition. Mainstream methods typically consist of two stages: text detection and text recognition. Text detection identifies the location of text regions in an image, while text recognition identifies the characters within those regions. Essentially, text recognition is the process of translating a text region image into a character sequence. In recent years, deep learning methods have made significant progress in OCR, greatly improving the accuracy and robustness of text recognition.

[0003] Existing deep learning-based text recognition methods typically employ models based on temporal feature classification and encoders / decoders. However, these methods heavily rely on large, manually labeled datasets, limiting the construction of large-scale, high-quality training sets. Consequently, they cannot guarantee the accuracy and generalization ability of text recognition in specific scenarios (such as special fonts, complex backgrounds, easily confused characters, and consecutive identical characters). Summary of the Invention

[0004] This invention provides a text recognition model training method, apparatus, electronic device, and storage medium to solve the problems of low text recognition accuracy and insufficient generalization ability of text recognition models in specific scenarios (such as special fonts, complex backgrounds, easily confused characters, and consecutive identical characters).

[0005] According to one aspect of the present invention, a method for training a text recognition model is provided, the method comprising:

[0006] Determine the first training sample set and the first text recognition model for the current training round; the first training sample set contains multiple first training samples; the first training samples include text-labeled images of a preset scene;

[0007] Based on the first text recognition model, text prediction is performed on the first training sample set to determine the first prediction result and the first confidence level of the first prediction result;

[0008] Based on the first confidence level corresponding to each of the first training samples, a first reference sample is selected from each of the first training samples;

[0009] The first loss value of the first text recognition model is determined based on the first prediction result corresponding to each of the first training samples. The parameters of the first text recognition model are adjusted based on the first loss value to obtain the second text recognition model for the next training round.

[0010] Based on the first recognition model and the first reference sample, a second reference sample is determined, and the first reference sample and the second reference sample are added to the first training sample set to obtain the second training sample set for the next training round, so as to train the second text recognition model based on the second training sample set.

[0011] According to another aspect of the present invention, a text recognition model training apparatus is provided, the apparatus comprising:

[0012] The information determination module is used to determine the first training sample set and the first text recognition model for the current training round; the first training sample set contains multiple first training samples; the first training samples include text-labeled images of a preset scene;

[0013] The prediction module is used to perform text prediction on the first training sample set based on the first text recognition model, and determine a first prediction result and a first confidence level of the first prediction result;

[0014] The first sample determination module is used to select a first reference sample from each of the first training samples based on the first confidence level corresponding to each of the first training samples.

[0015] The model determination module is used to determine the first loss value of the first text recognition model based on the first prediction result corresponding to each of the first training samples, and to adjust the parameters of the first text recognition model based on the first loss value to obtain the second text recognition model for the next training round.

[0016] The second sample determination module is used to determine a second reference sample based on the first recognition model and the first reference sample, add the first reference sample and the second reference sample to the first training sample set, obtain the second training sample set for the next training round, and train the second text recognition model based on the second training sample set.

[0017] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0018] At least one processor; and

[0019] A memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the text recognition model training method according to any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the text recognition model training method according to any embodiment of the present invention.

[0022] The technical solution of this invention involves determining a first training sample set and a first text recognition model for the current training round; the first training sample set includes multiple first training samples; the first training samples include text-labeled images of a preset scene; text prediction is performed on the first training sample set based on the first text recognition model to determine a first prediction result and a first confidence level of the first prediction result; a first loss value of the first text recognition model is determined based on the first prediction result corresponding to each first training sample; the parameters of the first text recognition model are adjusted based on the first loss value to obtain a second text recognition model for the next training round, ensuring the accuracy of the model in the next training round; based on the first confidence level corresponding to each first training sample, the second text recognition model is obtained from each first training sample set. In this paper, a first reference sample is selected, and then a second reference sample is determined based on the first recognition model and the first reference sample. The first and second reference samples are added to the first training sample set to obtain the second training sample set for the next training round. This allows the second text recognition model to be trained based on the second training sample set, enabling continuous optimization of the training set during model training. This effectively expands the coverage of the training set, allowing the model to adapt to complex scene requirements even under low-resource annotation conditions. It achieves model training without relying on a large number of manually annotated datasets and solves the problems of low text recognition accuracy and insufficient generalization ability of text recognition models in specific scenarios (such as special fonts, complex backgrounds, easily confused characters, and consecutive identical characters).

[0023] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart of a text recognition model training method provided according to an embodiment of the present invention;

[0026] Figure 2 This is a flowchart of another text recognition model training method provided by an embodiment of the present invention;

[0027] Figure 3 This is an architecture diagram of the first text recognition model applicable to embodiments of the present invention;

[0028] Figure 4 This is a schematic diagram of the structure of a text recognition model training device according to an embodiment of the present invention;

[0029] Figure 5 This is a schematic diagram of the structure of an electronic device that implements the text recognition model training method of the present invention, according to an embodiment of the present invention. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] Figure 1 This is a flowchart illustrating a text recognition model training method provided in an embodiment of the present invention. This embodiment is applicable to training text recognition models for specific scenarios (such as special fonts, easily confused characters, consecutive identical characters, and complex backgrounds). The method can be executed by a text recognition model training device, which can be implemented in hardware and / or software and can be configured in any electronic device with network communication capabilities. Figure 1 As shown, the present invention and the text recognition model training method include:

[0033] S110. Determine the first training sample set and the first text recognition model for the current training round; the first training sample set contains multiple first training samples; the first training samples include text-labeled images of a preset scene.

[0034] Text-tagged images can be understood as images with text content annotated. Text recognition models are a type of artificial intelligence model that can automatically detect and extract text information from visual carriers such as images, scanned documents, and video frames, and convert it into editable and understandable digital text formats. Its core technology is usually optical character recognition (OCR), combined with computer vision, deep learning, natural language processing and other technologies to achieve end-to-end text parsing.

[0035] Specifically, to ensure the richness of the training sample set in the preset scenario for each training round, determining the first training sample set for the current training round may include: if the current training round is the first time, a certain number of diverse text-labeled images in the preset scenario can be used as the first training sample set; if the current training round is not the first time, all samples in the training sample set of the first training round can be used as training samples for the current training round. Furthermore, samples that meet the preset training requirements from the training results of the previous training round can be augmented and added as new samples to the training sample set of the current training round, thereby constituting the first training sample set for the current training round.

[0036] Meanwhile, determining the first text recognition model for the current training round can include: if the current training round is the first time, the first text recognition model can directly obtain the text recognition model to be trained; if the current training round is not the first time, the third prediction result obtained by the third text recognition model of the previous training round on the third training sample set of the previous training round is obtained, the second loss value of the first text recognition model is determined based on the third prediction result corresponding to each third training sample, and the parameters of the third text recognition model are adjusted based on the second loss value to obtain the third text recognition model for the current training round; the third text recognition model and the first text recognition model are text recognition models with the same structure.

[0037] In an embodiment of the present invention, optionally, determining the first training sample set for the current training round may include steps A1-A2:

[0038] Step A1: If the current training round is the first one, acquire multiple first images of the preset scene, and construct the first training sample set of the current training round based on the first images.

[0039] The preset scenarios can be understood as scenarios targeting special fonts, easily confused characters, consecutive identical characters, and complex backgrounds.

[0040] Specifically, diverse background images of a preset scene are collected and extracted as the first image. Preset texts in the first image are labeled, with only one preset text labeled at a time, thereby obtaining multiple text-labeled images including the preset scene. The text-labeled images including the preset scene are then collected as the first training sample set for the first training round.

[0041] In addition, acquiring multiple first images of a preset scene and constructing the first training sample set for the current training round based on the first images may also include: determining the feature parameters of the preset scene and acquiring multiple first images covering all variation dimensions of the feature parameters; performing format standardization, noise removal, and data augmentation processing based on the preset scene constraints on the first images; generating annotation files according to the annotation specifications of the text recognition task; and dividing the preprocessed images into training subsets, validation subsets, and test subsets using stratified sampling to generate the first training sample set adapted to the training framework.

[0042] Step A2: If the current training round is not the first one, perform text prediction on the third training sample set of the previous training round based on the third text recognition model of the previous training round, determine the third prediction result and the third confidence level of the third prediction result, and select the seventh reference sample from each third training sample based on the third confidence level corresponding to each third training sample; determine the eighth reference sample based on the first recognition model and the seventh reference sample, and add the seventh reference sample and the eighth reference sample to the third training sample set to obtain the first training sample set of the current training round; the third text recognition model and the first text recognition model are text recognition models with the same structure.

[0043] The first recognition model can be understood as a multimodal artificial intelligence model that integrates computer vision and natural language processing capabilities. Its core objective is to achieve cross-modal understanding, association, and generation of visual content such as images / videos and text. It breaks down the barriers between visual pixels and linguistic symbols, enabling cross-modal understanding, reasoning, and generation of images, videos, and text. The architecture of the first recognition model revolves around three core modules: visual feature extraction, text feature extraction, and cross-modal fusion. The first recognition model can be a large-scale visual-language model, including but not limited to GPT-4o, Claude 4, and Gemini 2.5 Flash.

[0044] Specifically, selecting a seventh reference sample from the third training samples based on the third confidence level corresponding to each third training sample may include: determining the third training sample with the third prediction result corresponding to the third confidence level within a first preset range as the seventh reference sample; the first preset range may be a range smaller than the second preset confidence level, or the first preset range may be a range between the third preset confidence level and the fourth preset confidence level. The third confidence level can be used to describe the accuracy of the third prediction result. Further, text prediction is performed on the seventh reference sample based on the first recognition model to obtain the first reference prediction result, and the seventh reference sample in the first reference prediction result that meets the preset prediction conditions is determined as the eighth reference sample. Then, the seventh reference sample and the eighth reference sample are added to the third training sample set to obtain the first training sample set for the current training round.

[0045] In this embodiment of the invention, by determining whether the current training round is the first time, the training sample set for the corresponding training round is accurately obtained, thereby ensuring that the sample set for each training round is sufficiently rich. Further, if the current training round is not the first time, a seventh reference sample is selected from each of the third training samples based on the third confidence level corresponding to each of the third training samples in the previous training round. An eighth reference sample is determined based on the first recognition model and the seventh reference sample. The seventh and eighth reference samples are added to the third training sample set to obtain the first training sample set for the current training round. This indicates that the present invention can automatically enrich the training samples in the training set during the training process, and the training samples are more in line with the preset scenario. This not only ensures the sufficiency of the samples but also ensures the generalization ability of the samples, so that the text recognition model obtained by continuously iterating the training sample set of the present invention has high recognition ability and generalization ability.

[0046] S120. Based on the first text recognition model, perform text prediction on the first training sample set, and determine the first prediction result and the first confidence level of the first prediction result.

[0047] Specifically, the first prediction result can be the result of the first text recognition model annotating the first training samples in the first training sample set. The first confidence score can be used to describe the accuracy of the first prediction result.

[0048] S130. Based on the first confidence level corresponding to each first training sample, select the first reference sample from each first training sample.

[0049] Specifically, the first training sample corresponding to the first prediction result within the second preset range is determined as the first reference sample; the second preset range can be a range less than the fifth preset confidence level, or the second preset range can be a range between the sixth preset confidence level and the seventh preset confidence level.

[0050] S140. Determine the first loss value of the first text recognition model based on the first prediction result corresponding to each first training sample, and adjust the parameters of the first text recognition model based on the first loss value to obtain the second text recognition model for the next training round.

[0051] Specifically, if the first loss value is greater than the preset loss value or the number of training iterations is less than the third preset number of iterations, the parameters of the first text recognition model are adjusted based on the first loss value to obtain the second text recognition model for the next training round.

[0052] S150. Based on the first recognition model and the first reference sample, determine the second reference sample, add the first reference sample and the second reference sample to the first training sample set, and obtain the second training sample set for the next training round, so as to train the second text recognition model based on the second training sample set.

[0053] Specifically, determining the second reference sample based on the first recognition model and the first reference sample may include: acquiring the text data corresponding to the first reference sample; the text data can be understood as prompt words that indicate the text of a preset type; standardizing the first reference sample to obtain preprocessed visual data, and semantically cleaning the text data to obtain preprocessed text data; processing the preprocessed visual data using a visual encoder based on the first recognition model to generate visual feature vectors; processing the preprocessed text data using a text encoder based on the first recognition model to generate text feature vectors; processing the visual feature vectors and text feature vectors using a cross-modal fusion processor based on the first recognition model to obtain cross-modal fusion features; processing the cross-modal fusion features using a text prediction decoder based on the first recognition model to obtain the second reference prediction result, and determining the first reference sample in the second reference prediction result that meets the preset prediction conditions as the second reference sample. The preset prediction conditions may be that the accuracy of the prediction result meets the preset accuracy conditions.

[0054] Furthermore, the process of the cross-modal fusion processor based on the first recognition model processing visual feature vectors and text feature vectors may include: achieving modal information alignment and fusion through an attention mechanism.

[0055] Optionally, determining the second reference sample based on the first recognition model and the first reference sample may include steps B1-B3:

[0056] Step B1: Determine the fourth reference sample, which includes the first reference sample and the sample obtained by performing the first preset enhancement operation on the first reference sample.

[0057] The enhancement operations may include at least one of random scaling, color transformation, background expansion, and rotation.

[0058] Specifically, a first preset enhancement operation is performed on the first reference sample to obtain a first preset number of ninth reference samples, and the first reference sample and the ninth reference sample are determined as the fourth reference sample.

[0059] Furthermore, determining the fourth reference sample may also include: if there is a newly added unlabeled reference text image, using the first text recognition model to perform text prediction on the reference text image, obtaining the sixth prediction result and the sixth confidence of the sixth prediction result, and using the reference text image with the sixth prediction result corresponding to the sixth confidence greater than the seventh preset confidence as the first candidate text image, and labeling the candidate text image with pseudo-labels to generate the second candidate text image; performing the fourth preset enhancement operation on the second candidate text image to obtain the third candidate text image; finally, the first reference sample, the sample obtained by performing the first preset enhancement operation on the first reference sample, the second candidate text image, and the third candidate text image are also used as the fourth reference sample.

[0060] Step B2: Based on the first text recognition model, perform text prediction on the fourth reference sample to obtain the second prediction result and the second confidence level of the second prediction result. The fourth reference sample with the second prediction result corresponding to the second confidence level that is greater than the first preset confidence level is used as the fifth reference sample.

[0061] Step B3: Based on the first recognition model and the fifth reference sample, determine the second reference sample.

[0062] Specifically, based on the first recognition model, text prediction is performed on the fifth reference sample to obtain the third reference prediction result. The fifth reference sample that meets the preset prediction conditions in the third reference prediction result is determined as the second reference sample.

[0063] Optionally, determining the second reference sample based on the first recognition model and the fifth reference sample may include: determining the sample label of each fifth reference sample based on the first recognition model, and determining the fifth reference samples with the same sample label as the sixth reference sample; updating the fifth reference sample based on the number of sixth reference samples to obtain the second reference sample.

[0064] Specifically, the sample labels can reflect the position and text type of the text marked on the fifth reference sample. Therefore, based on the position and text type of the sample labels on the fifth reference sample, it can be determined whether the sample labels are the same. That is, if they are in the same position and have the same text type, it indicates that the sample labels of the fifth reference samples are the same, and the fifth reference sample with the same sample labels is determined as the sixth reference sample. Furthermore, updating the fifth reference sample based on the number of sixth reference samples can include: if the number of sixth reference samples is greater than or equal to a first preset value, then the sixth reference sample is retained; if the number of sixth reference samples is less than the first preset value, then the sixth reference sample is deleted, thereby obtaining the second reference sample.

[0065] In this embodiment of the invention, the sample label of each fifth reference sample is determined based on the first recognition model. The fifth reference samples with the same sample label are determined as the sixth reference samples. Then, the fifth reference samples are updated based on the number of sixth reference samples to obtain the second reference samples. This achieves continuous optimization of the training samples, reduces the cost of manual annotation, and maintains continuous improvement in model performance.

[0066] In this embodiment of the invention, a fourth reference sample is determined. The fourth reference sample includes a first reference sample and samples obtained by performing a first preset enhancement operation on the first reference sample, thereby improving the richness of the samples while ensuring that the samples fit the scene. Then, text prediction is performed on the fourth reference sample based on a first text recognition model to obtain a second prediction result and a second confidence level of the second prediction result. The fourth reference sample with the second prediction result corresponding to the second confidence level that is greater than the first preset confidence level is used as the fifth reference sample, thereby achieving the screening of high-quality training samples. Further, a second reference sample is determined based on the first recognition model and the fifth reference sample. The first recognition model can further screen high-value samples in the fifth reference sample to ensure the quality of the obtained second reference sample.

[0067] The technical solution of this invention involves determining a first training sample set and a first text recognition model for the current training round; the first training sample set includes multiple first training samples; the first training samples include text-labeled images of a preset scene; text prediction is performed on the first training sample set based on the first text recognition model to determine a first prediction result and a first confidence level of the first prediction result; a first loss value of the first text recognition model is determined based on the first prediction result corresponding to each first training sample; the parameters of the first text recognition model are adjusted based on the first loss value to obtain a second text recognition model for the next training round, ensuring the accuracy of the model in the next training round; and a first reference sample is selected from each first training sample based on the first confidence level corresponding to each first training sample, thereby achieving active learning to filter high-value samples. This paper proposes a method to optimize data distribution in a targeted manner, enabling the model to adapt to complex scenarios even under low-resource labeling conditions. Then, based on the first recognition model and the first reference sample, a second reference sample is determined. The first and second reference samples are added to the first training sample set to obtain the second training sample set for the next training round. This allows the second text recognition model to be trained based on the second training sample set, enabling continuous optimization of the training set during model training. This effectively expands the coverage of the training set, allowing the model to adapt to complex scenarios even under low-resource labeling conditions. It achieves model training without relying on a large amount of manually labeled datasets and solves the problems of low text recognition accuracy and insufficient generalization ability of text recognition models in specific scenarios (such as special fonts, complex backgrounds, easily confused characters, and consecutive identical characters).

[0068] Figure 2 This is a flowchart of another text recognition model training method provided by an embodiment of the present invention. The technical solution of this embodiment further optimizes the process in the aforementioned embodiments where, if the current training round is the first time, multiple first images of a preset scene are acquired, and a first training sample set for the current training round is constructed based on the first images. This embodiment can be combined with various optional solutions in one or more of the above embodiments. Figure 2 As shown, text recognition model training methods may include:

[0069] S210. If the current training round is the first one, acquire multiple first images of the preset scene, construct a preset text library of the preset scene based on the first images, and construct the first training sample set of the current training round based on the preset text library.

[0070] The preset scenarios can be understood as scenarios targeting special fonts, easily confused characters, consecutive identical characters, and complex backgrounds.

[0071] Specifically, diverse background images of a preset scene are collected and extracted as the first image. The text content of the preset scene in the first image is extracted and collected into a preset text library. Then, font colors are randomly applied, affine transformations are performed on the text content in the preset text library, and then it is superimposed on the diverse background images. Elements such as color blocks, gradients, watermarks, and noise are randomly added to simulate complex situations in real scenes, thereby obtaining the first training sample set for the current training round.

[0072] In an embodiment of the present invention, optionally, the first image includes a second image and a third image, wherein the second image is an image with text annotations and the third image is an image without text annotations; correspondingly, after constructing a preset text library for a preset scene based on the first image, the method further includes: obtaining a first text pair based on the second image; obtaining a second text pair based on the third image; both the first text pair and the second text pair are text pairs containing easily confused character pairs; and adding the first text pair and the second text pair to the preset text library.

[0073] Easily confused character pairs can be understood as two character pairs that are easily confused by text recognition models, such as O / 0, 1 / l, 8 / B, etc. Easily confused characters can be determined based on the visual similarity of the characters and the frequency of recognition errors.

[0074] Specifically, a special font library containing easily confused character pairs is obtained. The labeled characters in the second image are extracted, and it is determined whether each character is a confusing character. If so, the corresponding confusing character pair is matched from the special font library as the first text pair. Characters in the third image are labeled, and it is determined whether the labeled characters are confusing characters. If so, the corresponding confusing character pair is matched from the special font library as the second text pair.

[0075] As an optional embodiment, obtaining the first text pair based on the second image may include: performing text prediction on the second image based on a fourth text recognition model, obtaining a fourth prediction result and a fourth confidence level of the fourth prediction result, and determining the fourth prediction result corresponding to a fourth confidence level greater than a fourth preset confidence level as a first candidate result; determining the labeled text in the first candidate result as the first predicted text, determining the labeled text in the second image corresponding to the first candidate result as the first labeled text, and determining the first predicted text and the first labeled text as the first text pair. The fourth text recognition model can be any currently used text recognition model.

[0076] As an optional embodiment, obtaining the second text pair based on the third image may include: performing text prediction on the third image based on the fifth text recognition model, obtaining the fifth prediction result and the fifth confidence level of the fifth prediction result, and determining the fifth prediction result corresponding to the fifth confidence level that is greater than the fifth preset confidence level as the second candidate result, wherein the fifth text recognition model can be any text recognition model currently used.

[0077] Then, the third image corresponding to the second candidate result is subjected to a second preset enhancement operation to obtain a fourth image. The fourth image and the third image corresponding to the fourth image are determined as the fifth image. For example, the second preset enhancement operation to obtain the fourth image from the third image corresponding to the second candidate result may include enhancement methods including but not limited to random scaling, color transformation, background expansion, and rotation. Each enhancement operation can select one enhancement method until the second preset enhancement operation is completed and the fourth image is obtained.

[0078] Furthermore, based on the first recognition model, text prediction is performed on the fifth image to obtain multiple text prediction results. The fifth image with the same text prediction result is identified as the sixth image. If the number of sixth images is greater than or equal to a second preset value, the labeled text in the fifth prediction result of the third image corresponding to the sixth image is identified as the second predicted text, the text prediction result corresponding to the sixth image is identified as the second labeled text, and the second predicted text and the second labeled text are identified as the second text pair.

[0079] In this embodiment of the invention, a high-quality third image is first screened using a fifth text recognition model to ensure the quality of the subsequently extracted text pairs. Then, the number of images is increased through enhancement operations to enrich the image quantity. Furthermore, text prediction is performed on the fifth image based on the first recognition model. Based on the text prediction results, a more accurate second text pair is obtained to ensure the quality and quantity of the extracted text pairs, so as to ensure the quality of the training samples generated based on the text pairs.

[0080] As an optional embodiment, after obtaining the first text pair based on the second image and the second text pair based on the third image, the method further includes steps C1-C2:

[0081] Step C1: Based on the first text pair and the second text pair, determine the confusion frequency of different easily confused character pairs.

[0082] Specifically, text sequence alignment tools can be used to find the longest common subsequence of two sequences in a text pair and return a list of opcodes. The opcode list describes how to convert the first sequence into the second sequence, and counts the obfuscated character pairs corresponding to character substitution operations and the frequency of occurrence of obfuscated character pairs.

[0083] Furthermore, the frequency of occurrence of each obfuscated character pair is obtained from the opcode list, and the obfuscation frequency of different easily obfuscated character pairs is determined based on the frequency of occurrence of each obfuscated character pair.

[0084] Step C2: Identify easily confused character pairs with a confusion frequency greater than a preset frequency as target easily confused character pairs; construct a third text pair based on the target easily confused character pairs, and add the third text pair to the preset text library.

[0085] Specifically, constructing a third text pair based on a target easily confused character pair may include: directly using the target easily confused character pair as the third text pair, and / or, replacing characters or adding easily confused characters according to the confusion frequency of the target easily confused character pair, and using it as the third text pair.

[0086] In this embodiment of the invention, based on a first text pair and a second text pair, the confusion frequency of different easily confused character pairs is determined. The confusion frequency can reflect the probability of easy recognition errors in existing text recognition models. Easily confused character pairs with a confusion frequency greater than a preset frequency are identified as target easily confused character pairs. A third text pair is constructed based on the target easily confused character pairs to increase the richness of confused character pairs. The third text pair is added to a preset text library to increase the diversity of the preset text library, thereby ensuring the accuracy of the samples constructed based on the preset text library and better reflecting the defects of existing text models. Thus, defects can be more accurately compensated during model training, thereby improving the recognition accuracy of the model.

[0087] Optionally, after constructing a preset text library for a preset scene based on the first image, the method further includes: constructing a fourth text based on the text in the preset text library, and adding the fourth text to the preset text library; the fourth text is text containing at least a preset number of consecutive identical characters. In this embodiment of the invention, adding consecutive characters to the preset text library increases the richness of the text in the preset text library, thus covering text content for multiple specific scenes.

[0088] S220. If the current training round is not the first time, perform text prediction on the third training sample set of the previous training round based on the third text recognition model of the previous training round, determine the third prediction result and the third confidence of the third prediction result, and select the seventh reference sample from each third training sample based on the third confidence corresponding to each third training sample.

[0089] S230. Based on the first recognition model and the seventh reference sample, determine the eighth reference sample, add the seventh reference sample and the eighth reference sample to the third training sample set, and obtain the first training sample set for the current training round; the third text recognition model and the first text recognition model are text recognition models with the same structure.

[0090] S240. Determine the first text recognition model for the current training round, perform text prediction on the first training sample set based on the first text recognition model, and determine the first prediction result and the first confidence level of the first prediction result; select the first reference sample from each first training sample based on the first confidence level corresponding to each first training sample.

[0091] S250. Determine the first loss value of the first text recognition model based on the first prediction result corresponding to each first training sample, and adjust the parameters of the first text recognition model based on the first loss value to obtain the second text recognition model for the next training round.

[0092] S260. Based on the first recognition model and the first reference sample, determine the second reference sample, add the first reference sample and the second reference sample to the first training sample set, and obtain the second training sample set for the next training round, so as to train the second text recognition model based on the second training sample set.

[0093] In this embodiment of the invention, if the current training round is the first, multiple first images of a preset scene are acquired, and a preset text library of the preset scene is constructed based on the first images. This facilitates the construction of the first training sample set for the current training round based on the preset text library, ensuring the diversity of the training sample set for the first training round and improving the quality of the first training sample set. Then, in subsequent training processes, the training set samples are automatically enriched through the prediction results of the text recognition model, the confidence level of the prediction results, and the first recognition model, so as to continuously introduce new and valuable data during the model training process, thereby continuously improving the model's generalization ability and recognition accuracy.

[0094] Figure 3 This is an architecture diagram of the first text recognition model applicable to the embodiments of the present invention; the technical solution of this embodiment describes the first text recognition model in the foregoing embodiments in detail based on the above embodiments, and this embodiment can be combined with various optional solutions in one or more of the above embodiments.

[0095] The first text recognition model of the present invention may include a shared feature extraction backbone network, a main recognition branch, a first auxiliary branch, and a second auxiliary branch;

[0096] The shared feature extraction backbone network is a lightweight semantic segmentation neural network; for example, the shared feature extraction backbone network can adopt the LCNet network structure.

[0097] The main recognition branch is a recognition network based on an unaligned sequence labeling algorithm; for example, the main recognition branch can be a neural network based on Connectionist Temporal Classification (CTC).

[0098] The first auxiliary branch is a network based on the Transformer architecture for scene text recognition; for example, the first auxiliary branch can be NRTR (No Cyclic Sequence to Sequence Network); the second auxiliary branch is used to predict the number of characters of each class in the image, which can enhance the model's ability to perceive the number of characters.

[0099] The second auxiliary branch can be understood as a weakly supervised counting branch. It can contain a convolutional layer, a channel attention layer, a 1×1 convolution, and a sum pooling layer. The second auxiliary branch receives the feature map output from the shared feature extraction backbone network and transforms it into a counting map through the convolutional layer, channel attention layer, and 1×1 convolution. It then compresses this map into a fixed-length counting vector using sum pooling. The supervision signal for the second auxiliary branch is directly generated from the text labels; for example, for the text label "ABBA", the corresponding counting vector is [2,2,0,..].

[0100] The first text recognition model also includes a first loss function, a second loss function, and a third loss function; the first loss function is the loss function corresponding to the main recognition branch; the second loss function is the loss function corresponding to the first auxiliary branch; the first auxiliary branch introduces a loss weighting mechanism for consecutive identical characters; and the third loss function is the loss function corresponding to the second auxiliary branch.

[0101] Specifically, a first reference loss value for the main recognition branch is calculated using a first loss function, a second reference loss value for the first auxiliary branch is calculated using a second loss function, and a third reference loss value for the first auxiliary branch is calculated using a third loss function. The first loss value... Second loss value and the third loss value Perform a weighted summation to obtain the total loss value; total loss value This can be expressed using the following formula:

[0102] ;

[0103] Among them, α, β, and γ are hyperparameters used to balance the importance of the primary and secondary tasks.

[0104] Furthermore, when calculating the loss of the first auxiliary branch, a character-level loss weighting mechanism is introduced. This mechanism directly locates consecutive characters in the real text sequence and increases the weight of the cross-entropy loss at these specific character positions. This enables the attention mechanism of the first auxiliary branch to learn to distinguish adjacent but identical characters in the sequence, rather than treating them as a single entity. This refined supervision signal is transmitted through the shared feature extraction backbone network, thereby benefiting the main recognition branch as well.

[0105] Specifically, the calculation process for the second reference loss value of the first auxiliary branch may include: for each training sample in the training batch, firstly, obtain the positions of all consecutive identical characters based on the real text, generate a weight sequence, and for each position of consecutive identical characters, set its corresponding weight to a hyperparameter λ greater than 1, while the weights of the remaining characters are 1; then, calculate the character-by-character cross-entropy loss for each sample, and then sum all the cross-entropy losses by weight to obtain the second reference loss value of the first auxiliary branch.

[0106] Optionally, the first text recognition model of the present invention can be a model with a second auxiliary branch added to the baseline PP-OCRv4 recognition model.

[0107] Optionally, after determining the first loss value of the first text recognition model based on the first prediction result corresponding to each first training sample, the text recognition model training method further includes: if the first loss value is less than or equal to the preset loss value or the model training times reach the third preset number, indicating that the model has completed iterative training, then the model after deleting the first auxiliary branch and the second auxiliary branch in the first text recognition model is used as the target text recognition model. That is, the target text recognition model only retains the shared feature extraction backbone network and the main recognition branch, so as to use the target text recognition model to perform text recognition in the preset scenario, ensuring the model's extremely high inference efficiency.

[0108] The first auxiliary branch of the first text recognition model in this embodiment of the invention introduces a loss weighting mechanism for consecutive identical characters, which solves the problem that the model still has a high error rate in recognizing consecutive identical characters in specific scenarios. This enables the model to process and recognize consecutive identical characters more accurately, thereby improving the overall recognition accuracy and robustness. Meanwhile, the second auxiliary branch of the first text recognition model can predict the number of each type of character in the image, enhancing the model's ability to perceive the number of characters.

[0109] Figure 4 This is a schematic diagram of a text recognition model training device provided in an embodiment of the present invention. This embodiment is applicable to training text recognition models for specific scenarios (such as special fonts, easily confused characters, consecutive identical characters, and complex backgrounds). The text recognition model training device can be implemented in hardware and / or software, and can be configured in any electronic device with network communication capabilities. Figure 4 As shown, the text recognition model training device of the present invention includes:

[0110] The information determination module 310 is used to determine the first training sample set and the first text recognition model for the current training round; the first training sample set contains multiple first training samples; the first training samples include text-labeled images of a preset scene;

[0111] Prediction module 320 is used to perform text prediction on the first training sample set based on the first text recognition model, and determine a first prediction result and a first confidence level of the first prediction result;

[0112] The first sample determination module 330 is used to select a first reference sample from each of the first training samples based on the first confidence level corresponding to each of the first training samples.

[0113] The model determination module 340 is used to determine the first loss value of the first text recognition model based on the first prediction result corresponding to each of the first training samples, and to adjust the parameters of the first text recognition model based on the first loss value to obtain the second text recognition model for the next training round.

[0114] The second sample determination module 350 is used to determine a second reference sample based on the first recognition model and the first reference sample, add the first reference sample and the second reference sample to the first training sample set, obtain the second training sample set for the next training round, and train the second text recognition model based on the second training sample set.

[0115] Optionally, based on the above embodiments, the second sample determination module includes a first sample determination unit, a first prediction unit, and a second sample determination unit; the first sample determination unit is used to determine a fourth reference sample, the fourth reference sample including the first reference sample and a sample obtained by performing a first preset enhancement operation on the first reference sample; the first prediction unit is used to perform text prediction on the fourth reference sample based on the first text recognition model, obtain a second prediction result and a second confidence level of the second prediction result, and take the fourth reference sample with the second prediction result corresponding to the second confidence level that is greater than the first preset confidence level as the fifth reference sample; the second sample determination unit is used to determine a second reference sample based on the first recognition model and the fifth reference sample.

[0116] Based on the above embodiments, optionally, the second sample determination unit is configured to: determine the sample label of each fifth reference sample based on the first recognition model, determine the fifth reference samples with the same sample label as the sixth reference samples; update the fifth reference samples based on the number of the sixth reference samples, and obtain the second reference samples.

[0117] Based on the above embodiments, optionally, the second sample determination unit is further configured to: retain the sixth reference sample if the number of the sixth reference sample is greater than or equal to the first preset value; and delete the sixth reference sample if the number of the sixth reference sample is less than the first preset value.

[0118] Based on the above embodiments, optionally, the information determination module is used to: if the current training round is the first time, acquire multiple first images of a preset scene, and construct a first training sample set for the current training round based on the first images; if the current training round is not the first time, perform text prediction on the third training sample set of the previous training round based on the third text recognition model of the previous training round, determine the third prediction result and the third confidence level of the third prediction result, and select a seventh reference sample from each of the third training samples based on the third confidence level corresponding to each of the third training samples; the third text recognition model and the first text recognition model are text recognition models with the same structure; determine an eighth reference sample based on the first recognition model and the seventh reference sample, and add the seventh reference sample and the eighth reference sample to the third training sample set to obtain the first training sample set for the current training round.

[0119] Based on the above embodiments, optionally, the information determination module includes a text library construction unit and a sample set determination unit; the text library construction unit is used to construct a preset text library of a preset scene based on the first image; the sample set determination unit is used to construct a first training sample set for the current training round based on the preset text library.

[0120] Based on the above embodiments, optionally, the first image includes a second image and a third image, the second image is an image with text annotations, and the third image is an image without text annotations. The information determination module includes a text library update unit, which is used to: after constructing a preset text library for a preset scenario based on the first image, obtain a first text pair based on the second image and obtain a second text pair based on the third image; both the first text pair and the second text pair are text pairs containing easily confused character pairs; and add the first text pair and the second text pair to the preset text library.

[0121] Based on the above embodiments, optionally, the text library update unit includes a first text pair determination subunit, which is used to: perform text prediction on the second image based on the fourth text recognition model, obtain a fourth prediction result and a fourth confidence level of the fourth prediction result, determine the fourth prediction result corresponding to the fourth confidence level that is greater than the fourth preset confidence level as a first candidate result; determine the labeled text in the first candidate result as the first predicted text, determine the labeled text in the second image corresponding to the first candidate result as the first labeled text, and determine the first predicted text and the first labeled text as a first text pair.

[0122] Optionally, based on the above embodiments, the text library update unit includes a second text pair determination subunit, which is used to: perform text prediction on the third image based on the fifth text recognition model, obtain a fifth prediction result and a fifth confidence level of the fifth prediction result, and determine the fifth prediction result corresponding to a fifth confidence level greater than a fifth preset confidence level as a second candidate result; perform a second preset enhancement operation on the third image corresponding to the second candidate result to obtain a fourth image, and determine the fourth image and the third image corresponding to the fourth image as a fifth image; perform text prediction on the fifth image based on the first recognition model to obtain multiple text prediction results, and determine the fifth image with the same text prediction result as a sixth image; if the number of sixth images is greater than or equal to a second preset value, then determine the labeled text in the fifth prediction result of the third image corresponding to the sixth image as the second predicted text, determine the text prediction result corresponding to the sixth image as the second labeled text, and determine the second predicted text and the second labeled text as a second text pair.

[0123] Optionally, based on the above embodiments, the text library update unit is further configured to: after obtaining a first text pair based on the second image and a second text pair based on the third image, determine the confusion frequency of different easily confused character pairs based on the first text pair and the second text pair; determine easily confused character pairs with a confusion frequency greater than a preset frequency as target easily confused character pairs; construct a third text pair based on the target easily confused character pairs, and add the third text pair to the preset text library.

[0124] Optionally, based on the above embodiments, the text library update unit is further configured to: after constructing a preset text library for a preset scene based on the first image, construct a fourth text based on the text in the preset text library, and add the fourth text to the preset text library; the fourth text is a text containing at least a preset number of consecutive identical characters.

[0125] Based on the above embodiments, optionally, the first text recognition model includes a shared feature extraction backbone network, a main recognition branch, a first auxiliary branch, and a second auxiliary branch; the shared feature extraction backbone network is a lightweight semantic segmentation neural network; the main recognition branch is a recognition network based on an unaligned sequence labeling algorithm; the first auxiliary branch is a network based on the Transformer architecture for scene text recognition; and the second auxiliary branch is used to predict the number of characters of each class in the image.

[0126] The first text recognition model includes a first loss function, a second loss function, and a third loss function; the first loss function is the loss function corresponding to the main recognition branch; the second loss function is the loss function corresponding to the first auxiliary branch; the first auxiliary branch introduces a loss weighting mechanism for consecutive identical characters; and the third loss function is the loss function corresponding to the second auxiliary branch.

[0127] Optionally, based on the above embodiments, the text recognition model training device further includes a text recognition model determination module. The text recognition model determination module is used to determine the first loss value of the first text recognition model based on the first prediction result corresponding to each of the first training samples. If the first loss value is less than or equal to a preset loss value or the number of training times of the model reaches a third preset number, the model after deleting the first auxiliary branch and the second auxiliary branch in the first text recognition model is used as the target text recognition model, so as to use the target text recognition model to perform text recognition of the preset scenario.

[0128] The text recognition model training device provided in this embodiment of the invention can execute the text recognition model training method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0129] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0130] Figure 5 A schematic diagram of an electronic device that can be used to implement the text recognition model training method of embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0131] like Figure 5 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0132] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0133] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as text recognition model training methods.

[0134] In some embodiments, the text recognition model training method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via read-only memory (ROM) 12 and / or communication unit 19. When the computer program is loaded into random access memory (RAM) 13 and executed by processor 11, one or more steps of the text recognition model training method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to execute the text recognition model training method by any other suitable means (e.g., by means of firmware).

[0135] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0136] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0137] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0138] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0139] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0140] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0141] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and no limitation is imposed herein.

[0142] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for training a text recognition model, characterized in that, The method includes: Determine the first training sample set and the first text recognition model for the current training round; the first training sample set contains multiple first training samples; the first training samples include text-labeled images of a preset scene; Based on the first text recognition model, text prediction is performed on the first training sample set to determine the first prediction result and the first confidence level of the first prediction result; Based on the first confidence level corresponding to each of the first training samples, a first reference sample is selected from each of the first training samples; The first loss value of the first text recognition model is determined based on the first prediction result corresponding to each of the first training samples. The parameters of the first text recognition model are adjusted based on the first loss value to obtain the second text recognition model for the next training round. Based on the first recognition model and the first reference sample, a second reference sample is determined, and the first reference sample and the second reference sample are added to the first training sample set to obtain the second training sample set for the next training round, so as to train the second text recognition model based on the second training sample set.

2. The method according to claim 1, characterized in that, Based on the first recognition model and the first reference sample, a second reference sample is determined, including: A fourth reference sample is determined, the fourth reference sample including the first reference sample and a sample obtained by performing a first preset enhancement operation on the first reference sample; Based on the first text recognition model, text prediction is performed on the fourth reference sample to obtain a second prediction result and a second confidence level of the second prediction result. The fourth reference sample with the second prediction result corresponding to the second confidence level that is greater than the first preset confidence level is used as the fifth reference sample. Based on the first identification model and the fifth reference sample, a second reference sample is determined.

3. The method according to claim 2, characterized in that, Based on the first recognition model and the fifth reference sample, a second reference sample is determined, including: Based on the first recognition model, the sample label of each fifth reference sample is determined, and the fifth reference samples with the same sample label are determined as the sixth reference samples; The fifth reference sample is updated based on the number of the sixth reference sample to obtain the second reference sample.

4. The method according to claim 3, characterized in that, Updating the fifth reference sample based on the number of the sixth reference sample includes: If the number of the sixth reference samples is greater than or equal to the first preset value, then the sixth reference sample is retained; If the number of the sixth reference samples is less than the first preset value, then the sixth reference sample is deleted.

5. The method according to claim 1, characterized in that, The first training sample set for the current training round is determined, including: If the current training round is the first one, acquire multiple first images of a preset scene, and construct the first training sample set for the current training round based on the first images; If the current training round is not the first one, the text prediction of the third training sample set of the previous training round is performed based on the third text recognition model of the previous training round, the third prediction result and the third confidence of the third prediction result are determined, and the seventh reference sample is selected from each of the third training samples based on the third confidence corresponding to each of the third training samples; the third text recognition model and the first text recognition model are text recognition models with the same structure. Based on the first recognition model and the seventh reference sample, an eighth reference sample is determined, and the seventh reference sample and the eighth reference sample are added to the third training sample set to obtain the first training sample set for the current training round.

6. The method according to claim 5, characterized in that, Based on the first image, construct the first training sample set for the current training round, including: Based on the first image, a preset text library for a preset scene is constructed; Based on the preset text library, construct the first training sample set for the current training round.

7. The method according to claim 6, characterized in that, The first image includes a second image and a third image, wherein the second image is an image with text annotations, and the third image is an image without text annotations. After constructing a preset text library for a preset scene based on the first image, the method further includes: A first text pair is obtained based on the second image, and a second text pair is obtained based on the third image; both the first text pair and the second text pair are text pairs containing easily confused character pairs; Add the first text pair and the second text pair to the preset text library.

8. The method according to claim 7, characterized in that, The first text pair is obtained based on the second image, including: Based on the fourth text recognition model, text prediction is performed on the second image to obtain the fourth prediction result and the fourth confidence level of the fourth prediction result. The fourth prediction result corresponding to the fourth confidence level that is greater than the fourth preset confidence level is determined as the first candidate result. The labeled text in the first candidate result is determined as the first predicted text, the labeled text in the second image corresponding to the first candidate result is determined as the first labeled text, and the first predicted text and the first labeled text are determined as the first text pair.

9. The method according to claim 7, characterized in that, The second text pair is obtained based on the third image, including: Based on the fifth text recognition model, text prediction is performed on the third image to obtain the fifth prediction result and the fifth confidence level of the fifth prediction result. The fifth prediction result corresponding to the fifth confidence level that is greater than the fifth preset confidence level is determined as the second candidate result. Perform a second preset enhancement operation on the third image corresponding to the second candidate result to obtain the fourth image, and determine the fourth image and the third image corresponding to the fourth image as the fifth image; Based on the first recognition model, text prediction is performed on the fifth image to obtain multiple text prediction results. The fifth image with the same text prediction result is determined as the sixth image. If the number of the sixth images is greater than or equal to the second preset value, then the labeled text in the fifth prediction result of the third image corresponding to the sixth image is determined as the second predicted text, the text prediction result corresponding to the sixth image is determined as the second labeled text, and the second predicted text and the second labeled text are determined as the second text pair.

10. The method according to claim 7, characterized in that, After obtaining a first text pair based on the second image and a second text pair based on the third image, the method further includes: Based on the first text pair and the second text pair, determine the confusion frequency of different easily confused character pairs; The easily confused character pairs with a confusion frequency greater than a preset frequency are identified as target easily confused character pairs; A third text pair is constructed based on the target easily confused character pair, and the third text pair is added to the preset text library.

11. The method according to any one of claims 6-10, characterized in that, After constructing a preset text library for a preset scene based on the first image, the method further includes: Based on the text in the preset text library, a fourth text is constructed and added to the preset text library; the fourth text is a text containing at least a preset number of consecutive identical characters.

12. The method according to claim 1, characterized in that, The first text recognition model includes a shared feature extraction backbone network, a main recognition branch, a first auxiliary branch, and a second auxiliary branch; the shared feature extraction backbone network is a lightweight semantic segmentation neural network; the main recognition branch is a recognition network based on an unaligned sequence labeling algorithm; the first auxiliary branch is a network based on the Transformer architecture for scene text recognition; and the second auxiliary branch is used to predict the number of characters of each class in the image. The first text recognition model includes a first loss function, a second loss function, and a third loss function; the first loss function is the loss function corresponding to the main recognition branch; the second loss function is the loss function corresponding to the first auxiliary branch; the first auxiliary branch introduces a loss weighting mechanism for consecutive identical characters; and the third loss function is the loss function corresponding to the second auxiliary branch.

13. The method according to claim 12, characterized in that, After determining the first loss value of the first text recognition model based on the first prediction result corresponding to each of the first training samples, the method further includes: If the first loss value is less than or equal to the preset loss value or the number of model training times reaches the third preset number, then the model after deleting the first auxiliary branch and the second auxiliary branch in the first text recognition model is used as the target text recognition model, so as to use the target text recognition model to perform text recognition in the preset scenario.

14. A text recognition model training device, characterized in that, The device includes: The information determination module is used to determine the first training sample set and the first text recognition model for the current training round; the first training sample set contains multiple first training samples; the first training samples include text-labeled images of a preset scene; The prediction module is used to perform text prediction on the first training sample set based on the first text recognition model, and determine a first prediction result and a first confidence level of the first prediction result; The first sample determination module is used to select a first reference sample from each of the first training samples based on the first confidence level corresponding to each of the first training samples. The model determination module is used to determine the first loss value of the first text recognition model based on the first prediction result corresponding to each of the first training samples, and to adjust the parameters of the first text recognition model based on the first loss value to obtain the second text recognition model for the next training round. The second sample determination module is used to determine a second reference sample based on the first recognition model and the first reference sample, add the first reference sample and the second reference sample to the first training sample set, obtain the second training sample set for the next training round, and train the second text recognition model based on the second training sample set.

15. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the text recognition model training method according to any one of claims 1-13.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the text recognition model training method according to any one of claims 1-13.

Images