Data processing method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a data processing method and device, electronic equipment and storage medium, the method comprising: obtaining text image data to be identified; identifying the text image data according to a pre-trained text recognition model to determine an identification result, the trained text recognition model sharing character information with a single character recognition model; and outputting each text in the identification result for display.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data processing method, a data processing device, an electronic device, and a storage medium. Background Technology

[0002] Optical Character Recognition (OCR) technology refers to the recognition of text characters in an image.

[0003] Existing character recognition methods typically utilize pre-trained recognition models, but these models often suffer from low accuracy. Summary of the Invention

[0004] This application provides a data processing method to improve the recognition accuracy of a recognition model.

[0005] Accordingly, embodiments of this application also provide an electronic device and a storage medium to ensure the implementation and application of the above system.

[0006] To address the aforementioned issues, this application discloses a data processing method, which includes: acquiring text image data to be recognized; recognizing the text image data based on a pre-trained text recognition model; determining the recognition result; wherein the pre-trained text recognition model shares character information with a single-character recognition model; and outputting each text in the recognition result for display.

[0007] To address the aforementioned issues, this application discloses a data processing method. The method includes: providing an interactive page, the interactive page including a data upload control; acquiring text image data to be recognized based on triggering the data upload control, and uploading it to a server, so that the server can recognize the text image data based on a pre-trained text recognition model and determine the recognition result, wherein the pre-trained text recognition model shares character information with a single-character recognition model; receiving the recognition result and displaying it.

[0008] To address the aforementioned issues, this application discloses a data processing method, which includes: acquiring an image of an ancient character to be identified; identifying the ancient character image based on a pre-trained text recognition model to determine the recognition result, wherein the pre-trained text recognition model shares character information with a single-character recognition model; and outputting the recognition result for display.

[0009] To address the aforementioned issues, this application discloses an electronic device, including: a processor; and a memory storing executable code thereon, wherein when the executable code is executed, the processor performs one or more of the methods described in the above embodiments.

[0010] To address the aforementioned issues, embodiments of this application disclose one or more machine-readable media storing executable code thereon, which, when executed, causes a processor to perform one or more of the methods described in the above embodiments.

[0011] Compared with the prior art, the embodiments of this application have the following advantages:

[0012] In this embodiment, before recognizing the text image data, individual characters can be extracted from the labeled training text image to form a single-character image, which is then labeled. A text recognition model is then trained based on the labeled training text image, and a single-character recognition model is trained based on the labeled single-character images. Character information is shared between the text recognition model and the single-character recognition model. After the text recognition model training is complete, the text image data to be recognized can be obtained and input into the pre-trained text recognition model to extract the features of each character in the text image data and associate the features of the context characters of each character, thereby performing character classification, determining the recognition result, and outputting the recognition result for display. This embodiment can train the text recognition model using training text images and single-character images, thereby utilizing the character information of evenly distributed single characters in the single-character image to reduce the influence of factors such as uneven character distribution in the training text image on the recognition result, thus improving the recognition accuracy of the text recognition model. Attached Figure Description

[0013] Figure 1A This is a schematic flowchart of a data processing method according to an embodiment of this application;

[0014] Figure 1B This is a schematic diagram of the structure of a context modeling module according to an embodiment of this application;

[0015] Figure 1C This is a schematic diagram of the structure of a character enhancer according to an embodiment of this application;

[0016] Figure 1D This is a schematic flowchart illustrating the processing procedure of the discriminator according to an embodiment of this application;

[0017] Figure 1E This is a schematic flowchart of a data processing method according to another embodiment of this application;

[0018] Figure 2A This is a schematic flowchart of a data processing method according to another embodiment of this application;

[0019] Figure 2B This is a schematic diagram of the character enhancer processing flow according to an embodiment of this application;

[0020] Figure 3 This is a schematic flowchart of a data processing method according to another embodiment of this application;

[0021] Figure 4A This is a schematic flowchart of a data processing method according to another embodiment of this application;

[0022] Figure 4B This is a schematic flowchart of a data processing method according to another embodiment of this application;

[0023] Figure 5 This is a schematic flowchart of a data processing method according to another embodiment of this application;

[0024] Figure 6 This is a schematic flowchart of a data processing method according to another embodiment of this application;

[0025] Figure 7 This is a schematic diagram of the structure of a data processing apparatus according to an embodiment of this application;

[0026] Figure 8 This is a schematic diagram of the structure of a data processing apparatus according to another embodiment of this application;

[0027] Figure 9 This is a schematic diagram of the structure of a data processing apparatus according to another embodiment of this application;

[0028] Figure 10 This is a schematic diagram of the structure of an exemplary device provided in one embodiment of this application. Detailed Implementation

[0029] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0030] This application's embodiments can be applied to the field of character recognition. Optical Character Recognition (OCR) technology refers to the recognition of characters in an image. This application's embodiments can train a text recognition model using labeled training text images and single-character images. The single-character images can be determined in various ways, such as by extracting individual characters from the training text image, or by combining a single character with an image to form a single-character image. The single character can also be a character from a different font, ensuring a balanced distribution of categories in the single-character images. After the text recognition model is trained, the text image to be recognized can be input into the trained text recognition model to determine the recognition result. This application's embodiments can use single-character images to reduce the impact of factors such as the uneven distribution of characters in the training text image on the recognition result, thereby improving the recognition accuracy of the text recognition model.

[0031] Specifically, such as Figure 1A As shown, the training process in this embodiment involves three branches: a text line learning branch, a single character learning branch, and an adversarial learning branch. The text line learning branch is used to recognize images of a line of text (text line) containing one or more characters. The single character learning branch is used to recognize images of single characters. The adversarial learning branch is used to determine which branch the features extracted by the text line learning branch and the single character learning branch belong to, so as to adjust the other two branches. The text line learning branch can be used for recognition by a text recognition model. The text recognition model is used to recognize image data containing multiple characters to determine the recognition result. During training, training text images containing multiple characters can be input into the text recognition model to determine a first prediction result. Based on the difference between the first prediction result and the annotation of the training text image, the text recognition model is adjusted. The text recognition model may include a first feature extractor, a context modeling module, and a first character classifier. The first feature extractor is used to extract the feature information of each character in the image as the first feature of each character. The context modeling module is used to incorporate the features of the context (characters before and after the character) into the first feature of the character to determine the first feature data. The first character classifier is used to classify the character based on the first feature data to determine the first prediction result.

[0032] It should be noted that, as Figure 1B As shown, in addition to associating features of the character context, the context modeling module can also configure a character enhancer to enhance characters and reduce background interference in the image. Figure 1BIn the example shown, the context modeling module can be configured with two character enhancers and one encoder. The encoder is used to fuse character context features. In this embodiment, the character features can be enhanced by the first character enhancer to obtain the processed first feature. The character context features are then fused into the character features by the encoder to obtain the first feature data. The first feature data is further enhanced by the second character enhancer. Specifically, as shown... Figure 1C As shown, a character enhancer (taking the first character enhancer as an example) may include a background separator and a style extractor. The character enhancer can separate background features through the background separator, extract the style features of the characters through the style extractor, enhance the style features, and fuse them to form the processed first feature. This embodiment can enhance the style features of characters to reduce the interference of background features on the recognition results, thereby improving the recognition accuracy of the text recognition model.

[0033] The single-character recognition model in the single-character learning branch is used to recognize single-character images containing a single character to determine the recognition result. During training, single-character images can be input into the single-character recognition model to determine a second prediction result. The single-character recognition model is then adjusted based on the difference between the second prediction result and the corresponding annotation of the single-character image. The single-character recognition model includes a second feature extractor and a second character classifier. The second feature extractor extracts features from the single-character image as second feature data. The second character classifier classifies characters based on the second feature data to determine the recognition result (or prediction result). It should be noted that the first character classifier in the text recognition model and the second character classifier in the single-character recognition model are shared. The first character classifier and the second character classifier have the same structure and parameters. In this embodiment, the character information of the training text image and the character information of the single-character image can be shared by sharing the character classifier between the text recognition model and the single-character recognition model. Here, character information refers to various attributes of the character, such as character trajectory, character structure, etc.

[0034] Natural language processing often exhibits a long-tail effect, which refers to the imbalance in sample distribution. A small number of characters are frequently used, known as head characters, while a large number of characters are rarely used, known as tail characters. For example, in daily life, people typically use only about 3,000 Chinese characters, but the total number of Chinese characters is in the tens of thousands, representing a severe imbalance in character usage frequency. Therefore, classification methods that rely entirely on contextual features and related feature data will suffer from low accuracy in recognizing some rare characters due to the imbalanced character distribution.

[0035] In this embodiment, by sharing the first character classifier and the second character classifier, the first character classifier is also adjusted during the process of adjusting the second character classifier using single character images. This achieves the purpose of training the first character classifier of the text recognition model using single character images, which can reduce the impact of factors such as the uneven distribution of characters in the training text image on the recognition result, thereby improving the recognition accuracy of the text recognition model.

[0036] This application embodiment can acquire labeled training text images and input them into a text recognition model to extract the first features of characters through a first feature extractor. The processed first features are then determined by a context modeling module. Afterward, the processed first features can be input into a generator to determine character information and generate single-character images. It should be noted that this application embodiment describes generating single-character images using character features enhanced with character style. In some optional examples, single-character images can also be generated based on character features without character style enhancement for training. The specific configuration can be tailored to requirements. After determining the single-character image, annotations can be configured for it. This application embodiment can configure annotations for single-character images based on the annotations of the training text images. Alternatively, the single-character images can be arranged according to the positions of characters in the training text images to obtain a single-character image sequence, and the annotations of the training text images can be used as the annotations for the single-character image sequence.

[0037] In this embodiment, the single-character image is generated based on the character information of each character in the training text image. Therefore, the features of the characters in the single-character image should be consistent with the features of the corresponding characters in the training text image. However, during the actual feature extraction process, the text recognition model may extract some background features from the training text image as character features. Since the single-character image is a background-free image generated based on the character information, the features extracted from the single-character image do not include background features. Therefore, the features extracted by the text recognition model and the single-character recognition model may be inconsistent, or even orthogonal (the representation features are unrelated). Therefore, this embodiment can use a discriminator in the adversarial learning branch to determine the difference between the first and second feature data of the same character, and adjust the first feature extractor of the text recognition model based on the difference information. Specifically, as shown... Figure 1DAs shown, this embodiment of the application can acquire first feature data of the target character in the training text image and second feature data in the single character image, and input them into a discriminator to determine the difference information. The discriminator can score the feature data according to its data type, such as single character type (data corresponding to single character images) and character line type (data corresponding to training text images). The difference information is determined based on the first score of the first feature data and the second score of the second feature data; and the first feature extractor of the text recognition model is adjusted based on the difference information.

[0038] In addition, in order to further reduce the difference between the first feature data and the second feature data, this embodiment of the application can also generate a corresponding score sequence based on the first prediction result of the training text image, and extract the corresponding score in the score sequence to adjust the first score (such as taking the average of the two scores) to reduce the influence of background features in the first feature data on the first score, thereby improving the consistency between the first feature data and the second feature data, so as to improve the recognition accuracy of the text recognition model.

[0039] After determining the labeled training text images and labeled single-character images, the training text images can be input into the text recognition model to determine the first prediction result. Then, by combining the difference between the first prediction result and the labeled training text images, the character model adjustment amount is determined, and the text recognition model is adjusted accordingly. This character model adjustment includes adjustments for the first feature extractor, the context modeling module, and the character classification of the first character classifier. The first character classifier of the text recognition model is shared with the second character classifier of the single-character recognition model. Therefore, the second character classifier can be adjusted by adjusting the character classification of the first character classifier, thus ensuring that the parameters of the first and second character classifiers are consistent. Furthermore, besides the above method of sharing the first and second character classifiers, other methods can be used, such as configuring a shared character classifier for both the text recognition model and the single-character recognition model to achieve the effect of sharing a character classifier.

[0040] On the other hand, a single-character image can be input into a single-character recognition model to determine a second prediction result. Then, by combining the difference between the second prediction result and the annotation of the single-character image, the adjustment amount of the single-character model is determined, and the single-character recognition model is adjusted. The adjustment amount of the single-character model can include the adjustment amount corresponding to the second feature extractor and the single-character classification adjustment amount corresponding to the second character classifier. In this embodiment, the first character classifier can be adjusted based on the single-character classification adjustment amount to ensure that the parameters of the first character classifier and the second character classifier are consistent (shared).

[0041] It should be noted that during the training process of a text recognition model, the impact of character imbalance on the model is relatively small in the early stages of training, but becomes larger in the later stages. Therefore, this embodiment can configure corresponding weight information for single-character images, and continuously increase the influence of single-character images on the first character classifier of the text recognition model as the training process progresses, thereby improving the recognition accuracy of the text recognition model. For example, this embodiment can determine the corresponding weight information based on the number of iterations of the text recognition model, and then adjust the single-character classification adjustment amount based on the weight information, thereby continuously increasing the influence of the single-character classification adjustment amount on the first character classifier during the training process of the text recognition model, and improving the recognition accuracy of the text recognition model.

[0042] It should be noted that in the embodiments of this application, the text line learning branch, single character learning branch, and adversarial learning branch can be trained on one device or deployed on different devices, depending on the specific requirements. Furthermore, different components in the three branches can also be deployed on different computing devices. For example, the first feature extractor, context modeling module, and first character classifier can be configured on different computing devices, and connections can be established between the computing devices to complete the corresponding training.

[0043] After the text recognition model is trained, this embodiment can recognize the text image to be recognized based on the text recognition model. It should be noted that the single-character recognition model provides assistance to the first character classifier of the text recognition model during the training phase, and may not play a role in the model application phase. This embodiment can acquire the text image data to be recognized, and recognize the text image data based on the pre-trained text recognition model to determine the recognition result.

[0044] This application's embodiments can be applied to various scenarios involving text image recognition. Therefore, this application's embodiments can adopt data acquisition methods corresponding to different scenarios to acquire the text image data to be recognized. For example, this application's embodiments can be applied to big data processing scenarios, where target information can be set and text image data related to the target information can be acquired. This application's embodiments can also be applied to scenarios where text image recognition services are provided through a server, where a data upload interface can be provided to receive text image data based on the data upload interface, recognize the text image data, and provide feedback on the recognition results. Additionally, as... Figure 1E As shown, embodiments of this application can also be applied to the server side, where the server can interact with the terminal and provide an interactive page to the terminal to obtain text image data based on the interactive page, perform recognition, and then provide feedback on the recognition results. Embodiments of this application can also be applied to enterprise query scenarios, where text images containing enterprise names can be recognized, and based on the enterprise names in the recognition results, relevant enterprise data can be searched and output.

[0045] This application embodiment can output recognition results and display them to the user on an interactive page. The user can adjust the recognition results on the interactive page, generating result adjustment information to adjust the recognition results and, based on the adjusted recognition results, adjust the text recognition model. This application embodiment can adjust the text recognition model according to the user's adjustments to the recognition results, further improving the accuracy of the text recognition model. This application embodiment can also display training text images and arranged single-character images on the interactive page. The single-character images can be arranged according to the position of the characters in the single-character images in the training text images. The user can adjust the characters on the interactive page, obtain character adjustment instructions, and adjust the training text images or single-character images according to the character adjustment instructions to obtain adjusted images, so as to train the text recognition model based on the adjusted images.

[0046] Users can also personalize the text recognition model according to the corresponding scenario. Users can upload target training data for a specific domain, including target training text images and labeled data. This allows for the extraction of individual characters from the target training text images, the identification of target single-character images, and the configuration of labels for these images based on the labeled data. Subsequently, based on the labeled target training text images and labeled target single-character images, a target text recognition model is trained to recognize text image data within the target domain. Target domains can include education, healthcare, e-commerce, and other fields, allowing for the training of personalized text recognition models based on training data from the corresponding domain.

[0047] The embodiments of this application are optimized during the training phase of the text recognition model. Therefore, the embodiments of this application can be applied to scenarios where various types of text images are recognized. For example, the embodiments of this application can be applied to scenarios where text images containing information such as ancient characters, education-related information, medical-related information, e-commerce-related information, enterprise information, and meeting content are recognized.

[0048] For example, embodiments of this application can be applied to scenarios involving the recognition of text images containing ancient characters. The ancient character image can be acquired, and recognition can be performed using a pre-trained text recognition model to determine the recognition result. Based on the corresponding recognition result, the ancient characters can be converted into modern characters.

[0049] For another example, the embodiments of this application can also be applied to scenarios where educational text images containing educational information are recognized. The educational text image can be acquired, and the educational text image can be recognized according to a pre-trained text recognition model to determine the recognition result. The recognition result can then be processed accordingly. For example, when the educational information is a student's answer information, the embodiments of this application can score the answer based on the recognition result to determine the student's score.

[0050] In addition to the scenarios described above, this application embodiment can also be applied to scenarios involving text-image-based searches. This application embodiment can acquire a user-uploaded text image to be recognized, perform recognition based on a pre-trained text recognition model, and determine the recognition result. Then, a search can be performed based on the recognition result, and the search results can be displayed to the user.

[0051] Furthermore, by segmenting text images based on the text they contain, this application can also be applied to scenarios involving the recognition of text images containing different types of text. For example, this application can be applied to scenarios involving the recognition of text image data containing text in various languages ​​such as Chinese, English, and Japanese. Recognition of text images involving different types of characters within the same language is also possible. Taking Chinese characters as an example, this can include the recognition of simplified characters, traditional characters, seal script, and other types of Chinese characters. Similarly, it can also address the recognition of different fonts of Chinese characters, such as the recognition of Chinese text in various fonts like regular script, clerical script, cursive script, and willow script.

[0052] This application provides a data processing method corresponding to the training process of a text recognition model. This method can be applied to a processing end, which can be understood as a device for training the text recognition model. This application embodiment can extract individual characters from a training text image to form a single-character image, and then train the text recognition model based on the training text image and the single-character image, thereby improving the accuracy of the text recognition model. Specifically, as shown... Figure 2AAs shown, the method includes:

[0053] Step 202: Obtain the labeled training text image. Step 204: Determine and label the single-character image. The single-character image is determined based on the training text image or through synthesis. On one hand, this embodiment can extract a single character from the training text image to obtain a single-character image. On the other hand, this embodiment can also determine the single-character image through synthesis. This embodiment can combine a single character with an image to form a single-character image, or extract character information from other text images and synthesize a single-character image. This embodiment can synthesize a single-character image without a background based on character information, or synthesize a single-character image with a background based on character information and background information. This embodiment describes a single-character image without a background as an example. Regarding the method of determining single-character images based on training text images, this embodiment of the application can input the training text image into a text recognition model. The training text image is then processed by a first feature extractor and a context modeling module within the character model to determine the first features of each character after processing. The single-character image is then determined based on these processed first features. Specifically, as an optional embodiment, determining the single-character image includes: inputting the processed first features into a generator to determine character information and then determining the single-character image. Character information may include character trajectory information, character structure, and the character's position information in the training text image. After determining the single-character image, annotations can be configured for it. For single-character images determined based on training text images, this embodiment of the application can configure annotations for the single-character image based on the annotations in the training text image. This embodiment of the application can also determine the matching character in the training text image corresponding to the single-character image based on the character's position information in the training text image, and configure annotations for the single-character image based on the annotations of the matching character. For synthesized single-character images, the embodiments of this application can determine the annotation of the single-character image by means of text recognition or manual annotation. It should be noted that the embodiments of this application can also adopt other methods to configure annotations for single-character images, and the specific configuration can be made according to the requirements.

[0054] After determining the labeled training text images and labeled single character images, in step 206, a text recognition model can be trained based on the labeled training text images. The text recognition model includes a first feature extractor, a context modeling module, and a first character classifier.

[0055] The training text image can be input into the text recognition model. A first feature extractor extracts the first features of each character in the image. A context modeling module then associates the features of each character's context, thereby determining the first feature data. Subsequently, a first character classifier classifies the characters based on the first feature data, thus determining the first prediction result. After determining the first prediction result, the deviation between the first prediction result and the annotation of the training text image can be used to determine the character model adjustment amount, and the text recognition model can be adjusted accordingly.

[0056] To improve the accuracy of text recognition model classification, this embodiment can further enhance character features to reduce the interference of background features on the recognition results. Specifically, as an optional embodiment, the context modeling module is used to: acquire the first features of each character extracted by the first feature extractor, and perform feature separation to determine character features and background features; enhance the character features and fuse the background features to obtain the processed first features; perform context association on the processed first features to obtain first feature data, so as to determine the first prediction result based on the first feature data and the first character classifier, and thereby adjust the text recognition model based on the first prediction result and the annotation of the training text image. The context modeling module can be configured with a character enhancer and an encoder to classify the character features and background features in the first features and enhance the character features. Then, the enhanced character features are fused with the background features to obtain the processed first features. The encoder can then associate the features of the characters in the context to determine the first feature data. It should be noted that, in an optional example, after feature association, this embodiment can also configure a character enhancer to further enhance the character features to obtain the first feature data.

[0057] Furthermore, the characters input into the text recognition model and the single-character recognition model are essentially the same. However, due to inaccurate feature extraction, the features extracted by the two models may differ significantly. Therefore, this application embodiment can adjust the two models based on these differences. Specifically, as an optional embodiment, the method further includes: acquiring first feature data of the target character in the training text image and second feature data in the single-character image, and inputting them into a discriminator to determine the difference information; adjusting the first feature extractor of the text recognition model based on the difference information. This application embodiment can classify the input feature data (first feature data or second feature data) to determine whether the input feature data is a single-character type or a character line type, and adjust the first feature extractor based on the judgment result. It should be noted that this application embodiment can also adjust the second feature extractor based on the judgment result.

[0058] To reduce the interference of background features in the training text image on the discrimination result, this embodiment can also adjust the discrimination result of the first feature data based on the annotation of the training text image to reduce the difference between the first feature data and the second feature data. Specifically, as an optional embodiment, determining the difference information includes: scoring the first feature data and the second feature data according to data type, wherein the data type includes single character type and character line type; adjusting the first score of the first feature data according to the annotation of the training text image to reduce the influence of background features in the first feature data on the first score; and determining the difference information based on the adjusted first score and the second score of the second feature data. This embodiment can determine the data type of the feature data by scoring. For example, a score can be preset for the data type (e.g., 0 for single character type data and 1 for character line type data), thereby determining the type based on the score. After determining the scores corresponding to the first and second feature data, this embodiment can further generate a score sequence based on the first prediction result of the training text image, and adjust the first score of the first feature data according to the score sequence (e.g., taking the average of the corresponding scores in the score sequence and the first score) to obtain the adjusted first score. Then, based on the adjusted first score and the second score of the second feature data, the difference information is determined. In this embodiment, the smaller the difference between the first and second feature data (the less the discriminator can determine the data type), the more consistent the first and second feature data are, and the higher the recognition accuracy. Therefore, this embodiment can employ adversarial learning to adjust the first feature extractor based on the discriminator's judgment result, thereby reducing the interference of the background on the character features in the training text image and improving the accuracy of the text recognition model.

[0059] For single-character images, in step 208, the first character classifier of the text recognition model can be adjusted based on the labeled single-character image to obtain a trained text recognition model. In this embodiment, a single-character image can be input into the single-character recognition model to adjust the first character classifier based on the adjustment of the single-character recognition model. Specifically, as an optional embodiment, adjusting the first character classifier of the text recognition model based on the labeled single-character image includes: adjusting the single-character recognition model based on the labeled single-character image, wherein the single-character recognition model includes a second feature extractor and a second character classifier; adjusting the second character classifier based on the character information of the single-character image to determine the single-character classification adjustment amount; and adjusting the first character classifier based on the single-character classification adjustment amount.

[0060] This application embodiment can input a single-character image into a single-character recognition model, extract second feature data of the characters in the single-character image using a second feature extractor, and classify the characters based on the second feature data using a second character separator to determine a second prediction result. Then, the adjustment amount of the single-character model can be determined based on the second prediction result and the annotation of the single-character image, thereby adjusting the single-character recognition model. Furthermore, this application embodiment can determine the single-character classification adjustment amount based on the adjustment of the second character classifier, and adjust the first character classifier based on the single-character classification adjustment amount to achieve the purpose of sharing the first character classifier and the second character classifier, thereby achieving the sharing of character information (sharing character information of characters in training text images and character information of characters in single-character images) between the first character classifier and the second character classifier.

[0061] Furthermore, to control the influence of training text images and single-character images on the first character classifier at different stages, this embodiment can also configure weight information to adjust the first character classifier based on the weight information. Specifically, as an optional embodiment, adjusting the first character classifier based on the single-character classification adjustment amount includes: determining the weight information corresponding to the single-character classification adjustment amount; and adjusting the first character classifier according to the weight information and the single-character classification adjustment amount. The weight information can be determined based on the training process of the text recognition model. As the text recognition model iterates through training, the weight information can be continuously increased to enhance the influence of single-character images on the first character classifier. In the early stages of training, the first character classifier of the text recognition model is mainly adjusted using training text images; in the later stages of training, the first character classifier of the text recognition model is mainly adjusted using single-character images, thereby improving the accuracy of the text recognition model.

[0062] In this embodiment, labeled training text images can be acquired, individual characters can be extracted from the training text images to determine single-character images, or single-character images can be determined through synthesis; and labels can be configured for the single-character images. Then, on one hand, the training text images can be input into a text recognition model. A first feature extractor extracts the first features of each character, and a context modeling module associates the features of the character's context to obtain first feature data. A first character classifier then classifies the characters based on the first feature data to determine the prediction result. The text recognition model is then adjusted based on the prediction result and the labels of the training text images to train the model. On the other hand, the first character classifier can be adjusted using labeled single-character images to obtain a trained text recognition model. This embodiment can reduce the impact of factors such as uneven character distribution in the training text images on the recognition results by using single-character images that do not contain context features, thereby improving the recognition accuracy of the text recognition model.

[0063] The following describes the data processing method of this application embodiment using a specific example. Specifically, in this application embodiment, the first feature extractor of the text recognition model can adopt a residual network (ResNet) model, such as ResNet45. The residual network is a convolutional neural network. The residual network uses skip connections through its internal residual blocks to alleviate the gradient vanishing problem caused by increasing the depth in deep neural networks. The context modeling module of the text recognition model includes a character enhancer, an encoder, and a character enhancer connected in sequence. The character enhancer may include a background separator and a style extractor. In this application embodiment, the background classifier may include two fully connected layers and a sigmoid function. Each node of the fully connected layer is connected to all nodes of the previous layer to synthesize the features extracted by the nodes of the previous layer. In this embodiment, features can be extracted through two fully connected layers and mapped to background features through the sigmoid function. The sigmoid function is often used as an activation function of neural networks to map variables to the range of 0-1. The style extractor includes two fully connected layers to extract style features.

[0064] The first character classifier of the text recognition model can be understood as a fully connected layer followed by a normalized exponential function (softmax function). Furthermore, after determining the first prediction result and the annotation of the training text image, this embodiment can determine the character model adjustment amount (or the first loss function) based on the difference between the first prediction result and the annotation of the training text image. The character model adjustment amount can be determined using the following formula 1.

[0065] L Rline=Loss(P L ,y) Formula 1

[0066] Among them, P L y refers to the first prediction result, and y refers to the annotation result of the training text image.

[0067] Specifically, the character styles in training text images are usually uniform, while the background styles are inconsistent. Therefore, embodiments of this application can separate the background and characters, and process them according to their respective styles, thereby enhancing the character features and obtaining the processed first feature. Specifically, as... Figure 2B As shown in the embodiment of this application, local background features can be separated by a background separator, and local style features can be extracted by a style extractor. Then, for the local background features, feature concatenation can be performed between them to obtain a concatenated background feature. Feature concatenation involves concatenating the vectors of the local background features to form the concatenated background feature. After determining the concatenated background features and style features, an average style feature can be obtained by weighted averaging. The average style feature can include the average background style feature and the average character style feature (or foreground style feature). In this embodiment of the application, the average style feature S can be determined using Formula 2. a .

[0068]

[0069] Among them, W fb This represents the spliced ​​background features, which are based on the local background features W. b Confirmed, S L Z represents stylistic features, while W represents the spliced ​​background features. fb Summation of T-dimensions.

[0070] After determining the average style features, a weighted average can be used to fuse the average style features and the stitched background features to obtain the global features. This weighted average method can be understood as determining the background weight corresponding to each position in the stitched background features, and then fusing the foreground style features and background style features based on these background weights to obtain the global features. Global Feature S g It can be determined using the following formula 3.

[0071] S g =W fb S a Formula 3

[0072] After determining the global features, the elements of the global features and the first feature can be added one by one to obtain the processed first feature. Element-by-element addition can be understood as adding each element of the first feature (which can be understood as a matrix) to the corresponding element of the global feature (which can be understood as a matrix), thus obtaining the processed first feature (the matrix after addition). In this embodiment, averaging can be performed on uniform character styles, and averaging can also be performed on inconsistent background features, thereby enhancing character features and weakening the influence of background features on character features.

[0073] In this embodiment, the category corresponding to the background feature can be set to the blank class (corresponding to non-character features), and the category identifier (category id) can be set to 0. Therefore, in this embodiment, the probability sequence P0 of the blank class in the prediction result of the first character classifier can be used as the background separation result W of the background separator. b The approximate value is used to determine the enhancer adjustment amount of the character enhancer, and to adjust the parameters of the character enhancer, where the enhancer adjustment amount L is... loc It can be determined using the following formula 4.

[0074]

[0075] In this embodiment, the context modeling module has two character enhancers, and the processing results (processed character feature sequences) obtained by the two character classifiers are W, respectively. b (1) and W b (2) Then, the total localization loss function of the context modeling module can be determined by the following formula 5.

[0076] L loc =l loc (p0, W) b (1) )+l loc (p0, W) b (2) ) Formula 5

[0077] For a single-character recognition model, the model is used to classify single-character images. The input to the single-character recognition model is a uniformly distributed single-character image. The second feature extractor of the single-character recognition model can employ 41 convolutional layers, each extracting different features. The second character classifier can determine a second prediction result based on the second feature data extracted by the second feature extractor, and determine the single-character model adjustment amount (or second loss function) based on the second prediction result and the annotation of the single-character image. In this embodiment, a cross-entropy loss function can be used. Cross-entropy is an important concept in information theory, mainly used to measure the difference between two probability distributions. The cross-entropy loss function L... Rchar It can be determined according to the following formula 6.

[0078] L Rchar = -log P C y c Formula 6

[0079] Among them, P C This refers to the second prediction result, y c It refers to a single-character tag.

[0080] This application embodiment can share the first character classifier of the text recognition model and the second character classifier of the single character recognition model. The premise of sharing two classifiers is that the features extracted by the two models are highly correlated. However, without constraints, the extracted features may have poor correlation, or even be orthogonal (unrelated). Therefore, this application embodiment can introduce adversarial learning, using a discriminator to determine the differences between two features, thereby enhancing the correlation between the features extracted by the two models. Specifically, the discriminator can include two fully connected layers. The discriminator takes the extracted features as input and outputs a score indicating its judgment of which branch (text line learning branch or single character learning branch) the input feature comes from. In this embodiment, the labels for text lines and single characters are represented as 1 and 0, respectively. After passing through the discriminator, the second feature data in the single character recognition model is mapped to a score (second score), and the first feature data (sequence) of the text recognition model is mapped to a score sequence (sequence of the first score). In this application, the single-character recognition model does not include background features, while the text recognition model may extract features that include background features. Therefore, to reduce the influence of background features on the extracted features in the text recognition model, this embodiment can generate a character mask based on the first prediction result of the text recognition model, and adjust the discrimination result of the discriminator based on the character mask. The character mask can be represented by the following formula 7.

[0081]

[0082] Among them, P L This refers to the first prediction result, where 0 represents the category ID of the blank class (non-character class). In this embodiment, the prediction results of categories with characters greater than 0 can be filtered out from the prediction results, and the corresponding character mask M (score sequence) is determined. Then, the discrimination results (first score) of the discriminator are weighted and averaged using the character mask M as the weight to obtain the adjusted first score. The adjusted first score can be determined by the following formula 8.

[0083]

[0084] Among them, s l "m" refers to the first score after processing, and "m" refers to the character mask.

[0085] After determining the adjusted first score, the difference information can be determined based on the adjusted first score and second score, and the feature extractors (first feature extractor and / or second feature extractor) can be adjusted according to the difference information.

[0086] In this embodiment of the application, the adjustment amount (or binarized cross-entropy loss function L) for the feature extractor in the model can be determined by the following formula 9. D In this embodiment, the discriminator has poor discrimination ability in the initial stage. Therefore, after adjusting the discriminator with some data, the adjusted discriminator can be used to discriminate the feature data. The gradient of the parameter can be obtained by the following formula 9, and based on the corresponding gradient, it can be determined whether the discriminator can make a discrimination.

[0087] L D = -log S L -log(1-S C ) Formula 9

[0088] Among them, S L S refers to the sequence of first fractions. C It refers to the second fraction.

[0089] In this embodiment of the application, feature data (first feature data and second feature data) can be input into a discriminator, and a score (first score and second score) can be determined by the discriminator. Then, based on the first score and the second score, the adjustment amount for the feature extractor (first feature extractor and second feature extractor) can be determined, thereby adjusting the feature extractor. The adjustment amount for the first feature extractor can be determined by the following formula 10, and the adjustment amount for the second feature extractor can be determined by the following formula 11.

[0090] L Gline = -log(1-S L) Formula 10

[0091] L Gchar = -log S C Formula 11

[0092] In summary, the embodiments of this application use training text images to train the text recognition model and single character images to train the single character recognition model. The adjustment amount of the character model of the text recognition model can be represented by the following formula 12, and the adjustment amount of the single character model of the single character recognition model can be represented by the following formula 13.

[0093] L line =L Rline +α L L Gline +γL loc Formula 12

[0094] L char =L Rchar +α C L Gchar Formula 13

[0095] in,

[0096] α L =min(|S L -S C |+α L0 , 1.0)

[0097] α c =min(|S L -S C |+α C0 , 1.0)

[0098] After training is complete, this embodiment uses the trained text recognition model to recognize text images. Therefore, this embodiment mainly utilizes training text images, single-character images, and a discriminator to train the text recognition model. Correspondingly, from this perspective, the total adjustment amount L of the text recognition model... obj This can be expressed by the following formula 14.

[0099] L obj =L line +βL char +L D Formula 14

[0100] In the early stages of training, the first feature extractor and context modeling module of the text recognition model are adjusted more often by training text images. In the later stages of training, the first character classifier is adjusted more often by training single character images. Therefore, this embodiment sets a coefficient β as weight information to control the influence of training text images and single character images on the text recognition model at different stages based on the weight information. The weight information can be represented by the following formula 15.

[0101]

[0102] Where t is the iteration number of the current training step, T0 is the starting iteration step, T1 is the stopping iteration step, and B0 and B1 are the lower and upper bounds of β, respectively. After the text recognition model is trained, it can be used to recognize the text image to be recognized, determine the recognition result, and output it.

[0103] Based on the above embodiments, this application also provides a data processing method that can be applied to a processing end, such as... Figure 3 As shown, the method includes:

[0104] Step 302: Obtain the labeled training text image and extract individual characters from the training text image to obtain a single-character image. As an optional embodiment, the processed first feature is input into the generator to determine character information and thus the single-character image.

[0105] Step 304: Configure annotations for single-character images based on the annotations of the training text images. It should be noted that this embodiment describes the determination of single-character images based on training text images as an example. This embodiment can also determine single-character images in other ways; for example, in this application embodiment, single-character images can be synthesized through a synthesis method.

[0106] Step 306: Based on the labeled training text images, train a text recognition model. The text recognition model includes a first feature extractor, a context modeling module, and a first character classifier. As an optional embodiment, the context modeling module is used to: acquire the first features of each character extracted by the first feature extractor, and perform feature separation to determine character features and background features; enhance the character features and fuse them with background features to obtain processed first features; perform context association on the processed first features to obtain first feature data, so as to determine the prediction result based on the first feature data and the first character classifier, and thus adjust the text recognition model based on the prediction result and the annotations of the training text images. This application embodiment can also adjust the first feature extractor based on the difference between the first feature data and the second feature data. As an optional embodiment, the method further includes: acquiring the first feature data of the target character in the training text image and the second feature data in the single character image, and inputting them into a discriminator; scoring the first feature data and the second feature data according to data type, wherein the data type includes single character type and character line type; adjusting the first score of the first feature data according to the annotation of the training text image to reduce the influence of background features in the first feature data on the first score; determining the difference information based on the adjusted first score and the second score of the second feature data; and adjusting the first feature extractor of the text recognition model based on the difference information.

[0107] Step 308: Adjust the single-character recognition model based on the labeled single-character image. The single-character recognition model includes a second feature extractor and a second character classifier.

[0108] Step 310: Adjust the second character classifier based on the character information of the single character image to determine the single character classification adjustment amount.

[0109] Step 312: Determine the weight information corresponding to the single character classification adjustment amount.

[0110] Step 314: Adjust the first character classifier according to the weight information and single character classification adjustment amount to obtain the trained text recognition model.

[0111] In this embodiment, labeled training text images can be acquired, individual characters can be extracted from the training text images to determine single-character images, and annotations can be configured for the single-character images based on the annotations of the training text images. Then, on one hand, the training text images can be input into a text recognition model. A first feature extractor extracts the first features of each character, and a context modeling module associates the features of the character's context to obtain first feature data. A first character classifier classifies the characters based on the first feature data to determine the prediction result. Then, the text recognition model is adjusted based on the prediction result and the annotations of the training text images to train the text recognition model. On the other hand, single-character images can be input into a single-character recognition model. A second feature extractor extracts second feature data, and a second character classifier classifies the characters based on the second feature data to determine the second prediction result. Based on the second prediction result and the annotations of the single-character images, the single-character model adjustment amount is determined, and the single-character recognition model is adjusted based on the single-character model adjustment amount. Specifically, the single-character classification adjustment amount and corresponding weight information can be determined based on the adjustment of the second character classifier. Then, the first character classifier is adjusted based on the single-character classification adjustment amount and the weight information to obtain a trained text recognition model. In order to recognize the text image to be recognized based on the trained text recognition model.

[0112] Based on the above embodiments, this application also provides a data processing method corresponding to the text image recognition stage. This method can be applied to a processing end, which can be understood as a device that performs character recognition on text image data based on a pre-trained text recognition model. This method can acquire the text image data to be analyzed and perform recognition to determine the recognition result. Specifically, as shown... Figure 4A As shown, the method includes:

[0113] Step 402: Obtain the text image data to be recognized. The text image data may contain one character or multiple characters. After the text recognition model is trained, this embodiment can recognize the text image data to be recognized based on the text recognition model. This embodiment can configure the corresponding text image data acquisition method according to the specific scenario. Specifically, as an optional embodiment, obtaining the image data to be recognized includes at least one of the following steps: obtaining text image data related to the target information; providing a data upload interface to receive text image data based on the data upload interface; and providing an interactive page to obtain text image data based on the interactive page. This embodiment can set target information to obtain text image data related to the target information and perform recognition. This embodiment can also provide a data upload interface to obtain text image data, and this embodiment can also provide an interactive page so that users can upload text image data based on the interactive page.

[0114] After obtaining the text image data to be analyzed, in step 404, the text image data can be recognized according to the pre-trained text recognition model to determine the recognition result. The trained text recognition model shares character information with the single-character recognition model. In step 406, each text in the recognition result is output for display. It should be noted that in this embodiment, single-character images can be determined by synthesis. This embodiment only describes the determination of single-character images by training text images. The trained text recognition model is trained based on training text images during the training phase. During the training phase, it shares a character classifier with the single-character recognition model to assist in the training of the text recognition model based on the single-character recognition model. The single-character recognition model is trained based on single-character images.

[0115] This application embodiment can train a text recognition model based on single-character images and training text images, which can reduce the impact of factors such as the uneven distribution of characters in the training text images on the recognition results, thereby improving the recognition accuracy of the text recognition model. Specifically, as an optional embodiment, the method further includes the step of training the text recognition model: obtaining labeled training text images and extracting single characters from the training text images to obtain single-character images; configuring labels for the single-character images according to the labels of the training text images; training the text recognition model based on the labeled training text images; adjusting the single-character recognition model based on the labeled single-character images, wherein the single-character recognition model includes a second feature extractor and a second character classifier; adjusting the second character classifier based on the character information of the single-character images to determine the single-character classification adjustment amount, and adjusting the first character classifier based on the single-character classification adjustment amount. As an optional embodiment, the text recognition model includes: a first feature extractor for extracting first features from text image data; a context modeling module for performing feature separation on the first features, determining character features and background features, enhancing the character features, and fusing the background features to obtain processed first features; performing context association on the processed first features to obtain first feature data; and a first character classifier for classifying characters based on the first feature data to determine the recognition result. The training process of the text recognition model in this embodiment is similar to that in the above embodiments. Specific implementation details can be found in the training process of the text recognition model in the above embodiments, and will not be repeated here.

[0116] This application embodiment can also adjust single-character images manually to determine more accurate training data. Specifically, as an optional embodiment, the method further includes: providing an interactive page to display training text images and arranged single-character images; obtaining character adjustment instructions based on the interactive page, and adjusting the training text images and / or single-character images to obtain adjusted images. In this application embodiment, single-character images are determined by extracting individual characters from training text images. There may be a mismatch between single-character images and training text images. Therefore, this application embodiment can display training text images and arranged single-character images to the user and receive corresponding character adjustment instructions to adjust the training text images or single-character images. In this application embodiment, the user can adjust the order of characters, re-encode characters, delete characters, or mark incorrect characters.

[0117] This application embodiment can also be applied to scenarios where target objects are searched based on text images. Specifically, as an optional embodiment, the text image data package contains the name of the target object, and the method further includes: searching for relevant data of the target object based on the name of the target object in the recognition result, and outputting the data. This application embodiment can be applied to scenarios where target objects are searched. The target object can be a company name, place name, person name, item name, etc. Relevant data can be searched based on the corresponding text image data. For example, this embodiment can recognize text image data containing a company name, determine the recognition result, and search for relevant data of the company based on the company name in the recognition result, and display the searched relevant data to the user.

[0118] This application embodiment can also adjust the recognition results manually, thereby adjusting the text recognition model to improve its accuracy. Specifically, as an optional embodiment, the method further includes: receiving a result adjustment instruction for the recognition result; providing an editing control to obtain text adjustment operations based on the editing control; the text adjustment operations include at least one of position adjustment, text re-editing, text deletion, text addition, and text error marking; adjusting the recognition result according to the text adjustment operations; and adjusting the text recognition model according to the adjusted recognition result. This application embodiment can provide an interactive page to display various texts in the recognition results to the user. The user can adjust the texts in the recognition results on the interactive page. When the user makes adjustments, this application embodiment can configure editing controls on the interactive page. The editing controls can be understood as editing areas. The recognition results can be displayed in the editing areas, and the user can operate on the text in the editing areas to adjust the recognition results. Specifically, as an optional embodiment, the text adjustment operation includes: determining a position adjustment operation for the target text based on the operation of selecting and dragging the target text in the editing controls; determining a text re-editing operation for the target text based on the operation of selecting and re-editing the target text in the editing controls; determining a text deletion operation for the target text based on the operation of selecting and deleting the target text in the editing controls; determining a text addition operation for the target text based on the operation of selecting the target position and adding the target text in the editing controls; and determining a text error marking operation for the target text based on the operation of selecting the target text and adding an error mark in the editing controls. This application is applied to the processing end, which can interact with the terminal to receive operations from the terminal user on the interactive page, such as selecting and dragging target text, selecting and re-editing target text, selecting and deleting target text, selecting a target position and adding target text at the target position, selecting target text and adding an error mark, etc., to form corresponding text adjustment operations, and then adjusting the recognition results based on the text adjustment operations.

[0119] This application embodiment has a good effect on the recognition of rare characters. However, some rare characters are not commonly used, and users are not familiar with them. Therefore, in order to facilitate users' understanding of rare characters, this application embodiment can provide explanations for rare characters. Specifically, as an optional embodiment, the output of each text in the recognition result includes: classifying each text in the recognition result to determine rare texts and common texts; obtaining rare text explanation information for rare texts, wherein the rare text explanation information includes at least one of annotation information and phonetic information; and outputting common texts, rare texts, and rare text explanation information. Figure 4B As shown in the embodiment of this application, a table of uncommon texts can be pre-set on the server side. The server obtains text image data from the terminal and performs recognition using a trained text recognition model. After obtaining the recognition result, the text in the recognition result can be classified according to the uncommon text table and divided into uncommon texts (such as...). Figure 4B (in the text) and common texts (such as...) Figure 4B (The text includes characters such as "Hangzhou", "Gong", and "Si" in the text). This allows the acquisition of descriptive information about the uncommon text, such as phonetic transcription (e.g., Pinyin) or annotations (e.g., explanations of the text), which is then transmitted to the terminal for display. Users can understand the uncommon text based on the explanations, facilitating adjustments and confirmation of the recognition results.

[0120] To facilitate user adjustments to the recognition results, this embodiment can display recognition results with different confidence levels in different display methods. This allows users to easily determine characters with a certain confidence level and modify characters with low confidence levels. Specifically, as an optional embodiment, the method further includes: obtaining the recognition confidence level of each character's recognition result and determining the corresponding confidence level of the character's recognition result based on the recognition confidence level; determining the display method corresponding to the character's recognition result based on the confidence level, so that the character's recognition result is displayed in the interactive page according to the display method, facilitating user adjustments to the recognition results. This embodiment can pre-set a confidence threshold corresponding to the confidence level, determine the corresponding confidence level of the recognition result based on the confidence threshold and the recognition confidence level of the recognition result, and then determine the corresponding display method and issue the display method so that the recognition result is displayed in the interactive page according to the display method. For recognition results with low confidence levels, this embodiment can highlight them, thereby facilitating user adjustments and confirmation of the recognition results.

[0121] Furthermore, for recognition results with low confidence, this embodiment can also interact with humans through questioning, thereby improving the accuracy of the recognition results. Specifically, as an optional embodiment, the recognition confidence of each character's recognition result is obtained; when the recognition confidence of the recognition result is lower than a preset confidence threshold, the question information corresponding to the recognition result is output, so as to determine the recognition result based on the response information corresponding to the question information. This embodiment can compare the recognition confidence with a preset confidence threshold to determine whether the recognition result is credible. For recognition results with low confidence, this embodiment can generate question information, such as extracting the corresponding text image data to generate the question information: Is the text "Hangzhou weather 32 degrees"?, thereby obtaining the corresponding response information. Users can confirm the question information or adjust the recognition result in the question information to form response information, thereby improving the accuracy of the recognition result.

[0122] This application embodiment can also receive training data uploaded by users and train a personalized text recognition model. Specifically, as an optional embodiment, the method further includes: acquiring target training data in the target domain, the target training data including target training text images and annotation data; extracting a single character from the target training text image to determine a target single-character image, and configuring annotations for the target single-character image based on the annotation data; and training a target text recognition model based on the annotated target training text image and the annotated target single-character image, so as to recognize text image data in the target domain based on the target text recognition model. Users can upload target training text images related to the target domain. The target domain can be education, medical care, e-commerce, conferences, finance, etc., or other fields or sub-fields, which can be configured according to needs. This embodiment can receive target training data, determine target single-character images based on the target training data, and configure corresponding annotations. Then, based on the annotated target training text image and the annotated target single-character image, a target text recognition model is trained, so as to recognize text image data in the target domain based on the target text recognition model.

[0123] The implementation method of this embodiment is similar to that of the above embodiments. For specific implementation methods, please refer to the specific implementation methods of the above embodiments, which will not be repeated here.

[0124] In this embodiment, before recognizing the text image data, the embodiment can determine single-character images and corresponding annotations based on labeled training text images. A text recognition model is then trained based on the labeled training text images and labeled single-character images. After the text recognition model is trained, the text image data to be recognized can be obtained and input into the pre-trained text recognition model to extract the features of each character in the text image data and associate the features of the context characters of each character, thereby performing character classification and determining the recognition result. This embodiment can train the text recognition model using training text images and single-character images, thereby utilizing the evenly distributed single characters in the single-character images to reduce the influence of factors such as the uneven distribution of characters in the training text images on the recognition result, thus improving the recognition accuracy of the text recognition model.

[0125] This application also provides a data processing method that can be applied to a terminal. The terminal can be understood as a device that uploads a text image to be analyzed and receives the recognition result. The terminal can interact with a server to upload the text image to be analyzed. The server can be understood as a device that recognizes the text image based on a pre-trained text recognition model. After determining the recognition result of the text image, the server can feed back the recognition result to the terminal. Specifically, as shown... Figure 5 As shown, the method includes:

[0126] Step 502: Provide an interactive page, which includes a data upload control.

[0127] Step 504: Based on the triggering of the data upload control, acquire the text image data to be recognized and upload it to the server so that the server can recognize the text image data according to the pre-trained text recognition model and determine the recognition result. The trained text recognition model and the single-character recognition model share character information. Specifically, the trained text recognition model is trained based on training text images during the training phase. During the training phase, it shares a character classifier with the single-character recognition model to assist in the training of the text recognition model. The single-character recognition model is trained based on single-character images.

[0128] Step 506: Receive and display the recognition results.

[0129] The implementation method of this embodiment is similar to that of the above embodiments. For specific implementation methods, please refer to the specific implementation methods of the above embodiments, which will not be repeated here.

[0130] In this embodiment, the server can provide an interactive page to the terminal to offer text image recognition services. Users can trigger a data upload control on the interactive page to upload the text image data to be recognized to the server. The server can obtain the text image data and input it into a pre-trained text recognition model to extract features of each character in the text image data and associate them with the features of the context characters, thereby performing character classification and determining the recognition result. After determining the recognition result, the server can feed it back to the terminal for display.

[0131] This application also provides a data processing method that can be applied to scenarios involving the recognition of text images containing ancient characters to determine the recognition result. Specifically, for example... Figure 6 As shown, the method includes:

[0132] Step 602: Obtain the image of the ancient character to be identified.

[0133] Step 604: Recognize the ancient character image based on the pre-trained text recognition model and determine the recognition result. The pre-trained text recognition model and the single-character recognition model share character information. Specifically, the pre-trained text recognition model is trained on training text images during the training phase. During the training phase, it shares a character classifier with the single-character recognition model to assist in the training of the text recognition model. The single-character recognition model is trained on single-character images.

[0134] Step 606: Output the recognition results for display. As an optional embodiment, step 606 includes: obtaining text description information for each text in the recognition results, wherein the text description information includes at least one of annotation information and phonetic information. Output the text in the recognition results and the corresponding text description information for display.

[0135] This application demonstrates good performance in recognizing rare characters, thus it can be applied to ancient character recognition scenarios. However, ancient characters are often uncommon, and users may lack sufficient understanding of them. Therefore, to facilitate user understanding of ancient characters, this application provides explanations such as phonetic transcriptions or annotations (e.g., explanations of the text, its corresponding modern text, etc.) to aid user comprehension. Additionally, in an optional embodiment, this application can further categorize the ancient character texts in the recognition results. For example, a rare text table can be pre-configured to classify the ancient character texts in the recognition results into rare and common texts. This application can obtain and output the textual descriptions of rare texts for display, thereby reducing the amount of text requiring explanation and improving data processing efficiency.

[0136] The implementation method of this embodiment is similar to that of the above embodiments. For specific implementation methods, please refer to the specific implementation methods of the above embodiments, which will not be repeated here.

[0137] In this embodiment, an image containing ancient characters refers to an image containing ancient characters. Ancient characters refer to characters used before the existence of modern characters, such as ancient Chinese characters. Ancient Chinese characters can be understood as the Chinese characters used in China before the emergence of simplified characters. Simplified characters are the officially published simplified characters, usually referring to the characters in the "General List of Simplified Characters" published in 1956. Ancient Chinese characters can be understood as the Chinese characters used in China before the simplification of characters, and may include original ancient Chinese characters, traditional characters, oracle bone script, bronze script, Warring States script, Qin script, and ancient clerical script during the transition from Qin to early Han dynasty. Images containing ancient Chinese characters can be images related to ancient medicine, ancient education, and historical records. In this embodiment, images containing ancient Chinese characters can be identified by photographing books containing ancient Chinese characters, and the images can be input into a pre-trained text recognition model to determine the recognition result. After determining the recognition result, this embodiment can also convert the ancient characters based on the recognition result, such as converting ancient characters into modern characters. It should be noted that, in addition to the one-to-one correspondence conversion method, this embodiment can also perform semantic translation. Furthermore, embodiments of this application can also translate the translated modern text into text in other languages, which can be configured according to requirements.

[0138] This application also provides a data processing method applicable to text-image-based search scenarios. The method can recognize text images, search for relevant information based on the recognition results, and provide the relevant information back to the user. Specifically, the method includes:

[0139] Provide an interactive page to obtain the text image data to be recognized based on the interactive page.

[0140] The text image data is recognized based on a pre-trained text recognition model to determine the recognition result. The pre-trained text recognition model shares character information with the single-character recognition model. Specifically, the pre-trained text recognition model is trained on training text images during the training phase. During the training phase, it shares a character classifier with the single-character recognition model to assist in the training of the text recognition model. The single-character recognition model is trained on single-character images.

[0141] The system performs a search based on the identification results and returns the search results, which are then displayed on the interactive page.

[0142] The implementation method of this embodiment is similar to that of the above embodiments. For specific implementation methods, please refer to the specific implementation methods of the above embodiments, which will not be repeated here.

[0143] In this embodiment, an interactive page can be provided. Users upload text image data through the interactive page. After receiving the text image data, a pre-trained text recognition model can be used to recognize the text image data and determine the recognition result. Then, users can search based on the recognition result to find relevant information, which is then displayed on the interactive page. This embodiment can be applied to image-based search scenarios, such as searching based on text images containing company names. Based on the company name in the recognition result, relevant company information can be searched and displayed to the user.

[0144] This application also provides a data processing method that can be applied in educational scenarios. It can recognize text images containing education-related information based on a trained text recognition model to determine the recognition result. Specifically, the method includes:

[0145] Obtain the educational text image to be identified.

[0146] The educational text image is recognized based on a pre-trained text recognition model to determine the recognition result. The pre-trained text recognition model shares character information with the single-character recognition model. Specifically, the pre-trained text recognition model is trained on training text images during the training phase. During the training phase, it shares a character classifier with the single-character recognition model to assist in the training of the text recognition model. The single-character recognition model is trained on single-character images.

[0147] Output each text from the recognition results for display.

[0148] This application embodiment can display recognition results with different recognition confidence levels in different display methods, thereby facilitating users to modify characters with low confidence levels. Specifically, as an optional embodiment, the method further includes: obtaining the recognition confidence level of each character's recognition result, and determining the corresponding confidence level of the character's recognition result based on the recognition confidence level; determining the display method corresponding to the character's recognition result based on the confidence level, so that the character's recognition result is displayed in the interactive page according to the display method, facilitating users to adjust the recognition result. This application embodiment can pre-set a confidence threshold corresponding to the confidence level, so as to determine the corresponding confidence level of the recognition result based on the confidence threshold and the recognition confidence level of the recognition result, thereby determining the corresponding display method, and issuing the display method so that the recognition result is displayed in the interactive page according to the display method. For recognition results with low confidence levels, this application embodiment can highlight them, thereby facilitating users to adjust and confirm the recognition result.

[0149] Furthermore, for recognition results with low confidence, this embodiment of the application can also interact with humans through questioning, thereby improving the accuracy of the recognition results. Specifically, as an optional embodiment, the recognition confidence of each character's recognition result is obtained; when the recognition confidence of the recognition result is lower than a preset confidence threshold, the question information corresponding to the recognition result is output, so as to determine the recognition result based on the response information corresponding to the question information. This embodiment of the application can compare the recognition confidence with a preset confidence threshold to determine whether the recognition result is credible. For recognition results with low confidence, this embodiment of the application can generate question information, which the user can confirm, or adjust the recognition result in the question information to form response information, thereby improving the accuracy of the recognition result.

[0150] The implementation method of this embodiment is similar to that of the above embodiments. For specific implementation methods, please refer to the specific implementation methods of the above embodiments, which will not be repeated here.

[0151] In this embodiment, educational text images can be understood as images containing educational information, such as student answers, student registration information, information from books, and student homework. This embodiment can acquire educational text images, recognize them using a pre-trained text recognition model, determine the recognition result, and output it for display. Furthermore, appropriate processing can be performed based on the recognition result. For example, if the educational information in the educational text image is student answers, this embodiment can determine the student's corresponding score based on the recognition result.

[0152] This application also provides a data processing method applicable to e-commerce scenarios. It uses a trained text recognition model to recognize text images containing product-related information to determine the recognition result. Specifically, the method includes:

[0153] Obtain the product text image to be identified.

[0154] The product text image is recognized based on a pre-trained text recognition model to determine the recognition result. The text recognition model and the single-character recognition model share character information. Specifically, the pre-trained text recognition model is trained on training text images during the training phase. During the training phase, it shares a character classifier with the single-character recognition model to assist in the training of the text recognition model. The single-character recognition model is trained on single-character images.

[0155] Output each text from the recognition results for display.

[0156] The implementation method of this embodiment is similar to that of the above embodiments. For specific implementation methods, please refer to the specific implementation methods of the above embodiments, which will not be repeated here.

[0157] In this embodiment, the product text image can be understood as an image containing product-related information, such as product name, production date, product description, and promotional materials. This embodiment can acquire a product text image, recognize it using a pre-trained text recognition model, determine the recognition result, and output it for display. Furthermore, appropriate processing can be performed based on the recognition result. For example, if the recognition result is a product name, a search for the corresponding product can be conducted based on the product name, and the search results can be displayed to the user.

[0158] This application also provides a data processing method applicable to medical scenarios. It uses a trained text recognition model to recognize text images containing medical information to determine the recognition result. Specifically, the method includes:

[0159] Obtain the medical text image to be recognized.

[0160] A pre-trained text recognition model is used to recognize medical text images and determine the recognition result. The text recognition model shares character information with the single-character recognition model. Specifically, the pre-trained text recognition model is trained on training text images during the training phase. During the training phase, it shares a character classifier with the single-character recognition model to assist in the training of the text recognition model. The single-character recognition model is trained on single-character images.

[0161] Output each text from the recognition results for display.

[0162] The implementation method of this embodiment is similar to that of the above embodiments. For specific implementation methods, please refer to the specific implementation methods of the above embodiments, which will not be repeated here.

[0163] In this embodiment, a medical text image can be understood as an image containing medical-related information, such as medical records, medical documents, electronic medical records, and medical image text. This embodiment can acquire a medical text image, recognize it using a pre-trained text recognition model, determine the recognition result, and output it for display. Furthermore, appropriate processing can be performed based on the recognition result. For example, if the recognition result is a disease name, relevant information about that disease can be searched, and the search results can be displayed to the user.

[0164] This application also provides a data processing method that can be applied in a meeting scenario. It can recognize text images containing meeting-related information based on a trained text recognition model to determine the recognition result. Specifically, the method includes:

[0165] Obtain the image of the meeting text to be identified;

[0166] The meeting text image is recognized based on a pre-trained text recognition model to determine the recognition result. The text recognition model and the single-character recognition model share character information. Specifically, the pre-trained text recognition model is trained on training text images during the training phase. During the training phase, it shares a character classifier with the single-character recognition model to assist in the training of the text recognition model. The single-character recognition model is trained on single-character images.

[0167] Output each text from the recognition results for display.

[0168] The implementation method of this embodiment is similar to that of the above embodiments. For specific implementation methods, please refer to the specific implementation methods of the above embodiments, which will not be repeated here.

[0169] In this embodiment, the meeting text image can be understood as an image containing medical-related information, such as meeting content. This embodiment can acquire the meeting text image, recognize it using a pre-trained text recognition model, determine the recognition result, and output it for display. Furthermore, corresponding processing can be performed based on the recognition result. For example, the meeting content can be recorded based on the recognition result, and related meetings can be associated.

[0170] This application also provides a data processing method applicable to traffic scenarios. Based on a trained text recognition model, it identifies text images containing traffic-related information to determine the recognition result. Specifically, the method includes:

[0171] Obtain the traffic text image to be identified.

[0172] Traffic text images are recognized using a pre-trained text recognition model to determine the recognition result. The pre-trained text recognition model shares character information with the single-character recognition model. Specifically, the pre-trained text recognition model is trained using training text images during the training phase. During this phase, it shares a character classifier with the single-character recognition model to assist in the training of the text recognition model. The single-character recognition model is trained using single-character images.

[0173] Output each text from the recognition results for display.

[0174] The implementation method of this embodiment is similar to that of the above embodiments. For specific implementation methods, please refer to the specific implementation methods of the above embodiments, which will not be repeated here.

[0175] In this embodiment, a traffic text image can be understood as an image containing traffic-related information, such as license plate information, billboard information, etc. This embodiment can acquire a traffic text image, recognize it using a pre-trained text recognition model, determine the recognition result, and output it for display. Furthermore, corresponding processing can be performed based on the recognition result. For example, traffic violations can be recorded based on the license plate information in the recognition result.

[0176] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of this application.

[0177] Based on the above embodiments, this embodiment also provides a data processing apparatus, referring to... Figure 7 Specifically, it can include the following modules:

[0178] The text image acquisition module 702 is used to acquire text image data to be recognized;

[0179] The text image recognition module 704 is used to recognize text image data based on a pre-trained text recognition model and determine the recognition result. The text recognition model shares character information with the single-character recognition model. It should be noted that in this embodiment, single-character images can be determined through synthesis; this embodiment only describes the determination of single-character images by training text images. Specifically, the trained text recognition model is trained based on training text images during the training phase. During the training phase, it shares a character classifier with the single-character recognition model to assist in the training of the text recognition model. The single-character recognition model is trained based on single-character images.

[0180] The recognition result output module 706 is used to output the text in the recognition result for display.

[0181] In summary, in this embodiment of the application, before recognizing the text image data, the embodiment can determine single-character images and corresponding annotations based on the labeled training text images. A text recognition model is then trained based on the labeled training text images and labeled single-character images. After the text recognition model is trained, the text image data to be recognized can be obtained and input into the pre-trained text recognition model to extract the features of each character in the text image data and associate the features of the context characters of each character, thereby performing character classification and determining the recognition result. This embodiment of the application can train the text recognition model using training text images and single-character images, thereby utilizing the evenly distributed single characters in the single-character images to reduce the influence of factors such as the uneven distribution of characters in the training text images on the recognition result, thus improving the recognition accuracy of the text recognition model.

[0182] Based on the above embodiments, this embodiment also provides a data processing apparatus, referring to... Figure 8 Specifically, it can include the following modules:

[0183] The single-character image determination module 802 is used to obtain the labeled training text image.

[0184] The single-character image annotation module 804 is used to determine and annotate single-character images, wherein the single-character images are determined based on training text images or through synthesis.

[0185] The training text image training module 806 is used to train a text recognition model based on the labeled training text images. The text recognition model includes a first feature extractor, a context modeling module, and a first character classifier.

[0186] The single-character image training module 808 is used to adjust the first character classifier of the text recognition model based on the labeled single-character images to obtain the trained text recognition model.

[0187] In summary, in this embodiment, labeled training text images can be obtained, individual characters can be extracted from the training text images to determine single-character images, or single-character images can be synthesized; and labels can be configured for the single-character images. Then, on one hand, the training text images can be input into a text recognition model. The first feature extractor extracts the first features of each character, and the context modeling module associates the features of the character's context to obtain first feature data. The first character classifier then classifies the characters based on the first feature data to determine the prediction result. The text recognition model is then adjusted based on the prediction result and the labels of the training text images to train the model. On the other hand, the first character classifier can be adjusted using labeled single-character images to obtain a trained text recognition model. This embodiment can reduce the impact of uneven character distribution in the training text images on the recognition results by using single-character images that do not contain context features, thereby improving the recognition accuracy of the text recognition model.

[0188] Based on the above embodiments, this embodiment also provides a data processing device, which may specifically include the following modules:

[0189] The single-character image acquisition and processing module is used to acquire labeled training text images and extract individual characters from the training text images to obtain single-character images. As an optional embodiment, the processed first feature is input into the generator to determine character information and thus the single-character image.

[0190] The single-character annotation acquisition and processing module is used to configure annotations for single-character images based on the annotations of the training text images.

[0191] A training text image processing module is used to train a text recognition model based on labeled training text images. The text recognition model includes a first feature extractor, a context modeling module, and a first character classifier. As an optional embodiment, the context modeling module is used to: acquire the first features of each character extracted by the first feature extractor, and perform feature separation to determine character features and background features; enhance the character features and fuse them with background features to obtain processed first features; perform context association on the processed first features to obtain first feature data, so as to determine the prediction result based on the first feature data and the first character classifier, thereby adjusting the text recognition model based on the prediction result and the annotations of the training text images. This application embodiment can also adjust the first feature extractor based on the difference between the first feature data and the second feature data. As an optional embodiment, the device further includes: a discriminator analysis and processing module, used to acquire the first feature data of the target character in the training text image and the second feature data in the single character image, and input them into the discriminator to score the first feature data and the second feature data according to data type, wherein the data type includes single character type and character line type; a first score adjustment processing module, used to adjust the first score of the first feature data according to the annotation of the training text image to reduce the influence of background features in the first feature data on the first score; and a difference information acquisition and processing module, used to determine difference information based on the adjusted first score and the second score of the second feature data. Based on the difference information, the first feature extractor of the text recognition model is adjusted.

[0192] The single-character image training and processing module is used to adjust the single-character recognition model based on the labeled single-character images. The single-character recognition model includes a second feature extractor and a second character classifier.

[0193] The single-classification adjustment amount acquisition and processing module is used to determine the single-character classification adjustment amount based on the adjustment of the second character classifier.

[0194] The single-class weight acquisition and processing module is used to determine the weight information corresponding to the single-character classification adjustment amount.

[0195] The first character classifier adjustment module is used to adjust the first character classifier according to the weight information and the single character classification adjustment amount to obtain the trained text recognition model.

[0196] In this embodiment, labeled training text images can be acquired, individual characters can be extracted from the training text images to determine single-character images, and annotations can be configured for the single-character images based on the annotations of the training text images. Then, on one hand, the training text images can be input into a text recognition model. A first feature extractor extracts the first features of each character, and a context modeling module associates the features of the character's context to obtain first feature data. A first character classifier classifies the characters based on the first feature data to determine the prediction result. Then, the text recognition model is adjusted based on the prediction result and the annotations of the training text images to train the text recognition model. On the other hand, single-character images can be input into a single-character recognition model. A second feature extractor extracts second feature data, and a second character classifier classifies the characters based on the second feature data to determine the second prediction result. Based on the second prediction result and the annotations of the single-character images, the single-character model adjustment amount is determined, and the single-character recognition model is adjusted based on the single-character model adjustment amount. Specifically, the single-character classification adjustment amount and corresponding weight information can be determined based on the adjustment of the second character classifier. Then, the first character classifier is adjusted based on the single-character classification adjustment amount and the weight information to obtain a trained text recognition model. In order to recognize the text image to be recognized based on the trained text recognition model.

[0197] Based on the above embodiments, this embodiment also provides a data processing device, such as... Figure 9 As shown, it can specifically include the following modules:

[0198] The interactive page providing module 902 is used to provide an interactive page, which includes a data upload control.

[0199] The text image upload module 904 is used to acquire text image data to be recognized based on the triggering of the data upload control, and upload it to the server so that the server can recognize the text image data according to a pre-trained text recognition model and determine the recognition result. The trained text recognition model shares character information with the single-character recognition model. Specifically, the trained text recognition model is trained on training text images during the training phase, and it shares a character classifier with the single-character recognition model during the training phase to assist the text recognition model in training. The single-character recognition model is trained on single-character images.

[0200] The recognition result receiving module 906 is used to receive and display the recognition results.

[0201] In this embodiment, the server can provide an interactive page to the terminal to offer text image recognition services. Users can trigger a data upload control on the interactive page to upload the text image data to be recognized to the server. The server can obtain the text image data and input it into a pre-trained text recognition model to extract features of each character in the text image data and associate them with the features of the context characters, thereby performing character classification and determining the recognition result. After determining the recognition result, the server can feed it back to the terminal for display.

[0202] Based on the above embodiments, this embodiment also provides a data processing device that can be applied to scenarios involving the recognition of images of ancient characters. Specifically,

[0203] The text image acquisition module 702 is specifically used to acquire images of ancient characters to be identified.

[0204] The text image recognition module 704 is specifically used to recognize ancient character images based on a pre-trained text recognition model and determine the recognition result. The trained text recognition model shares character information with the single-character recognition model. Specifically, the trained text recognition model is trained on training text images during the training phase, and it shares a character classifier with the single-character recognition model during the training phase to assist in the training of the text recognition model based on the single-character recognition model. The single-character recognition model is trained on single-character images. The text image recognition module 704 is also used to obtain text description information for each text in the recognition result, including at least one of annotation information and phonetic information. The text in the recognition result and the corresponding text description information are output for display.

[0205] In this embodiment, an image containing ancient characters refers to an image containing ancient characters. Ancient characters refer to characters used before the existence of modern characters, such as ancient Chinese characters. Ancient Chinese characters can be understood as the Chinese characters used in China before the emergence of simplified characters. Simplified characters are the officially published simplified characters, usually referring to the characters in the "General List of Simplified Characters" published in 1956. Ancient Chinese characters can be understood as the Chinese characters used in China before the simplification of characters, and may include original ancient Chinese characters, traditional characters, oracle bone script, bronze script, Warring States script, Qin script, and ancient clerical script during the transition from Qin to early Han dynasty. Images containing ancient Chinese characters can be images related to ancient medicine, ancient education, and historical records. In this embodiment, images containing ancient Chinese characters can be identified by photographing books containing ancient Chinese characters, and the images can be input into a pre-trained text recognition model to determine the recognition result. After determining the recognition result, this embodiment can also convert the ancient characters based on the recognition result, such as converting ancient characters into modern characters. It should be noted that, in addition to the one-to-one correspondence conversion method, this embodiment can also perform semantic translation. Furthermore, embodiments of this application can also translate the translated modern text into text in other languages, which can be configured according to requirements.

[0206] Based on the above embodiments, this embodiment also provides a data processing device that can be applied to scenarios involving text-image-based searches. Specifically,

[0207] The text image acquisition module 702 is specifically used to provide an interactive page to acquire text image data to be recognized based on the interactive page.

[0208] The text image recognition module 704 is specifically used to recognize text image data based on a pre-trained text recognition model, determine the recognition result, and the trained text recognition model shares character information with the single-character recognition model. It performs a search based on the recognition result and returns the search results to be displayed on the interactive page. Specifically, the trained text recognition model is trained on training text images during the training phase, and it shares a character classifier with the single-character recognition model during the training phase to assist in the training of the text recognition model. The single-character recognition model is trained on single-character images.

[0209] In this embodiment, an interactive page can be provided. Users upload text image data through the interactive page. After receiving the text image data, a pre-trained text recognition model can be used to recognize the text image data and determine the recognition result. Then, users can search based on the recognition result to find relevant information, which is then displayed on the interactive page. This embodiment can be applied to image-based search scenarios, such as searching based on text images containing company names. Based on the company name in the recognition result, relevant company information can be searched and displayed to the user.

[0210] Based on the above embodiments, this embodiment also provides a data processing device that can be applied in educational scenarios. Specifically,

[0211] The text image acquisition module 702 is specifically used to acquire educational text images to be recognized.

[0212] The text image recognition module 704 is specifically used to recognize educational text images based on a pre-trained text recognition model and determine the recognition result. The trained text recognition model shares character information with the single-character recognition model. Specifically, the trained text recognition model is trained on training text images during the training phase, and it shares a character classifier with the single-character recognition model during the training phase to assist in the training of the text recognition model. The single-character recognition model is trained on single-character images.

[0213] The recognition result output module 706 is specifically used to output each text in the recognition result for display.

[0214] In this embodiment, educational text images can be understood as images containing educational information, such as student answers, student registration information, or information from books. This embodiment can acquire educational text images, recognize them using a pre-trained text recognition model, determine the recognition result, and perform corresponding processing based on the result. For example, if the educational information in the educational text image is student answers, this embodiment can determine the student's score based on the recognition result.

[0215] Based on the above embodiments, this embodiment also provides a data processing device that can be applied in e-commerce scenarios. Specifically,

[0216] The text image acquisition module 702 is specifically used to acquire the product text image to be identified.

[0217] The text image recognition module 704 is specifically used to recognize product text images based on a pre-trained text recognition model and determine the recognition result. The trained text recognition model shares character information with the single-character recognition model. Specifically, the trained text recognition model is trained on training text images during the training phase, and it shares a character classifier with the single-character recognition model during the training phase to assist in the training of the text recognition model. The single-character recognition model is trained on single-character images.

[0218] The recognition result output module 706 is specifically used to output each text in the recognition result for display.

[0219] In this embodiment, the product text image can be understood as an image containing product-related information, such as product name, production date, and product description. This embodiment can acquire a product text image, recognize it using a pre-trained text recognition model, determine the recognition result, and perform corresponding processing based on the result. For example, if the recognition result is a product name, the system can search for the corresponding product based on the name and display the search results to the user.

[0220] Based on the above embodiments, this embodiment also provides a data processing device that can be applied in medical scenarios. Specifically,

[0221] The text image acquisition module 702 is specifically used to acquire medical text images to be recognized.

[0222] The text image recognition module 704 is specifically used to recognize medical text images based on a pre-trained text recognition model and determine the recognition result. The pre-trained text recognition model shares character information with the single-character recognition model. Specifically, the pre-trained text recognition model is trained on training text images during the training phase, and it shares a character classifier with the single-character recognition model during the training phase to assist in the training of the text recognition model. The single-character recognition model is trained on single-character images.

[0223] The recognition result output module 706 is specifically used to output each text in the recognition result for display.

[0224] In this embodiment, a medical text image can be understood as an image containing medical-related information, such as medical records or medical documents. This embodiment can acquire a medical text image, recognize it using a pre-trained text recognition model, determine the recognition result, and perform corresponding processing based on the result. For example, if the recognition result is a disease name, relevant information about that disease can be searched, and the search results can be displayed to the user.

[0225] Based on the above embodiments, this embodiment also provides a data processing device that can be applied in a conference scenario. Specifically,

[0226] The text image acquisition module 702 is specifically used to acquire the meeting text image to be recognized.

[0227] The text image recognition module 704 is specifically used to recognize meeting text images based on a pre-trained text recognition model and determine the recognition result. The trained text recognition model shares character information with the single-character recognition model. Specifically, the trained text recognition model is trained on training text images during the training phase, and it shares a character classifier with the single-character recognition model during the training phase to assist in the training of the text recognition model. The single-character recognition model is trained on single-character images.

[0228] The recognition result output module 706 is specifically used to output each text in the recognition result for display.

[0229] In this embodiment, the meeting text image can be understood as an image containing medical-related information, such as meeting content. This embodiment can acquire the meeting text image, recognize it using a pre-trained text recognition model, determine the recognition result, and perform corresponding processing based on the result. For example, the meeting content can be recorded based on the recognition result, and related meetings can be associated.

[0230] Based on the above embodiments, this embodiment also provides a data processing device that can be applied in traffic scenarios. Specifically,

[0231] The text image acquisition module 702 is specifically used to acquire traffic text images to be recognized.

[0232] The text image recognition module 704 is specifically used to recognize traffic text images based on a pre-trained text recognition model and determine the recognition result. The trained text recognition model shares character information with the single-character recognition model. Specifically, the trained text recognition model is trained on training text images during the training phase, and it shares a character classifier with the single-character recognition model during the training phase to assist in the training of the text recognition model. The single-character recognition model is trained on single-character images.

[0233] The recognition result output module 706 is specifically used to output each text in the recognition result for display.

[0234] In this embodiment, a traffic text image can be understood as an image containing traffic-related information, such as license plate information, billboard information, etc. This embodiment can acquire a traffic text image, recognize it using a pre-trained text recognition model, determine the recognition result, and perform corresponding processing based on the result. For example, traffic violations can be recorded based on the license plate information in the recognition result.

[0235] This application also provides a non-volatile readable storage medium storing one or more modules (programs). When these modules are applied to a device, they enable the device to execute the instructions for the method steps in this application.

[0236] This application provides one or more machine-readable media storing instructions that, when executed by one or more processors, cause an electronic device to perform one or more of the methods described in the above embodiments. In this application, the electronic device includes devices such as servers and terminal devices.

[0237] Embodiments of this disclosure can be implemented as an apparatus with any suitable hardware, firmware, software, or any combination thereof, configured as desired, and the apparatus may include electronic devices such as servers (clusters) and terminals. Figure 10 An exemplary apparatus 1000 is schematically shown that can be used to implement the various embodiments described in this application.

[0238] In one embodiment, Figure 10An exemplary device 1000 is shown, which includes one or more processors 1002, a control module (chipset) 1004 coupled to at least one of the processors 1002, a memory 1006 coupled to the control module 1004, a non-volatile memory (NVM) / storage device 1008 coupled to the control module 1004, one or more input / output devices 1010 coupled to the control module 1004, and a network interface 1012 coupled to the control module 1004.

[0239] Processor 1002 may include one or more single-core or multi-core processors, and processor 1002 may include any combination of general-purpose processors or special-purpose processors (e.g., graphics processors, application processors, baseband processors, etc.). In some embodiments, device 1000 can serve as a server, terminal, or other device as described in the embodiments of this application.

[0240] In some embodiments, apparatus 1000 may include one or more computer-readable media (e.g., memory 1006 or NVM / storage device 1008) having instructions 1014 and one or more processors 1002 that are combined with the one or more computer-readable media and configured to execute instructions 1014 to implement modules and thereby perform the actions described in this disclosure.

[0241] In one embodiment, the control module 1004 may include any suitable interface controller to provide any suitable interface to at least one of the processors 1002 and / or any suitable device or component communicating with the control module 1004.

[0242] The control module 1004 may include a memory controller module to provide an interface to the memory 1006. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0243] Memory 1006 may be used, for example, to load and store data and / or instructions 1014 for device 1000. In one embodiment, memory 1006 may include any suitable volatile memory, such as suitable DRAM. In some embodiments, memory 1006 may include double data rate type quad synchronous dynamic random access memory (DDR4 SDRAM).

[0244] In one embodiment, the control module 1004 may include one or more input / output controllers to provide interfaces to the NVM / storage device 1008 and (one or more) input / output devices 1010.

[0245] For example, NVM / storage device 1008 may be used to store data and / or instructions 1014. NVM / storage device 1008 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable (one or more) non-volatile storage devices (e.g., one or more hard disk drives (HDDs), one or more optical disc drives (CDs), and / or one or more digital universal optical disc (DVD) drives).

[0246] NVM / storage device 1008 may include storage resources that are part of a device on which device 1000 is mounted, or that can be accessed by the device without being part of the device. For example, NVM / storage device 1008 may be accessed via a network via one or more input / output devices 1010.

[0247] One or more input / output devices 1010 may provide an interface for device 1000 to communicate with any other suitable device. Input / output devices 1010 may include communication components, audio components, sensor components, etc. Network interface 1012 may provide an interface for device 1000 to communicate via one or more networks. Device 1000 may wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, such as accessing wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G, 5G, etc., or combinations thereof.

[0248] In one embodiment, at least one of the processors 1002 may be logically packaged with one or more controllers (e.g., memory controller modules) of the control module 1004. In one embodiment, at least one of the processors 1002 may be logically packaged with one or more controllers of the control module 1004 to form a system-in-package (SiP). In one embodiment, at least one of the processors 1002 may be integrated with the logic of one or more controllers of the control module 1004 on the same die. In one embodiment, at least one of the processors 1002 may be integrated with the logic of one or more controllers of the control module 1004 on the same die to form a system-on-a-chip (SoC).

[0249] In various embodiments, device 1000 may be, but is not limited to, a terminal device such as a server, desktop computing device, or mobile computing device (e.g., laptop computing device, handheld computing device, tablet computer, netbook, etc.). In various embodiments, device 1000 may have more or fewer components and / or different architectures. For example, in some embodiments, device 1000 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touch screen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0250] The detection device can use a main control chip as a processor or control module, and sensor data, position information, etc. can be stored in a memory or NVM / storage device. The sensor group can be used as an input / output device, and the communication interface can include a network interface.

[0251] This application also provides an electronic device, including: a processor; and a memory storing executable code thereon, which, when executed, causes the processor to perform one or more methods as described in this application.

[0252] This application also provides one or more machine-readable media having executable code stored thereon, which, when executed, causes a processor to perform one or more of the methods described in this application.

[0253] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0254] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0255] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0256] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0257] These computer program instructions may also be loaded onto a computer or other programmable data processing terminal equipment to cause a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable terminal equipment, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0258] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0259] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0260] The above provides a detailed description of a data processing method, a data processing device, an electronic device, and a storage medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A data processing method, characterized in that, The method includes: Obtain the text image data to be recognized; The text image data is recognized based on a pre-trained text recognition model to determine the recognition result. The pre-trained text recognition model shares character information with the single character recognition model during the training process. The text recognition model includes a first character classifier, and the single character recognition model includes a second character classifier. During the training phase, the first character classifier in the text recognition model and the second character classifier in the single character recognition model share character information. The single character recognition model provides an auxiliary function to the first character classifier of the text recognition model. Output each text in the recognition results for display; It also includes the step of training the text recognition model: Train a text recognition model based on the labeled training text images; Based on the labeled single-character images, the single-character recognition model is adjusted; The second character classifier is adjusted based on the character information of the single character image to determine the single character classification adjustment amount, and the first character classifier is adjusted based on the single character classification adjustment amount.

2. The method according to claim 1, characterized in that, The text image data packet contains the name of the target object, and the method further includes: Based on the name of the target object in the recognition results, search for relevant data of the target object and output it.

3. The method according to claim 1, characterized in that, The method further includes: The system receives an instruction to adjust the recognition result and provides an editing control to obtain text adjustment operations based on the editing control. The text adjustment operations include at least one of the following: position adjustment operation, text re-editing operation, text deletion operation, text addition operation, and text error marking operation. Adjust the recognition results based on the text adjustment operation; Based on the adjusted recognition results, the text recognition model is adjusted.

4. The method according to claim 1, characterized in that, The text in the output recognition result includes: The texts in the recognition results are classified to identify rare and common texts; Obtain obscure text description information, wherein the obscure text description information includes at least one of annotation information and phonetic information; Output common text, rare text, and descriptions of rare text.

5. The method according to claim 1, characterized in that, The method further includes: Obtain the recognition confidence score of each character; When the confidence level of the recognition result is lower than the preset confidence threshold, the question information corresponding to the recognition result is output so as to determine the recognition result based on the response information corresponding to the question information.

6. The method according to claim 1, characterized in that, The method further includes: Obtain the recognition confidence score of each character and determine the credibility level of the character recognition result based on the recognition confidence score; Based on the aforementioned confidence level, the display method corresponding to the character recognition result is determined so that the character recognition result can be displayed on the interactive page in the appropriate manner, making it easier for users to adjust the recognition result.

7. The method according to claim 1, characterized in that, The text recognition model includes: The first feature extractor is used to extract the first features from the text image data; The context modeling module is used to perform feature separation on the first feature, determine character features and background features, enhance the character features, and fuse the background features to obtain the processed first feature; and perform context association on the processed first feature to obtain the first feature data. The first character classifier is used to classify characters based on the first feature data and determine the recognition result.

8. The method according to claim 7, characterized in that, The method further includes: Obtain the labeled training text images; Single-character images are identified and labeled, and these images are determined based on training text images or through synthesis.

9. A data processing method, characterized in that, The method includes: Provide an interactive page, which includes a data upload control; Based on the triggering of the data upload control, the text image data to be recognized is obtained and uploaded to the server so that the server can recognize the text image data according to the pre-trained text recognition model and determine the recognition result. The trained text recognition model shares character information with the single character recognition model during the training process. The text recognition model includes a first character classifier, and the single character recognition model includes a second character classifier. During the training phase, the first character classifier in the text recognition model and the second character classifier in the single character recognition model share character information. The single character recognition model provides an auxiliary function to the first character classifier of the text recognition model. Receive and display the recognition results; The text recognition model is trained through the following steps: Train a text recognition model based on the labeled training text images; Based on the labeled single-character images, the single-character recognition model is adjusted; The second character classifier is adjusted based on the character information of the single character image to determine the single character classification adjustment amount, and the first character classifier is adjusted based on the single character classification adjustment amount.

10. A data processing method, characterized in that, The method includes: Obtain images of the ancient characters to be identified; Ancient character images are identified based on a pre-trained text recognition model to determine the recognition result. The pre-trained text recognition model shares character information with the single-character recognition model during the training process. The text recognition model includes a first character classifier, and the single-character recognition model includes a second character classifier. During the training phase, the first character classifier in the text recognition model and the second character classifier in the single-character recognition model share character information. The single-character recognition model provides an auxiliary function to the first character classifier of the text recognition model. Output the recognition results for display; The text recognition model is trained through the following steps: A text recognition model is trained based on labeled ancient character images. Based on the labeled single-character images, the single-character recognition model is adjusted; The second character classifier is adjusted based on the character information of the single character image to determine the single character classification adjustment amount, and the first character classifier is adjusted based on the single character classification adjustment amount.

11. An electronic device, characterized in that, include: processor; and A memory having executable code stored thereon, which, when executed, causes the processor to perform the method as described in one or more of claims 1-10.

12. One or more machine-readable media having executable code stored thereon, which, when executed, causes a processor to perform the method as described in one or more of claims 1-10.

Images