Data processing method, device, electronic device and computer storage medium

By using multiple text sequence recognition models to identify and evaluate text sequence images, the problems of inaccurate identification results and high annotation cost in the prior art are solved, and a more efficient text sequence recognition and annotation process is achieved.

CN113761968BActive Publication Date: 2024-11-12ALIBABA GROUP HOLDING LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010486511.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-06-01
Publication Date
2024-11-12
Estimated Expiration
2040-06-01

AI Technical Summary

Technical Problem

The existing text sequence recognition model relies on artificially synthesized images during training, resulting in inaccurate recognition results and high manual labeling cost, making it difficult to achieve efficient training data labeling.

Method used

The text sequence recognition model with at least two different structures is used to identify the text sequence images. By evaluating the consistency and confidence of the recognition results, the target recognition results are obtained, and image annotation is performed based on this, reducing manual annotation costs and improving labeling efficiency.

Benefits of technology

It improves the accuracy and efficiency of text sequence recognition, reduces the annotation cost, realizes accurate annotation of text sequence images, and enhances the quality of model training data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113761968B_ABST
    Figure CN113761968B_ABST
Patent Text Reader

Abstract

The embodiment of the present invention provides a data processing method, device, electronic device and computer storage medium, wherein the data processing method comprises: obtaining a text sequence image to be processed; performing text sequence recognition on the text sequence image using at least two text sequence recognition models with different structures to obtain at least two corresponding recognition results; and evaluating the at least two recognition results to obtain a target recognition result. Through the embodiment of the present invention, a more accurate recognition result can be obtained.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of computer technology, and in particular to a data processing method, device, electronic device and computer storage medium. Background Art

[0002] Text sequence recognition is a technology that identifies the text sequence contained in an input image. Different from single text or single character recognition, text sequence recognition aims at the recognition problem from pixel sequence to text sequence.

[0003] At present, the industry mostly implements text sequence recognition based on the sequence recognition model CRNN (Convolutional Recurrent Neural Network). The overall architecture of this type of recognition model is relatively similar. In this case, the training effect of the recognition model depends largely on the quantity and quality of the training data. Since the training of the text sequence recognition model requires a huge amount of training data, the model is generally trained by using artificially synthesized text sequence images plus real text sequence images. However, the synthesized text sequence images are quite different from the real images in terms of fonts, backgrounds, etc., resulting in poor overall training effects and inaccurate text sequence recognition results. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a data processing solution to at least partially solve the above problems.

[0005] According to a first aspect of an embodiment of the present invention, a data processing method is provided, comprising: obtaining a text sequence image to be processed; performing text sequence recognition on the text sequence image using at least two text sequence recognition models with different structures to obtain at least two corresponding recognition results; and evaluating the at least two recognition results to obtain a target recognition result.

[0006] According to a second aspect of an embodiment of the present invention, another data processing method is provided, comprising: receiving a processing request sent by a client for requesting data processing of a text sequence image; obtaining the text sequence image to be processed according to the processing request; and performing data processing on the text sequence image by the data processing method as described in the first aspect.

[0007] According to a third aspect of an embodiment of the present invention, a data processing device is provided, comprising: a first acquisition module, used to obtain a text sequence image to be processed; a second acquisition module, used to perform text sequence recognition on the text sequence image using at least two text sequence recognition models with different structures, and obtain at least two corresponding recognition results; and a labeling module, used to evaluate the at least two recognition results obtained, and obtain a target recognition result.

[0008] According to a fourth aspect of an embodiment of the present invention, there is provided an electronic device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the data processing method described in the first aspect or the second aspect.

[0009] According to a fifth aspect of an embodiment of the present invention, there is provided a computer storage medium on which a computer program is stored. When the program is executed by a processor, the data processing method as described in the first aspect or the second aspect is implemented.

[0010] According to the data processing scheme provided by the embodiment of the present invention, at least two text sequence recognition models with different structures are used to perform text sequence recognition on text sequence images to obtain corresponding recognition results; then, the obtained recognition results are evaluated to determine the target recognition results. On the one hand, the text sequence recognition model can mine effective information in the text sequence images, and through at least two text sequence recognition models, the text sequence recognition results can be obtained from different dimensions; on the other hand, if the evaluation results are consistent, it means that the recognition results of at least two text sequence recognition models are consistent, and a more accurate recognition result can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0012] Figure 1A is a flowchart of a data processing method according to Embodiment 1 of the present invention;

[0013] Figure 1B for Figure 1A A schematic diagram of an example scenario in the illustrated embodiment;

[0014] Figure 2A is a flowchart of a data processing method according to Embodiment 2 of the present invention;

[0015] Figure 2B for Figure 2A A schematic diagram of the structure of a text sequence recognition model in the illustrated embodiment;

[0016] Figure 2C for Figure 2A An overview diagram of data processing in the illustrated embodiment;

[0017] Figure 2D for Figure 2A A schematic diagram of analysis logic of a processing result in the illustrated embodiment;

[0018] Figure 3A is a flowchart of a data processing method according to Embodiment 3 of the present invention;

[0019] Figure 3B for Figure 3A A schematic diagram of an example scenario in the illustrated embodiment;

[0020] Figure 4 is a structural block diagram of a data processing device according to a fourth embodiment of the present invention;

[0021] Figure 5 FIG. 4 is a schematic diagram of the structure of an electronic device according to Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in the field based on the embodiments in the embodiments of the present invention should fall within the scope of protection of the embodiments of the present invention.

[0023] The specific implementation of the embodiment of the present invention is further described below in conjunction with the accompanying drawings of the embodiment of the present invention.

[0024] Embodiment 1

[0025] Reference Figure 1A , shows a step flow chart of a data processing method according to embodiment 1 of the present invention.

[0026] The data processing method of this embodiment includes the following steps:

[0027] Step S102: Obtaining a text sequence image to be processed.

[0028] A text sequence refers to text, characters, or symbols written in rows or columns according to natural writing habits, and an image containing a text sequence is a text sequence image. In an embodiment of the present invention, a text sequence image can be an image with a pure color background (such as black text on a white background or white text on a black background), or an image with a non-pure color background (such as a background image).

[0029] The text sequence image may be an image formed by cutting the text sequence from the original image, but is not limited thereto, and may also be an original image containing the text sequence.

[0030] Step S104: using at least two text sequence recognition models with different structures to perform text sequence recognition on the text sequence image, and obtaining at least two corresponding recognition results.

[0031] The text sequence recognition model can be any appropriate model, and different text sequence models can have different structures. Different text sequence recognition models can perform text sequence recognition on text sequence images from different dimensions and / or focus points, and the recognition results obtained may or may not meet the same standards. If the same standards are met, it means that the results recognized by different text sequence recognition models are the same or similar, and the consistency of the results is higher and more accurate. Otherwise, it means that the consistency is low and further processing is required.

[0032] In one feasible manner, the text sequence recognition performed on the text sequence image includes: text sequence content recognition, single word confidence recognition, and whole line average confidence recognition. The recognition result obtained by the above recognition can effectively characterize the content of the text sequence and the information of the content accuracy. Correspondingly, the recognition result includes: the recognized text sequence content, the confidence of a single word or character or symbol in the text sequence content, and the average confidence of the entire text sequence. However, it is not limited to this. In practical applications, those skilled in the art can also recognize more content through the text sequence recognition model according to actual needs and obtain more recognition results.

[0033] Step S106: Evaluate the at least two obtained recognition results to obtain a target recognition result.

[0034] The evaluation of the recognition result can be mainly used to evaluate the difference or consistency between at least two recognition results. The smaller the difference and the higher the consistency, the more accurate the recognition result is. Otherwise, the recognition result is not accurate enough.

[0035] In one feasible manner, at least one of the following evaluations is performed on the at least two recognition results obtained: consistency evaluation, confidence evaluation. Among them, the consistency evaluation can be used to evaluate whether the recognition results of the at least two text sequence recognition models for the text sequence content recognition are consistent; the confidence evaluation can be used to evaluate whether the single word confidence and the average confidence of the entire line output by the at least two text sequence recognition models meet the preset confidence standards. Furthermore, based on the above consistency evaluation and / or confidence evaluation, the target recognition result can be obtained, for example, whether the content of the recognized text sequence is consistent, whether the confidence of the single word meets certain standards, whether the average confidence of the entire line meets certain standards, and so on.

[0036] Through the above process, at least two text sequence recognition models with different structures are used to perform text sequence recognition on the text sequence image to obtain corresponding recognition results; then, the obtained recognition results are evaluated to determine the target recognition results. On the one hand, the text sequence recognition model can mine the effective information in the text sequence image, and through at least two text sequence recognition models, the text sequence recognition results can be obtained from different dimensions; on the other hand, if the evaluation results are consistent, it means that the recognition results of at least two text sequence recognition models are consistent, and a more accurate recognition result can be obtained.

[0037] The target recognition results obtained can be applied to different scenarios according to actual needs, such as AR (augmented reality) scenarios based on target recognition results, information search, feedback, recommendation and other scenarios based on target recognition results.

[0038] In an optional manner, the target recognition result can also be used to annotate the scene, that is, after obtaining the target recognition result, the text sequence image can also be annotated according to the target recognition result.

[0039] As mentioned above, since the training of the text sequence recognition model requires a large amount of training data, the model is generally trained by using artificially synthesized text sequence images plus real text sequence images. However, the synthetic text sequence images are quite different from the real images in terms of fonts, backgrounds, etc., which not only leads to inaccurate text sequence recognition results, but also the training effect of artificially synthesized text sequence images is not as good as that of real images. Therefore, it is particularly important to annotate the real images. However, it is not realistic and costly to manually annotate all the massive training data. Therefore, the industry is in urgent need of a data annotation solution with low annotation cost. In addition, the data processing solution provided by the embodiment of the present invention is adopted to obtain the final and more accurate target recognition result by evaluating the text sequence recognition results of at least two text sequence recognition models, and annotate the text sequence images based on the target recognition result. For example, the text sequence content can be used as the content annotation of the text sequence image.

[0040] The following is an example scenario to illustrate the above process. Figure 1B shown.

[0041] Figure 1B In the example, it is assumed that three text sequence recognition models A, B and C with different structures are used, and the text sequence image X to be processed contains a line of text "Example Scene". First, the text sequence image X is obtained; after the text sequence image X is obtained, it is input into the text sequence recognition models A, B and C respectively for text sequence recognition.

[0042] In one case, if Figure 1B As shown in, if the recognition result of the text sequence recognition model A input is: the text sequence content is "example scene", "example scene", the single word confidence of each of the four characters is 0.9, and the average confidence of the entire text sequence "example scene" is also 0.9; the recognition result of the text sequence recognition model B input is: the text sequence content is "example scene", "example scene", the single word confidence of each of the four characters is 0.92, 0.94, 0.94, 0.96 respectively, and the average confidence of the entire text sequence "example scene" is 0.94; the recognition result of the text sequence recognition model C input is: the text sequence content is "example scene", "example scene", the single word confidence of each of the four characters is 0.9, and the average confidence of the entire text sequence "example scene" is also 0.9. The analysis shows that (1) the recognition results of the text sequence content of the three text sequence recognition models are consistent; (2) the confidence of the single word recognized by each text sequence recognition model is greater than 0.9; (3) the average confidence of the entire text sequence recognized by each text sequence recognition model is 0.9, 0.94 and 0.9 respectively, which is greater than or equal to 0.9.

[0043] Based on this, it can be determined that the text sequence recognition results have a high degree of uniformity, and the annotation of the text sequence image can be determined based on the results, for example, the recognized text sequence content is used as the content annotation of the text sequence image.

[0044] It should be noted that, in the above-mentioned multiple text sequence recognition models, the single-word confidences recognized by each text sequence recognition model are taken as an example for simplified description, but those skilled in the art should understand that in actual applications, the single-word confidences corresponding to each word may be different, as shown in text sequence recognition model B.

[0045] In another case ( Figure 1B(not shown in the figure), if the recognition result of the text sequence recognition model A input is: the text sequence content is "example scene", "example scene", the single word confidence of each of the four characters is 0.4, and the average confidence of the entire text sequence "example scene" is also 0.4; the recognition result of the text sequence recognition model B input is: the text sequence content is "example soup scene", "example soup scene", the single word confidence of each of the four characters is 0.3, and the average confidence of the entire text sequence "example soup scene" is also 0.3; the recognition result of the text sequence recognition model C input is: the text sequence content is "example scene", "example scene", the single word confidence of each of the four characters is 0.6, and the average confidence of the entire text sequence "example scene" is also 0.6. The analysis shows that (1) the recognition results of the text sequence content of the three text sequence recognition models are inconsistent; (2) the confidence of the single word recognized by each text sequence recognition model is greater than 0.2; (3) the average confidence of the entire text sequence recognized by each text sequence recognition model is 0.4, 0.3 and 0.6 respectively, which is greater than or equal to 0.3.

[0046] Based on this, it can be determined that the text sequence recognition results have a low degree of uniformity and cannot be accurately classified, which may be a difficult sample. In this case, a labeling interface or interface can be provided to the user, and the user can perform manual labeling, and the manual labeling result can be used as the labeling result of the text sequence image.

[0047] In another case ( Figure 1B (not shown in the figure), if the recognition result of the text sequence recognition model A input is: the text sequence content is "example scene", "example scene", the single word confidence of each of the four characters is 0.4, and the average confidence of the entire text sequence "example scene" is also 0.4; the recognition result of the text sequence recognition model B input is: the text sequence content is "example scene", "example scene", the single word confidence of each of the four characters is 0.3, and the average confidence of the entire text sequence "example scene" is also 0.3; the recognition result of the text sequence recognition model C input is: the text sequence content is "example scene", "example scene", the single word confidence of each of the four characters is 0.6, and the average confidence of the entire text sequence "example scene" is also 0.6. The analysis shows that (1) the recognition results of the text sequence content of the three text sequence recognition models are consistent; (2) the confidence of each word recognized by each text sequence recognition model is greater than 0.2 and the confidence is low; (3) the average confidence of the entire text sequence recognized by each text sequence recognition model is 0.4, 0.3 and 0.6 respectively, which is greater than or equal to 0.3 and the confidence is low.

[0048] In this case, although the three text sequence recognition models have consistent recognition results for the text content, both the confidence of individual words and the confidence of the entire text sequence are low, and they cannot be accurately classified. They can be put back into the training data pool and re-labeled in the future to obtain different results.

[0049] Thus, different annotation processing of text sequence images in different situations is achieved, and the processing results are more objective and accurate.

[0050] It can be seen that through this example, for unlabeled text sequence images, at least two text sequence recognition models with different structures are used to perform text sequence recognition on them to obtain corresponding recognition results; then, according to the obtained recognition results, the text sequence images can be labeled. On the one hand, the text sequence recognition model can mine effective information in the text sequence images, and through at least two text sequence recognition models, the text sequence recognition results can be obtained from different dimensions; on the other hand, if the text sequence recognition results are consistent, the text sequence images can be labeled accordingly to obtain more accurate labeling results; on the other hand, this data processing solution does not require manual labeling, which not only reduces the labeling cost, but also greatly improves the labeling efficiency.

[0051] The data processing method of this embodiment can be executed by any appropriate electronic device with data processing capability, including but not limited to: a server, a PC, etc.

[0052] Embodiment 2

[0053] Reference Figure 2A , shows a step flow chart of a data processing method according to embodiment 2 of the present invention.

[0054] The data processing method of this embodiment takes the annotation of text sequence images as an example scenario, but as described in the first embodiment, the solution provided by the embodiment of the present invention is not limited to the annotation scenario. The data processing method of this embodiment includes the following steps:

[0055] Step S202: Acquire the original image containing the text sequence.

[0056] In this embodiment, the original image containing the text sequence can be any appropriate image containing the text sequence. In the training data of the text sequence recognition model, due to the cost of manual annotation, in practical applications, only a small part of it has manual annotation, which makes it difficult to train the text sequence recognition model. To this end, the embodiment of the present invention provides a data processing solution for this problem to achieve automatic image annotation.

[0057] In practical applications, the original image can be any appropriate image containing a text sequence, and in addition to conventional text images, it can also be a text table image, a text contract image, or a technical document image. It can be seen that the solution provided by the embodiment of the present invention can be widely applied to various scenarios and has good compatibility.

[0058] In addition, specifically in this embodiment, by processing the original image in step S204 below, a text sequence image can be obtained, that is, the text sequence image in this embodiment is a part of the original image.

[0059] Step S204: obtaining a text sequence image to be processed.

[0060] As described above, the text sequence image in this step is obtained by processing the original image. In one feasible manner, this step can be implemented as follows: obtaining the original image containing the text sequence; performing text sequence detection on the original image, and segmenting the original image according to the detection result to obtain the text sequence image. In this way, the corresponding text sequence image can be obtained for the text sequence, making the recognition of the text sequence recognition model more targeted and processing less data, which not only improves the recognition accuracy, but also improves the recognition speed and efficiency.

[0061] For example, a text sequence detection module can be used to predict the position of the text sequence in the original image, and the original image can be segmented according to the position to obtain a text sequence image. The text sequence detection module can be implemented as, for example, TextBoxes text detection, CTPN text detection, PixelLink text detection, etc.

[0062] Step S206: using at least two text sequence recognition models with different structures to perform text sequence recognition on the text sequence image, and obtaining at least two corresponding recognition results.

[0063] In this embodiment, three text sequence recognition models with different structures are set to ensure the objectivity of the recognition results and the accuracy of the subsequent voting classification.

[0064] The text sequence recognition model is mostly implemented as a model structure based on a convolutional neural network structure combined with a bidirectional long short-term memory network structure and a decoder based on temporal classification. The three text sequence recognition models in this embodiment are also based on this model structure. Through this model structure, text sequence sequences can be effectively identified. The model based on this structure can be implemented as a CNN+BLSTM+CTC structure, or as a CNN+BLSTM+ATTENTION structure. Among them, the CNN+BLSTM+CTC structure has better recognition results for Chinese text sequence sequences.

[0065] In the above structure, the CNN (convolutional neural network) part is mainly used for feature extraction, the BLSTM (bidirectional long short-term memory network) part realizes text sequence recognition based on the result of feature extraction of CNN, and the CTC decoder can be used for text alignment. Among them, BLSTM determines the final text corresponding to the current position by predicting the probability value, and its output includes the final text and the probability value corresponding to the text, which can be used as the single word confidence in the embodiment of the present invention, and the average confidence of the entire text sequence can be obtained by summing the single word confidence of all the characters in the text sequence and taking the average.

[0066] Optionally, the above convolutional neural network structure can be implemented as one of the following: VGG network, ResNet network, Inception network, DenseNet network. As mentioned above, the decoder based on temporal classification can be implemented as one of the following: CTC decoder, Attention decoder.

[0067] The structures of the above-mentioned various networks can be described with reference to the structures in the related art, and will not be repeated here. For example, the structure of a text sequence recognition model in this embodiment is as follows: Figure 2B As shown in the figure, the structure is composed of CNN part, BLSTM part and CTC part connected in sequence, and the text sequence image input content is "SANTOSA" Figure 2B After entering the model structure shown, the text sequence content "SANTOSA" is recognized.

[0068] But not limited to this, in the present embodiment, the text sequence recognition performed on the text sequence image by the text sequence recognition model of different structures can include the recognition of the text sequence content and the recognition of the single word confidence of each text in the text sequence. Optionally, after obtaining the single word confidence, the whole line average confidence of the whole text sequence can also be obtained based on the single word confidence, that is, the average value of the sum of the single word confidence of each text in the text sequence. Through these recognition results, effective recognition information of the text sequence image can be obtained.

[0069] For example, after obtaining the text sequence images, three text sequence recognition models are used to perform text sequence recognition on the text sequence images to predict the text sequence content and the corresponding single-word confidence. The three text sequence recognition models all adopt the CNN+BLSTM+CTC structure. The difference is that the CNN structure uses VGG, ResNet and Inception respectively, that is, the three structures are: VGG+BLSTM+CTC structure, ResNet+BLSTM+CTC structure, and Inception+BLSTM+CTC structure, so as to obtain more diverse text features through different structures and provide richer information for confidence analysis.

[0070] Step S208: Evaluate the at least two obtained recognition results to obtain a target recognition result.

[0071] In this embodiment, the evaluation includes consistency evaluation and confidence evaluation, but those skilled in the art should understand that in actual applications, only one of them can be selected for use as needed.

[0072] In addition, in order to enable users to obtain target recognition results more effectively and flexibly, in a feasible manner, after obtaining at least two recognition results, a user's selection operation on the at least two recognition results can be received, and the at least two recognition results are evaluated according to the selection operation to obtain the target recognition result. In this case, a display interface is provided to the user, and the obtained recognition results are displayed to the user through the display interface, and the user's selection operation on the at least two recognition results is received through the display interface, that is, the evaluation of the at least two recognition results is implemented as the user's selection operation, and the recognition result selected by the user is the target recognition result. In this way, the determination of the target recognition result that meets the user's needs is achieved.

[0073] Step S210: labeling the text sequence images according to the target recognition result.

[0074] For example, in the first case, if the target recognition result indicates that at least two recognition results are consistent, and the confidence indicated by each recognition result is greater than the first preset threshold, the recognition result is used to annotate the text sequence image. This situation indicates that the recognition results of multiple text sequence recognition models have a high degree of consistency and are relatively accurate. Based on this, the text sequence images are annotated, which can realize automatic annotation of massive text sequence images and improve annotation efficiency.

[0075] Among them, the first preset threshold can be appropriately set by those skilled in the art according to actual conditions. Optionally, the first preset threshold may include: a first sub-threshold for indicating the confidence of a single word, and a second sub-threshold for indicating the average confidence of a text sequence.

[0076] For example, if the text sequence content recognition results of the VGG+BLSTM+CTC structure, the ResNet+BLSTM+CTC structure, and the Inception+BLSTM+CTC structure for the text sequence image are consistent, such as "example scenes", and the single word confidence of the text sequence output by the three structures is greater than 0.8 (first sub-threshold), and the average confidence of the entire line of the text sequence output by the three structures is greater than 0.9 (second sub-threshold). In this case, the recognition results can be used to annotate the text sequence image. For example, the recognized text sequence content is used as the content annotation of the text sequence image.

[0077] In the second case, if the target recognition result indicates that at least two recognition results are inconsistent, and the confidence indicated by each recognition result is less than or equal to the first preset threshold and greater than the second preset threshold, the text sequence image is annotated according to the annotation data input by the user. In this way, difficult samples, that is, image samples that cannot be classified normally or are difficult to classify, can be effectively distinguished for subsequent processing, and they are effectively annotated by the annotation data input by the user.

[0078] Among them, the second preset threshold can also be appropriately set by technical personnel in this field according to actual conditions and the setting of the first preset threshold. Optionally, the second preset threshold includes: a third sub-threshold for indicating the confidence of a single word, and a fourth sub-threshold for indicating the average confidence of a text sequence.

[0079] For example, if the text sequence content recognition results of the VGG+BLSTM+CTC structure, the ResNet+BLSTM+CTC structure, and the Inception+BLSTM+CTC structure for the text sequence image are inconsistent, such as the text sequence content recognized by the VGG+BLSTM+CTC structure and the ResNet+BLSTM+CTC structure is "Example Scene", while the text sequence content recognized by the Inception+BLSTM+CTC structure is "Example Tang Scene"; and the single word confidence of the text sequence output by the three structures is greater than 0.2 (third sub-threshold) and less than or equal to 0.8 (first sub-threshold), and the average confidence of the entire line of the text sequence output by the three structures is greater than 0.4 (fourth sub-threshold) and less than or equal to 0.9 (second sub-threshold). In this case, the text sequence image can be determined as a difficult sample and manually annotated to obtain accurate annotation data.

[0080] In addition, in the second case, after the user has annotated the text sequence images, the aforementioned at least two text sequence recognition models with different structures (such as the aforementioned text sequence recognition models with three structures) can be used to re-perform text sequence recognition on the text sequence images with the annotation data input by the user; according to the difference between the recognition result and the annotation data input by the user, the parameters of some or all of the aforementioned at least two text sequence recognition models with different structures are trained. It can be seen that there is no obvious boundary between the training phase and the application phase of the text sequence recognition model in the embodiment of the present invention, that is, the text sequence recognition model in the embodiment of the present invention updates its own parameters while performing application recognition to improve its own recognition ability and improve the accuracy of the recognition result. With the improvement of the recognition ability of the text sequence recognition model itself, the number of images that need to be manually annotated and the number of images that are put back into the data pool for re-recognition will be less and less, and the automatic annotation of text sequence images will be more and more, which can effectively improve the efficiency of automatic annotation of text sequence images.

[0081] All cases except the first and second cases mentioned above can be classified as other cases, and the text sequence images belonging to other cases are returned to the data pool for subsequent re-identification and labeling processing.

[0082] In other cases, there is a third case, that is, if the target recognition result indicates that at least two recognition results are consistent, and the confidence indicated by each recognition result is less than or equal to the first preset threshold and greater than the second preset threshold, at least two text sequence recognition models with different structures are used to re-recognize and annotate the text sequence image. The text sequence image in this case is neither an accurately recognized image nor a difficult image, so it can be put back into the data pool and re-recognized and annotated using the multiple text sequence recognition models next time to ensure recognition accuracy and annotation accuracy, and to allow multiple text sequence recognition models to be further trained.

[0083] Through the above process, multiple text sequence recognition models with different structures are used to mine effective information in text sequence images. On the one hand, the text sequence images that can be accurately recognized are accurately annotated, and on the other hand, difficult samples that are helpful for training the text sequence recognition model can be mined for manual annotation. The realization is to build a text sequence image training set at a relatively low cost, which greatly improves the effect of the text sequence recognition model. And because it can be widely applied to various text sequence images, it will greatly enrich the training data set of the text sequence recognition model, thereby improving the recognition effect of the text sequence recognition model.

[0084] In addition, in another application scenario, after obtaining the target recognition result, the target recognition result can also be compared with the preset content to obtain a comparison result. The preset content is any appropriate content set by a person skilled in the art according to actual needs. For example, if the text sequence image is a pirated screenshot novel image, the target recognition result can be compared with the novel content of the regular novel to obtain a comparison result. If the comparison results are consistent, it can be determined that the text sequence image is an illegal pirated content image, and then corresponding processing can be performed, such as an alarm prompt or reporting to a relevant address, etc.

[0085] The above process is exemplified below with a specific example scenario. Figure 2C shown.

[0086] In this example, we select the VGG+BLSTM+CTC structure ( Figure 2C The structure of ResNet+BLSTM+CTC is shown in the figure ( Figure 2C Indicated as text sequence recognition model B), and the structure of Inception+BLSTM+CTC ( Figure 2C Three text sequence recognition models marked as text sequence recognition model C).

[0087] For the original image containing a text sequence, a text sequence detection module is first used to predict the position of the text sequence in the original image, and the original image is segmented according to the position to obtain a text sequence image. After obtaining the text sequence image, the text sequence recognition models A, B and C are used to predict the text sequence content and the corresponding single word confidence in the text sequence image. After obtaining the recognition results of the text sequence recognition models A, B and C, including the recognized text sequence content, single word confidence, and obtaining the average confidence of the entire text sequence (such as a whole line) based on the single word confidence, the text sequence recognition model A can be used to predict the text sequence content and the corresponding single word confidence in the text sequence image. Figure 2C The confidence analysis model shown in the figure analyzes and processes the three recognition results output by the three models to obtain the processing results. Figure 2D The processing logic of the confidence analysis model shown in , according to the recognition results, the text sequence images and recognition results can be divided into three parts: (1) Text sequence images that can be accurately pre-labeled using the recognition results: Figure 2D As shown in the top row of , this method can directly use the recognition results of the text sequence recognition model as the annotations of the text sequence images, and then add the annotated text sequence images to the training set for subsequent training of the text sequence recognition model; (2) Valuable difficult samples (text sequence images): Figure 2D As shown in the middle row of , this type of image requires manual annotation, and then the manually annotated text sequence images are added to the training set for subsequent training of the text sequence recognition model; (3) Other text sequence images: Figure 2D As shown in the bottom row of , those that do not belong to (1) and (2) can be classified as this case. The text sequence images can be returned to the data pool. After the next round of iterative model update, these images can be further mined. By mining a large number of text sequence images in the above manner, a large-scale text sequence image training set can be obtained. On the one hand, these images can be used to train the text sequence recognition model required online. On the other hand, these text sequence images can be used to update the three text sequence recognition models, and then continue to mine images, forming an automated mining process to obtain a steady stream of training data at a low cost.

[0088] As can be seen from the above, by using the recognition results of multiple text sequence recognition models to accurately annotate unannotated text sequence images and mining valuable difficult samples for manual annotation, the cost of obtaining text sequence image training data can be greatly reduced. In addition, the automatic mining of text sequence images and the automatic update process of text sequence recognition model iteration can be realized. The text sequence images can be automatically annotated with the newly generated images and recognition results, and then the text sequence recognition model can be trained. The updated text sequence recognition model can further mine richer image samples. The entire process can realize the iterative upgrade of the text sequence recognition model with minimal human intervention, improving the efficiency of model development.

[0089] The data processing method of this embodiment can be executed by any appropriate electronic device with data processing capability, including but not limited to: a server, a PC, etc.

[0090] Embodiment 3

[0091] Reference Figure 3A , shows a step flow chart of a data processing method according to embodiment 3 of the present invention.

[0092] In this embodiment, the data processing method provided by the embodiment of the present invention is described by taking the recognition and annotation of text sequence images through a service end (such as a cloud or a server or a SaaS platform) as an example.

[0093] The data processing method of this embodiment includes the following steps:

[0094] Step S302: receiving a processing request sent by a client for requesting data processing of a text sequence image.

[0095] The processing request may be in any appropriate form.

[0096] Step S304: according to the processing request, the text sequence image to be processed is obtained.

[0097] The text sequence image to be processed may be the aforementioned original image containing the text sequence, or may be the aforementioned text sequence image.

[0098] In one feasible manner, when a processing request from a client is received through a SaaS (Software-as-a-Service) platform, the text sequence image to be processed can be obtained locally from the SaaS platform according to the processing request. In this case, if the SaaS platform has the text sequence image stored locally, it can be directly obtained, thereby improving the speed and efficiency of data processing.

[0099] In another feasible manner, when receiving a processing request from a client through the SaaS platform, the SaaS platform can collect the text sequence images to be processed from a third party according to the processing request. For example, the text sequence images can be obtained from a third-party website or from a data interface provided by a third-party application through the network. In this case, the SaaS platform obtains the text sequence images through the third party, and does not need to store them locally, thus saving storage resources of the SaaS platform.

[0100] In another feasible manner, when receiving a processing request from a client via the SaaS platform, the SaaS platform may obtain a text sequence image from the client according to the processing request. In this case, the text sequence image is stored in the client, and the SaaS platform obtains the text sequence image from the client, thereby obtaining a data processing result that better meets the needs of the client.

[0101] Step S306: performing data processing on the text sequence image to be processed.

[0102] For example, the text sequence image may be processed by the data processing method described in Embodiment 1 or 2, which will not be described in detail here.

[0103] The following uses the data processing through the SaaS platform as an example to illustrate the above process. Figure 3B shown.

[0104] Figure 3B In the example, multiple (two or more) text sequence recognition models with different structures are deployed in SaaS. If the client sends a processing request to the SaaS platform, after receiving the processing request, the SaaS platform first obtains the text sequence image to be processed from the local first storage device by the processing device; then, the SaaS platform performs text sequence recognition on the obtained text sequence image through multiple text sequence recognition models with different structures, and evaluates the multiple recognition results output by multiple models to obtain the target recognition result. In addition, in this example, after obtaining the target recognition result, the text sequence image is also annotated, or, if a difficult sample is found, it is sent back to the client for manual annotation by the user, or put back into the data pool for subsequent re-identification and annotation; the SaaS platform stores the annotated text sequence image in the local second storage device. After completing the data processing of the text sequence image obtained this time, the SaaS platform will send a processing completion message to the client.

[0105] In the above, the example of labeling through the SaaS platform is taken, but those skilled in the art should understand that the solution of this embodiment is also applicable to other forms of server ends.

[0106] It can be seen that through this embodiment, the server calls the text sequence recognition model to perform data processing and annotation of text sequence images based on the client's data processing request, thereby achieving accurate text sequence recognition and text sequence image annotation without any requirements on client resources or performance, and ensuring the effect and efficiency of processing and annotation.

[0107] The data processing method of this embodiment can be executed by any appropriate electronic device with data processing capability, including but not limited to: a server, a PC, etc.

[0108] Embodiment 4

[0109] Reference Figure 4 , shows a structural block diagram of a data processing device according to Embodiment 4 of the present invention.

[0110] The data processing device of this embodiment includes: a first acquisition module 402, used to obtain a text sequence image to be processed; a second acquisition module 404, used to perform text sequence recognition on the text sequence image using at least two text sequence recognition models with different structures to obtain at least two corresponding recognition results; an evaluation module 406, used to evaluate the at least two recognition results obtained to obtain a target recognition result.

[0111] Optionally, the evaluation module 406 is used to perform at least one of the following evaluations on the at least two recognition results obtained: consistency evaluation, confidence evaluation, to obtain a target recognition result.

[0112] Optionally, the data processing device of this embodiment further includes: a labeling module 408, which is used to label the text sequence image according to the target recognition result.

[0113] Optionally, the labeling module 408 is configured to label the text sequence image using the recognition result if the processing result indicates that the at least two recognition results are consistent and the confidence level indicated by each recognition result is greater than a first preset threshold.

[0114] Optionally, the labeling module 408 is used to label the text sequence image according to the labeling data input by the user if the target recognition result indicates that the at least two recognition results are inconsistent, and the confidence indicated by each recognition result is less than or equal to a first preset threshold and greater than a second preset threshold.

[0115] Optionally, the device also includes: a retraining module 410, which is used to use the at least two text sequence recognition models with different structures to re-perform text sequence recognition on the text sequence image with the annotation data input by the user; and perform parameter training on part or all of the at least two text sequence recognition models with different structures based on the difference between the recognition result and the annotation data input by the user.

[0116] Optionally, the labeling module 408 is used to use the at least two text sequence recognition models with different structures to re-perform text sequence recognition and labeling on the text sequence image if the target recognition result indicates that the at least two recognition results are consistent, and the confidence indicated by each recognition result is less than or equal to a first preset threshold and greater than a second preset threshold.

[0117] Optionally, the first preset threshold includes: a first sub-threshold for indicating the confidence of a single word, and a second sub-threshold for indicating the average confidence of a text sequence; the second preset threshold includes: a third sub-threshold for indicating the confidence of a single word, and a fourth sub-threshold for indicating the average confidence of a text sequence.

[0118] Optionally, the at least two text sequence recognition models with different structures are both models based on a convolutional neural network structure combined with a bidirectional long short-term memory network structure and a decoder based on temporal classification.

[0119] Optionally, the convolutional neural network structure includes one of the following: VGG network, ResNet network, Inception network, DenseNet network; and / or, the decoder based on temporal classification includes one of the following: CTC decoder, Attention decoder.

[0120] Optionally, the first acquisition module 402 is used to obtain an original image containing a text sequence; perform text sequence detection on the original image, and segment the original image according to the detection result to obtain the text sequence image.

[0121] Optionally, the original image containing the text sequence includes one of the following: a text table image, a text contract image, and a technical document image.

[0122] Optionally, the evaluation module 406 is used to receive a selection operation of the user on the at least two recognition results, and obtain a target recognition result according to the selection operation.

[0123] Optionally, the evaluation module 406 is further configured to compare the target recognition result with preset content to obtain a comparison result.

[0124] The data processing device of this embodiment is used to implement the corresponding data processing methods in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be described in detail here. In addition, the functional implementation of each module in the data processing device of this embodiment can refer to the description of the corresponding parts in the aforementioned method embodiments, which will not be described in detail here.

[0125] Embodiment 5

[0126] Reference Figure 5 , shows a schematic diagram of the structure of an electronic device according to Embodiment 5 of the present invention. The specific embodiment of the present invention does not limit the specific implementation of the electronic device.

[0127] like Figure 5 As shown, the electronic device may include: a processor (processor) 502 , a communication interface (Communications Interface) 504 , a memory (memory) 506 , and a communication bus 508 .

[0128] in:

[0129] The processor 502 , the communication interface 504 , and the memory 506 communicate with each other via a communication bus 508 .

[0130] The communication interface 504 is used to communicate with other electronic devices or servers.

[0131] The processor 502 is used to execute the program 510, and specifically can execute the relevant steps in the above data processing method embodiment.

[0132] Specifically, the program 510 may include program codes, which include computer operation instructions.

[0133] The processor 502 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0134] The memory 506 is used to store the program 510. The memory 506 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0135] The program 510 can be specifically used to enable the processor 502 to perform the following operations: obtain a text sequence image to be processed; use at least two text sequence recognition models with different structures to perform text sequence recognition on the text sequence image to obtain at least two corresponding recognition results; and evaluate the at least two recognition results obtained to obtain a target recognition result.

[0136] In an optional embodiment, the program 510 is further configured to cause the processor 502 to evaluate the at least two recognition results obtained to obtain the target recognition result: perform at least one of the following evaluations on the at least two recognition results obtained: consistency evaluation, confidence evaluation, and obtain the target recognition result.

[0137] In an optional implementation, the program 510 is further configured to enable the processor 502 to label the text sequence image according to the target recognition result after obtaining the target recognition result.

[0138] In an optional embodiment, the program 510 is also used to enable the processor 502 to annotate the text sequence image according to the target recognition result: if the target recognition result indicates that the at least two recognition results are consistent, and the confidence indicated by each recognition result is greater than a first preset threshold, then the recognition result is used to annotate the text sequence image.

[0139] In an optional embodiment, the program 510 is also used to enable the processor 502 to annotate the text sequence image according to the target recognition result: if the target recognition result indicates that the at least two recognition results are inconsistent, and the confidence indicated by each recognition result is less than or equal to the first preset threshold and greater than the second preset threshold, then annotate the text sequence image according to the annotation data input by the user.

[0140] In an optional embodiment, the program 510 is also used to enable the processor 502 to use the at least two text sequence recognition models with different structures to re-perform text sequence recognition on the text sequence image with the annotation data input by the user; and perform parameter training on part or all of the at least two text sequence recognition models with different structures based on the difference between the recognition result and the annotation data input by the user.

[0141] In an optional embodiment, the program 510 is also used to enable the processor 502 to annotate the text sequence image according to the target recognition result: if the target recognition result indicates that the at least two recognition results are consistent, and the confidence indicated by each recognition result is less than or equal to the first preset threshold and greater than the second preset threshold, then use the at least two text sequence recognition models with different structures to re-perform text sequence recognition and annotate the text sequence image.

[0142] In an optional embodiment, the first preset threshold includes: a first sub-threshold for indicating the confidence of a single word, and a second sub-threshold for indicating the average confidence of a text sequence; the second preset threshold includes: a third sub-threshold for indicating the confidence of a single word, and a fourth sub-threshold for indicating the average confidence of a text sequence.

[0143] In an optional embodiment, the at least two text sequence recognition models with different structures are both models based on a convolutional neural network structure combined with a bidirectional long short-term memory network structure and a decoder based on temporal classification.

[0144] In an optional embodiment, the convolutional neural network structure includes one of the following: VGG network, ResNet network, Inception network, DenseNet network; and / or, the time series classification based decoder includes one of the following: CTC decoder, Attention decoder.

[0145] In an optional embodiment, the program 510 is also used to enable the processor 502 to: obtain an original image containing a text sequence when obtaining a text sequence image to be processed; perform text sequence detection on the original image, and segment the original image according to the detection result to obtain the text sequence image.

[0146] In an optional implementation, the original image containing the text sequence includes one of the following: a text table image, a text contract image, and a technical document image.

[0147] In an optional embodiment, the program 510 is also used to enable the processor 502 to receive a user's selection operation on the at least two recognition results when evaluating the at least two recognition results to obtain a target recognition result, and obtain the target recognition result according to the selection operation.

[0148] In an optional implementation, the program 510 is further configured to enable the processor 502 to evaluate the at least two recognition results obtained to obtain a target recognition result, and then compare the target recognition result with preset content to obtain a comparison result.

[0149] The specific implementation of each step in program 510 can refer to the corresponding description of the corresponding steps and units in the above data processing method embodiment, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described devices and modules can refer to the corresponding process description in the above method embodiment, which will not be repeated here.

[0150] Through the electronic device of this embodiment, at least two text sequence recognition models with different structures are used to perform text sequence recognition on the text sequence image to obtain corresponding recognition results; then, the obtained recognition results are evaluated to determine the target recognition results. On the one hand, the text sequence recognition model can mine effective information in the text sequence image, and through at least two text sequence recognition models, the text sequence recognition results can be obtained from different dimensions; on the other hand, if the evaluation results are consistent, it means that the recognition results of at least two text sequence recognition models are consistent, and a more accurate recognition result can be obtained.

[0151] It should be pointed out that, according to the needs of implementation, the various components / steps described in the embodiments of the present invention can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present invention.

[0152] The above-described method according to an embodiment of the present invention may be implemented in hardware, firmware, or as software or computer code that may be stored in a recording medium (such as a CD ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code that is originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded over a network and will be stored in a local recording medium, so that the method described herein may be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, a processor, or hardware, the data processing method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the data processing method shown herein, the execution of the code converts the general-purpose computer into a dedicated computer for executing the data processing method shown herein.

[0153] Those of ordinary skill in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present invention.

[0154] The above implementation methods are only used to illustrate the embodiments of the present invention, and are not limitations of the embodiments of the present invention. Ordinary technicians in the relevant technical field may make various changes and modifications without departing from the spirit and scope of the embodiments of the present invention. Therefore, all equivalent technical solutions also belong to the scope of the embodiments of the present invention. The patent protection scope of the embodiments of the present invention should be defined by the claims.

Claims

1. A data processing method, comprising: Obtaining the text sequence image to be processed; Using at least two text sequence recognition models with different structures to perform text sequence recognition on the text sequence image, and obtaining at least two corresponding recognition results; Perform consistency evaluation and confidence evaluation on the at least two recognition results obtained to obtain a target recognition result, wherein if the at least two recognition results are consistent, and the confidence indicated by each recognition result is greater than a first preset threshold, the recognition result is used as the target recognition result, wherein the consistency evaluation is used to evaluate whether the at least two recognition results are consistent, and the confidence evaluation is used to evaluate the single-word confidence of the at least two recognition results, and whether the average confidence of the entire row of the at least two recognition results meets the preset confidence standard.

2. The method according to claim 1, wherein: After obtaining the target recognition result, the method further includes: The text sequence images are labeled according to the target recognition result.

3. The method according to claim 2, wherein: The step of labeling the text sequence images according to the target recognition result includes: If the target recognition result indicates that the at least two recognition results are consistent, and the confidence level indicated by each recognition result is greater than a first preset threshold, the recognition result is used to annotate the text sequence image.

4. The method according to claim 2, wherein: The step of labeling the text sequence images according to the target recognition result includes: If the target recognition result indicates that the at least two recognition results are inconsistent, and the confidence indicated by each recognition result is less than or equal to a first preset threshold and greater than a second preset threshold, the text sequence image is annotated according to the annotation data input by the user.

5. The method according to claim 4, wherein: The method further comprises: Re-performing text sequence recognition on the text sequence image having the annotation data input by the user using the at least two text sequence recognition models with different structures; According to the difference between the recognition result and the annotation data input by the user, parameter training is performed on part or all of the at least two text sequence recognition models with different structures.

6. The method according to claim 2, wherein: The step of labeling the text sequence images according to the target recognition result includes: If the target recognition result indicates that the at least two recognition results are consistent, and the confidence indicated by each recognition result is less than or equal to the first preset threshold and greater than the second preset threshold, the at least two text sequence recognition models with different structures are used to re-perform text sequence recognition and annotation on the text sequence image.

7. The method according to any one of claims 4 to 6, wherein: The first preset threshold comprises: a first sub-threshold for indicating the confidence of a single word, and a second sub-threshold for indicating the average confidence of a text sequence; The second preset threshold includes: a third sub-threshold for indicating the confidence of a single word, and a fourth sub-threshold for indicating the average confidence of a text sequence.

8. The method according to claim 1, wherein: The at least two text sequence recognition models with different structures are both models based on a convolutional neural network structure combined with a bidirectional long short-term memory network structure and a decoder based on temporal classification.

9. The method according to claim 8, wherein: The convolutional neural network structure includes one of the following: VGG network, ResNet network, Inception network, DenseNet network; and / or, The decoder based on temporal classification includes one of the following: a CTC decoder and an Attention decoder.

10. The method according to claim 1, wherein: The step of obtaining a text sequence image to be processed comprises: Obtain the original image containing the text sequence; The text sequence detection is performed on the original image, and the original image is segmented according to the detection result to obtain the text sequence image.

11. The method according to claim 10, wherein: The original image containing the text sequence includes one of the following: a text table image, a text contract image, and a technical document image.

12. The method according to claim 1, wherein: The step of evaluating the at least two recognition results to obtain a target recognition result includes: A selection operation of the user on the at least two recognition results is received, and a target recognition result is obtained according to the selection operation.

13. The method according to claim 1, wherein: After evaluating the at least two recognition results to obtain a target recognition result, the method further includes: The target recognition result is compared with the preset content to obtain a comparison result.

14. A data processing method, comprising: receiving a processing request sent by a client for requesting data processing of a text sequence image; According to the processing request, obtaining a text sequence image to be processed; The text sequence image is processed by the data processing method according to any one of claims 1 to 13.

15. The method according to claim 14, wherein: The step of obtaining a text sequence image to be processed according to the processing request includes: According to the processing request, the text sequence image to be processed is obtained locally from the SaaS platform; or, According to the processing request, the SaaS platform obtains the text sequence image to be processed from the third party; or, According to the processing request, the SaaS platform obtains the text sequence image to be processed from the client.

16. A data processing device, comprising: A first acquisition module is used to obtain a text sequence image to be processed; A second acquisition module is used to perform text sequence recognition on the text sequence image using at least two text sequence recognition models with different structures to obtain at least two corresponding recognition results; An evaluation module is used to perform consistency evaluation and confidence evaluation on the at least two recognition results obtained to obtain a target recognition result, wherein if the at least two recognition results are consistent and the confidence indicated by each recognition result is greater than a first preset threshold, the recognition result is used as the target recognition result, wherein the consistency evaluation is used to evaluate whether the at least two recognition results are consistent, and the confidence evaluation is used to evaluate the single-word confidence of the at least two recognition results, and whether the average confidence of the entire line of the at least two recognition results meets a preset confidence standard.

17. An electronic device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute any one of claims 1-13, or execute an operation corresponding to the data processing method according to any one of claims 14-15.

18. A computer storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the program implements any one of claims 1 to 13, or implements the data processing method according to any one of claims 14 to 15.