Text recognition method and related apparatus

Abstract

This application relates to a text recognition method and related apparatus, which pertain to the field of artificial intelligence technology. In this application, the position of a text line is indicated by its start and end positions, making it applicable to the annotation of text lines with arbitrary structures and layouts. During model training, only the start and end positions of the text lines need to be annotated, eliminating the need for bounding boxes and reducing the cost of text line position annotation. Determining the reading order by the start and end positions on both sides of the text line applies to reading orders of any version and rule, improving the robustness of reading order determination. Simplifying the text line localization problem into the problem of recognizing the start and end positions of the text line simplifies the problems the model needs to handle, improves the model's inference efficiency, and enhances the stability of the model training process. The model only needs to determine the text content between the start and end positions, reducing or even eliminating the possibility of illusions in the model's recognition results.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to text recognition methods and related devices. Background Technology

[0002] With the development of large-scale knowledge base models and retrieval-augmented generation (RAG) technology, users increasingly need to quickly and accurately identify document content from complex documents. Most Optical Character Recognition (OCR) systems employ a modeling paradigm based on text line detection and recognition, which can already identify text content in documents quite well. However, in scenarios such as seals, traffic signs, merchandise, and trademarks, text lines conveying key information are more complex, and existing systems often struggle to recognize them. Summary of the Invention

[0003] To address the aforementioned problems, this application provides a text recognition method and related apparatus, which solves the problem of recognizing complex text lines.

[0004] Therefore, the following technical solution is adopted in this application:

[0005] Firstly, this application provides a text recognition method, comprising: acquiring a first image containing a text line to be recognized; inputting the first image into an encoder to output encoded features; inputting the encoded features into a decoder to output position pairs, each position pair including a first position and a second position, the first position representing the position of the first character of the text line, the second position representing the position of the last character of the text line, and the text line order representing the sequence from the first position to the second position; and obtaining the text recognition result of the text line based on the position pairs and the encoded features. The first image may be, for example, a seal image, a traffic sign image, or a product packaging image. The first position may also be a start position, beginning position, or other descriptions, and the second position may also be an end position, termination position, or other descriptions. The text line order refers to the order in which the text line is read, a natural word order. By parsing the text content at the positions indicated by the position pairs in the encoded features under the guidance of the position pairs, the text recognition result of the text line can be obtained, thus improving the accuracy of the text recognition result.

[0006] In this application, the position of a text line is indicated by its start and end positions. This method is applicable to the labeling of text lines with any structure and layout, reducing interference introduced by rectangular detection boxes when detecting text lines with special structures, thus improving the accuracy of text recognition within the text line. During model training, only the start and end positions of the text line need to be labeled, eliminating the need for detection boxes and reducing the labeling cost of text line positions. Determining the reading order by the start and end positions on both sides of the text line is applicable to reading orders of any version and rule, improving the robustness of reading order determination. Simplifying the text line localization problem into the identification problem of the start and end positions of the text line simplifies the problem that the model needs to handle, improves the model's inference efficiency, and enhances the stability of the model training process. The model only needs to determine the text content between the start and end positions, reducing or even eliminating the illusion of model recognition results.

[0007] In one possible implementation, inputting coded features into a decoder to output position pairs includes: inputting coded features into a decoder to output a first position; and inputting the coded features and the first position into a decoder to output a second position.

[0008] This implementation provides an autoregressive position generation process. By employing this process, position information can be predicted segment by segment in an orderly manner, fully utilizing the prediction results of preceding positions as prior information for subsequent predictions. This improves the continuity, accuracy, and spatial consistency of position prediction, and is beneficial for enhancing the model's localization accuracy for text line position pairs.

[0009] In one possible implementation, inputting encoded features into a decoder to output position pairs includes: inputting encoded features into a decoder to output hidden layer features; and inputting hidden layer features into a classification layer to output position pairs.

[0010] In this implementation, the task of recognizing position pairs can be achieved by configuring a classification layer on the output side of the decoder. The hidden layer features output by the decoder are intermediate features, which contain both visual features of the first image (such as contours and textures) and detailed information of the first image (such as text line content and text line position). During the position recognition process, the decoder decodes the input encoded features once, extracts information related to the text line position from the hidden layer features, and outputs the position pairs corresponding to the text lines, ensuring the accuracy of the position pairs.

[0011] In one possible implementation, the text recognition result of the text line is obtained based on the position pairs and encoding features, including: obtaining the position representation based on the position pairs and hidden layer features; and obtaining the text recognition result of the text line based on the position representation and encoding features.

[0012] In this implementation, hidden layer features are fused into position pairs to obtain position representations. During text recognition, the decoder's input includes encoded features and position representations. Guided by the position representations, the decoder performs secondary decoding on the encoded features, extracting text content information at the positions indicated by the position representations from the hidden layer features, and outputting the text recognition result. Hidden layer features represent high-dimensional, macroscopic features of the first image. Thus, the position representations not only contain information about the position pairs representing text line locations but also macroscopic, high-dimensional semantic information such as contextual information representing the position pairs. This improves the completeness and semantic richness of the information represented by the position representations of the text line locations, thereby improving the accuracy of the text recognition results. For example, the contextual information of the first position includes a seal border on the left, text above, and a blank area below.

[0013] In one possible implementation, the location representation is obtained based on the location pairs and hidden layer features, including: inputting the location pairs into the embedding layer to output embedded features; and fusing the embedded features and hidden layer features to obtain the location representation.

[0014] In this implementation, the embedding layer maps discrete position pairs to a continuous high-dimensional vector space, generating embedding features with semantic representation capabilities. These embedding features can be fused with the hidden layer features of the decoder, supplementing prior information and improving the model's ability to learn and represent the positional representations of text lines.

[0015] In one possible implementation, the text recognition result of the text line is obtained based on the position representation and encoding features, including: inputting the position representation and encoding features into an encoder to output the text recognition result of the text line.

[0016] In this implementation, the position representation and encoded features are input into the encoder to output the text recognition result of the text line. This enables the encoder to learn the spatial position features and visual semantic features of the text line at the same time, strengthens the spatial correlation and semantic recognition of the text line features, and effectively improves the accuracy, continuity and anti-interference ability of the text recognition result.

[0017] In one possible implementation, acquiring the first image includes: acquiring a second image to be identified, the second image including a target region containing text lines; and extracting the first image from the second image, the first image representing a portion of the image corresponding to the target region.

[0018] In this implementation, a first image corresponding to the target region containing the text line is extracted from the original second image input by the user. The encoder and decoder only need to process the image of the target region (such as the inside of the seal), which can reduce the computational load of the encoder and decoder, and also reduce noise interference in areas outside the target region, thereby improving the accuracy of the text recognition results.

[0019] In one possible implementation, the first image represents an image containing a seal, with the text line located inside the seal.

[0020] In this implementation, the seal is merely an illustrative example and does not constitute a limitation on the scenario of this application.

[0021] In one possible implementation, the first position represents the position point around the first character of the text line, and the second position represents the position point around the last character of the text line.

[0022] This implementation provides an illustrative example of the first and second positions, but this does not constitute a limitation on the first and second positions. Any method that can identify the position of a text line is a feasible implementation, such as marker lines, special symbols, etc.

[0023] In one possible implementation, the first image contains multiple lines of text to be recognized and multiple position pairs, where each position pair represents a first position and a second position of one of the multiple text lines. Based on the position pairs and encoded features, the text recognition result of the text lines is obtained, including: obtaining the text recognition result of the text lines corresponding to each position pair in the multiple position pairs in parallel, based on the multiple position pairs and encoded features.

[0024] In this implementation, in scenarios with multiple lines of text, the text recognition results of multiple lines of text can be recognized in parallel based on the start and end positions of the multiple lines of text, thereby improving the efficiency of predicting text recognition results.

[0025] Secondly, this application provides a model training method, including: acquiring a training dataset. The training dataset includes image samples, location labels, and text labels. The image samples are input into a text recognition model to output location information and text information. The parameters of the text recognition model are adjusted based on the loss between location information and location labels, and the loss between text information and text labels. Here, the location labels represent the ground truth values ​​of the start and end positions of text lines in the image samples; the start position represents the position of the first character of the text line; the end position represents the position of the last character of the text line; the order of the text lines represents the sequence from the start position to the end position; and the text labels represent the ground truth values ​​of the text content of the text lines. The text recognition model includes at least an encoder and a decoder. The encoder is used to generate encoded features of the image samples, the decoder is used to generate location information based on the encoded features, and the decoder is also used to generate text information based on the location information and encoded features. The location information represents the predicted results of the start and end positions of the text lines, and the text information represents the predicted results of the text content of the text lines.

[0026] In one possible implementation, location labels are obtained by annotating the start and end positions of text lines in the image sample. This implementation uses location pairs including the start and end positions to label the text lines, requiring only the start and end positions of the text lines to be labeled, eliminating the need to label bounding boxes of rectangular, arc-shaped, or other similar shapes, thus reducing the labeling cost of bounding boxes.

[0027] In one possible implementation, the process by which the decoder generates position information based on the encoded features includes: inputting the encoded features into the decoder to output a predicted value for the start position; and inputting the encoded features and the start position into the decoder to output a predicted value for the end position.

[0028] In one possible implementation, the process of the decoder generating location information based on the encoded features includes: inputting the encoded features into the decoder to output hidden layer features; and inputting the hidden layer features into the classification layer to output location information.

[0029] In one possible implementation, the process by which the decoder obtains the text recognition result of the text line based on the position information and the encoding features includes: obtaining the position representation based on the position pairs and the hidden layer features; and obtaining the text recognition result of the text line based on the position representation and the encoding features.

[0030] In one possible implementation, the process by which the decoder obtains a position representation based on the position pairs and hidden layer features includes: inputting the position pairs into the embedding layer to output embedded features; and fusing the embedded features and hidden layer features to obtain the position representation.

[0031] In one possible implementation, the process by which the decoder obtains the text recognition result of the text line based on the position representation and encoding features includes: inputting the position representation and encoding features into the encoder to output the text recognition result of the text line.

[0032] In one possible implementation, the process of generating an image sample includes: acquiring an image to be identified, the image to be identified including a target region containing text lines; and extracting an image sample from the image to be identified, the image sample representing a portion of the image corresponding to the target region.

[0033] In one possible implementation, the image sample represents an image containing a seal, with the text line located inside the seal.

[0034] In one possible implementation, the start position represents the position point around the first character of the text line, and the end position represents the position point around the last character of the text line.

[0035] In one possible implementation, the image sample contains multiple lines of text to be identified. There are also multiple location labels, each representing the ground truth value of the start and end positions of one of the multiple text lines. The decoder generates the text information by: obtaining the text information of each line corresponding to each of the multiple location information and encoded features in parallel. Each location information represents the predicted start and end positions of one of the multiple text lines.

[0036] Thirdly, this application provides a text recognition device, including an image acquisition module and a processing module. The image acquisition module acquires a first image containing a text line to be recognized. The first image may be, for example, a seal image, a traffic sign image, or a product packaging image. The processing module inputs the first image into an encoder to output encoded features. The encoded features are input into a decoder to output position pairs, each position pair including a first position and a second position. The first position represents the position of the first character of the text line, and the second position represents the position of the last character of the text line. The order of the text line represents the sequence from the first position to the second position. Based on the position pairs and the encoded features, the text recognition result of the text line is obtained. Guided by the position pairs, the text content at the positions indicated by the position pairs in the encoded features is decoded to obtain the text recognition result of the text line, which can improve the accuracy of the text recognition result.

[0037] In one possible implementation, the processing module is specifically used to: input the encoded features into the decoder to output a first position; and input the encoded features and the first position into the decoder to output a second position.

[0038] In one possible implementation, the processing module is specifically used to: input the encoded features into the decoder to output hidden layer features; and input the hidden layer features into the classification layer to output position pairs.

[0039] In one possible implementation, the processing module is specifically used to: obtain a position representation based on the position pairs and hidden layer features; and obtain the text recognition result of the text line based on the position representation and the encoded features.

[0040] In one possible implementation, the processing module is specifically used to: input the location pairs into the embedding layer to output embedded features; and fuse the embedded features and hidden layer features to obtain a location representation.

[0041] In one possible implementation, the processing module is specifically used to: input position representations and encoded features into the encoder to output the text recognition result of the text line.

[0042] In one possible implementation, the image acquisition module is specifically used to: acquire a second image to be identified, the second image including a target region containing text lines; and extract a first image from the second image, the first image representing a portion of the image corresponding to the target region.

[0043] In one possible implementation, the first image represents an image containing a seal, with the text line located inside the seal.

[0044] In one possible implementation, the first position represents the position point around the first character of the text line, and the second position represents the position point around the last character of the text line.

[0045] In one possible implementation, the first image contains multiple lines of text to be recognized and multiple position pairs. Each position pair represents the first and second positions of one of the multiple text lines. The processing module is specifically used to: obtain the text recognition result of the text line corresponding to each position pair in the multiple position pairs in parallel, based on the multiple position pairs and encoded features.

[0046] Fourthly, this application provides a model training apparatus, including a dataset acquisition module and a training module. The dataset acquisition module is used to acquire a training dataset. The training dataset includes image samples, position labels, and text labels. The position labels represent the ground truth values ​​of the start and end positions of text lines in the image samples. The start position represents the position of the first character of the text line, and the end position represents the position of the last character of the text line. The order of the text lines represents the sequence from the start position to the end position. The text labels represent the ground truth values ​​of the text content of the text lines. The training module is used to input image samples into a text recognition model to output position information and text information. The text recognition model includes at least an encoder and a decoder. The encoder is used to generate encoded features of the image samples, and the decoder is used to generate position information based on the encoded features. The decoder is also used to generate text information based on the position information and encoded features. The position information represents the predicted results of the start and end positions of the text lines, and the text information represents the predicted results of the text content of the text lines. Furthermore, the text recognition model is trained based on the loss between position information and position labels, and the loss between text information and text labels.

[0047] In one possible implementation, location labels are obtained by annotating the start and end positions of text lines in the image sample. This implementation uses location pairs including the start and end positions to label the text lines, requiring only the start and end positions of the text lines to be labeled, eliminating the need to label bounding boxes of rectangular, arc-shaped, or other similar shapes, thus reducing the labeling cost of bounding boxes.

[0048] In one possible implementation, the training module is specifically used to: input the encoded features into the decoder to output the predicted value of the start position; and input the encoded features and the start position into the decoder to output the predicted value of the end position.

[0049] In one possible implementation, the training module is specifically used to: input encoded features into the decoder to output hidden layer features; and input the hidden layer features into the classification layer to output location information.

[0050] In one possible implementation, the training module is specifically used to: obtain position representations based on position pairs and hidden layer features; and obtain text recognition results for text lines based on position representations and encoded features.

[0051] In one possible implementation, the training module is specifically used to: input the location pairs into the embedding layer to output embedded features; and fuse the embedded features and hidden layer features to obtain a location representation.

[0052] In one possible implementation, the training module is specifically used to: input positional representations and encoded features into the encoder to output the text recognition results of the text lines.

[0053] In one possible implementation, the dataset acquisition module is specifically used to: acquire an image to be identified, the image to be identified including a target region containing text lines; and extract an image sample from the image to be identified, the image sample representing a portion of the image corresponding to the target region.

[0054] In one possible implementation, the image sample represents an image containing a seal, with the text line located inside the seal.

[0055] In one possible implementation, the start position represents the position point around the first character of the text line, and the end position represents the position point around the last character of the text line.

[0056] In one possible implementation, the image sample contains multiple lines of text to be identified. There are also multiple location labels, each representing the ground truth value of the start and end positions of one of the multiple text lines. The training module is specifically used to: obtain the text information of the text line corresponding to each of the multiple location information and encoded features in parallel. Each location information is used to represent the predicted start and end positions of one of the multiple text lines.

[0057] Fifthly, this application provides a chip system including a processor and a power supply circuit, the power supply circuit being used to supply power to the processor, and the processor being used to execute the method described in any possible implementation of the first and second aspects.

[0058] Sixthly, this application provides a computing device including a processor and a memory. The processor is configured to execute instructions stored in the memory, causing the computing device to perform the methods described in any possible implementation of the first and second aspects.

[0059] In a seventh aspect, this application provides a computing device cluster, including at least one computing device, each computing device including a processor and a memory; the processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, such that the computing device cluster performs the method described in any possible implementation of the first and second aspects.

[0060] Eighthly, this application provides a computer-readable storage medium including computer program instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method described in any possible implementation of the first and second aspects, wherein the cluster of computing devices includes at least one computing device.

[0061] Ninthly, this application provides a computer program product containing instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method described in any possible implementation of the first and second aspects, wherein the cluster of computing devices includes at least one computing device.

[0062] It is understood that the beneficial effects of aspects two through nine above can be found in the relevant descriptions in aspect one above, and will not be repeated here. Attached Figure Description

[0063] The accompanying drawings used in the embodiments or technical description are briefly introduced below.

[0064] Figure 1a A schematic diagram of a two-stage approach provided for related technologies;

[0065] Figure 1b A schematic diagram of the two-dimensional CTC method in the single-stage approach provided for related technologies;

[0066] Figure 1c A schematic diagram of a Transformer-based method in a single-stage approach provided for related technologies;

[0067] Figure 2a This is a schematic diagram of the structure of the seal recognition model provided in the embodiments of this application;

[0068] Figure 2b A schematic diagram illustrating the training process of the seal recognition model provided in the embodiments of this application;

[0069] Figure 3A schematic diagram illustrating an example of a seal recognition model provided in an embodiment of this application;

[0070] Figure 4 A schematic diagram illustrating the process of recognizing seal position information provided in an embodiment of this application;

[0071] Figure 5 A schematic diagram illustrating the process of recognizing seal text information provided in an embodiment of this application;

[0072] Figure 6 A schematic diagram illustrating an application scenario provided in an embodiment of this application;

[0073] Figure 7 A schematic diagram illustrating an example of a text recognition system provided in an embodiment of this application;

[0074] Figure 8 This is a schematic diagram illustrating a user interacting with a cloud computing platform, provided in an embodiment of this application.

[0075] Figure 9 A flowchart illustrating a text recognition method provided in an embodiment of this application;

[0076] Figure 10 A schematic flowchart illustrating a model training method provided in an embodiment of this application;

[0077] Figure 11 This is a schematic diagram illustrating the composition of a text recognition device provided in an embodiment of this application;

[0078] Figure 12 This is a schematic diagram of the composition of a model training device provided in an embodiment of this application;

[0079] Figure 13 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0080] Figure 14 This is a schematic diagram of the structure of a computing device cluster provided in an embodiment of this application;

[0081] Figure 15 This is a schematic diagram of another computing device cluster provided in an embodiment of this application. Detailed Implementation

[0082] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0083] In this article, the term "and / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three cases: A exists alone, A and B exist simultaneously, and B exists alone. The symbol " / " in this article indicates that the related objects have an "or" relationship; for example, A / B means A or B.

[0084] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.

[0085] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0086] In the description of the embodiments in this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, etc.; multiple elements means two or more elements, etc. "Several" means one or more.

[0087] With the development of large-scale knowledge base models and retrieval-augmented generation (RAG) technology, users increasingly need to quickly and accurately identify document content from complex documents. Most Optical Character Recognition (OCR) systems employ a modeling paradigm based on text line detection and recognition, which can already identify text content in documents quite well. However, in scenarios such as seals, traffic signs, commodities, and trademarks, existing systems often struggle to recognize text lines that convey crucial information.

[0088] Taking seals as an example, seals differ from regular text content, possessing complex and varied two-dimensional structures, text shapes, and reading orders. This presents significant challenges compared to the scenarios where existing OCR systems can handle horizontally distributed text within a single line or vertically distributed text across multiple lines. The text position representation methods of existing OCR systems suffer from insufficient granularity or excessive cost when applied to seal scenarios. Furthermore, the two-stage framework of text line detection and recognition also leads to error accumulation and inefficiency. In addition, background interference or text degradation commonly found in seals challenge the robustness of existing OCR systems. Therefore, current seal recognition technologies still have shortcomings in terms of accuracy, efficiency, and cost.

[0089] In related technologies, seal recognition schemes can be divided into two categories according to the processing procedure: two-stage methods and single-stage methods.

[0090] Figure 1a A schematic diagram of a two-stage approach provided for related technologies. (e.g.) Figure 1a As shown, in the two-stage method, the position of each line of text in the seal is first detected, then the text slices are cropped from the entire seal image based on the text position, and then the text recognition algorithm is used for recognition. Finally, the recognition results of each text are combined to determine their reading order based on manual rules to obtain the seal recognition result.

[0091] However, two-stage seal recognition methods have several problems. Firstly, the granularity of text detection is insufficient or the cost is too high. For example, the detection box granularity of a horizontal rectangular text detector is insufficient, meaning the detection box doesn't fit the text well enough, introducing a lot of background noise. Similarly, while the detection box of an arbitrary-shaped text detector can fit the text closely, training this detector requires labeling polygonal boxes containing multiple coordinate points, resulting in extremely high labeling costs. Secondly, the accuracy of text recognition is insufficient. For example, text lines cropped by horizontal rectangular boxes contain a lot of irrelevant background interference; arbitrary-shaped bounding boxes cause text distortion due to perspective transformations during cropping. Thirdly, rule-based reading order judgment has poor robustness. For example, simple top-to-bottom or layout-specific reading order rules have poor generalization across different layouts or rotation angles. Finally, two-stage methods suffer from error accumulation. For example, prediction errors in earlier steps can propagate to later steps, affecting the accuracy of the final recognition result.

[0092] In single-stage methods, there is no distinction between text detection and text recognition stages; instead, a single model identifies the content of the seal image end-to-end. Single-stage methods can be further divided into methods based on two-dimensional connectionist temporal classification (CTC) and methods based on Transformers.

[0093] Figure 1bA schematic diagram of the two-dimensional CTC method in a single-stage approach provided for related technologies. (Example) Figure 1b As shown, in the two-dimensional CTC-based method, the seal features are first extracted by CNN, and then processed alternately by self-attention and cross-attention modules. The obtained features are classified frame by frame and then CTC decoded to obtain the final recognition result.

[0094] However, single-stage stamp recognition methods based on 2D CTC have some problems. On the one hand, they have poor layout generalization. For example, for complex layouts (such as mixed horizontal and vertical layouts), the model struggles to implicitly and accurately focus on the corresponding text frame by frame according to the reading order through alternating attention mechanisms. On the other hand, training is unstable. For example, the complex 2D structure, text shapes, and reading order make it difficult for the model to converge in the early stages of training, easily leading to model degradation or crashes during the training process.

[0095] Figure 1c A schematic diagram of a Transformer-based method in a single-stage approach provided for related technologies. (Example) Figure 1c As shown, in the Transformer-based method, the seal features are first extracted using a CNN, and then the encoder and decoder of the Transformer are used to predict the text content in the seal in an autoregressive manner according to the reading order.

[0096] However, Transformer-based seal recognition methods have some problems. Firstly, they suffer from the illusion problem. When recognizing seals, especially when multiple repeating characters appear consecutively or the layout is complex, the output is prone to illusions due to the lack of positional information. Secondly, they are inefficient. For example, these methods use a serial autoregressive approach to predict the seal content character by character, which is time-consuming.

[0097] In view of this, embodiments of this application provide a text recognition method. During text recognition, an encoder extracts features from an image, and the encoder outputs an encoded feature vector (i.e., encoded features). The decoder first uses the encoded features as input to gradually "decode" the position information of text lines in the image. Based on this position information and the encoded features, text information at the position indicated by this position information can be generated. The position information includes one or more position pairs, each including a start position and an end position. The start position refers to the beginning of a text line in the seal, and the end position refers to the end of that text line. Furthermore, in each position pair, the start position comes first, followed by the end position, thus determining the reading order of the text line. In any position pair, the reading order of the text line is from the previous position to the next position in the position pair.

[0098] In this embodiment, the position of a text line is indicated by its start and end positions. This method is applicable to the labeling of text lines with any structure and layout, reducing interference introduced by rectangular detection boxes when detecting text lines with special structures, and thus improving the accuracy of text recognition within the text line. During model training, only the start and end positions of the text line need to be labeled, eliminating the need for complex detection boxes and reducing the labeling cost of text line positions. Determining the reading order by the start and end positions on both sides of the text line is applicable to reading orders of any version and rule, improving the robustness of reading order determination.

[0099] In this embodiment, the problem of locating text lines is simplified to identifying the start and end positions of the text lines. This simplifies the problem the model needs to handle, improves the model's inference efficiency, and enhances the stability of the model training process. The model only needs to determine the text content between the start and end positions, reducing or even eliminating the possibility of misleading recognition results.

[0100] In this application embodiment, a seal is used as an example for illustrative purposes, but this does not constitute a limitation on the scenario of this application embodiment.

[0101] Figure 2a This is a schematic diagram of the structure of the seal recognition model provided in an embodiment of this application. Figure 2a As shown, a seal recognition model includes an encoder and a decoder. During inference, after the seal image is input into the seal recognition model, the encoder extracts visual features of the seal image (such as the seal's texture, outline, and contrast with the background), outputting an encoded feature vector (i.e., encoded features). The decoder first uses the encoded features as input to gradually "decode" the seal's detailed information (such as text content, text position coordinates, and text arrangement order), outputting hidden layer features. These hidden layer features are the intermediate features generated during decoding, preceding the "final output."

[0102] The hidden layer features output by the decoder are intermediate features, containing both visual features of the seal (such as outline and texture) and detailed information about the seal (such as text content and text position). Therefore, the decoder can be used for both location recognition and text recognition tasks. In the location recognition task, the encoded features are input into the decoder, which performs a first decoding, extracting information related to the seal's position from the hidden layer features and outputting the seal's position information. In the text recognition task, the encoded features and the seal's position information are input into the decoder. Guided by the seal's position information, the decoder performs a second decoding, extracting text content information corresponding to the seal's position from the hidden layer features and outputting the seal's text content. In the diagram, the black triangle arrows correspond to the first decoding process for generating the seal's position information, and the broken line arrows correspond to the second decoding process for generating the seal's text information (i.e., the seal's text content).

[0103] In the location recognition task, the hidden layer features output by the decoder serve as input to the classification layer, which outputs the seal location information. The seal location information includes one or more location pairs, each pair containing a start position and an end position. The start position refers to the beginning of a text line on the seal, and the end position refers to the end of that text line. Furthermore, within each location pair, the start position comes first, followed by the end position, thus determining the reading order of the text line. In any location pair, the reading order of the text line is from the previous position to the next position in the pair.

[0104] For example, the position information of a text line can be represented by the endpoints of its start and end positions. The endpoint of the start position is located in the middle before the first character of the text line, and the endpoint of the end position is located in the middle after the last character of the text line.

[0105] In this embodiment, the position representation at the middle before the text content start field and the middle after the text content end field is merely illustrative. In other possible implementations, the same position recognition purpose can be achieved by using other positions such as the bottom or top of the text for text position annotation. This approach should be considered within the scope of protection of this embodiment. The main purpose of the dual-end fixed-point text position representation method proposed in this embodiment is to introduce text start and end related position information to the subsequent recognition module. Changing the aforementioned positions can also achieve the same purpose and belongs to the same inventive concept as this embodiment.

[0106] It should be noted that in related technologies, rectangular detection boxes can be used to identify the position of the text lines on the seal. However, rectangular detection boxes may have poor fit with text lines of special structures (such as curved text lines), introducing a large amount of background noise. Detection boxes of arbitrary shapes can be used to identify the position of the text lines on the seal, but the annotation cost of arbitrary shapes is extremely high. In the embodiments of this application, the position pairs including the start and end positions indicate the position of the text lines on the seal, reducing the annotation cost of detection boxes of arbitrary shapes and also reducing the interference information introduced when rectangular detection boxes detect text lines of special structures, thus helping to improve the seal recognition effect.

[0107] In the text recognition task, the seal location information output by the classification layer is used as input to the embedding layer, which outputs the embedding features of the seal location information. Endpoint representations are obtained by fusing the embedding features of the seal location information with the hidden layer features output by the decoder. The endpoint representations and encoded features are then used as input to the decoder to output the seal text information.

[0108] Incorporating the embedding features of the seal location information into the hidden layer features generated by the decoder is an optional implementation. Alternatively, the embedding and encoded features of the seal location information can be directly used as input to the decoder.

[0109] The hidden layer features generated by the decoder are intermediate representations that combine "image visual features" and "detail information prediction state". They retain the visual features of the seal from the encoder (such as the shape, edges, and pixel distribution of the seal) and also include the decoder's prediction state of the seal details (for example, the coordinates of the upper left corner of the seal have been predicted, and the next step is to predict the coordinates of the lower left corner).

[0110] For example, endpoint representation (also known as location representation) refers to the fused residual feature obtained by element-wise addition of embedding features and hidden layer features. This feature is not a simple information superposition, but a deep complementary fusion of the basic information of the original input and the higher-order features calculated by the sub-layers, possessing excellent information integrity, feature dimensionality consistency, gradient propagation preservation, and feature distribution stability. Element-wise addition is a lightweight and efficient fusion method that allows the decoded features to carry prior information. The fused features are then re-inputted into the decoder, allowing the decoder to perform a second decoding, outputting a more robust stamp recognition result.

[0111] After element-wise addition and fusion of the embedding features of the seal location information with the hidden layer features output by the decoder, the fused features are re-inputted into the decoder. This process forms the feature enhancement and cyclic decoding features in the decoding stage, increasing the contextual information of the location information and improving the accuracy of the seal recognition results. By supplementing the hidden layer features of the decoder with the prior information of the embedding features of the location information, the decoder can combine the visual features corresponding to the encoded features and the location information corresponding to the embedding features when generating the final output, accurately identifying the text recognition result at the location indicated by the location information, thus improving the accuracy of the text recognition results.

[0112] Optionally, to reduce the computational load on the seal recognition model and decrease input noise interference, the image to be recognized can be preprocessed. For example, the image to be recognized can be input into a seal detection module, which performs seal detection and target region cropping, outputting a local image containing only the seal area. Since this local image only retains the area where the seal is located, the amount of data is significantly reduced, effectively reducing model processing overhead and suppressing the influence of irrelevant noise.

[0113] In one possible implementation, the stamp detection module can be an image recognition system based on Optical Character Recognition (OCR) technology. Alternatively, it can be an image recognition system based on color thresholding. Finally, it can be a deep learning model for image recognition.

[0114] Figure 2b This is a schematic diagram illustrating the training process of the seal recognition model provided in an embodiment of this application. Figure 2b As shown, the training data for the seal recognition model includes seal image samples and their corresponding seal location labels and seal text labels. After inputting the seal image samples into the seal recognition model, the model outputs predicted values ​​for both seal location information and seal text information. The loss value A between the predicted seal location information and the seal location label is calculated, and the loss value B between the predicted seal text information and the seal text label is calculated. Based on the loss values ​​A and B, the parameters of the seal recognition model are adjusted. For example, the parameters can be adjusted by weighted summation of the loss values ​​A and B. More important loss values ​​can be assigned greater weight. The internal processing of the seal recognition model is described in other sections and will not be repeated here.

[0115] Figure 3 This is a schematic diagram illustrating an example of a seal recognition model provided in an embodiment of this application. For example... Figure 3As shown in the example, the image to be recognized is input into the seal detection module. The seal detection module obtains the seal image by detecting and cropping the seal in the image. The seal image is then input into the encoder in the encoding module to obtain encoded features. The encoded features are then input into the decoder to obtain hidden layer features. These hidden layer features are used, on the one hand, as input to the classification layer to output seal location information, and on the other hand, they are fused with the embedding features of the seal location information to obtain endpoint representations. The endpoint representations are input into the decoder, and the decoder outputs the text recognition result. The final text recognition result output by the decoder is the seal recognition result.

[0116] The decoder and classification layer, used to identify the seal location, are also known as the two-endpoint positioning module, which identifies the endpoints of the text lines containing seal location information. The decoder and embedding layer, used to identify the seal text, are also known as the parallel recognition module, which identifies the specific content of the text lines containing seal text information. The decoder shares parameters between the two-endpoint positioning module and the parallel recognition module, reducing the parameter size of the seal recognition model and improving its training efficiency.

[0117] Combination Figure 4 The first decoding process for generating seal position information will be further explained. For example... Figure 4 As shown, the dual-ended fixed-point module, based on the encoding features output by the encoding module, uses a specific start of sequence (SOS) as the first token and regressively predicts the coordinates of the two ends of each text line according to the reading order of the text lines, until a specific end of sequence (EOS) appears.

[0118] For example, the predicted paired-end fixed-point sequence { , … , },in The total number of text lines in the seal, { } represents the coordinates of a fixed point (e.g., the middle of a Chinese character) at either end of the nth line of text, in the order of reading. } represents the coordinates of a single point on the left. } represents the coordinates of a single point on the right. For example, the coordinates of a seal. Viewed as a sequence of 4-tokens, the decoder starts with the SOS token and predicts the next output token by token until the end-of-sequence token (EOS) is generated. The whole process is an autoregressive process of predicting the "next token" based on the "generated tokens".

[0119] The hidden layer features of the decoder are the intermediate features generated at each step of the decoding process, falling between the "encoder features" and the "final position output." For example, the decoder follows the stamp coordinates... Decoding and prediction are performed sequentially. In the first step of decoding, prediction... During the process, hidden layer features This includes the "edge features on the left side of the first token in the first text line along the x-axis" extracted by the encoder. This is used in the second step of decoding prediction. During the process, hidden layer features This includes the "edge features on the left side of the first token in the first text line along the y-axis" extracted by the encoder and the prediction from the previous step. The characteristics of [the target]. Predicting in the third step of decoding. During the process, hidden layer features This includes the "edge features on the right side of the last token in the first text line along the x-axis" extracted by the encoder and the predictions from the first two steps. , Features. Predicting in the fourth step of decoding. During the process, hidden layer features This includes the "edge features on the right side of the last token in the first text line along the y-axis" extracted by the encoder and the predictions from the first three steps. , , The characteristics of [the text]. It should be noted that in other examples, if the text is read from right to left, the relevant descriptions in the above examples need to be swapped.

[0120] For example, the image to be identified can be an invoice, document, etc., containing a seal. The seal could be, for instance, the "Financial Seal of [Company Name] in [City Name]," where "[Company Name]" is an example of the company name, and "Financial Seal" is an example of the seal type. In the seal location information, the first pair of locations corresponding to the company name is... The second set of position pairs corresponding to the seal category is: . The x-coordinate represents the starting position of the nth line of text. The y-coordinate represents the starting position of the nth line of text. The x-coordinate representing the end position of the nth line of text. This represents the y-coordinate of the end position of the nth text line. The decoder autoregressively presses... Predict them one by one in sequence.

[0121] Combination Figure 5This section further explains the first decoding process for generating the seal text information (i.e., text line recognition results and text recognition results). For example... Figure 5 As shown, after obtaining the two-ended fixed points output in the reading order, the coordinates of the fixed points are then input into { , … , The data is then transferred to the parallel recognition module. In this module, endpoint representations are first extracted, as shown on the left side of the figure. Endpoint coordinates are first processed through an embedding layer to obtain embedding features, which are then added to the hidden layer features corresponding to the decoder in the endpoint localization module to obtain the final endpoint representation. For example, position coordinates... The corresponding endpoints are represented as Then, parallel text recognition is performed. Utilizing a decoder that shares parameters with the endpoint localization module, the content of each text line is recognized in parallel based on the encoded features output by the encoder and the endpoint features extracted from multiple sets of dual-end single-point localization, thus improving the recognition efficiency of the autoregressive model. Finally, the parallel text recognition module outputs the complete text line recognition results within the seal in reading order.

[0122] Since seals are often used as key information on documents, the need for seal recognition is widespread. Embodiments of this application can be used as components of OCR-related products, serving as one of the fundamental capabilities for document recognition. Furthermore, due to the development of large-scale knowledge base models and RAG technology, users need to extract relevant content from complex documents; therefore, embodiments of this application can also be used in the technical field of large-scale knowledge base products.

[0123] For example, the technical solutions of this application embodiment can be provided to users as cloud service interfaces. For instance, the seal recognition technical solution provided in this application embodiment can be directly provided to users as an independent cloud service capability through an Application Programming Interface (API). Furthermore, the seal recognition technical solution provided in this application embodiment can also be integrated as a sub-module of general document recognition into a complete solution, and provided to users as a cloud service.

[0124] For example, the technical solutions of this application embodiment can also provide services to users through user-local integration. For instance, since seal documents typically contain important and critical data, some users may have data security needs, and the data cannot be uploaded to the cloud. To address this need, the technical solutions and framework proposed in this application embodiment also support user-side edge deployment. By providing a complete integration solution, it enables customers to achieve a full-process application from data annotation and model training to local deployment of seal recognition services at the customer's site. Furthermore, the dual-end fixed-point seal text annotation method proposed in this application embodiment can effectively reduce annotation costs and improve annotation efficiency. Therefore, in this scenario, the technical solutions of this application embodiment are more competitive.

[0125] This application mainly proposes a dual-end fixed-point position information representation method for text in seal recognition scenarios, and combines it with a parallel text recognition method to effectively reduce seal recognition costs and improve seal recognition efficiency.

[0126] Figure 6 This is a schematic diagram illustrating an application scenario provided by an embodiment of this application. For example... Figure 6 As shown, in one application scenario, the technical solution implemented in this application embodiment can be carried out via cloud services. The implementation process can be divided into three stages. First, after obtaining training data samples, the positions of the seal text in the image samples are marked using a dual-end fixed-point method. Then, the seal recognition model is trained to obtain a seal recognition model that meets the performance requirements. Next, the seal recognition model is deployed to a cloud server, allowing different clients to access it. Subsequently, the seal recognition model is provided to users via API calls. Users send the data to be recognized to the server through the API interface, and then can obtain the seal recognition results output by the model through the network.

[0127] In another application scenario, since the content of the seal may involve some critical outputs, users may have concerns about data privacy. Therefore, the embodiments of this application can also be used for users to train, deploy, and invoke the seal recognition model locally. During the training phase, users need to collect training samples themselves, which can be labeled by themselves or by purchasing labeling services; then, the training method proposed in the embodiments of this application is used to train the seal recognition model. After training is completed, the seal recognition model is deployed to the user's local server, and then the seal recognition model can be invoked through code or by building a graphical user interface.

[0128] It should be noted that the technical solution proposed in this application is mainly used to solve text content recognition scenarios with complex layouts and critical content, such as seal recognition. Therefore, for other text content with complex layouts, the method proposed in this application can also be used for position representation to provide information for subsequent recognition modules. In particular, for non-horizontal text layouts such as slanted or curved lines, traditional two-point horizontal rectangular text position annotation introduces a large amount of redundant information, and polygonal position annotation methods are costly. The embodiments of this application have significant advantages in position representation for these special text arrangements. Based on this idea, the technical solution of this application can also be extended to some other similar scenarios.

[0129] For example, the technical solutions of this application embodiment can also be extended to the transportation field for tasks such as traffic sign recognition, road sign content recognition, and license plate recognition. As another example, the technical solutions of this application embodiment can also be extended to the retail field for tasks such as product packaging content recognition and product label content recognition. Furthermore, the technical solutions of this application embodiment can also be extended to the engineering field for tasks such as special format annotation recognition of engineering drawings and construction drawing content recognition. The embodiments of this application can be used in the scenarios listed above, but are not limited to the enumerated cases; other scenarios can be specifically determined according to the actual situation.

[0130] Figure 7 This is a schematic diagram illustrating an example of a text recognition system provided in an embodiment of this application. For example... Figure 7 As shown, in one example, the text recognition system 700 includes a terminal device 710 and a server 720.

[0131] For example, terminal device 710 acquires a first image input by the user, the first image containing a text line to be recognized, and sends the first image to server 720; server 720 inputs the first image into encoder to output encoded features; server 720 inputs the encoded features into decoder to output position pairs, the position pairs including a first position and a second position, the first position representing the position of the first character of the text line, the second position representing the position of the last character of the text line, and the order of the text line representing the order from the first position to the second position; server 720 performs secondary decoding based on the position pairs and encoded features to obtain the text recognition result of the text line; server 720 sends the text recognition result to terminal device 710; terminal device 710 displays the text recognition result to the user, realizing user interaction.

[0132] Optionally, server 720 may include one or more servers; and / or, server 720 may provide one or more terminal devices 710 with the methods or apparatus provided in the embodiments of this application.

[0133] Optionally, a relevant application may be installed on the terminal device 710, and the application or webpage may provide an interface; the terminal device 710 may receive inference requests input by the user on the interface and send them to the server 720.

[0134] It should be understood that in some optional implementations, the terminal device 710 may also act as a node and complete the work itself without the cooperation of the server 720, and this application embodiment is not limited in this regard. In some optional implementations, the server 720 may also rely on the first image received from other devices besides the terminal or locally stored, without the cooperation of the terminal device 710, and this application embodiment is not limited in this regard.

[0135] The following description Figure 7 The product form of the terminal device 710. In this application embodiment, the terminal device 710 can be a mobile phone, tablet computer, wearable device, in-vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc., and this application embodiment does not impose any restrictions on it.

[0136] It can be further understood that server 720 can be various types of servers, such as x89 architecture servers, specifically rack servers, blade servers, high-density servers, rack servers, or high-performance servers, etc. In other words, this application embodiment does not specifically limit the specific type of server. Furthermore, it can be understood that... Figure 7 The server structure shown does not constitute a limitation on the server structure. A server may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0137] Furthermore, the server 720 can be configured as an independent physical server, or as a server cluster or distributed system consisting of multiple physical servers. It can also be configured as a cloud server or cloud server cluster that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The cloud server cluster is deployed in several cloud data centers. The software can be an application that implements object control methods, but it is not limited to the above forms.

[0138] In one possible scenario, server 720 can function as a cloud platform (a software platform employing application virtualization technology, integrating multiple functions such as software search, download, use, management, and backup). In practical use, server 720 can deploy a cloud management platform and a data center, with terminal device 710 interacting with the cloud through the cloud management platform. Additionally, the data center can deploy nodes, which can be virtual machine instances, container instances, physical servers, etc.

[0139] Furthermore, the systems that can implement the technical solutions of this application embodiment include cloud servers, web pages, application programming interfaces (APIs), mobile application software, etc. The inference model of this application embodiment can be deployed on the server side, website, or mobile application software to meet the text generation needs of enterprise users and individual users, and can also be used by developers for secondary development through open API interfaces. Furthermore, the implementation of this application embodiment mainly involves deploying the relevant platforms and applications of the inference model of this application embodiment, such as cloud platforms, websites, and applications (APPs).

[0140] In another possible scenario, the method provided in this application embodiment can be implemented by software. The software has a terminal and a server; the terminal device 710 runs the software, and the server 720 runs the software's server. While the terminal device 710 is running the software, it can call the server running on the server 720 to implement the method provided in this application embodiment.

[0141] In other words, the method provided in this application embodiment can be applied to the aforementioned terminal device 710 to realize inference between multiple terminal devices; the method provided in this application embodiment can also be applied to the server 720 to realize inference between multiple server nodes. In specific implementation, it can run on the terminal device 710 or the server 720 in a software manner. For example, the software can be a service or an application. This application embodiment can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application embodiment can also be practiced in a distributed computing environment, where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can reside in local and remote computer storage media, including storage devices.

[0142] In some embodiments, each component of the solution framework described above can be configured on a cloud computing platform, for example, deployed on at least one instance such as a virtual machine or container, so that the cloud computing platform can provide text recognition services. Of course, each component of the solution framework described above can also be configured on nodes other than the cloud computing platform, for example, deployed in at least one data center or on at least one server, depending on the actual situation, and is not limited here.

[0143] The cloud computing platform can provide pages related to public cloud services for users to remotely access. In this embodiment, users can pre-purchase text recognition services on the cloud computing platform. For ease of understanding, the interaction between the user and the cloud computing platform is described below.

[0144] Figure 8 This is a schematic diagram illustrating a user interacting with a cloud computing platform, as provided in an embodiment of this application. Figure 8 As shown, the interaction between the user and the cloud computing platform mainly includes: the user logs into the cloud computing platform 800 through a client webpage, selects and purchases a text recognition service on the cloud computing platform 800, and after purchase, the user can perform inference on multiple computing nodes based on the functions provided by the text recognition service, and use one or more computing nodes on the cloud computing platform 800 to perform text recognition. During the text recognition process, the cloud computing platform 800 executes the method of this application embodiment, indicating the position of the text line through the start and end positions. This method is applicable to the position annotation of text lines with any structure and any layout, reducing the interference information introduced by rectangular detection boxes when detecting text lines with special structures, reducing the annotation cost of text line positions, improving the robustness of reading order judgment, improving the inference efficiency of the model, improving the stability of the model training process, and reducing or even avoiding illusions in the model's recognition results.

[0145] The cloud computing platform 800 is primarily used to manage the infrastructure for running text recognition services. For example, the infrastructure for running text recognition services can include multiple data centers located in different regions, each containing multiple servers. Data centers can provide basic resources for text recognition services, such as computing resources and storage resources. Therefore, when users purchase and use text recognition services, they primarily pay for the resources they use.

[0146] For example, when using the text recognition service, users can input text recognition tasks through the configuration interface, application programming interface, or user interaction interface provided by the cloud computing platform 800. Then, the cloud computing platform 800 can perform text recognition on one or more computing nodes according to the text recognition task input by the user.

[0147] In some embodiments, the components of the scheme framework described above can also be configured on a local server, depending on the actual situation, and are not limited here.

[0148] Figure 9 This is a flowchart illustrating a text recognition method provided in an embodiment of this application. Figure 9 As shown, this application embodiment provides a text recognition method, which mainly includes steps S910 to S940.

[0149] Step S910: Obtain a first image, which contains the text line to be recognized. The first image may be, for example, a stamp image, a traffic sign image, or a product packaging image.

[0150] Step S920: Input the first image into the encoder to output encoded features.

[0151] Step S930: Input the encoded features into the decoder to output position pairs. The position pairs include a first position and a second position. The first position represents the position of the first character of the text line, and the second position represents the position of the last character of the text line. The order of the text lines represents the order from the first position to the second position.

[0152] For example, the first position can also be a start position, beginning position, or other descriptions, and the second position can also be an end position, termination position, or other descriptions. The order of text lines refers to the order in which the text lines are read, which is a natural word order.

[0153] Step S940: Based on the position pairs and encoding features, obtain the text recognition result of the text line.

[0154] In this step, guided by the position pairs, the text content at the positions indicated by the position pairs in the encoded features is parsed to obtain the text recognition results of the text lines, which can improve the accuracy of the text recognition results.

[0155] In this embodiment, the position of a text line is indicated by its start and end positions. This method is applicable to the labeling of text lines with any structure and layout, reducing interference introduced by rectangular detection boxes when detecting text lines with special structures, and thus improving the accuracy of text recognition within the text line. During model training, only the start and end positions of the text line need to be labeled, eliminating the need for detection boxes and reducing the labeling cost. Determining the reading order by the start and end positions on both sides of the text line is applicable to reading orders of any version and rule, improving the robustness of reading order determination. Simplifying the text line localization problem to the identification of the start and end positions of the text line simplifies the problems the model needs to handle, improves the model's inference efficiency, and enhances the stability of the model training process. The model only needs to determine the text content between the start and end positions, reducing or even eliminating the illusion of model recognition results.

[0156] For example, this application aims to address the problems of insufficient granularity or excessive cost associated with horizontal text detectors or arbitrary-shape text detectors in the field of seal recognition. Horizontal text detectors locate the text detection box using the coordinates of two diagonally opposite points. This method satisfies the detection requirements for horizontal text while reducing annotation costs. However, it easily introduces redundant background information in non-horizontal text scenarios, leading to insufficient segmentation granularity and decreased recognition accuracy. While arbitrary-polygon text detection boxes can meet the segmentation requirements for arbitrary-shaped text, they require multiple points for selection, resulting in high annotation costs. To address the problems of these two text detectors, this application innovatively proposes a text position representation method based on dual-end fixed points, which reduces annotation costs while avoiding the decrease in accuracy caused by insufficient granularity.

[0157] Furthermore, existing autoregressive stamp extraction methods based on Transformers require serial output of all text in the stamp, waiting for the previous text sequence to complete before outputting subsequent text sequences, resulting in low efficiency and long processing time. This application's embodiment introduces a parallel text recognition module, allowing each text recognition process to be output in parallel, reducing the dependence of the output process on previous text outputs, decreasing error accumulation, and accelerating the text output process.

[0158] In summary, by combining a dual-end fixed-point text location representation module and a parallel text recognition module, this embodiment of the application reduces the problem of decreased recognition accuracy caused by the introduction of redundant information and improves recognition efficiency compared with existing technologies, while ensuring low annotation costs.

[0159] This application primarily proposes an end-to-end text recognition method combining dual-end point localization and parallel text recognition. On one hand, it proposes a text representation method based on dual-end point localization. Dual-end point localization uses the start and end points of a text line to represent its positional information, where the start point is located in the middle before the first character, and the end point is located in the middle of the last character's end. This reduces the cost of polygonal bounding boxes and minimizes interference introduced by parallel bounding boxes, thus improving the seal recognition effect. Correspondingly, a dual-end point localization prediction module is proposed. For the proposed dual-end point localization text representation method, an autoregressive recognition module is implemented. This module, based on image encoding features, uses an encoder to autoregressively predict the coordinates of the dual-end point of each text line according to the reading order, serving as the positional representation for subsequent parallel text recognition.

[0160] On the other hand, the embodiments of this application mainly propose a parallel recognition module. This module first uses double-ended fixed points as parallel input, then extracts the endpoint representations of each text line, and then uses a decoder that shares parameters with the double-ended fixed point module to recognize the content of each text line in parallel based on the endpoint representations of each text line. By recognizing multiple text lines within the seal in parallel, the recognition efficiency is improved.

[0161] This application proposes a seal recognition framework combining two-end fixed-point mapping and parallel text recognition. The framework first obtains the encoded features of the seal image through an encoding module. Then, it uses a two-end fixed-point mapping module to predict the coordinates of the two ends of each text line according to the reading order. Finally, based on these coordinates, a parallel recognition module identifies the content of each text line, ultimately yielding the seal recognition result. The modules are coupled end-to-end at the feature level, and the decoder modules for two-end fixed-point mapping and parallel text recognition share parameters, improving parameter utilization.

[0162] In summary, this application's embodiments innovatively propose a dual-end fixed-point representation method for text location in seal recognition scenarios. It also enables the dual-end fixed-point prediction module to output the dual-end fixed-point coordinates of text lines in the seal according to the text reading order based on the seal image encoding features. Furthermore, by combining the decoupling characteristics of dual-end fixed-point recognition, a parallel seal text recognition module is implemented, improving recognition efficiency. Based on the seal recognition process proposed in this application's embodiments, the above modules are integrated. By sharing the decoder parameters for dual-end fixed-point recognition and parallel text recognition, a complete end-to-end framework from seal image encoding features to seal recognition results is achieved.

[0163] In related technologies, rectangular detection boxes are used to identify the position of text lines. However, rectangular detection boxes may not fit well with text lines of certain special structures (such as curved text lines), introducing a large amount of background noise interference. In the embodiments of this application, position pairs including start and end positions are used to indicate the position of text lines. This can be applied to the position annotation of text lines of any structure, reducing the interference information introduced by rectangular detection boxes when detecting text lines of special structures, and helping to improve the accuracy of text recognition in text lines.

[0164] In related technologies, arbitrary-shaped detection boxes are used to identify the position of text lines, but the annotation cost of arbitrary shapes is extremely high. In the embodiments of this application, position pairs including start and end positions are used to indicate the position of text lines. When annotating the position of text lines, only the start and end positions of the text lines need to be annotated, and complex detection boxes do not need to be annotated, thus reducing the annotation cost of arbitrary-shaped detection boxes.

[0165] In related technologies, rule-based reading order determination suffers from poor robustness. For example, simple top-to-bottom reading order rules or rules specific to a particular layout have poor generalization across different layouts or rotation angles. In the embodiments of this application, each position is centered, with the start position preceding the end position. This determines the reading order of this text line as starting from the start position and ending at the end position. Thus, determining the reading order by the start and end positions on both sides of the text line can be applied to reading orders of any version and rule, improving the robustness of reading order determination.

[0166] In related technologies, the two-stage method of detecting the position of a text line and recognizing its content suffers from error accumulation. For example, prediction errors in the earlier steps can propagate to later steps, affecting the accuracy of the final text recognition result. In this embodiment, the start position indicates the approximate position of the first character of the text line, and the end position indicates the approximate position of the last character of the text line. In this way, even with a small error in the start and end positions, the model can still recognize the correct text content, reducing the adverse impact of text line position errors on the accuracy of the text recognition result.

[0167] In related technologies, there is a problem with poor layout generalization. For example, for complex layouts (such as mixed horizontal and vertical layouts), the model struggles to implicitly and accurately focus on the corresponding text frame by frame according to the reading order through alternating attention mechanisms. In the embodiments of this application, the position of a text line is determined by the start and end positions on both sides of the text line. In complex layouts, the start and end positions of each text line can also be identified, thereby determining the position of each text line.

[0168] In related technologies, there is a problem of unstable training. For example, complex two-dimensional structures, text shapes, and reading orders make it difficult for the model to converge in the early stages of training, which can easily lead to model degradation or crashes during the training process. In the embodiments of this application, in scenarios with complex two-dimensional structures, text shapes, and reading orders, the problem of locating text lines can be simplified to the problem of identifying the start and end positions of text lines. This simplifies the problem that the model needs to handle, resulting in faster model training convergence and a more stable training process.

[0169] In related technologies, Transformer-based seal recognition methods suffer from the illusion problem. When recognizing seals, especially when multiple repeating characters appear consecutively or the layout is complex, the output is prone to illusions due to the lack of positional guidance. In this embodiment, the position of the text line is determined by the start and end positions on both sides. The model only needs to determine the text content between the start and end positions, reducing or even eliminating the illusion in the model's recognition results.

[0170] In related technologies, Transformer-based seal recognition methods suffer from low efficiency. For example, these methods predict seal content character by character using a serial autoregressive approach, which is time-consuming. In this embodiment, in scenarios with multiple text lines, the start and end positions of both sides of each text line are predicted sequentially. The model can recognize the text content of multiple text lines in parallel based on their start and end positions, thus improving the efficiency of text content prediction.

[0171] In one possible implementation, inputting coded features into a decoder to output position pairs includes: inputting coded features into a decoder to output a first position; and inputting the coded features and the first position into a decoder to output a second position.

[0172] This implementation provides an autoregressive position generation process. By employing this process, position information can be predicted segment by segment in an orderly manner, fully utilizing the prediction results of preceding positions as prior information for subsequent predictions. This improves the continuity, accuracy, and spatial consistency of position prediction, and is beneficial for enhancing the model's localization accuracy for text line position pairs.

[0173] In one possible implementation, inputting encoded features into a decoder to output position pairs includes: inputting encoded features into a decoder to output hidden layer features; and inputting hidden layer features into a classification layer to output position pairs.

[0174] In this implementation, the task of recognizing position pairs can be achieved by configuring a classification layer on the output side of the decoder. The hidden layer features output by the decoder are intermediate features, which contain both visual features of the first image (such as contours and textures) and detailed information of the first image (such as text line content and text line position). During the position recognition process, the decoder decodes the input encoded features once, extracts information related to the text line position from the hidden layer features, and outputs the position pairs corresponding to the text lines, ensuring the accuracy of the position pairs.

[0175] In one possible implementation, the text recognition result of the text line is obtained based on the position pairs and encoding features, including: obtaining the position representation based on the position pairs and hidden layer features; and obtaining the text recognition result of the text line based on the position representation and encoding features.

[0176] In this implementation, hidden layer features are fused into position pairs to obtain position representations. During text recognition, the decoder's input includes encoded features and position representations. Guided by the position representations, the decoder performs secondary decoding on the encoded features, extracting text content information at the positions indicated by the position representations from the hidden layer features, and outputting the text recognition result. Hidden layer features represent high-dimensional, macroscopic features of the first image. Thus, the position representations not only contain information about the position pairs representing text line locations but also macroscopic, high-dimensional semantic information such as contextual information representing the position pairs. This improves the completeness and semantic richness of the information represented by the position representations of the text line locations, thereby improving the accuracy of the text recognition results. For example, the contextual information of the first position includes a seal border on the left, text above, and a blank area below.

[0177] In one possible implementation, the location representation is obtained based on the location pairs and hidden layer features, including: inputting the location pairs into the embedding layer to output embedded features; and fusing the embedded features and hidden layer features to obtain the location representation.

[0178] In this implementation, the embedding layer maps discrete position pairs to a continuous high-dimensional vector space, generating embedding features with semantic representation capabilities. These embedding features can be fused with the hidden layer features of the decoder, supplementing prior information and improving the model's ability to learn and represent the positional representations of text lines.

[0179] In one possible implementation, the text recognition result of the text line is obtained based on the position representation and encoding features, including: inputting the position representation and encoding features into an encoder to output the text recognition result of the text line.

[0180] In this implementation, the position representation and encoded features are input into the encoder to output the text recognition result of the text line. This enables the encoder to learn the spatial position features and visual semantic features of the text line at the same time, strengthens the spatial correlation and semantic recognition of the text line features, and effectively improves the accuracy, continuity and anti-interference ability of the text recognition result.

[0181] In one possible implementation, acquiring the first image includes: acquiring a second image to be identified, the second image including a target region containing text lines; and extracting the first image from the second image, the first image representing a portion of the image corresponding to the target region.

[0182] In this implementation, a first image corresponding to the target region containing the text line is extracted from the original second image input by the user. The encoder and decoder only need to process the image of the target region (such as the inside of the seal), which can reduce the computational load of the encoder and decoder, and also reduce noise interference in areas outside the target region, thereby improving the accuracy of the text recognition results.

[0183] In one possible implementation, the first image represents an image containing a seal, with the text line located inside the seal.

[0184] In this implementation, the seal is merely an illustrative example and does not constitute a limitation on the scenarios of the embodiments of this application.

[0185] In one possible implementation, the first position represents the position point around the first character of the text line, and the second position represents the position point around the last character of the text line.

[0186] This implementation provides an illustrative example of the first and second positions, but this does not constitute a limitation on the first and second positions. Any method that can identify the position of a text line is a feasible implementation, such as marker lines, special symbols, etc.

[0187] In one possible implementation, the first image contains multiple lines of text to be recognized and multiple position pairs, where each position pair represents a first position and a second position of one of the multiple text lines. Based on the position pairs and encoded features, the text recognition result of the text lines is obtained, including: obtaining the text recognition result of the text lines corresponding to each position pair in the multiple position pairs in parallel, based on the multiple position pairs and encoded features.

[0188] In this implementation, in scenarios with multiple lines of text, the text recognition results of multiple lines of text can be recognized in parallel based on the start and end positions of the multiple lines of text, thereby improving the efficiency of predicting text recognition results.

[0189] Figure 10This is a flowchart illustrating a model training method provided in an embodiment of this application. Figure 10 As shown, this application embodiment provides a model training method, which mainly includes steps S1010 to S1030.

[0190] Step S1010: Obtain the training dataset. The training dataset includes image samples, position labels, and text labels. The position labels represent the truth values ​​of the start and end positions of the text lines in the image samples. The start position represents the position of the first character of the text line, and the end position represents the position of the last character of the text line. The order of the text lines represents the order from the start position to the end position. The text labels represent the truth values ​​of the text content of the text lines.

[0191] Step S1020: Input the image sample into the text recognition model to output location information and text information. The text recognition model includes at least an encoder and a decoder. The encoder is used to generate encoded features of the image sample, and the decoder is used to generate location information based on the encoded features. The decoder is also used to generate text information based on the location information and encoded features. The location information represents the predicted start and end positions of the text line, and the text information represents the predicted text content of the text line.

[0192] Step S1030: Train the text recognition model based on the loss between location information and location labels, and the loss between text information and text labels.

[0193] In one possible implementation, location labels are obtained by annotating the start and end positions of text lines in the image sample. This implementation uses location pairs including the start and end positions to label the text lines, requiring only the start and end positions of the text lines to be labeled, eliminating the need to label bounding boxes of rectangular, arc-shaped, or other similar shapes, thus reducing the labeling cost of bounding boxes.

[0194] In one possible implementation, the process by which the decoder generates position information based on the encoded features includes: inputting the encoded features into the decoder to output a predicted value for the start position; and inputting the encoded features and the start position into the decoder to output a predicted value for the end position.

[0195] In one possible implementation, the process of the decoder generating location information based on the encoded features includes: inputting the encoded features into the decoder to output hidden layer features; and inputting the hidden layer features into the classification layer to output location information.

[0196] In one possible implementation, the process by which the decoder obtains the text recognition result of the text line based on the position information and the encoding features includes: obtaining the position representation based on the position pairs and the hidden layer features; and obtaining the text recognition result of the text line based on the position representation and the encoding features.

[0197] In one possible implementation, the process by which the decoder obtains a position representation based on the position pairs and hidden layer features includes: inputting the position pairs into the embedding layer to output embedded features; and fusing the embedded features and hidden layer features to obtain the position representation.

[0198] In one possible implementation, the process by which the decoder obtains the text recognition result of the text line based on the position representation and encoding features includes: inputting the position representation and encoding features into the encoder to output the text recognition result of the text line.

[0199] In one possible implementation, the process of generating an image sample includes: acquiring an image to be identified, the image to be identified including a target region containing text lines; and extracting an image sample from the image to be identified, the image sample representing a portion of the image corresponding to the target region.

[0200] In one possible implementation, the image sample represents an image containing a seal, with the text line located inside the seal.

[0201] In one possible implementation, the start position represents the position point around the first character of the text line, and the end position represents the position point around the last character of the text line.

[0202] In one possible implementation, the image sample contains multiple lines of text to be identified. There are also multiple location labels, each representing the ground truth value of the start and end positions of one of the multiple text lines. The decoder generates the text information by: obtaining the text information of each line corresponding to each of the multiple location information and encoded features in parallel. Each location information represents the predicted start and end positions of one of the multiple text lines.

[0203] Figure 11 This is a schematic diagram illustrating the composition of a text recognition device provided in an embodiment of this application. Figure 11 As shown, this application embodiment provides a text recognition device 1100, which mainly includes an image acquisition module 1110 and a processing module 1120.

[0204] Image acquisition module 1110 is used to acquire a first image, which contains the text line to be recognized. The first image may be, for example, a seal image, a traffic sign image, or a product packaging image.

[0205] The processing module 1120 is used to input the first image into the encoder to output encoded features. The encoded features are then input into the decoder to output position pairs, each pair including a first position and a second position. The first position represents the position of the first character of the text line, and the second position represents the position of the last character of the text line. The order of the text line represents the sequence from the first position to the second position. Based on the position pairs and the encoded features, the text recognition result of the text line is obtained. By decoding the text content at the positions indicated by the position pairs in the encoded features under the guidance of the position pairs, the accuracy of the text recognition result can be improved.

[0206] In one possible implementation, the processing module 1120 is specifically used to: input the encoded features into the decoder to output a first position; and input the encoded features and the first position into the decoder to output a second position.

[0207] In one possible implementation, the processing module 1120 is specifically used to: input the encoded features into the decoder to output hidden layer features; and input the hidden layer features into the classification layer to output position pairs.

[0208] In one possible implementation, the processing module 1120 is specifically used to: obtain a position representation based on the position pairs and hidden layer features; and obtain the text recognition result of the text line based on the position representation and the encoding features.

[0209] In one possible implementation, the processing module 1120 is specifically used to: input the position pair into the embedding layer to output the embedded feature; and fuse the embedded feature and the hidden layer feature to obtain the position representation.

[0210] In one possible implementation, the processing module 1120 is specifically used to: input the position representation and encoding features into the encoder to output the text recognition result of the text line.

[0211] In one possible implementation, the image acquisition module 1110 is specifically used to: acquire a second image to be recognized, the second image including a target region containing text lines; and extract a first image from the second image, the first image representing a portion of the image corresponding to the target region.

[0212] In one possible implementation, the first image represents an image containing a seal, with the text line located inside the seal.

[0213] In one possible implementation, the first position represents the position point around the first character of the text line, and the second position represents the position point around the last character of the text line.

[0214] In one possible implementation, the first image contains multiple lines of text to be recognized and multiple position pairs, where each position pair represents the first and second positions of one of the multiple text lines. The processing module 1120 is specifically used to: obtain, in parallel, the text recognition result of the text line corresponding to each position pair in the multiple position pairs based on the multiple position pairs and the encoded features.

[0215] Figure 12 This is a schematic diagram illustrating the composition of a model training device provided in an embodiment of this application. Figure 12 As shown in the figure, this application embodiment provides a model training device 1200, which mainly includes a dataset acquisition module 1210 and a training module 1220.

[0216] The dataset acquisition module 1210 is used to acquire the training dataset. The training dataset includes image samples, position labels, and text labels. The position labels represent the truth values ​​of the start and end positions of the text lines in the image samples. The start position represents the position of the first character of the text line, and the end position represents the position of the last character of the text line. The order of the text lines represents the order from the start position to the end position. The text labels represent the truth values ​​of the text content of the text lines.

[0217] Training module 1220 is used to input image samples into the text recognition model to output location information and text information. The text recognition model includes at least an encoder and a decoder. The encoder generates encoded features of the image samples, the decoder generates location information based on the encoded features, and the decoder also generates text information based on the location information and encoded features. The location information represents the predicted start and end positions of text lines, and the text information represents the predicted text content of the text lines. The text recognition model is trained using losses between location information and location labels, and between text information and text labels.

[0218] In one possible implementation, location labels are obtained by annotating the start and end positions of text lines in the image sample. This implementation uses location pairs including the start and end positions to label the text lines, requiring only the start and end positions of the text lines to be labeled, eliminating the need to label bounding boxes of rectangular, arc-shaped, or other similar shapes, thus reducing the labeling cost of bounding boxes.

[0219] In one possible implementation, the training module 1220 is specifically used to: input the encoded features into the decoder to output the predicted value of the start position; and input the encoded features and the start position into the decoder to output the predicted value of the end position.

[0220] In one possible implementation, the training module 1220 is specifically used to: input the encoded features into the decoder to output hidden layer features; and input the hidden layer features into the classification layer to output location information.

[0221] In one possible implementation, the training module 1220 is specifically used to: obtain a position representation based on the position pairs and hidden layer features; and obtain the text recognition result of the text line based on the position representation and the encoded features.

[0222] In one possible implementation, the training module 1220 is specifically used to: input the position pairs into the embedding layer to output embedded features; and fuse the embedded features and hidden layer features to obtain a position representation.

[0223] In one possible implementation, the training module 1220 is specifically used to: input position representations and encoded features into the encoder to output the text recognition result of the text line.

[0224] In one possible implementation, the dataset acquisition module 1210 is specifically used to: acquire an image to be identified, the image to be identified including a target region, the target region containing text lines; and extract an image sample from the image to be identified, the image sample representing a portion of the image corresponding to the target region.

[0225] In one possible implementation, the image sample represents an image containing a seal, with the text line located inside the seal.

[0226] In one possible implementation, the start position represents the position point around the first character of the text line, and the end position represents the position point around the last character of the text line.

[0227] In one possible implementation, the image sample contains multiple lines of text to be identified. There are also multiple location labels, each representing the ground truth value of the start and end positions of one of the multiple text lines. The training module 1220 is specifically used to: obtain the text information of the text line corresponding to each of the multiple location information and encoded features in parallel. Each location information is used to represent the predicted start and end positions of one of the multiple text lines.

[0228] The software and hardware implementation of the text recognition device 1100 and the model training device 1200 (hereinafter referred to as: the devices of this application embodiment) will be further explained below.

[0229] In some embodiments, each module of the apparatus of this application can be implemented in software or in hardware. For example, the implementation of image acquisition module 1110 will be described below. Similarly, the implementation of processing module 1120, dataset acquisition module 1210, and training module 1220 can refer to the implementation of image acquisition module 1110.

[0230] As an example of a software functional unit, the image acquisition module 1110 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, or a container. Further, the aforementioned computing instance may be one or more. For example, the image acquisition module 1110 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0231] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0232] As an example of a hardware functional unit, the image acquisition module 1110 may include at least one computing device, such as a server. Alternatively, the image acquisition module 1110 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0233] The image acquisition module 1110 includes multiple computing devices that can be distributed within the same region or in different regions. Similarly, the image acquisition module 1110 can be distributed within the same Availability Zone (AZ) or in different AZs. Likewise, the image acquisition module 1110 can be distributed within the same Virtual Private Cloud (VPC) or in multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0234] It should be noted that, in other embodiments, the image acquisition module 1110 can be used to execute any step in the text recognition method described in the above embodiments, and the processing module 1120 can also be used to execute any step in the text recognition method described in the above embodiments. Furthermore, the steps implemented by the image acquisition module 1110 and the processing module 1120 can be specified as needed, and the text recognition device 1100 can achieve some or all of its functions by implementing different steps in the text recognition method described in the above embodiments through the image acquisition module 1110 and the processing module 1120 respectively. Similarly, the embodiments of this application do not limit the steps implemented by the dataset acquisition module 1210 and the training module 1220 in the model training device 1200.

[0235] This application also provides a computing device 1300. For example... Figure 13As shown, the computing device 1300 includes a bus 1302, a processor 1304, a memory 1306, and a communication interface 1308. The processor 1304, memory 1306, and communication interface 1308 communicate with each other via the bus 1302. The computing device 1300 can be a server, such as a central server, an edge server, or a local server in a local data center, or it can be an electronic device such as a desktop computer, laptop computer, or smartphone. It should be understood that this application does not limit the number of processors and memories in the computing device 1300.

[0236] Bus 1302 can be a Peripheral Component Interconnect (PCI) bus, a Peripheral Component Interconnect Express (PCIe) bus, an Extended Industry Standard Architecture (EISA) bus, a Unified Bus (Ubus or UB), a Compute Express Link (CXL) bus, a Cache Coherent Interconnect for Accelerators (CCIX) bus, etc. The Unified Bus is also known as the Lingqu Bus. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 13 The bus 1304 may be represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 1304 may include a path for transmitting information between various components of the computing device 1300 (e.g., memory 1306, processor 1304, communication interface 1308).

[0237] The processor 1304 may include any one or more of the following computing devices: central processing unit (CPU), graphics processing unit (GPU), microprocessor (MP) or digital signal processor (DSP), ASIC, FPGA, CPLD, NPU, SoC, offload card, accelerator card, etc.

[0238] Memory 1306 may include volatile memory, such as random access memory (RAM). Processor 1304 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD). Furthermore, memory 1306 may also be implemented using storage class memory (SCM), phase change memory (PCM), or other types of storage media.

[0239] It is worth noting that the same type of storage medium can be configured in the same computing device to realize the function of memory 1306, or two or more types of storage media can be configured to realize the function of memory 1306. This application does not limit this.

[0240] The memory 1306 stores executable program code, and the processor 1304 executes the executable program code to implement the functions of the apparatus in the embodiments of this application, thereby implementing the methods described in the above embodiments. That is, the memory 1306 stores instructions for executing the methods described in the above embodiments.

[0241] The communication interface 1308 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the computing device 1300 and other devices or communication networks.

[0242] As one possible implementation, the computing device 1300 may also include a chip system, which includes a processor and a power supply circuit. The power supply circuit supplies power to the processor, and the processor executes the operation steps corresponding to the methods in the embodiments of this application. For simplicity, further details are omitted here. The processor can be implemented using a GPU, or it can be implemented using computing devices or AI chips such as a DPU, NPU, XPU, SoC, offloading card, or accelerator card.

[0243] As one possible implementation, the computing device 1300 may include various types of processors 1304, that is, the computing device 1300 is a heterogeneous device. For example, the computing device 1300 includes a CPU and a GPU, and at least one of the processors 1304 can execute the operation steps corresponding to the method of the embodiments of this application. For the sake of brevity, further details are omitted here.

[0244] This application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be an electronic device such as a desktop computer, a laptop computer, or a smartphone.

[0245] like Figure 14 As shown, the computing device cluster includes at least one computing device 1300. The memory 1306 of one or more computing devices 1300 in the computing device cluster may store the same instructions for performing the methods described in the above embodiments.

[0246] In some possible implementations, the memory 1306 of one or more computing devices 1300 in the computing device cluster may also store partial instructions for executing the methods described in the above embodiments. In other words, a combination of one or more computing devices 1300 can jointly execute instructions for performing the methods described in the above embodiments.

[0247] It should be noted that the memory 1306 in different computing devices 1300 within the computing device cluster can store different instructions, each used to execute a portion of the functions of the apparatus in this application embodiment. That is, the instructions stored in the memory 1306 of different computing devices 1300 can implement the functions of one or more modules of the apparatus in this application embodiment.

[0248] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 15 One possible implementation is shown. For example... Figure 15 As shown, two computing devices 1300A and 1300B are connected via a network. Exemplarily, they are connected to the network through communication interfaces in each computing device. In this type of possible implementation, the memory 1306 in computing device 1300A stores instructions for the functions of one or more modules of the apparatus of this application embodiment. Simultaneously, the memory 1306 in computing device 1300B stores instructions for the functions of another one or more modules of the apparatus of this application embodiment.

[0249] It should be understood that Figure 15 The functions of computing device 1300A shown can also be performed by multiple computing devices 1300. Similarly, the functions of computing device 1300B can also be performed by multiple computing devices 1300.

[0250] This application also provides another computing device cluster. The connection relationships between the computing devices in this computing device cluster can be similarly referred to... Figure 14 and Figure 15 The connection method of the computing device cluster is different in that the memory 1306 of one or more computing devices 1300 in the computing device cluster can store the same instructions for executing the methods in the above embodiments.

[0251] In some possible implementations, the memory 1306 of one or more computing devices 1300 in the computing device cluster may also store partial instructions for executing the aforementioned data processing method. In other words, a combination of one or more computing devices 1300 can jointly execute instructions for performing the aforementioned method.

[0252] Based on the methods in the above embodiments, this application provides a computer-readable storage medium including computer program instructions. When executed by a cluster of computing devices including at least one computing device, the computer program instructions cause the cluster of computing devices to perform the methods in the above embodiments. Exemplarily, the computer-readable storage medium can be any available medium capable of being stored in the cluster of computing devices or a data storage device such as a data center containing one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives).

[0253] Based on the methods in the above embodiments, this application provides a computer program product containing instructions. The computer program product may be software or program products containing instructions, capable of running on a computing device or stored on any available medium. When the instructions are executed by a cluster of computing devices containing at least one computing device, at least one computing device in the cluster of computing devices performs the methods in the above embodiments.

[0254] It is understood that the processor in the embodiments of this application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor may be a microprocessor or any conventional processor.

[0255] The method steps in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0256] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0257] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application.

[0258] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. A text recognition method, characterized by, The method includes: Acquire a first image, which contains the text line to be recognized; The first image is input into the encoder to output encoded features; The encoded features are input into the decoder to output position pairs, which include a first position and a second position. The first position represents the position of the first character of the text line, and the second position represents the position of the last character of the text line. The order of the text line represents the order from the first position to the second position. Based on the position pairs and the encoding features, the text recognition result of the text line is obtained.

2. The method of claim 1, wherein, The step of inputting the encoded features into the decoder to output position pairs includes: The encoded features are input into the decoder to output the first position; The encoded feature and the first position are input into the decoder to output the second position.

3. The method according to claim 1 or 2, characterized in that, The step of inputting the encoded features into the decoder to output position pairs includes: The encoded features are input into the decoder to output hidden layer features; The hidden layer features are input into the classification layer to output the position pairs.

4. The method of claim 3, wherein, The step of obtaining the text recognition result of the text line based on the position pair and the encoded features includes: Based on the location pairs and the hidden layer features, a location representation is obtained; Based on the location representation and the encoding features, the text recognition result of the text line is obtained.

5. The method of claim 4, wherein, The step of obtaining the position representation based on the position pair and the hidden layer features includes: The location is input to the embedding layer to output embedded features; The location representation is obtained by fusing the embedded features and the hidden layer features.

6. The method according to claim 4 or 5, characterized in that, The step of obtaining the text recognition result of the text line based on the position representation and the encoding feature includes: The location representation and the encoded features are input into the encoder to output the text recognition result of the text line.

7. The method according to any one of claims 1 to 6, characterized in that, The acquisition of the first image includes: Acquire a second image to be identified, the second image including a target region, the target region containing the text line; The first image is extracted from the second image, and the first image represents a portion of the image corresponding to the target region.

8. The method according to any one of claims 1 to 7, characterized in that, The first image represents an image containing a seal, with the text line located inside the seal.

9. The method according to any one of claims 1 to 8, characterized in that, The first position represents the position point around the first character of the text line, and the second position represents the position point around the last character of the text line.

10. The method according to any one of claims 1 to 9, characterized in that, The first image contains multiple lines of text to be recognized, and the number of position pairs is also multiple. Each position pair represents a first position and a second position of one of the multiple text lines. The step of obtaining the text recognition result of the text lines based on the position pairs and the encoded features includes: Based on multiple position pairs and the encoded features, the text recognition results of the text lines corresponding to each position pair are obtained in parallel.

11. A model training method, comprising: The method includes: Obtain a training dataset, which includes image samples, location labels, and text labels. The location labels represent the truth values ​​of the start and end positions of text lines in the image samples. The start position represents the position of the first character of the text line, and the end position represents the position of the last character of the text line. The order of the text lines represents the order from the start position to the end position. The text labels represent the truth values ​​of the text content of the text lines. The image sample is input into a text recognition model to output location information and text information. The text recognition model includes at least an encoder and a decoder. The encoder is used to generate encoded features of the image sample. The decoder is used to generate the location information based on the encoded features. The decoder is also used to generate the text information based on the location information and the encoded features. The location information represents the predicted start and end positions of the text line. The text information represents the predicted text content of the text line. The text recognition model is trained based on the loss between the location information and the location label, and the loss between the text information and the text label.

12. The method of claim 11, wherein, The location labels are obtained by marking the start and end positions of the text lines in the image sample.

13. A text recognition apparatus characterized by comprising: The device includes: An image acquisition module is used to acquire a first image, wherein the first image contains a line of text to be recognized; The processing module is configured to input the first image into an encoder to output encoded features; and input the encoded features into a decoder to output position pairs, the position pairs including a first position and a second position, the first position representing the position of the first character of the text line, the second position representing the position of the last character of the text line, the order of the text line representing the order from the first position to the second position; and obtain the text recognition result of the text line based on the position pairs and the encoded features.

14. A model training apparatus, comprising: The device includes: The dataset acquisition module is used to acquire a training dataset, which includes image samples, position labels, and text labels. The position labels represent the truth values ​​of the start and end positions of the text lines in the image samples. The start position represents the position of the first character of the text line, and the end position represents the position of the last character of the text line. The order of the text lines represents the order from the start position to the end position. The text labels represent the truth values ​​of the text content of the text lines. A training module is used to input the image samples into a text recognition model to output location information and text information. The text recognition model includes at least an encoder and a decoder. The encoder is used to generate encoded features of the image samples. The decoder is used to generate the location information based on the encoded features. The decoder is also used to generate the text information based on the location information and the encoded features. The location information represents the predicted start and end positions of the text line, and the text information represents the predicted text content of the text line. The module also trains the text recognition model based on the loss between the location information and the location labels, and the loss between the text information and the text labels.

15. A chip system, characterized by The chip system includes a processor and a power supply circuit, the power supply circuit being used to supply power to the processor, the processor being used to perform the method as described in any one of claims 1-12.

16. A computing device, comprising: The computing device includes a processor and memory; The processor is configured to execute instructions stored in the memory to cause the computing device to perform the method as described in any one of claims 1-12.

17. A cluster of computing devices, characterized in that, It includes at least one computing device, each computing device including a processor and memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method as described in any one of claims 1-12.

18. A computer-readable storage medium, characterized in that, The method includes computer program instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method as described in any one of claims 1-12, wherein the cluster of computing devices includes at least one computing device.

19. A computer program product containing instructions, characterized in that, When the instruction is executed by the computing device cluster, the computing device cluster causes the computing device cluster to perform the method as described in any one of claims 1-12, wherein the computing device cluster includes at least one computing device.

Images