Business card text recognition method and device, electronic equipment and readable storage medium

Abstract

The invention discloses a business card text recognition method and device, electronic equipment and a readable storage medium, and belongs to the technical field of image processing. The method comprises the following steps: acquiring a business card image to be identified; performing text detection on the business card image through a text detection model of a double-branch architecture, and determining a target image area in which a text exists in the business card image in combination with the output of each branch; and according to the relative position of the target image area in the business card image, performing text recognition on the business card image through a text recognition model of an encoder-decoder architecture, and determining a text recognition result of the business card image. According to the invention, the accuracy of text recognition is improved.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and in particular relates to a method, apparatus, electronic device and readable storage medium for recognizing text on business cards. Background Technology

[0002] In the digital age, images are an important carrier of textual information. Text recognition technology, which extracts textual information from images, has been widely used in many fields. Traditional business card text recognition methods employ a full-image recognition approach, such as processing the entire image indiscriminately. However, because the background of the image contains a large amount of information unrelated to the text, it can severely interfere with the recognition process, leading to a decrease in recognition accuracy. Furthermore, some business card text recognition methods in related technologies have poor robustness when dealing with images of varying quality and in different scenarios. Summary of the Invention

[0003] This application aims to address at least one of the technical problems existing in the related art. To this end, this application proposes a business card text recognition method, apparatus, electronic device, and readable storage medium, which improves the accuracy of text recognition.

[0004] Firstly, this application provides a method for recognizing business card text, the method comprising: Obtain the image of the business card to be recognized; The business card image is used to detect text using a dual-branch text detection model, and the target image region containing text in the business card image is determined by combining the outputs of each branch. Based on the relative position of the target image region in the business card image, a text recognition model with an encoder-decoder architecture is used to perform text recognition on the business card image to determine the text recognition result of the business card image.

[0005] In the above technical solution, the text detection model with a dual-branch architecture accurately determines the region where the text is located in the business card image. Compared with full-image recognition, it can reduce background interference and avoid wasting computing resources, thus improving the efficiency of text recognition. Moreover, the dual-branch architecture allows each branch to be trained for a single task, avoiding conflicts in the optimization objectives of different tasks and helping to improve the overall detection accuracy. Combining the feature information of the target image region with the text recognition model with an encoder-decoder architecture, it can better adapt to business card images of different qualities and scenarios, improving the accuracy and robustness of text recognition and meeting the text recognition needs of various practical application scenarios.

[0006] According to some embodiments of this application, the text detection model includes a coordinate prediction branch and a confidence prediction branch; the text detection model using a dual-branch architecture performs text detection on the business card image, and determines the target image region containing text in the business card image by combining the outputs of each branch, including: The relative positions of each candidate image region in the business card image are determined by the coordinate prediction branch, and the confidence of each relative position in the business card image is determined by the confidence prediction branch. The candidate image regions are filtered according to the confidence level corresponding to each candidate image region to obtain the target image region in the business card image.

[0007] According to some embodiments of this application, the coordinate prediction branch includes a bidirectional recurrent neural network, and the confidence prediction branch includes a channel attention mechanism; The step of determining the relative positions of each candidate image region in the business card image through the coordinate prediction branch, and determining the confidence level of each relative position in the business card image through the confidence prediction branch, includes: The business card image is input into the front backbone network of the text detection model to extract features from the business card image, thereby obtaining deep semantic features and shallow semantic features respectively; Enhanced features are obtained by enhancing the deep semantic features output by the pre-backbone network; The enhanced features are processed by the bidirectional recurrent neural network to obtain hidden features carrying bidirectional contextual information of the business card image, and the relative positions of candidate image regions in the business card image are predicted by regression based on the hidden features. The shallow semantic features output by the pre-backbone network are recalibrated using the channel attention mechanism, and the confidence of each relative position of the business card image is predicted based on the recalibrated shallow semantic features.

[0008] According to some embodiments of this application, the step of performing text recognition on the business card image using an encoder-decoder architecture text recognition model based on the relative position of the target image region in the business card image, and determining the text recognition result of the business card image, includes: Based on the relative position of the target image region in the business card image, the target image region is identified in the business card image; The labeled business card image is input into the text recognition model. The encoder and decoder of the text recognition model perform text recognition on the labeled target image region to obtain the text recognition result of the target image region. Based on the text recognition results of the target image region, the text recognition results of the business card image are determined.

[0009] According to some embodiments of this application, the step of inputting the marked business card image into a text recognition model, and performing text recognition on the marked target image region through the encoder and decoder of the text recognition model to obtain the text recognition result of the target image region includes: Feature extraction is performed on the target image region in the business card image, and position encoding is added to the extracted features to obtain the position encoding features of the target image region; The location-encoded features are input into the encoder to perform multi-layer processing on the location-encoded features; Based on the output of the encoder, a multi-head cross-attention calculation is performed in the decoder to obtain the decoded feature sequence of the target image region; The decoded feature sequence is mapped to the vocabulary probability distribution corresponding to the target image region using a multilayer perceptron, and the text recognition result of the target image region is determined based on the vocabulary probability distribution.

[0010] According to some embodiments of this application, the decoder consists of multiple decoders; the step of performing multi-head cross-attention calculation in the decoder to obtain the decoded feature sequence of the target image region based on the output of the encoder includes: In each layer of the multi-layer decoder, a query matrix and a key matrix are determined based on the positional encoding features, and a value matrix is ​​determined based on the input of the decoder. Based on the query matrix, key matrix, and value matrix, multi-head cross-attention calculation is performed, and the output features of each attention head are integrated to obtain the output of the decoder. The decoding feature sequence of the target image region is determined based on the output of the last layer decoder in the multi-layer decoder. The input of the first layer decoder in the multi-layer decoder is the output of the encoder; the input of each layer decoder other than the first layer decoder is the output of the layer above the decoder.

[0011] According to some embodiments of this application, the method further includes: The text recognition results of the business card image are subjected to structured processing to convert the text recognition results into values ​​of preset standard fields.

[0012] Secondly, this application provides a business card text recognition device, the device comprising: The acquisition unit is used to acquire the image of the business card to be recognized; The detection unit is used to perform text detection on the business card image using a dual-branch architecture text detection model, and combine the outputs of each branch to determine the target image region in the business card image where text exists. The recognition unit is used to perform text recognition on the business card image based on the relative position of the target image region in the business card image, using a text recognition model with an encoder-decoder architecture, and to determine the text recognition result of the business card image.

[0013] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the business card text recognition method as described in the first aspect above.

[0014] Fourthly, this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the business card text recognition method as described in the first aspect above.

[0015] Fifthly, this application provides a chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the business card text recognition method as described in the first aspect.

[0016] In a sixth aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the business card text recognition method as described in the first aspect above.

[0017] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0018] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is one of the flowcharts illustrating a business card text recognition method provided in some embodiments of this application; Figure 2 This is a schematic diagram of the structure of a text detection model provided in some embodiments of this application; Figure 3 This is a schematic diagram of the structure of a text recognition model provided in some embodiments of this application; Figure 4 This is a second schematic flowchart of a business card text recognition method provided in some embodiments of this application; Figure 5 This is a schematic diagram of the structure of a business card text recognition device provided in some embodiments of this application; Figure 6 These are schematic diagrams of the structure of electronic devices provided in some embodiments of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0020] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0021] The following description, in conjunction with the accompanying drawings, details the business card text recognition method, apparatus, electronic device, and readable storage medium provided in this application through specific embodiments and application scenarios.

[0022] The business card text recognition method can be applied to a terminal, specifically executed by the hardware or software within the terminal.

[0023] The business card text recognition method provided in this application embodiment can be executed by an electronic device or a functional module or entity in an electronic device that can implement the business card text recognition method. The electronic devices mentioned in this application embodiment include, but are not limited to, mobile phones, tablets, computers, cameras, and wearable devices. The business card text recognition method provided in this application embodiment will be described below using an electronic device as the execution subject as an example.

[0024] Figure 1 This is one of the flowcharts illustrating a business card text recognition method provided in some embodiments of this application. For example... Figure 1 As shown, the business card text recognition method includes steps 110, 120 and 130.

[0025] Step 110: Obtain the image of the business card to be recognized.

[0026] It is understood that a business card image is typically an image of a business card containing text information, which can be a scanned document image or a photograph taken in a natural scene. Obtaining a business card image can be done directly through an image acquisition device (such as a scanner or camera), or by reading an existing image file from a storage device (such as a hard drive or USB flash drive). This application does not specifically limit the method used.

[0027] Step 120: Perform text detection on the business card image using a dual-branch architecture text detection model, and combine the outputs of each branch to determine the target image region in the business card image where text exists.

[0028] Understandably, a dual-branch text detection model typically refers to a deep learning model that includes two functionally independent but collaborative sub-networks (branches). The two branches focus on different core objectives of text detection (such as coordinate prediction and confidence prediction), and the model achieves accurate localization of target image regions containing text by fusing the outputs of the two branches. Compared to single-branch detection models, dual-branch models allow each branch to be trained for a single task, avoiding conflicts in optimization objectives across different tasks and contributing to improved overall detection accuracy.

[0029] For example, a text detection model is a machine learning model trained on a large amount of image data with labeled text regions. It is usually a deep learning model that can learn the feature patterns of text regions in an image and is used to detect whether there is text in the input image and to locate the region where the text is located.

[0030] Text detection refers to the process of locating regions containing text in an image; the target image region is the image region where text is determined to exist after text detection, and it is the specific object for subsequent text recognition. For example, determining the target image region containing text in a business card image can be done using a pre-trained text detection model, such as a deep learning-based target detection model. The business card image is input into the first branch of the text detection model, which performs feature extraction, region proposal generation, classification, and regression to determine the relative positions of each candidate image region within the business card image. The second branch of the text detection model then calculates the confidence scores of each relative position within the business card image, and the target image region is selected based on the confidence scores. Alternatively, traditional text detection methods, such as connected component analysis, can be used. These methods use two branches to determine whether a region is a text region based on features of connected components in the image (such as aspect ratio and area).

[0031] Understandably, accurately detecting the target image region can avoid unnecessary recognition of non-text regions, thus improving the efficiency and accuracy of text recognition.

[0032] Step 130: Based on the relative position of the target image region in the business card image, perform text recognition on the business card image using an encoder-decoder architecture text recognition model to determine the text recognition result of the business card image.

[0033] As can be understood, a text recognition model refers to a model that has been pre-trained on a large amount of text image data, learning the mapping relationship between the features of the text image and the corresponding text, and is used to recognize the text content in the target image region. For example, the encoder is used to perform deep extraction and encoding of the input image features (or target region features), and the decoder is used to generate the corresponding text sequence based on the encoded features.

[0034] The feature information of the target image region is information used to describe the characteristics of the target image region. It may include the relative position of the target image region in the business card image, as well as other information such as the size, shape, and color distribution of the region.

[0035] For example, text recognition of a business card image based on the feature information of the target image region can be achieved by preprocessing the image according to the relative position of the target image region within the business card image, such as cropping or scaling to highlight the target image region. Then, a pre-trained text recognition model is used to extract features from the target image region, adding positional encoding before inputting it into an encoder for multi-layer processing. Finally, a decoder generates the text recognition result. In some embodiments, traditional optical character recognition (OCR) methods, such as feature-matching methods, can also be used. These methods extract features of characters in the target image region and match them with a pre-established feature library to find the most similar character as the recognition result.

[0036] In this embodiment of the application, preprocessing with the feature information of the target image region can further improve the accuracy of text recognition, and finally obtain accurate business card image text recognition results. Among them, the business card text recognition method based on deep learning can learn the contextual information and semantic information of the text, and also has a good recognition effect on poor-quality text such as blurry or deformed text.

[0037] The business card text recognition method provided in this application accurately determines the text region in a business card image through a dual-branch architecture text detection model. Compared with full-image recognition, it can reduce background interference and avoid wasting computational resources, thus improving the efficiency of text recognition. Furthermore, the dual-branch approach allows each branch to be trained for a single task, avoiding conflicts in optimization objectives for different tasks and helping to improve overall detection accuracy. Combining the feature information of the target image region with an encoder-decoder architecture text recognition model, it can better adapt to business card images of different qualities and scenarios, improving the accuracy and robustness of text recognition and meeting the text recognition needs of various practical application scenarios.

[0038] In some embodiments of this application, the text detection model includes a coordinate prediction branch and a confidence prediction branch; the text detection of the business card image using the dual-branch architecture text detection model, and the determination of the target image region containing text in the business card image by combining the outputs of each branch, includes: The relative positions of each candidate image region in the business card image are determined by the coordinate prediction branch, and the confidence of each relative position in the business card image is determined by the confidence prediction branch. The candidate image regions are filtered according to the confidence level corresponding to each candidate image region to obtain the target image region in the business card image.

[0039] In this embodiment, the text detection model employs a multi-task learning paradigm. It uses a dual-branch architecture to jointly optimize two objectives: classification score and coordinate regression. These objectives predict the foreground / background confidence of the text region and its precise bounding box coordinates, respectively. Specifically, the coordinate prediction branch predicts the relative position of candidate regions (i.e., candidate image regions) within the entire image, typically outputting the coordinate parameters of the candidate region's bounding box. The confidence prediction branch calculates the confidence that each candidate image region output by the coordinate prediction branch is actually a text region, typically outputting a confidence value between 0 and 1.

[0040] Candidate image regions refer to image regions that the text detection model initially determines may contain text when detecting business card images; target image regions are those that, after confidence filtering, are confirmed to actually contain text. The relative position of a candidate image region refers to its coordinate position within the business card image, typically represented by the coordinates of its bounding box, such as the coordinates of the top-left and bottom-right corners.

[0041] For example, a deep learning-based object detection model can be used, such as using a convolutional neural network to extract features from a business card image to generate a feature map, then generating a series of candidate regions that may contain text on the feature map, and classifying and regressing each candidate region to determine whether the region is a text region, further refining the position of the candidate regions, and outputting the confidence score of each candidate region as a text region.

[0042] Confidence score can be considered as the probability value by which a text detection model judges whether a candidate image region actually contains text. For example, the confidence score can be set between 0 and 1, with the value closer to 1 indicating that the model believes the region is more likely to contain text.

[0043] Candidate image regions are filtered based on their confidence levels to obtain the target image region in the business card image. For example, a confidence threshold can be set, retaining candidate image regions with confidence levels higher than this threshold as the target image region. For instance, a threshold of 0.8 would be set, with candidate regions having confidence levels greater than 0.8 designated as the target text region. In some embodiments, algorithms such as Non-Maximum Suppression (NMS) can also be used. For example, candidate regions are first sorted by confidence level from highest to lowest, then each candidate region is iterated through sequentially, calculating the overlap between the current candidate region and other candidate regions. If the overlap exceeds a set threshold, candidate regions with lower confidence levels are suppressed (deleted), and the final retained candidate region is the target image region.

[0044] The business card text recognition method provided in this application uses a text detection model to detect business card images. The coordinate prediction branch and the confidence prediction branch respectively obtain candidate image regions, their relative positions, and confidence scores. Then, the target image region is obtained through confidence score filtering. The text detection model can quickly and accurately locate regions that may contain text, and the confidence score filtering can further remove false detection regions. This method can effectively extract the real text regions from complex business card images, which helps to improve the accuracy and efficiency of text detection.

[0045] In some embodiments of this application, the coordinate prediction branch includes a bidirectional recurrent neural network, and the confidence prediction branch includes a channel attention mechanism; the step of determining the relative positions of each candidate image region in the business card image through the coordinate prediction branch, and determining the confidence of each relative position of the business card image through the confidence prediction branch, includes: The business card image is input into the front backbone network of the text detection model to extract features from the business card image, thereby obtaining deep semantic features and shallow semantic features respectively; Enhanced features are obtained by enhancing the deep semantic features output by the pre-backbone network; The enhanced features are processed by a bidirectional recurrent neural network to obtain hidden features carrying bidirectional contextual information of the business card image, and the relative positions of candidate image regions in the business card image are predicted by regression based on the hidden features. The shallow semantic features output by the front backbone network are recalibrated using a channel attention mechanism, and the confidence level of each relative position of the business card image is predicted based on the recalibrated shallow semantic features.

[0046] The pre-backbone network refers to the basic feature extraction network in a text detection model. It is used to extract preliminary feature information from business card images. Using operations such as convolutional layers in the pre-backbone network, the business card image is converted into a feature representation with semantic information. Deep semantic features refer to the features output from the deeper layers of the pre-backbone network, containing high-level semantic information of the image. These are usually obtained through further processing of the extracted features, improving their expressive power and highlighting information relevant to text detection. Shallow semantic features refer to the features output from the shallower layers of the pre-backbone network, preserving more detailed image information, such as edges and textures.

[0047] For example, classic convolutional neural network structures, such as the ResNet series, can be used as the front-end backbone network. After the business card image is input, it goes through multiple convolutional layers, pooling layers, and other operations to gradually extract the image features. Alternatively, lightweight convolutional neural networks such as MobileNet and EfficientNet can be used, which, while ensuring a certain feature extraction capability, have fewer parameters and less computation, thereby improving the applicability of the text detection model to resource-constrained devices.

[0048] Enhanced features are features that have undergone feature enhancement operations, making them more effective at distinguishing between text and non-text regions compared to the original features. For example, a Feature Pyramid Network (FPN) can be used to enhance the first feature. The FPN fuses features from different levels through a top-down path and lateral connections, enhancing the expressive power of the features. Alternatively, dilated convolutions can be used to expand the receptive field, acquiring more contextual information without increasing the number of parameters, thereby enhancing the features.

[0049] Understandably, in text recognition, the text in a business card image is usually not isolated; the semantics of each character or word and the recognition result are influenced by the content before and after it. Two-way contextual information refers to comprehensive information that considers all relevant information before (forward context) and after (backward context) the current position in the text sequence. It can more comprehensively reflect the dependencies and semantic relationships between characters or words in the text, thus providing strong support for more accurate text recognition.

[0050] A Bidirectional Recurrent Neural Network (BiRNN) consists of two layers of recurrent neural networks running in opposite directions: a forward recurrent neural network (RNN) and a backward recurrent neural network (RNN). The forward RNN starts from the beginning of the text sequence and processes the augmented features corresponding to each character or image region sequentially, considering both the features at the current position and the features from all previous positions. The backward RNN, on the other hand, starts from the end of the text sequence and processes the augmented features corresponding to each character or image region in reverse order, again considering both the features at the current position and the features from all subsequent positions.

[0051] For example, in the text sequence "ABCDE", the feedforward recurrent neural network (RNN) processes the character "C" by utilizing not only the enhanced features of "C" itself but also the feature information from the preceding "A" and "B", thus obtaining a hidden feature representation containing forward contextual information. Similarly, the feedback recurrent neural network (RNN) processes the character "C" by utilizing not only the enhanced features of "C" itself but also the feature information from the following "D" and "E", thus obtaining a hidden feature representation containing backward contextual information. By fusing the hidden features obtained at each position from the feedforward and feedback recurrent neural networks (e.g., through concatenation, addition, etc.), a hidden feature carrying bidirectional contextual information is obtained. For the character "C", the fused hidden feature contains information from both the preceding "A" and "B" and the following "D" and "E", more comprehensively reflecting the semantics and contextual relationships of "C" within the entire text sequence.

[0052] In some texts, dependencies between characters or words can span long distances. Traditional unidirectional recurrent neural networks may suffer from vanishing or exploding gradients when processing long sequences, making it difficult to effectively capture long-distance dependencies. Bidirectional recurrent neural networks, by processing sequences simultaneously from two directions, can better capture long-distance contextual information, improving their ability to recognize long texts.

[0053] Hidden features refer to the feature representation obtained after a bidirectional recurrent neural network processes the augmented features, carrying bidirectional contextual information about the business card image. Based on the hidden features, a regression model can predict the relative positions of candidate image regions, such as the coordinates of bounding boxes.

[0054] The channel attention mechanism is used to adjust the weights of each channel of the feature. Specifically, it enhances the feature representation of key channels and suppresses unimportant channels. The second feature output by the pre-backbone network is recalibrated using the channel attention mechanism, and the confidence of each relative position in the business card image is predicted based on the recalibrated second feature. For example, the channel attention module in SENet is used to perform global average pooling on the second feature to obtain the channel description vector. Then, the weights of each channel are learned through a fully connected layer, and finally, the weights are multiplied by the original second feature for recalibration.

[0055] Figure 2 This is a schematic diagram of the structure of a text detection model provided in some embodiments of this application. For example... Figure 2 As shown, the text detection model uses MobileNet-V3 (a lightweight convolutional neural network architecture launched by the Google team in 2019) as the front backbone network; The output of MobileNet-V3 Stage 4 is used as the first feature, and the output of MobileNet-V3 Stage 3 is used as the second feature; The text detection model includes a coordinate prediction branch and a foreground / background confidence score branch; In the coordinate prediction branch, the first feature output by the front backbone network is enhanced by the cross-modal transformation (CMT) module to obtain enhanced features. The enhanced features are then processed by the bidirectional recurrent neural network (composed of multiple basic building blocks of recurrent neural networks, RNN Cells) of the text detection model to inject temporal information, thereby obtaining hidden features carrying bidirectional contextual information of the business card image. Based on the hidden features, the relative positions (coordinates) of candidate image regions in the business card image are predicted by regression. In the foreground / background confidence score branch, it is first composed of one The convolutional layer adjusts the channel dimension and compresses features. Then, the features are recalibrated through the channel attention mechanism (SENet), and the confidence score (cls score) of each relative position of the business card image is predicted based on the recalibrated second feature.

[0056] Specifically, the enhanced feature is obtained by enhancing the first feature output by the front backbone network through the cross-modal transfer (CMT) module, as follows:

[0057] in, Represents backbone network The output characteristics, For use in feature transformation and dimensionality reduction Convolutional layer Represents a non-linear activation function. This refers to the output characteristics enhanced by the CMT module.

[0058] The enhanced features are processed by a bidirectional recurrent neural network of the text detection model to obtain hidden features carrying bidirectional contextual information of the business card image, represented as follows:

[0059]

[0060]

[0061] in, and These represent the forward and backward RNNs at time points, respectively. The hidden state, This indicates a tensor splicing operation. This is a hidden feature that incorporates bidirectional contextual information.

[0062] The relative positions of candidate image regions in a business card image predicted by latent feature regression are represented as follows:

[0063] in, For time step Hidden features, and For learnable weights and bias parameters, This is the final text coordinate sequence predicted by the model (i.e., the relative position of the selected image region).

[0064] Through a Convolutional layers perform channel dimension adjustment and feature compression, as shown below:

[0065] in, Represents backbone network The output characteristics, For use in feature transformation and dimensionality reduction Convolutional layer These are the features after feature compression.

[0066] Features are recalibrated using the channel attention mechanism (SENet), as follows:

[0067] in, Indicates global average pooling. Represents the ReLU activation function. This represents the Sigmoid activation function; and These are two learnable weight matrices used to perform a linear transformation on the pooled features; and These are two bias terms, used together with the weight matrix for linear transformation. This is the second feature after recalibration.

[0068] Understandably, by fusing feature maps at different levels to adaptively process text regions of various scales; introducing a bidirectional RNN structure to effectively model the temporal dependencies of text sequences; and combining a channel attention mechanism to dynamically focus on key text regions, the accuracy and robustness of detection are significantly improved.

[0069] The business card text recognition method provided in this application extracts features through a pre-backbone network, enhances the deep semantic features output by the pre-backbone network, and then uses a bidirectional recurrent neural network to capture contextual information and predict candidate region locations. The application of feature enhancement and bidirectional RNN enables the model to better handle text regions of different scales and shapes and capture the contextual information of the text. The shallow semantic features are recalibrated through a channel attention mechanism to predict confidence. The channel attention mechanism enables the model to dynamically focus on key text regions, reduce background interference, and more accurately predict confidence. This allows the text detection model to effectively detect target image regions containing text in various complex scenarios, providing a reliable foundation for subsequent text recognition.

[0070] In some embodiments of this application, the step of performing text recognition on the business card image using an encoder-decoder architecture text recognition model based on the relative position of the target image region in the business card image, and determining the text recognition result of the business card image, includes: Based on the relative position of the target image region in the business card image, the target image region is identified in the business card image; The labeled business card image is input into the text recognition model. The encoder and decoder of the text recognition model perform text recognition on the labeled target image region to obtain the text recognition result of the target image region. The text recognition result of the business card image is based on the text recognition result of the target image region.

[0071] In some embodiments, the relative position of the candidate image region in the business card image includes at least the boundary coordinates of the candidate image region in the business card image.

[0072] It is understandable that preprocessing refers to a series of processing operations performed on a business card image based on the relative position and other feature information of the target image region before text recognition is performed on the target image region. The purpose is to optimize the image conditions so as to facilitate subsequent text recognition of the target image region.

[0073] The target image region is identified in the business card image based on its relative position within the business card image. For example, this can be achieved by using an image processing library to draw a rectangle on the business card image based on the bounding box coordinates of the target image region. In some embodiments, the identified image can be cropped to retain only the portion containing the target image region, reducing the computational load for subsequent recognition. Alternatively, an image mask can be used, generating a corresponding mask image based on the position of the target image region, and then merging the mask image with the business card image to highlight the target image region.

[0074] For example, a text recognition model could be a sequence recognition model based on the Transformer architecture. First, features are extracted from the target image region, such as using a convolutional neural network to extract visual features. Then, positional encoding is added to preserve the character order information. The processed features are input into an encoder for multi-layer feature encoding, and then a decoder gradually generates a text sequence, ultimately yielding the text recognition result for the target image region.

[0075] Determining the text recognition result of a business card image based on the text recognition results of the target image regions can be achieved by either concatenating the text recognition results of all target image regions according to their relative positions within the business card image, or by arranging the text recognition results according to the coordinates of the bounding boxes of the target image regions, thus forming a complete text recognition result for the business card image. For images with specific structures, such as tables, the text recognition results of the target image regions can be reorganized and rearranged based on the table's structural information to restore the original table structure. Organizing and arranging text based on structural information can more accurately restore the information in the image, improving the readability and usability of the text recognition results.

[0076] The business card text recognition method provided in this application preprocesses the business card image and identifies the target region based on the relative position of the target image region. By highlighting the target text region through preprocessing, background interference is reduced, and the accuracy of text recognition is improved. Then, the encoder and decoder of the text recognition model are used to recognize the text in the target region. The pre-trained text recognition model can adapt to the text recognition needs in various complex scenarios, which improves the reliability of the recognition results. Finally, the text recognition result of the entire image is determined based on the recognition results of each target region. Through a reasonable result integration method, the final text recognition result has good readability and practicality, and can meet various needs for image text recognition in practical applications.

[0077] In some embodiments of this application, the step of inputting the marked business card image into a text recognition model, and performing text recognition on the marked target image region through the encoder and decoder of the text recognition model to obtain the text recognition result of the target image region includes: Feature extraction is performed on the target image region in the business card image, and position encoding is added to the extracted features to obtain the position encoding features of the target image region; The location-encoded features are input into the encoder to perform multi-layer processing on the location-encoded features; Based on the output of the encoder, a multi-head cross-attention calculation is performed in the decoder to obtain the decoded feature sequence of the target image region; The decoded feature sequence is mapped to the vocabulary probability distribution corresponding to the target image region using a multilayer perceptron, and the text recognition result of the target image region is determined based on the vocabulary probability distribution.

[0078] Positional encoding refers to adding positional information to extracted features to enable a model to perceive the positional information of characters or text elements in a target image region. This allows the model to understand the relative or absolute position of each feature within the sequence. Positionally encoded features are features that have undergone positional encoding processing, incorporating positional information into the original feature information of the target image region.

[0079] Feature extraction is performed on the target image region in the business card image, and positional encoding is added to the extracted features to obtain the positional encoding features of the target image region. For example, a convolutional neural network can be used to extract features from the target image region. This can be achieved by using multiple convolutional and pooling layers to progressively extract local and global features of the image, and then using sine and cosine functions to generate positional encoding vectors, which are then added to or concatenated with the extracted feature vectors. Alternatively, a visual Transformer can be used for feature extraction. The target image region is segmented into multiple image patches, and after linear transformation and positional encoding, the patches are input into the Transformer encoder for feature extraction. The positional encoding can also use a learnable positional embedding method, allowing the model to automatically learn positional information during training.

[0080] An encoder is a part of a text recognition model, used to perform multi-layer processing on the positional encoding features of the input. Through a series of transformations and calculations, it extracts higher-level semantic features. Multi-layer processing refers to the encoder gradually transforming and abstracting the input features through multiple layers of neural networks (such as self-attention layers, feedforward neural network layers, etc.) to capture more complex feature relationships.

[0081] For example, a Transformer encoder can be used to input position-encoded features into the encoder of a text recognition model for multi-layer processing of the position-encoded features. A Transformer encoder consists of multiple identical encoder layers stacked together. Each encoder layer contains a multi-head self-attention layer and a feedforward neural network layer. After the position-encoded features are input, the correlation between features is calculated by the multi-head cross-attention layer, and then a non-linear transformation is performed by the feedforward neural network layer. After multi-layer processing, a higher-level feature representation is obtained.

[0082] Understandably, the decoder also consists of multiple decoder layers, each of which can contain a multi-head cross-attention layer, an encoder-decoder attention layer, or a feedforward neural network layer. The encoder's output serves as one of the decoder's inputs. In the decoder's multi-head cross-attention layer, the decoder's own features interact with the encoder's output, generating a decoded feature sequence through multi-head cross-attention calculation. The decoded feature sequence, obtained after multi-head cross-attention calculation by the decoder, can be considered to represent the semantic and structural information of the text in the target image region.

[0083] For example, a multilayer perceptron (MLP) takes the decoded feature sequence as input, processes it through multiple fully connected layers and nonlinear activation functions, and finally outputs a vector the same size as the vocabulary, representing the probability of each word in the vocabulary. The word with the highest probability is selected as the recognition result at the current time step by step to generate the entire text sequence. In some embodiments, a conditional random field (CRF) can be used as the final mapping layer to consider the transition probabilities between words and predict the word sequence more accurately. The MLP can learn the complex mapping relationship between the decoded feature sequence and the probability distribution of the vocabulary through fully connected layers and nonlinear activation functions, effectively realizing text recognition.

[0084] The vocabulary probability distribution refers to the probability of each word appearing in the vocabulary. This distribution can be used to determine the text content corresponding to the target image region.

[0085] The business card text recognition method provided in this application extracts features from the target image region in the business card image and adds positional encoding to the extracted features to obtain the positional encoding features of the target image region. Positional encoding enables the model to adaptively learn positional representations based on data, and the introduction of positional encoding improves the model's adaptability to irregular text arrangements. The positional encoding features are processed in multiple layers by an encoder, and multi-head cross-attention calculation is performed in the decoder of the text recognition model based on the encoder output to obtain the decoded feature sequence of the target image region. It has efficient parallel processing capabilities and strong long-range dependency modeling capabilities. The multi-head cross-attention mechanism can quickly capture global contextual information, improving the recognition accuracy of long and complex text. Through multi-layer processing and complex mapping relationships, the text content can be accurately identified from the target image region, improving the adaptability of the business card text recognition method to scenarios with diverse layouts and irregular text arrangements.

[0086] In some embodiments of this application, the step of inputting the location-encoded features into the encoder of the text recognition model to perform multi-layer processing on the location-encoded features includes: In each layer of the encoder, a query matrix, a key matrix, and a value matrix corresponding to the location encoding features are determined; Based on the query matrix, key matrix, and value matrix, multi-head self-attention calculation is performed, and the attention outputs of each attention head are concatenated to obtain the global context features of the target image region. The global context features are used to generate the value matrix required for the decoder to perform multi-head self-attention computation.

[0087] In the attention mechanism, the query matrix (Q) is a matrix used to match queries with other features, indicating the information that needs to be focused on at the moment; the key matrix (K) is a matrix used to perform matching calculations with the query matrix, measuring the relevance of features at different positions to the query; the value matrix (V) is a matrix containing actual feature information, which is weighted and summed according to the matching results of the query and the key to obtain the final attention output.

[0088] Multi-head self-attention computation refers to dividing the self-attention computation process into multiple heads (subspaces) and performing them in parallel. Each head captures the dependencies between features from different perspectives, and then the outputs of each head are concatenated to obtain a more comprehensive feature representation. Global context features are the features obtained after multi-head self-attention computation and concatenation of the outputs of each attention head. Global context features can be considered to include the global dependencies and contextual information between all positional features in the target image region.

[0089] At each layer of the encoder, a query matrix, key matrix, and value matrix corresponding to the positional encoded features are determined. For example, these matrices can be obtained by linearly transforming the positional encoded features using learnable weight matrices. Alternatively, convolution operations can be used to generate these matrices. Different one-dimensional convolution kernels are used to perform convolution operations on the positional encoded features to obtain the corresponding matrices.

[0090] Multi-head self-attention computation is performed based on the query matrix, key matrix, and value matrix. Specifically, for each attention head, the product of the query matrix and key matrix is ​​first calculated and scaled (divided by the square root of the key matrix dimension). Then, the attention weights are obtained through the softmax function. Finally, the attention weights are multiplied by the value matrix to obtain the output of that attention head. The attention outputs of all attention heads are concatenated to obtain the global contextual features of the target image region.

[0091] Understandably, multi-head self-attention computation (scaled dot product attention) is computationally efficient, can be computed in parallel, and the scaling operation can prevent the dot product result from being too large, which would cause the gradient of the softmax function to be too small. Multiple attention heads can capture feature dependencies from different subspaces, and the global context features obtained after concatenation contain richer and more comprehensive information.

[0092] Global context features can be used for further transformations, such as linear or nonlinear transformations, and then used as the input value matrix of the decoder. The global context features, after processing, can be used as the value matrix, enabling the decoder to make full use of the global information extracted by the encoder, better understand the contextual relationships of the target image region, and thus generate a more accurate decoding feature sequence.

[0093] Figure 3 These are schematic diagrams illustrating the structure of text recognition models provided in some embodiments of this application. For example... Figure 3 As shown, the text recognition model uses a convolutional neural network to extract features from the target image region in the business card image:

[0094] in, For sequence length, For feature dimensions; The sine-cosine function is then used to generate position encoding. The positional encoding is added to the extracted features to preserve the positional encoding features of the character sequence:

[0095] in, For location-encoded features; The encoder consists of multiple Transformer modules. Position-encoded features are input into the encoder of the text recognition model for multi-layered processing. At each layer of the encoder, a query matrix corresponding to the position-encoded features is determined. Key matrix Sum matrix :

[0096] in, , and It is a learnable weight matrix used to encode features at different locations. Mapped to query, key, and value spaces; Based on query matrix Key matrix Sum matrix Perform multi-head self-attention computation:

[0097] in, For the first The output of each attention head, , and It corresponds to the first A query, key, and value matrix for each attention head. It is the dimension of query and key; The global contextual features h of the target image region are obtained by concatenating the attention outputs of each attention head. : h

[0098] in, For the first The output of each attention head, Indicates splicing, To output the weight matrix; The global context features are used to generate the value matrix required for the decoder to perform multi-head self-attention computation; specifically, the output of the multi-head attention mechanism. The residual connection and layer normalization are used as the value matrix of the decoder. enter:

[0099] in, Representation layer normalization; The decoded feature sequence is mapped to the vocabulary probability distribution corresponding to the target image region by a multilayer perceptron (MPL Head), and the text recognition result (Predicted Sequence) of the target image region is determined based on the vocabulary probability distribution corresponding to the target image region.

[0100] Understandably, the encoder-decoder architecture based on Transformer achieves parallel capture of global contextual information through a multi-head self-attention mechanism, which completely solves the bottleneck of CRNN in long sequence processing and parallel computing; explicit positional encoding makes the model naturally adaptable to irregular text arrangements.

[0101] The business card text recognition method provided in this application determines the query matrix, key matrix, and value matrix at each layer of the encoder, then obtains global context features through multi-head self-attention calculation, and finally uses these features for multi-head self-attention calculation in the decoder. The multi-head self-attention mechanism enables the encoder to efficiently capture the global dependencies between features of the target image region, generating rich global context features. These global context features can be applied to the decoder, helping the decoder to better understand the contextual information of the image and improve the accuracy and coherence of text recognition. Multi-layer processing of the positional encoding features can ensure recognition performance while maintaining certain computational efficiency, helping to adapt to various complex text recognition scenarios.

[0102] In some embodiments of this application, the decoder consists of multiple decoders; the step of performing multi-head cross-attention calculation in the decoder to obtain the decoded feature sequence of the target image region based on the output of the encoder includes: In each layer of the multi-layer decoder, a query matrix and a key matrix are determined based on the positional encoding features, and a value matrix is ​​determined based on the input of the decoder. Based on the query matrix, key matrix, and value matrix, multi-head cross-attention calculation is performed, and the output features of each attention head are integrated to obtain the output of the decoder. The decoding feature sequence of the target image region is determined based on the output of the last layer decoder in the multi-layer decoder. The input of the first layer decoder in the multi-layer decoder is the output of the encoder; the input of each layer decoder other than the first layer decoder is the output of the layer above the decoder.

[0103] It is understandable that a multi-layer decoder refers to a decoder module composed of multiple decoders (sub-decoders) with the same or similar structures cascaded together. Each unit has independent attention calculation and feature processing functions, and the feature is gradually optimized and accurately decoded through iterative processing layer by layer.

[0104] As can be understood, multi-head cross-attention computation refers to the parallel expansion of cross-attention, typically using 8-16 independent attention heads to capture information from different subspaces. Each head independently calculates query-key similarity and weights the results into a matrix, which is then concatenated and fused. For example, some attention heads can focus on character shapes, while others focus on contextual semantics. Cross-attention enables dynamic interaction between the encoder and decoder. Compared to single-head attention, it can simultaneously capture multi-dimensional and multi-scale feature dependencies, improving the richness and comprehensiveness of feature representation and effectively addressing complex feature scenarios such as multiple fonts and layouts in business card text.

[0105] In some embodiments, determining the query matrix and key matrix based on the location encoding features includes: Apply a convolution operation to the position-encoded features to obtain the transformed features; The transformed features are mapped to the query space based on the query weight matrix to obtain the query matrix; The transformed features are mapped to the key space based on the key weight matrix to obtain the key matrix.

[0106] The business card text recognition method provided in this application embodiment The following is an example of determining the query matrix:

[0107] in, For querying the matrix, for Convolutional layer For location encoding features, It is a learnable weight matrix used to encode location features. Mapped to the query space.

[0108] In this embodiment, the first-layer decoder directly receives the output of the encoder after multi-layer processing as input, and outputs 3D features after multi-head cross-attention calculation and feature processing. The second-layer decoder takes the output features of the first layer as input and repeats attention calculation and feature optimization. The features output by the last layer decoder are the final decoded feature sequence. For example, residual connections and layer normalization can be used to ensure stable gradient propagation. The output of each layer is processed by a feedforward network before being input to the next layer, improving the stability and convergence speed of the current layer's decoding.

[0109] The business card text recognition method provided in this application determines the query matrix and key matrix based on the positional encoding features in each layer of the multi-layer decoder, and determines the value matrix based on the input of the decoder. Then, it performs multi-head cross-attention calculation and integrates the output features of each attention head to obtain the decoded feature sequence of the target image region. Through the hierarchical design of the decoder, the synergy of multi-head cross-attention and positional encoding, multi-layer optimization, and multi-dimensional attention focusing, it can effectively cope with complex feature scenarios such as multiple fonts and multiple layouts of business card text, which helps to improve the accuracy of text recognition.

[0110] In some embodiments of this application, the method further includes: The text recognition results of the business card image are subjected to structured processing to convert the text recognition results into values ​​of preset standard fields.

[0111] It is understandable that structured processing refers to the process of converting unstructured or semi-structured text recognition results into structured data with specific formats and semantics according to preset rules and standards.

[0112] Preset standard fields refer to predefined fields used to store and represent different aspects of text recognition results. For example, when recognizing invoice information, preset standard fields may include invoice number, invoice date, amount, etc.; and when recognizing business card information, preset standard fields may include name, contact information, address, etc.

[0113] For example, a series of rules, such as regular expression rules, can be pre-defined. For the identified text, matching and extraction can be performed according to these rules. Alternatively, natural language processing technology can be used to perform semantic understanding on the text recognition results. By constructing a semantic model, entities (such as names of people, places, and organizations) and relationships in the text can be identified. Then, the identified entities and relationships are filled into the corresponding positions according to a pre-defined structured template. During the structured processing, a mapping relationship is established between the information in the text recognition results and pre-defined standard fields.

[0114] In some embodiments, the identified information is converted and formatted according to the requirements of preset standard fields.

[0115] Figure 4 This is a second schematic flowchart of a business card text recognition method provided in some embodiments of this application. Figure 4 As shown, the business card text recognition method includes: Users upload images of business cards that require text recognition to the MINIO file server; A pre-trained text detection model was used to perform text detection on business card images in a file server. The text recognition model is pre-trained to perform text recognition based on the target image region determined in the text detection step. The text recognition results output by the text recognition model are structured. The structured data is manually reviewed for quality control, and feedback is used to optimize the data that fails the manual review. The text recognition results that have passed quality control will be output.

[0116] The business card text recognition method provided in this application can convert the text recognition result into the value of a preset standard field through structured processing, which improves the readability and usability of the data, helps to improve the practicality of the business card text recognition method, and meets the needs of different fields for the structuring and standardization of text data.

[0117] The business card text recognition method provided in this application can be executed by a business card text recognition device. This application uses a business card text recognition device executing the method as an example to illustrate the business card text recognition device provided in this application.

[0118] Figure 5 This is a schematic diagram of the structure of a business card text recognition device provided in some embodiments of this application. For example... Figure 5 As shown, the business card text recognition device 500 includes: an acquisition unit 501, a detection unit 502, and a recognition unit 503.

[0119] Acquisition unit 501 is used to acquire the image of the business card to be recognized; The detection unit 502 is used to perform text detection on the business card image using a dual-branch architecture text detection model, and combine the outputs of each branch to determine the target image region in the business card image where text exists. The recognition unit 503 is used to perform text recognition on the business card image based on the relative position of the target image region in the business card image, using a text recognition model with an encoder-decoder architecture, and to determine the text recognition result of the business card image.

[0120] In some embodiments, the text detection model includes a coordinate prediction branch and a confidence prediction branch; the detection unit 502 is used for: The relative positions of each candidate image region in the business card image are determined by the coordinate prediction branch, and the confidence of each relative position in the business card image is determined by the confidence prediction branch. The candidate image regions are filtered according to the confidence level corresponding to each candidate image region to obtain the target image region in the business card image.

[0121] In some embodiments, the coordinate prediction branch includes a bidirectional recurrent neural network, and the confidence prediction branch includes a channel attention mechanism; determining the relative positions of each candidate image region in the business card image through the coordinate prediction branch, and determining the confidence of each relative position of the business card image through the confidence prediction branch, includes: The business card image is input into the front backbone network of the text detection model to extract features from the business card image, thereby obtaining deep semantic features and shallow semantic features respectively; Enhanced features are obtained by enhancing the deep semantic features output by the pre-backbone network; The enhanced features are processed by the bidirectional recurrent neural network to obtain hidden features carrying bidirectional contextual information of the business card image, and the relative positions of candidate image regions in the business card image are predicted by regression based on the hidden features. The shallow semantic features output by the pre-backbone network are recalibrated using the channel attention mechanism, and the confidence scores of each relative position in the business card image are predicted based on the recalibrated shallow semantic features. In some embodiments, the identification unit 503 is used for: Based on the relative position of the target image region in the business card image, the target image region is identified in the business card image; The labeled business card image is input into the text recognition model. The encoder and decoder of the text recognition model perform text recognition on the labeled target image region to obtain the text recognition result of the target image region. Based on the text recognition results of the target image region, the text recognition results of the business card image are determined.

[0122] In some embodiments, inputting the identified business card image into a text recognition model, and performing text recognition on the identified target image region through the encoder and decoder of the text recognition model to obtain the text recognition result of the target image region includes: Feature extraction is performed on the target image region in the business card image, and position encoding is added to the extracted features to obtain the position encoding features of the target image region; The location-encoded features are input into the encoder to perform multi-layer processing on the location-encoded features; Based on the output of the encoder, a multi-head cross-attention calculation is performed in the decoder to obtain the decoded feature sequence of the target image region; The decoded feature sequence is mapped to the vocabulary probability distribution corresponding to the target image region using a multilayer perceptron, and the text recognition result of the target image region is determined based on the vocabulary probability distribution.

[0123] In some embodiments, the decoder consists of multiple decoders; the step of performing multi-head cross-attention calculation in the decoder to obtain the decoded feature sequence of the target image region based on the output of the encoder includes: In each layer of the multi-layer decoder, a query matrix and a key matrix are determined based on the positional encoding features, and a value matrix is ​​determined based on the input of the decoder. Based on the query matrix, key matrix, and value matrix, multi-head cross-attention calculation is performed, and the output features of each attention head are integrated to obtain the output of the decoder. The decoding feature sequence of the target image region is determined based on the output of the last layer decoder in the multi-layer decoder. The input of the first layer decoder in the multi-layer decoder is the output of the encoder; the input of each layer decoder other than the first layer decoder is the output of the layer above the decoder.

[0124] In some embodiments, inputting the location-encoded features into the encoder of the text recognition model to perform multi-layer processing on the location-encoded features includes: In each layer of the encoder, a query matrix, a key matrix, and a value matrix corresponding to the location encoding features are determined; Based on the query matrix, key matrix, and value matrix, multi-head self-attention calculation is performed, and the attention outputs of each attention head are concatenated to obtain the global context features of the target image region. The global context features are used to generate the value matrix required for the decoder to perform multi-head self-attention computation.

[0125] In some embodiments, the business card text recognition device 500 further includes a conversion module, which is used for: The text recognition results of the business card image are subjected to structured processing to convert the text recognition results into values ​​of preset standard fields.

[0126] The business card text recognition device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0127] The business card text recognition device in this application embodiment can be a device with an operating system. This operating system can be a Microsoft (Windows) operating system, an Android operating system, an iOS operating system, or other possible operating systems; this application embodiment does not specifically limit it.

[0128] The business card text recognition device provided in this application embodiment can implement all the processes implemented in the above-described business card text recognition method embodiment and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0129] Figure 6 These are schematic diagrams of the structure of an electronic device provided in some embodiments of this application. In some embodiments, such as Figure 6 As shown, this application embodiment also provides an electronic device 600, including a processor 601, a memory 602, and a computer program stored in the memory 602 and executable on the processor 601. When the program is executed by the processor 601, it implements the various processes of the above-described business card text recognition method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0130] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0131] This application also provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described business card text recognition method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0132] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0133] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described business card text recognition method.

[0134] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0135] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described business card text recognition method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0136] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0137] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0138] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the related technology, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0139] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0140] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0141] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for recognizing text on a business card, characterized in that, include: Obtain the image of the business card to be recognized; The business card image is used to detect text using a dual-branch text detection model, and the target image region containing text in the business card image is determined by combining the outputs of each branch. Based on the relative position of the target image region in the business card image, a text recognition model with an encoder-decoder architecture is used to perform text recognition on the business card image to determine the text recognition result of the business card image.

2. The business card text recognition method according to claim 1, characterized in that, The text detection model includes a coordinate prediction branch and a confidence prediction branch; the text detection model using a dual-branch architecture performs text detection on the business card image and combines the outputs of each branch to determine the target image region containing text in the business card image, including: The relative positions of each candidate image region in the business card image are determined by the coordinate prediction branch, and the confidence of each relative position in the business card image is determined by the confidence prediction branch. The candidate image regions are filtered according to the confidence level corresponding to each candidate image region to obtain the target image region in the business card image.

3. The business card text recognition method according to claim 2, characterized in that, The coordinate prediction branch includes a bidirectional recurrent neural network, and the confidence prediction branch includes a channel attention mechanism; The step of determining the relative positions of each candidate image region in the business card image through the coordinate prediction branch, and determining the confidence level of each relative position in the business card image through the confidence prediction branch, includes: The business card image is input into the front backbone network of the text detection model to extract features from the business card image, thereby obtaining deep semantic features and shallow semantic features respectively; Enhanced features are obtained by enhancing the deep semantic features output by the pre-backbone network; The enhanced features are processed by the bidirectional recurrent neural network to obtain hidden features carrying bidirectional contextual information of the business card image, and the relative positions of candidate image regions in the business card image are predicted by regression based on the hidden features. The shallow semantic features output by the pre-backbone network are recalibrated using the channel attention mechanism, and the confidence of each relative position of the business card image is predicted based on the recalibrated shallow semantic features.

4. The business card text recognition method according to any one of claims 1 to 3, characterized in that, The step of performing text recognition on the business card image based on the relative position of the target image region within the business card image, using an encoder-decoder architecture text recognition model, and determining the text recognition result of the business card image, includes: Based on the relative position of the target image region in the business card image, the target image region is identified in the business card image; The labeled business card image is input into the text recognition model. The encoder and decoder of the text recognition model perform text recognition on the labeled target image region to obtain the text recognition result of the target image region. Based on the text recognition results of the target image region, the text recognition results of the business card image are determined.

5. The business card text recognition method according to claim 4, characterized in that, The step involves inputting the labeled business card image into a text recognition model, and then using the encoder and decoder of the text recognition model to perform text recognition on the labeled target image region to obtain the text recognition result of the target image region, including: Feature extraction is performed on the target image region in the business card image, and position encoding is added to the extracted features to obtain the position encoding features of the target image region; The location-encoded features are input into the encoder to perform multi-layer processing on the location-encoded features; Based on the output of the encoder, a multi-head cross-attention calculation is performed in the decoder to obtain the decoded feature sequence of the target image region; The decoded feature sequence is mapped to the vocabulary probability distribution corresponding to the target image region using a multilayer perceptron, and the text recognition result of the target image region is determined based on the vocabulary probability distribution.

6. The business card text recognition method according to claim 5, characterized in that, The decoder consists of multiple decoders; the step of performing multi-head cross-attention calculation in the decoder to obtain the decoded feature sequence of the target image region based on the output of the encoder includes: In each layer of the multi-layer decoder, a query matrix and a key matrix are determined based on the positional encoding features, and a value matrix is ​​determined based on the input of the decoder. Based on the query matrix, key matrix, and value matrix, multi-head cross-attention calculation is performed, and the output features of each attention head are integrated to obtain the output of the decoder. The decoding feature sequence of the target image region is determined based on the output of the last layer decoder in the multi-layer decoder. The input of the first layer decoder in the multi-layer decoder is the output of the encoder; the input of each layer decoder other than the first layer decoder is the output of the layer above the decoder.

7. The business card text recognition method according to claim 1, characterized in that, The method further includes: The text recognition results of the business card image are subjected to structured processing to convert the text recognition results into values ​​of preset standard fields.

8. A business card text recognition device, characterized in that, include: The acquisition unit is used to acquire the image of the business card to be recognized; The detection unit is used to perform text detection on the business card image using a dual-branch architecture text detection model, and combine the outputs of each branch to determine the target image region in the business card image where text exists. The recognition unit is used to perform text recognition on the business card image based on the relative position of the target image region in the business card image, using a text recognition model with an encoder-decoder architecture, and to determine the text recognition result of the business card image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the business card text recognition method as described in any one of claims 1-7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the business card text recognition method as described in any one of claims 1-7.

Images