Text recognition method and system
The text features are extracted through deep learning models and combined with the feature fusion of the prompt encoder and attention mechanism, the accuracy problem of traditional text recognition methods in complex scenarios is solved, and the accuracy and flexibility of text recognition are improved.
Patent Information
- Application Number
- CN202411050260.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2024-12-03
AI Technical Summary
In complex scenarios such as handwritten or scene text, the accuracy of character segmentation is difficult to guarantee, resulting in a significant decrease in recognition accuracy.
The deep learning model is used to extract key features of text images, and the prompt encoder is generated by the prompt encoder, and feature fusion is performed in combination with the attention mechanism, dynamically adjust the importance of different features, and make full use of the context and semantic information of the text.
It improves the accuracy and flexibility of text recognition, enhances the richness and diversity of feature representations, and improves the text recognition performance in complex scenarios.
Smart Images

Figure CN119068470A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of text recognition, and in particular to a text recognition method and system. Background Art
[0002] With the continuous development of artificial intelligence technology, text recognition technology has been widely used in all walks of life. Text recognition refers to the technology of automatically detecting and identifying text information from images or videos. Traditional text recognition methods usually adopt a method based on character segmentation, that is, the text image is first preprocessed, such as binarization, denoising, etc., and then character segmentation is performed, and then the segmented individual characters are recognized. This method has achieved good results in printed text recognition, but in complex scenes such as handwriting or scene text, the accuracy of character segmentation is difficult to guarantee, resulting in a significant decrease in recognition accuracy. Summary of the invention
[0003] The present application provides a text recognition method and system to improve the accuracy of text recognition.
[0004] In a first aspect, the present application provides a text recognition method, the method comprising: Acquire a picture book image to be recognized, and extract text data in the picture book image to be recognized by OCR to obtain picture book text data; Extracting features of the picture book image to be identified to obtain image features; Fusing the picture book text data and the image features to obtain a multimodal representation of the picture book; Comparing the multimodal representation of the picture book with a preset picture book feature library to obtain a picture book matching result; Text recognition is performed on the picture book image to be recognized according to the picture book matching result and the multimodal representation of the picture book to obtain a picture book recognition result.
[0005] In the above technical solution, by inputting the text image to be recognized into the image encoder in the preset target image recognition model, the key features of the text image can be efficiently extracted to obtain a compact and information-rich text feature representation. This feature extraction method fully utilizes the powerful learning ability of the deep learning model, and can automatically learn the significant patterns and distinguishing information in the text image, laying a good foundation for subsequent text recognition.
[0006] After obtaining the text features, the present invention further inputs them into the prompt encoder and generates prompt features through operations such as feature selection and fusion. The prompt feature plays a role in guiding text recognition. It selects the most relevant information for the current recognition task from the entire text features and encodes it into a compact vector representation. This prompt mechanism enables the model to adaptively adjust the focus on text features according to different recognition requirements, thereby improving the accuracy and flexibility of recognition.
[0007] Finally, the present invention inputs the text features and the prompt features into the predictive encoder, and obtains the final text recognition result through attention processing and multi-layer perception processing. Among them, the attention mechanism enables the model to comprehensively consider the text features and the prompt features when recognizing each character, and dynamically adjust the degree of emphasis on different features. This adaptive feature fusion method enables the model to make full use of the contextual information and semantic information of the text, further improving the recognition accuracy.
[0008] In a second aspect of the present application, a text recognition system is provided, the system comprising: A data acquisition module is used to acquire a text image to be recognized, and input the text image to be recognized into an image encoder in a preset target image recognition model to obtain text features; A first processing module, configured to input the text feature into a prompt encoder in a preset target text recognition model to obtain a prompt feature; The second processing module is used to input the text features and prompt features into a predictive encoder in a preset target text recognition model. The specific processing process is: attention processing and multi-layer perception processing are performed on the text features and the prompt features in turn to obtain text recognition results.
[0009] In a third aspect of the present application, a computer storage medium is provided, wherein the computer storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the above method steps.
[0010] In the fourth aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the above method.
[0011] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. The present invention can efficiently extract the key features of the text image by inputting the text image to be recognized into the image encoder in the preset target image recognition model, and obtain a compact and information-rich text feature representation. This feature extraction method makes full use of the powerful learning ability of the deep learning model, and can automatically learn the significant patterns and distinguishing information in the text image, laying a good foundation for subsequent text recognition. After obtaining the text features, the present invention further inputs them into the prompt encoder, and generates prompt features through operations such as feature selection and fusion. The prompt feature plays a role in guiding text recognition. It selects the most relevant information for the current recognition task from the entire text feature and encodes it into a compact vector representation. This prompt mechanism enables the model to adaptively adjust the focus on text features according to different recognition requirements, thereby improving the accuracy and flexibility of recognition. Finally, the present invention inputs the text features and prompt features into the predictive encoder together, and obtains the final text recognition result through attention processing and multi-layer perception processing. Among them, the attention mechanism enables the model to comprehensively consider the text features and prompt features when recognizing each character, and dynamically adjust the degree of attention to different features. This adaptive feature fusion method enables the model to fully utilize the contextual and semantic information of the text, further improving the recognition accuracy.
[0012] 2. This application processes text features through an attention mechanism to obtain a text attention vector, which specifically includes the following steps: First, the text features are linearly transformed to obtain a text query vector, a text key vector, and a text value vector. The purpose of this step is to map text features to three different semantic spaces, representing query information, key information, and value information, respectively, to lay the foundation for subsequent attention calculations. Among them, the text query vector represents the current query or question, the text key vector represents the key information related to the query, and the text value vector represents the specific content or answer related to the query. Through linear transformation, these three vectors can extract semantic information of different aspects and levels in the text features, enhancing the richness and diversity of feature representation.
[0013] 3. Through a series of steps such as normalization, feature fusion, feature vector selection and weighted summation, the application prompts the encoder to convert text features into high-quality and highly discriminative prompt features, providing key prior knowledge and guidance information for the training and prediction of the target text recognition model. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] Figure 1 A flowchart of a text recognition method provided in an embodiment of the present application; Figure 2 An architectural diagram of a text recognition system provided in an embodiment of the present application; Figure 3 A schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION
[0015] In order to enable technicians in this field to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.
[0016] In the description of the embodiments of the present application, words such as "for example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "for example" or "for example" is intended to present related concepts in a specific way.
[0017] In the description of the embodiments of the present application, the meaning of the term "multiple" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.
[0018] In order to facilitate understanding of the method and system provided by the embodiments of the present application, before introducing the embodiments of the present application, the background of the embodiments of the present application is first introduced.
[0019] With the rapid development of artificial intelligence technology, text recognition technology has become an indispensable key technology and has been widely used in many fields such as industrial production, financial services, transportation, medical education, etc. Text recognition is a technology that uses computer vision and pattern recognition algorithms to automatically detect and identify text information in images or videos. It can help us quickly and accurately extract and digitize text content on various carriers, greatly improving the efficiency and accuracy of information processing.
[0020] Most traditional text recognition methods use a processing flow based on character segmentation. First, a series of preprocessing operations are performed on the input text image, such as image binarization, denoising and enhancement, to improve image quality and contrast. Then, the image segmentation algorithm is used to locate and extract the text area and segment it into independent characters. Finally, feature extraction and pattern recognition are performed on these segmented character images to obtain the final recognition result. This method has achieved good results in the task of printed text recognition, and the character recognition accuracy can reach more than 90%. However, in practical applications, we often encounter various complex text scenes, such as handwritten text, natural scene text, etc. In these scenes, the content of the text area is often diverse and uncertain, and there may be adhesion, overlap, deformation and other phenomena between characters, which brings great challenges to the text recognition method based on character segmentation. It is difficult for the character segmentation algorithm to accurately separate the adhesion characters, or it over-segments many meaningless fragments, resulting in frequent subsequent recognition errors. According to relevant research statistics, in handwriting and scene text recognition tasks, the character recognition accuracy of traditional methods will drop significantly to about 70%, seriously restricting the application scope and effect of text recognition technology.
[0021] Therefore, there is an urgent need for a new text recognition method that can break through the bottleneck of character segmentation and improve the overall text recognition performance in complex scenarios.
[0022] After the background introduction of the above content, those skilled in the art can understand the problems existing in the prior art. The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.
[0023] Based on the above background technology, please refer to Figure 1 , Figure 1 A flowchart of a text recognition method provided in an embodiment of the present application is provided. The system can be implemented by a computer program or can be run as an independent tool application. Specifically, in an embodiment of the present application, the method can be applied on a server, but can also be applied on electronic devices such as a server. A text recognition method includes the following steps: S101, obtaining a text image to be recognized, and inputting the text image to be recognized into an image encoder in a preset target image recognition model to obtain text features; Specifically, in order to extract effective feature representation from text images, the present invention utilizes a preset target image recognition model. The model is a deep learning model trained with a large amount of text data and has powerful feature extraction and learning capabilities. We input the text image to be recognized into the image encoder of the model for processing.
[0024] The image encoder is a key component of the target image recognition model, which is mainly composed of a multi-layer convolutional neural network. It maps the text image to a high-dimensional feature space layer by layer to extract the significant patterns and key information in the image. During the encoding process, both the local and global features of the image can be effectively captured to form a comprehensive text feature representation.
[0025] Through the image encoder, the present invention can convert text images of different sizes and resolutions into a unified text feature vector. This compact feature representation not only contains visual information such as character shape and stroke structure, but also contains high-level information such as character combination and contextual semantics. It lays a solid foundation for subsequent text recognition.
[0026] Based on the above embodiment, as an optional embodiment, the step of inputting the to-be-recognized text image into a text encoder in a preset target text recognition model to obtain text features includes: S601, extracting features from the text image to be recognized to obtain preliminary text features; Specifically, the process of extracting features from the text image to be identified to obtain preliminary text features is as follows: First, the text image to be identified is preprocessed, such as size normalization, noise removal, binarization, etc., to eliminate interference factors in the image and improve the stability and reliability of feature extraction. Secondly, the preprocessed text image is extracted using classic image feature extraction algorithms, such as SIFT, SURF, HOG, etc. These algorithms obtain a set of local feature descriptors with excellent properties such as scale invariance, rotation invariance, and illumination invariance by performing multi-scale and multi-directional gradient calculations and statistics on the image, which can effectively characterize the spatial structure and visual features of the text area. Thirdly, the extracted local feature descriptors are summarized and integrated to obtain a feature vector of fixed length, namely the preliminary text feature. This feature summary usually uses methods such as bag-of-words model and convolutional neural network, with the aim of mapping local features into a high-dimensional feature space to enhance the discriminability and robustness of the features. Finally, the obtained preliminary text features are used as the input of the text encoder for subsequent feature encoding and optimization.
[0027] S602, performing attention mechanism processing on the preliminary text features to obtain preliminary text attention features; Specifically, the process of applying the attention mechanism to the preliminary text features to obtain the preliminary text attention features is as follows: First, the preliminary text features are input into an attention module, which is usually composed of a multi-layer fully connected neural network or a convolutional neural network. The purpose is to learn a mapping function to convert the preliminary text features into attention weights. This mapping function can adaptively adjust the importance of different feature dimensions according to the current recognition task and context information, highlight key information, and suppress noise interference. Secondly, the learned attention weights are element-wise multiplied with the preliminary text features to obtain a weighted feature representation, namely the preliminary text attention features. Through this weight multiplication operation, the attention mechanism can selectively amplify or reduce the amplitude of different feature dimensions, making the feature representation more refined and effective. Thirdly, the preliminary text attention features are input into the subsequent feature encoding and fusion modules for further feature optimization and abstraction. The introduction of the attention mechanism enables the model to more flexibly and intelligently utilize feature information at different levels and in different regions, and improve the discriminability and generalization ability of feature representation.
[0028] S603: Perform feature processing on the preliminary text attention feature to obtain the text feature.
[0029] Specifically, the process of feature processing the preliminary text attention features to obtain text features is as follows: First, the preliminary text attention features are input into a feature encoder, which is usually composed of multiple convolutional layers, pooling layers and fully connected layers, with the purpose of hierarchical and structured feature extraction and abstraction of the preliminary text attention features. The convolutional layer can effectively extract local features and contextual information in the text image through local receptive field and weight sharing; the pooling layer can reduce the size of the feature map through downsampling and feature selection, and improve the translation invariance and robustness of the features; the fully connected layer can integrate local features into global features through nonlinear transformation and feature combination, and improve the discriminative power and generalization ability of the features. Secondly, a feature fusion module is added to the top of the feature encoder, which is usually implemented by an attention mechanism or a gating mechanism, with the purpose of adaptively fusing and optimizing feature information at different levels and in different regions. The feature fusion module can dynamically adjust the importance weights of different features according to the current recognition task and contextual information, highlight key information, suppress noise interference, and obtain a more refined and effective feature representation. Finally, the output of the feature fusion module is used as the final text feature to guide and optimize the subsequent text recognition process. Text features integrate the results of preliminary feature extraction, attention mechanism optimization and feature encoding fusion. They have higher discriminability, robustness and generalization ability, and are the core input and decision-making basis of the text recognition model.
[0030] S102, inputting the text feature into a prompt encoder in a preset target text recognition model to obtain a prompt feature; Specifically, the hint encoder receives text features as input and obtains a compact hint feature vector through a series of feature transformation and information extraction operations. In this process, the hint encoder automatically identifies key patterns and salient areas in the text features and encodes them into hint features. Hint features contain the most helpful information for the current recognition task, such as character type, text layout, language attributes, etc. The internal structure of the hint encoder usually consists of an attention mechanism and a feedforward neural network. The attention mechanism enables the hint encoder to adaptively allocate the degree of attention to different features according to the requirements of the recognition task when processing text features. This dynamic feature selection and weighting mechanism enables the hint feature to highlight the most discriminative information and reduce the interference of redundancy and noise. At the same time, the feedforward neural network further performs nonlinear transformation and feature fusion on the attention features to form the final hint feature representation.
[0031] By introducing the hint encoder, the present invention can fully mine the semantic information and context clues in the text features and generate targeted hint features. This hint feature guides and constrains the subsequent recognition process, allowing the text recognition model to infer the correct character sequence more accurately and efficiently. The addition of hint features greatly reduces the difficulty and uncertainty of recognition and improves the generalization ability and robustness of the model.
[0032] Based on the above embodiment, as an optional embodiment, the step of inputting the text feature into a prompt encoder in a preset target text recognition model to obtain the prompt feature includes: S501, normalizing the prompt feature to obtain a normalized feature; Specifically, normalization processing usually adopts methods such as minimum-maximum normalization or Z-score normalization. Minimum-maximum normalization maps each element of the prompt feature to the interval [0, 1], and the formula is: (element value - minimum value) / (maximum value - minimum value). Among them, the minimum value and the maximum value correspond to the minimum value and the maximum value of all elements in the prompt feature, respectively. Through this linear transformation, the value of the prompt feature is compressed into a unified scale, eliminating the magnitude difference between different dimensions. Z-score normalization converts the feature into a standard normal distribution with a mean of 0 and a standard deviation of 1 by subtracting the feature mean from each element of the prompt feature and dividing it by the feature standard deviation. This normalization method not only eliminates the difference in feature values, but also makes the feature present good statistical characteristics.
[0033] The normalized features obtained after normalization have the following advantages: First, the value range of the normalized features is more standardized and unified, eliminating the magnitude differences between different dimensions and making the features more comparable. This helps the subsequent recognition model treat different features more fairly and consistently, improving the accuracy and stability of recognition. Second, the statistical characteristics of the normalized features are better, especially after applying Z-score normalization, the features present a standard normal distribution with a mean of 0 and a standard deviation of 1. This standardized feature distribution is conducive to the convergence and generalization of the recognition model, accelerates the model training process, and improves the performance and robustness of the model.
[0034] S502, performing feature fusion on the normalized feature and a preset second learning parameter to obtain a point mapping; Specifically, feature fusion usually adopts methods such as additive fusion or multiplicative fusion. In additive fusion, the preset second learning parameter is added element by element to the normalized feature to obtain a point mapping. This process can be expressed as: point mapping = normalized feature + preset second learning parameter. The dimension of the preset second learning parameter is the same as that of the normalized feature, and each element is a learnable parameter, which is trained and updated through optimization algorithms such as gradient descent. Through additive fusion, the preset second learning parameter can translate and bias the normalized feature to better adapt it to the characteristics of the current recognition task. In multiplicative fusion, the preset second learning parameter is multiplied element by element to obtain a point mapping. This process can be expressed as: point mapping = normalized feature * preset second learning parameter. Similar to additive fusion, the dimension of the preset second learning parameter is the same as that of the normalized feature, and each element is a learnable parameter. Through multiplicative fusion, the preset second learning parameter can scale and weight the normalized feature, highlight important features, suppress minor features, and improve the discriminability of features.
[0035] S503, selecting a corresponding selected feature vector from the normalized features according to the value of each element in the point map; Specifically, the process of selecting feature vectors can be divided into the following steps: First, traverse each element in the point map and determine the selection threshold or ratio according to the size of its value. For example, you can select the feature vectors corresponding to the first k elements with the largest value in the point map, or select the feature vectors corresponding to all elements in the point map whose values are greater than a certain threshold. This threshold or ratio can be set and adjusted based on experience or cross-validation to achieve the optimal selection effect. Secondly, according to the determined selection criteria, extract the corresponding feature vector from the normalized features to form a selected feature vector. This process can be implemented by means of indexing or masking, and the normalized feature vectors corresponding to the elements in the point map that meet the selection conditions are extracted to form a new feature representation. Finally, the selected feature vectors are combined into a new feature matrix or feature vector as the input for subsequent recognition tasks.
[0036] S504: weighting the selected feature vector according to the value of the point mapping to obtain the prompt feature.
[0037] Specifically, the weighting process can be achieved by multiplying the value of the point mapping by the selected feature vector element by element. This process can be expressed as: prompt feature = point mapping * selected feature vector, where * represents element-by-element multiplication. Each element of the point mapping is a real value, indicating the importance or weight of the corresponding feature. By multiplying the value of the point mapping by the selected feature vector, each element in the selected feature vector can be scaled and adjusted so that important features are amplified and minor features are reduced, thereby improving the discriminative power and generalization ability of the feature. This weighting process is performed feature by feature. For each feature vector in the selected feature vector, it is multiplied by the element value of the corresponding position in the point mapping to obtain a weighted feature vector. Finally, all weighted feature vectors are combined into a new feature representation, which is the prompt feature.
[0038] S103, inputting the text features and the prompt features into a predictive encoder in a preset target text recognition model, and the specific processing process is: performing attention processing and multi-layer perception processing on the text features and the prompt features in turn to obtain a text recognition result.
[0039] Specifically, the predictive encoder receives text features and prompt features as input and performs a series of feature transformation and information fusion operations on them. First, the predictive encoder performs attention processing on the text features and prompt features respectively. The attention mechanism enables the predictive encoder to comprehensively consider the text features and prompt features when decoding each character, and dynamically adjust the importance of different features. This adaptive feature weight allocation enables the predictive encoder to make full use of feature information from different sources to improve the accuracy and reliability of recognition.
[0040] After attention processing, the predictive encoder will further input the obtained attention features into a multi-layer perceptron (MLP) for processing. MLP is a feed-forward neural network composed of multiple fully connected layers that can perform highly nonlinear transformations and abstractions on input features. Through MLP, the predictive encoder can further refine and fuse the attention features to extract more abstract and robust high-level feature representations. This multi-level feature transformation enables the predictive encoder to better capture the contextual dependencies between characters, thereby making more accurate sequence predictions.
[0041] Finally, the predictive encoder passes the high-level features output by the MLP through a softmax layer to map them into the probability distribution of character categories. For each time step, the predictive encoder selects the character with the highest probability as the recognition result of the current position, and concatenates it with the previous recognition result to form a complete character sequence. This sequence is the final text recognition result obtained by the method of the present invention.
[0042] Based on the above embodiment, as an optional embodiment, the text feature and the prompt feature are sequentially subjected to attention processing and multi-layer perception processing to obtain a text recognition result, including: S201, processing the text feature and the prompt feature through an attention mechanism to obtain an attention text feature; Specifically, the predictive encoder inputs the text features and the prompt features into the attention mechanism module. The attention mechanism dynamically assigns a weight coefficient to each feature by calculating the similarity and correlation between different features. This weight coefficient reflects the importance of the feature to the current recognition decision. The larger the weight, the more critical the information contained in the feature, which needs to be paid more attention and utilized.
[0043] When calculating the attention weights, the predictive encoder takes into account the content of the text features and the prompt features. It measures the degree of match between different features through methods such as dot product and cosine similarity. At the same time, the predictive encoder also adjusts the distribution of attention weights based on the current decoding state and contextual information. This enables the attention mechanism to dynamically adjust the emphasis on different features according to the progress of recognition, and achieve adaptive feature selection and fusion. After obtaining the attention weights, the predictive encoder performs a weighted summation with the original text features to obtain the attention text feature. The attention text feature is a filtered and emphasized feature representation that highlights the key information that is most helpful for the current recognition, while suppressing the interference of noise and redundancy. This feature enhancement and noise suppression effect makes the subsequent recognition process more accurate and efficient.
[0044] Based on the above embodiment, as an optional embodiment, the text feature and the prompt feature are processed by the attention mechanism to obtain the attention text feature, including: S301, processing the text feature through an attention mechanism to obtain a text attention vector; Specifically, in the process of attention mechanism processing, the text features need to be processed first to obtain the text attention vector. The purpose of this step is to enable the attention mechanism to adaptively extract and weight the key information in the text features, highlight the feature components that are most helpful for the current recognition task, and thus improve the accuracy and efficiency of subsequent recognition.
[0045] Specifically, the attention mechanism receives text features as input, and through a series of calculations and transformations, generates a text attention vector with the same dimension as the text features. Each element in this vector is a real value, indicating the importance or attention of the text feature at the corresponding position. The larger the value, the greater the impact of the feature on the current recognition decision, and it needs to be paid special attention to and utilized.
[0046] When calculating the text attention vector, the attention mechanism takes into account the content and contextual information of the text features. It linearly transforms the text features through a weight matrix to obtain an intermediate representation. Then, the attention mechanism applies a softmax function to this intermediate representation, normalizes it to a probability distribution, and obtains the final text attention vector. This process can be seen as a soft feature selection and weighting mechanism that dynamically adjusts the emphasis on different features according to the current recognition task and context.
[0047] Through the processing of the attention mechanism, the text attention vector can effectively highlight the key information in the text features and suppress the influence of noise and redundant components. This adaptive feature enhancement and noise suppression makes the subsequent recognition process more accurate and efficient. At the same time, the calculation process of the text attention vector also provides interpretability for the attention mechanism, allowing us to intuitively understand the model's focus and decision-making basis during recognition.
[0048] Based on the above embodiment, as an optional embodiment, the text feature is processed by the attention mechanism to obtain a text attention vector, including: S401, performing linear transformation on the text features to obtain a text query vector, a text key vector and a text value vector; Specifically, the attention mechanism inputs the text features into three different linear transformations to obtain the text query vector, text key vector, and text value vector. The dimensions of these three vectors are usually the same as the dimensions of the original text features, but they play different roles in semantics.
[0049] The text query vector can be regarded as a query signal that specifies the focus or target of the current attention mechanism. It maps the original text features to a new space through linear transformation to highlight the feature components most relevant to the current recognition task. The generation process of the text query vector usually involves the learning of weight matrices and bias terms, which enables it to adaptively adjust the degree of attention to different features.
[0050] The text key vector can be regarded as the key in a key-value pair, which is used to measure the correlation or similarity between different text positions. It maps the original text features to a new space through linear transformation to facilitate matching and similarity calculation with the text query vector. The generation process of the text key vector also involves the learning of weight matrices and bias terms, which enables it to effectively capture the dependencies and contextual information within the text.
[0051] The text value vector can be regarded as the value in a key-value pair, which is used to represent the actual content or information of different text positions. It maps the original text features to a new space through linear transformation to facilitate interaction and aggregation with the text query vector and the text key vector. The generation process of the text value vector also involves the learning of weight matrices and bias terms, which enables it to effectively extract and represent key information in text features.
[0052] By linearly transforming the text features to obtain these three vectors, the attention mechanism provides a more suitable and effective representation for subsequent attention weight calculation and feature aggregation. The introduction of text query vectors, text key vectors, and text value vectors enables the attention mechanism to more flexibly and adaptively capture the complex dependencies within the text and highlight the feature information most relevant to the current recognition task. At the same time, this linear transformation process also provides greater freedom for the learning and optimization of the attention mechanism, enabling it to automatically adjust the attention and utilization strategies for different features according to different task requirements and data characteristics.
[0053] S402, calculating the similarity between the text query vector and the text key vector, and converting the similarity into a text attention score using a softmax function; Specifically, the attention mechanism uses methods such as dot product and cosine similarity to calculate the similarity between the text query vector and the text key vector. Dot product is the most commonly used similarity calculation method. It multiplies the corresponding elements of two vectors and sums them to obtain a scalar value representing the similarity of the vectors. Cosine similarity measures the similarity of two vectors by calculating the cosine value of the angle between them and normalizes the length of the vector. Regardless of the similarity calculation method used, the purpose is to quantify the degree of match between the text query vector and the text key vector and highlight the text position that is most relevant to the current recognition task.
[0054] The calculated similarity value is usually a real number with a large range of values, which is not easy to use directly. In order to convert the similarity into a more standardized and interpretable form, the attention mechanism inputs it into the softmax function for processing. The softmax function is a commonly used normalization function that maps a set of real values to a probability distribution so that the sum of all values is 1 and each value is between 0 and 1. Through the action of the softmax function, the similarity value is converted into a text attention score, which indicates the relative importance of different text positions to the current recognition task.
[0055] The calculation and normalization process of the text attention score enables the attention mechanism to allocate the degree of attention to different text positions in a soft and differentiable way. Unlike the hard binary attention, the text attention score generated by the softmax function is continuous and differentiable, which facilitates the end-to-end learning and optimization of the attention mechanism. At the same time, the text attention score also provides weight information for subsequent feature aggregation, allowing the attention mechanism to adaptively adjust the information utilization strategy for different text positions and improve the accuracy and robustness of recognition.
[0056] S403: Perform weighted summation on the text value vector according to the text attention score to obtain a text attention vector.
[0057] Specifically, the attention mechanism traverses each text position and multiplies its corresponding text attention score with the text value vector element by element. This multiplication operation causes each element in the text value vector to be scaled by the attention score of its corresponding position, reflecting the importance of that position to the current recognition task. The higher the attention score, the larger the result after multiplication, indicating that the text value vector at that position contributes more to the subsequent aggregation. Conversely, for positions with lower attention scores, the result after multiplication is smaller, and the impact on the aggregation result is smaller.
[0058] After completing the position-by-position multiplication operation, the attention mechanism adds all scaled text value vectors element-by-element to obtain a vector with the same dimension as a single text value vector, namely the text attention vector. This vector combines the information of all text positions and weights them according to their attention scores, highlighting the features most relevant to the current recognition task while suppressing the influence of minor or noisy information. The text attention vector can be regarded as an adaptive representation of the original text features under the guidance of the attention mechanism, with higher discriminability and representation ability.
[0059] Through the weighted summation operation, the attention mechanism realizes the selective aggregation of information at different text positions. Unlike simple averaging or concatenation, the weighted summation can dynamically adjust the contribution of each position according to the attention score, so that the aggregation result pays more attention to the information related to the current task and ignores irrelevant or interfering information. This adaptive feature aggregation mechanism enables the text attention vector to better characterize the relationship between text content and recognition tasks, improving the accuracy and generalization ability of recognition.
[0060] S302, processing the prompt feature through an attention mechanism to obtain a prompt attention vector; Specifically, after obtaining the text attention vector, the attention mechanism also needs to process the prompt feature to obtain the prompt attention vector. The purpose of this step is to enable the attention mechanism to make full use of the prior knowledge and contextual information in the prompt feature to guide and constrain the subsequent text recognition process and improve the accuracy and reliability of recognition.
[0061] Specifically, the attention mechanism receives the cue feature as input, and through a series of calculations and transformations, generates a cue attention vector with the same dimension as the cue feature. Similar to the text attention vector, each element in the cue attention vector is also a real value, indicating the importance or attention of the cue feature at the corresponding position. The larger the value, the greater the impact of the cue feature on the current recognition decision, and it needs to be considered and utilized.
[0062] When calculating the cue attention vector, the attention mechanism comprehensively analyzes the content of the cue feature and its relevance to the current recognition task. It linearly transforms the cue feature through a weight matrix to obtain an intermediate representation. The attention mechanism then applies a softmax function to this intermediate representation, normalizes it to a probability distribution, and obtains the final cue attention vector. This process can be viewed as a soft cue feature selection and weighting mechanism that dynamically adjusts the emphasis on different cue features based on the current recognition progress and context.
[0063] Through the processing of the attention mechanism, the cue attention vector can effectively highlight the key information in the cue features and suppress the influence of irrelevant or redundant cue components. This adaptive cue feature enhancement and noise suppression enables the subsequent recognition process to more accurately and efficiently utilize prior knowledge, reducing the uncertainty and error rate of recognition. At the same time, the calculation process of the cue attention vector also provides interpretability for the attention mechanism, allowing us to intuitively understand how the model utilizes and weighs different cue information during recognition.
[0064] S303, concatenating the text attention vector and the prompt attention vector to obtain the attention text feature.
[0065] Specifically, after obtaining the text attention vector and the prompt attention vector, the attention mechanism needs to concatenate them to obtain the final attention text feature. The purpose of this step is to fuse and integrate the attention information of the text feature and the prompt feature to form a more comprehensive and informative feature representation, providing a more accurate and reliable basis for subsequent text recognition.
[0066] Specifically, the attention mechanism will concatenate the text attention vector and the prompt attention vector in a certain way to obtain a new vector representation, namely the attention text feature. The concatenation method may vary depending on the specific application scenario and model design. Common concatenation methods include simple vector concatenation, weighted summation, element-wise multiplication, etc. Regardless of the concatenation method used, the purpose is to organically combine the information in the text attention vector and the prompt attention vector, and make full use of their contributions to the recognition task in different aspects.
[0067] Through the concatenation operation, the attention text feature can contain the key information of the text feature and the hint feature at the same time, reflecting the attention mechanism's ability to adaptively integrate different information sources. The text attention vector highlights the important features of the text content itself, while the hint attention vector introduces the guidance of prior knowledge and contextual information. Their combination enables the attention text feature to more comprehensively and accurately depict the key information required for the current recognition task, improving the reliability and generalization ability of recognition.
[0068] Compared with using text attention vector or prompt attention vector alone, the concatenated attention text feature has stronger representation ability and discrimination. It combines the advantages of text content and prompt information, and can better cope with complex and changing text recognition scenarios. Through the adaptive fusion of the attention mechanism, the attention text feature can dynamically adjust the attention and utilization of different information, and always maintain a high recognition performance. At the same time, the formation process of the attention text feature also provides greater flexibility and scalability for the attention mechanism, which facilitates the introduction of other types of auxiliary information and knowledge, and further improves the intelligence level of text recognition.
[0069] S202, performing multi-layer perception processing on the attention text feature to obtain the text recognition result.
[0070] Specifically, after obtaining the attention text features, the predictive encoder will input them into the multi-layer perceptron (MLP) for further feature transformation and abstraction, and finally obtain the text recognition result. The purpose of this step is to map the attention text features to a higher-dimensional feature space and extract more abstract and robust feature representations, thereby improving the recognition accuracy and generalization ability.
[0071] Specifically, MLP consists of multiple fully connected layers, each of which contains a certain number of neurons. These neurons are connected through nonlinear activation functions (such as ReLU, sigmoid, etc.), which can perform complex nonlinear transformations on input features. When the attention text feature is input to MLP, it goes through a series of fully connected layers and activation functions to gradually map the feature to a higher dimensional space.
[0072] In this process, MLP automatically learns and adjusts the weight and bias parameters of each neuron to optimize the effect of feature transformation. Through gradient back propagation and optimization algorithms (such as Adam, SGD, etc.), MLP can continuously update these parameters so that the transformed features can distinguish different character categories to the greatest extent and improve recognition accuracy.
[0073] The multi-layer structure and nonlinear transformation of MLP enable the predictive encoder to extract more abstract and advanced feature representations. Compared with shallow models, MLP can better characterize the complex dependencies and contextual information between characters and capture the semantic and structural features of text. This high-level feature representation enables the predictive encoder to decode and recognize character sequences more accurately and robustly.
[0074] In the last layer of the MLP, the predictive encoder passes the transformed features through a softmax layer to convert them into probability distributions of character categories. For each time step, the predictive encoder selects the character with the highest probability as the recognition result at the current position and concatenates it with the previous recognition result to form a complete character sequence. This sequence is the final text recognition result obtained by the predictive encoder based on the attention text features.
[0075] Based on the above embodiment, as an optional embodiment, the training process of the target text recognition model includes: S701, obtaining a sample data set, wherein the sample data set includes text image sample data and text recognition result sample data; Specifically, when training the target text recognition model, the first step is to obtain a sample data set. The sample data set is the basis and basis for model training. It contains a large amount of text image sample data and corresponding text recognition result sample data. The purpose of obtaining a sample data set is to provide sufficient training materials so that the model can learn and master the rules and characteristics of text recognition, continuously optimize and improve recognition performance, and ultimately achieve accurate, stable, and generalized recognition results.
[0076] Specifically, the process of obtaining a sample data set can be carried out in the following ways: First, collect a large amount of text image sample data. These text image sample data can come from various channels, such as public data sets, self-collection, and partners. The text image sample data should cover as many scenes, fonts, and languages as possible to ensure that the model can adapt to different recognition tasks. Secondly, annotate each text image sample data to obtain the corresponding text recognition result sample data. The text recognition result sample data is an accurate transcription of the text content contained in the text image sample data. It corresponds to the text image sample data one by one to form a complete sample data pair. The annotation process can be completed by manual annotation or automatic annotation using an existing high-precision text recognition model. The quality of the annotation directly affects the effect of model training. Finally, the collected text image sample data and text recognition result sample data are organized into a standardized format to construct a sample data set. The sample data set is usually stored in the form of a data file or a database. Each sample data pair contains text image data and corresponding text recognition result data, which is convenient for subsequent reading and use.
[0077] After obtaining a high-quality sample data set, it can be used to train the target text recognition model. The quality and quantity of the sample data set have an important impact on the effect of model training. Therefore, the following points should be noted when obtaining the sample data set: First, the sample data set should have sufficient scale and diversity, covering as many scenes, fonts, languages, etc. as possible, to ensure that the model can learn rich features and rules and have good generalization ability. Secondly, the text image sample data in the sample data set should be of high quality, with clear images, no noise, and no distortion, to avoid introducing unnecessary interference and errors. Thirdly, the text recognition result sample data in the sample data set should be accurate and correspond one-to-one with the text image sample data to ensure that the model learns the correct mapping relationship. Finally, the sample data set should undergo necessary preprocessing and cleaning to eliminate invalid, repeated, and erroneous sample data to improve data quality and efficiency.
[0078] S702, inputting the text image sample data into an image encoder in a preset initial image recognition model to obtain text sample features; Specifically, the process of inputting the text image sample data into the image encoder is as follows: First, each text image sample data in the sample data set is sequentially input into the image encoder in the preset initial image recognition model. The image encoder is usually a convolutional neural network (CNN) or its variants, such as ResNet, Inception, etc. The image encoder receives the text image sample data as input, and extracts the multi-level and multi-scale features of the text image through a series of operations such as convolution, pooling, and nonlinear transformation. Secondly, the image encoder integrates and encodes the extracted features to obtain a feature vector of a fixed length, namely the text sample feature. The text sample feature is a highly compressed and abstract version of the original text image sample data. It retains the key information and semantic content of the text image while removing redundancy and noise. Finally, the obtained text sample feature is output as the input of the subsequent recognition model for model training and optimization.
[0079] By inputting text image sample data into the image encoder to obtain text sample features, the following advantages can be brought: First, the text sample feature is a high-level abstraction and compression of the original text image sample data, which retains the key information and semantic content of the text image while removing redundancy and noise. This feature representation is more compact, efficient, and discriminative, and can provide a more accurate and reliable basis for subsequent recognition decisions. Secondly, by using the image encoder in the preset initial image recognition model for feature extraction, the existing mature models and experience can be fully utilized, avoiding the huge overhead of training from scratch. This transfer learning method can greatly accelerate the training process of the model and improve training efficiency and performance. Thirdly, the extraction process of text sample features is end-to-end trainable, and the parameters of the image encoder can be automatically learned and adjusted through optimization algorithms such as back propagation to adapt to the current recognition task. This data-driven feature optimization method can automatically capture and utilize the potential patterns and laws in the data, and continuously improve the quality and efficiency of feature representation.
[0080] S703, inputting the text sample feature into a prompt encoder in a preset initial text recognition model to obtain a prompt sample feature; Specifically, the process of inputting the text sample features into the prompt encoder is as follows: First, the text sample features obtained in the previous step are sequentially input into the prompt encoder in the preset initial text recognition model. The prompt encoder is usually a recurrent neural network (RNN) or its variants, such as LSTM, GRU, etc. The prompt encoder receives the text sample features as input, and extracts the semantic information and contextual relationships in the text sample features through iterative calculations of a series of recurrent units. Secondly, the prompt encoder integrates and encodes the extracted semantic information and contextual relationships to obtain a feature vector of fixed length, namely the prompt sample feature. The prompt sample feature is a highly abstract and semantic version of the text sample feature. It retains the key semantic information and contextual relationships in the text image while removing redundancy and noise. Finally, the obtained prompt sample feature is output as the input of the subsequent recognition model for model training and optimization.
[0081] S704, inputting the text features and the prompt features into a predictive encoder in a preset initial text recognition model to obtain a predictive text recognition result; Specifically, the process of inputting text sample features into the prompt encoder is as follows: First, the text sample features output by the image encoder are sequentially input into the prompt encoder in the preset initial text recognition model. The prompt encoder is usually a recurrent neural network (RNN) or its variants, such as LSTM, GRU, etc. The prompt encoder receives the text sample features as input, and through a series of loops and gating operations, it performs sequence modeling and information extraction on the text sample features to capture the contextual dependencies and semantic information of the text content. Secondly, the prompt encoder integrates and encodes the extracted contextual information and semantic information to obtain a feature vector of fixed length, namely the prompt sample feature. The prompt sample feature is a highly refined and optimized text sample feature. It retains the key information and semantic structure of the text content, while removing redundancy and noise, which is more in line with the characteristics and laws of natural language. Finally, the obtained prompt sample feature is output as the input of the subsequent prediction encoder to guide and optimize the prediction process.
[0082] S705, based on the predicted text recognition result and the text recognition result sample data, obtain the value of the loss function, use the value of the loss function to adjust the parameters of the preset initial text recognition model, and obtain the target text recognition model when the preset training conditions are met.
[0083] Specifically, the process of obtaining the loss function value and optimizing the model based on the predicted text recognition result and the text recognition result sample data is as follows: First, the predicted text recognition result output by the predictive encoder is compared with the corresponding text recognition result sample data, and the difference metric between them is calculated to obtain the value of the loss function. Commonly used loss functions include cross entropy loss, CTC loss, edit distance loss, etc., which measure the degree of match between the predicted result and the true result from different angles. The smaller the value of the loss function, the closer the predicted result is to the true result, and the better the performance of the model. Secondly, the value of the loss function is used to adjust and optimize the parameters of the preset initial text recognition model. This process usually uses the back propagation algorithm to pass the gradient information of the loss function to each parameter of the model, and update the value of the parameter according to optimization strategies such as gradient descent. By continuously iterating this process, the parameters of the model will gradually be adjusted to the optimal state, so that the predicted result is as close to the true result as possible. Thirdly, during the model training process, some preset training conditions need to be set as the criteria for stopping iteration, such as the maximum number of iterations, the threshold of the loss function, and the improvement of the performance of the validation set. When the model training process meets these preset conditions, we believe that the model has reached the optimal state and can stop training to obtain the final target text recognition model.
[0084] On the other hand, the present invention also provides a text recognition system, such as Figure 2 , the system comprises: Data acquisition module 1, used to acquire a text image to be recognized, and input the text image to be recognized into an image encoder in a preset target image recognition model to obtain text features; A first processing module 2 is used to input the text feature into a prompt encoder in a preset target text recognition model to obtain a prompt feature; The second processing module 3 is used to input the text features and prompt features into a predictive encoder in a preset target text recognition model. The specific processing process is: the text features and the prompt features are sequentially subjected to attention processing and multi-layer perception processing to obtain text recognition results.
[0085] Please refer to Figure 3 The application also discloses an electronic device. Figure 3 The electronic device 300 may include: at least one processor 301 , at least one network interface 304 , a user interface 303 , a memory 305 , and at least one communication bus 302 .
[0086] The communication bus 302 is used to realize the connection and communication between these components.
[0087] The user interface 303 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.
[0088] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).
[0089] Among them, the processor 301 may include one or more processing cores. The processor 301 uses various interfaces and lines to connect various parts in the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 305, and calling data stored in the memory 305. Optionally, the processor 301 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field~Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 301 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 301, and it can be implemented separately through a chip.
[0090] Among them, the memory 305 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read~Only Memory). Optionally, the memory 305 includes a non-transitory computer-readable medium. The memory 305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 305 may optionally be at least one storage system located away from the aforementioned processor 301. Refer to Figure 3The memory 305 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application program for a flocculant addition analysis method.
[0091] exist Figure 3 In the electronic device 300 shown, the user interface 303 is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor 301 can be used to call the application program storing the road assessment method in the memory 305. When executed by one or more processors 301, the electronic device 300 executes one or more methods in the above-mentioned embodiments. It should be noted that for the aforementioned method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application. In the above embodiments, the description of each embodiment has its own emphasis. For the parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.
[0092] In the several implementations provided in this application, it should be understood that the disclosed system can be implemented in other ways. For example, the system embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of the system or unit can be electrical or other forms.
[0093] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0094] The present application also provides a computer storage medium that can store multiple instructions, which are suitable for being loaded and executed by a processor as described above. Figure 1 The road assessment method of the embodiment shown in the figure can be found in the specific implementation process. Figure 1 The specific description of the illustrated embodiment will not be repeated here.
[0095] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0096] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory, including a number of instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: various media that can store program codes, such as USB flash drives, mobile hard drives, magnetic disks or optical disks.
[0097] The above are only exemplary embodiments of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, those skilled in the art will easily think of other embodiments of the present disclosure.
[0098] This application is intended to cover any variation, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the art not described in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.
Claims
1. A text recognition method, characterized in that: The method comprises: Acquire a text image to be recognized, and input the text image to be recognized into an image encoder in a preset target image recognition model to obtain text features; Inputting the text features into a prompt encoder in a preset target text recognition model to obtain prompt features; The text features and the prompt features are input into a predictive encoder in a preset target text recognition model. The specific processing process is: attention processing and multi-layer perception processing are performed on the text features and the prompt features in turn to obtain a text recognition result.
2. The method according to claim 1, characterized in that The step of sequentially performing attention processing and multi-layer perception processing on the text features and the prompt features to obtain a text recognition result includes: Processing the text feature and the prompt feature through an attention mechanism to obtain an attention text feature; The attention text features are subjected to multi-layer perception processing to obtain the text recognition result.
3. The method according to claim 2, characterized in that The text feature and the prompt feature are processed by the attention mechanism to obtain the attention text feature, including: Processing the text features through an attention mechanism to obtain a text attention vector; Processing the prompt feature through an attention mechanism to obtain a prompt attention vector; The text attention vector and the prompt attention vector are concatenated to obtain the attention text feature.
4. The method according to claim 3, characterized in that The text feature is processed by the attention mechanism to obtain a text attention vector, including: Performing a linear transformation on the text features to obtain a text query vector, a text key vector, and a text value vector; Calculate the similarity between the text query vector and the text key vector, and convert the similarity into a text attention score using a softmax function; The text value vectors are weighted and summed according to the text attention scores to obtain a text attention vector.
5. The method according to claim 1, characterized in that The step of inputting the text features into a prompt encoder in a preset target text recognition model to obtain prompt features includes: Normalizing the prompt feature to obtain a normalized feature; Performing feature fusion on the normalized feature and the preset second learning parameter to obtain a point mapping; According to the value of each element in the point map, selecting a corresponding selected feature vector from the normalized features; The selected feature vector is weighted according to the value of the point mapping to obtain the prompt feature.
6. The method according to claim 1, characterized in that The step of inputting the to-be-recognized text image into a text encoder in a preset target text recognition model to obtain text features includes: Extracting features of the text image to be recognized to obtain preliminary text features; Performing attention mechanism processing on the preliminary text features to obtain preliminary text attention features; The preliminary text attention feature is subjected to feature processing to obtain the text feature.
7. The method according to claim 1, characterized in that The training process of the target text recognition model includes: Acquire a sample data set, wherein the sample data set includes text image sample data and text recognition result sample data; Inputting the text image sample data into an image encoder in a preset initial image recognition model to obtain text sample features; Inputting the text sample features into a prompt encoder in a preset initial text recognition model to obtain prompt sample features; Inputting the text features and the prompt features into a predictive encoder in a preset initial text recognition model to obtain a predictive text recognition result; Based on the predicted text recognition result and the text recognition result sample data, the value of the loss function is obtained, and the parameters of the preset initial text recognition model are adjusted using the value of the loss function. When the preset training conditions are met, the target text recognition model is obtained.
8. A text recognition system, characterized in that: include: A data acquisition module is used to acquire a text image to be recognized, and input the text image to be recognized into an image encoder in a preset target image recognition model to obtain text features; A first processing module, configured to input the text feature into a prompt encoder in a preset target text recognition model to obtain a prompt feature; The second processing module is used to input the text features and prompt features into a predictive encoder in a preset target text recognition model. The specific processing process is: attention processing and multi-layer perception processing are performed on the text features and the prompt features in turn to obtain text recognition results.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the method according to any one of claims 1 to 7.
10. An electronic device, characterized in that: It includes a processor, a memory and a transceiver, the memory is used to store instructions, the transceiver is used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the method according to any one of claims 1 to 7.
Citation Information
Cited By
Endoscope image quality information generation method and device
CN119251655A
