A method for retrieving a capacitive detection report document based on text image alignment

Abstract

The application discloses a capacitive detection report document retrieval method based on text image alignment, extracts semantic features of a document image and semantic features of a to-be-searched keyword; image semantic features and text semantic features are fused to obtain a probability feature map after self-attention alignment correction; a binary image Bm of the document image is obtained based on the probability feature map by setting a threshold Hr to obtain a text box position; feature selection is performed based on the probability feature map by setting a threshold Ht, a feature area of each text line is extracted, and a selected feature set of all existing texts on the probability feature map is constructed; GLR prediction is performed based on the selected feature set to obtain position information containing the to-be-searched keyword, the position information is mapped into the text box of the binary image Bm, and thus the position of the to-be-searched keyword in the input document image is obtained. This method can realize efficient and accurate keyword retrieval and positioning of a capacitive detection report.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of information retrieval, and particularly relates to a capacitor detection report document retrieval method based on text image alignment. BACKGROUND

[0002] With the popularization and complication of electronic devices, as an important component in electronic devices, the performance detection and fault diagnosis of capacitors become increasingly important. The capacitor detection report document records the key data and analysis results of capacitor detection, which is of great significance for the quality control, fault troubleshooting and product improvement of capacitors. However, with the increasing number of detection documents, how to efficiently and accurately retrieve and locate keywords from massive capacitor detection reports has become a problem to be solved.

[0003] The traditional capacitor detection report document retrieval method is usually based on text line detection and recognition technology, and key word matching or simple text similarity calculation is performed according to the recognized text line text content. These methods have achieved good results in content retrieval of large-scale document sets, but these technologies are for positioning and processing of text lines. Once the text field retrieval and positioning in the text line are involved, the traditional OCR technology cannot solve it. In addition, due to the professionalism and complexity of capacitor detection reports, it is difficult to fully express the content and intention of the document by relying only on keywords or simple text semantic features, thereby affecting the accuracy and effect of retrieval. SUMMARY

[0004] The purpose of the present application is to provide a capacitor detection report document retrieval method based on text image alignment, which can realize efficient and accurate keyword retrieval and positioning of capacitor detection reports.

[0005] In order to achieve the above purpose, the solution of the present application is:

[0006] A capacitor detection report document retrieval method based on text image alignment, comprising the following steps:

[0007] Step 1, extracting the semantic features F of the input document image i and the semantic features F of the keyword to be searched s ;

[0008] Step 2, fusing the image semantic features F i and the text semantic features F s to obtain the probability feature map F after self-attention alignment correction p ;

[0009] Step 3, based on the probability feature map F pBy setting a threshold Hr, a binary image Bm of the document image is obtained, and the text box position is acquired; based on the probability feature map F... p Feature selection is performed by setting a threshold Ht to extract feature regions for each line of text. These regions are then used as boundaries to construct a probabilistic feature map F. p The selection feature set of all existing text is used; GLR prediction is performed based on the selection feature set to obtain the location information containing the keyword to be searched, and the location information is mapped to the text box of the binary image Bm to obtain the location of the keyword to be searched in the input document image.

[0010] In step 1 above, the semantic features F of the document image are extracted. i The specific process is as follows:

[0011] First, feature layers of 1 / 2, 1 / 4, 1 / 8, and 1 / 16 of the document image size are extracted respectively, and these four feature layers are combined into four feature maps with the same scale.

[0012] Then, the four feature maps with consistent scales are merged along the channel dimension to obtain the semantic features F of the document image. i .

[0013] In step 1 above, the semantic features F of the keyword to be searched are extracted. s The specific process is as follows:

[0014] First, semantic analysis is performed on the text containing the search keywords to obtain the 1×1024 dimension text semantic features F. t ;

[0015] Then, for the feature F t A nonlinear transformation is performed to obtain a 1×512 dimension semantic feature F. s .

[0016] The specific process of step 2 above is as follows:

[0017] First, an attention weight alpha is generated using a specific attention calculation process.

[0018] alpha = softmax(Q*K) T )

[0019] Where * denotes matrix multiplication, T denotes transpose operation; Q is the characteristic F s K is a feature F i ;

[0020] Then, the attention weight alpha is applied to the image feature K to obtain the aligned feature F after attention. a ,

[0021] F a = alpha * K

[0022] where * denotes element-wise multiplication;

[0023] Finally, the channel number of F a is reduced from 512 to 1, obtaining the probability feature map F p .

[0024] In step 3, the probability feature map F p is upsampled to the same size as the input document image, and threshold processing is applied to obtain a binary map Bm by setting a threshold Hr.

[0025] In step 3, GLR prediction is performed based on the selected feature set to obtain the position information of the keywords to be searched. The specific process is as follows: the selected feature set is subjected to a pooling operation, reducing the dimension from T x 1 x MaxH x MaxW to T x 1 x 1 x MaxW, where T is the number of text entities detected in the document image, and MaxH and MaxW are the maximum height and width, respectively, present in the feature region; then the pooled features are mapped to a feature space with dimensions Tx1x1x512 using a multi-layer perception (MLP) network; finally, the prediction values G, L, and R are obtained using a fully connected layer, where G represents whether the text box is similar to the text, and L and R represent the left and right boundaries, respectively, of the region where the search text is located.

[0026] After adopting the above scheme, the present application adopts a text image alignment feature fusion and retrieval mechanism to fuse the text semantic features of the user input search keywords with the features of the text segments in the document image, thereby outputting the position information of the text segment most relevant to the query keywords, achieving accurate positioning and efficient retrieval of the text segments within the text lines of the document image. The beneficial effects of the present application include:

[0027] (1) Efficient retrieval capability: By combining text positioning technology and text image self-attention alignment algorithm, the present application can quickly locate the key text segment position in the capacitance detection report document and extract features that are distinctive and expressive, thereby achieving efficient document retrieval. This is very useful for processing capacitance detection report documents and can significantly improve retrieval speed and work efficiency.

[0028] (2) Improved accuracy of accurate positioning: The present application uses accurate text segment positioning technology to avoid the retrieval errors caused by inaccurate positioning in traditional retrieval methods. At the same time, by optimizing the feature extraction and similarity calculation process, the accuracy of the retrieval is further improved. This means that users can more accurately obtain the required information, reducing the risk of false positives and missed detection.

[0029] (3) Flexibility and scalability: The retrieval method of the present application has good flexibility and scalability. No matter how the format of the capacitance detection report document changes, the present application can adapt and provide stable and accurate retrieval results. In addition, this method can also be easily applied to other types of document retrieval tasks to meet the needs of different industries and fields.

[0030] (4) Improve user experience: By directly outputting the text segment location information most relevant to the query keyword, the present application provides users with a more convenient and efficient retrieval experience. Users do not need to manually browse the entire document to quickly find the required text and location information, thereby saving time and effort.

[0031] 5. Promote technological progress and industry development: The implementation of the present application not only improves the analysis efficiency and accuracy of the capacitance detection report, but also brings new breakthroughs and developments to the field of document retrieval. By promoting the progress of related technologies, the present application is expected to have a positive impact on data processing and analysis in the capacitance manufacturing and testing industry, promoting the intelligent and information development of the industry. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 is a flowchart of the present application;

[0033] Figure 2 is a principle diagram of extracting document image features in the present application;

[0034] wherein Input image is the input picture, conv is a 3*3 convolutional neural network, upxN is N times up-sampling, Element-wise sum is element-wise addition, and 1 / M on the picture is 1 / M size dimension of the input picture;

[0035] Figure 3 is a principle diagram of extracting semantic features of the keyword to be searched in the present application;

[0036] wherein Semantic feature is the learned semantic feature;

[0037] Figure 4 is a principle diagram of obtaining a probability feature map in the present application;

[0038] wherein Image Feature is the learned image feature map, Semantic Feature is the learned semantic feature, Aligned Feature is the aligned feature map, and Probability Feature is the probability feature map; Figure 3

[0039] Figure 5 is a principle diagram of predicting text location in the present application;​

[0040] wherein, Probability Feature is a probability feature map, BinaryMap is a binary map, Select Feature is a feature selection process, Selected Feature Map is a selected feature map, Pooling is a pooling operation, MLP is a multi-layer perceptron, and Location Predict is a location prediction (GLR prediction). DETAILED DESCRIPTION

[0041] In recent years, deep learning technology has made significant progress in image processing and natural language processing, providing new solutions for document retrieval, where document image processing technology can locate more accurate document positions, and text image alignment as an effective self-attention learning method can extract more discriminative feature representations by learning the similarities and differences between different samples, thereby improving the performance of retrieval tasks.

[0042] The present application provides a capacitive detection report document retrieval method based on text image alignment, comprising the following steps:

[0043] Step one, extracting the semantic features F of the document image i , extracting the semantic features F of the keyword to be searched s ;

[0044] As shown in Figure 2 , the present application extracts the semantic features of the document image based on ResNet and FPN (Feature Pyramid Network); comprising the following process:

[0045] a) using the depth characteristics of the ResNet network, extracting multi-scale features of the document image, which correspond to different receptive field sizes, thereby effectively capturing rich information from low-level details to high-level semantics; in this embodiment, the ResNet network is configured to input image size 1 / 2, 1 / 4, 1 / 8, 1 / 16, these feature layers are then fine-tuned and combined into four scale-consistent feature maps through a series of upsampling and convolution operations;

[0046] b) using the ConCat module to merge the four scale-consistent feature maps in the channel dimension to obtain the image semantic features F i ; in this embodiment, in order to better learn deep image features, an adaptive hyperparameter λ is used in the ConCat module, which is obtained by globally pooling the four different size feature maps to get the average value of each feature map, and then the four average values are taken to get the adaptive hyperparameters λ1, λ2, λ ",λ # , and then the four adaptive hyperparameters are multiplied by the corresponding features respectively, and then combined to obtain the image semantic feature F i with a dimension of BxNx(H / 4)x(W / 4), where B represents the number of images in the batch, N represents the depth of the output mask layer (set to 512 layers in this embodiment); H and W represent the height and width of the input image respectively, and H / 4 and W / 4 indicate that the height and width of the image semantic feature map are 1 / 4 of the original input image size.

[0047] As shown in Figure 3 , the semantic features of the search keyword are extracted, including the following processes:

[0048] a) Extracting the features F t of the search keyword; in this embodiment, the BERT (Bidirectional Encoder Representations from Transformers) model is used as the core component to extract the semantic features of the search keyword, which is a deep learning model based on the Transformer architecture and has been widely applied and has excellent performance in the field of natural language processing. The BERT model can use its self-attention mechanism and bidirectional encoding capability to perform in-depth semantic analysis and understanding of the text of the search keyword, generating a text semantic feature F t with a dimension of 1x1024, which can effectively capture the key information and context relationship in the text, providing strong support for subsequent feature fusion and task processing; t

[0049] b) Nonlinear transformation is performed on the feature F t to obtain a semantic feature F s with a dimension of 1x512; in this embodiment, a layer of MLP (Multi-Layer Perceptron) model is introduced after the BERT model, which receives the semantic feature F t output by the BERT model as input and maps it to a lower-dimensional feature space through nonlinear transformation, finally outputting a refined semantic feature F s with a dimension of 1x512; this dimension reduction process helps to reduce computational complexity while retaining the most representative semantic information in the text.

[0050] Step two, fuse the text semantic feature with the image semantic feature to obtain the probability feature map F p after self-attention alignment correction;

[0051] As shown in Figure 4 , the text semantic feature F s ​(Feature of Semantic, denoted as Q, with a dimension of 1x512) and image semantic features F i (Image Feature, denoted as K, with dimensions 1x512x1 / 4Hx1 / 4W) First, an attention weight alpha is generated through a specific attention calculation process. Here, Q and K act as the query and key, respectively, and are used to calculate similarity in the attention mechanism. The formula for calculating the attention weight alpha is as follows:

[0052] alpha = softmax(Q*K) T )

[0053] Here, * denotes matrix multiplication, and T denotes transpose. The softmax function normalizes the similarity scores into a weight distribution, where the alpha dimension is (H / 4)×(W / 4)×1. After obtaining the attention weight alpha, it is applied to the image feature K to generate the alignment feature F after attention. a The calculation formula is:

[0054] F a =alpha*K

[0055] Here, * denotes element-wise multiplication. Through this process, the model can focus on the regions in the image most relevant to the text semantics, improving the discriminative power of features and task performance. At this point, F... a The feature dimension is B×512×(H / 4)×(W / 4). Next, a 1x1 convolutional layer can reduce the number of channels in the feature map from 512 to 1, resulting in the probabilistic feature map F. p The dimension of the 1x1 convolutional layer is B×1×(H / 4)×(W / 4), and its size is 1 / 4 of the original image. In this way, the 1x1 convolutional layer can maintain the spatial dimension but reduce the number of channels, thus obtaining the desired dimension and a probabilistic feature map that preserves spatial information.

[0056] In step two, by performing attention-weighted processing on the image semantic features under the supervision of text semantics, the model can focus on the image region most relevant to the text content, thereby improving the accuracy of cross-modal alignment.

[0057] Step 3, based on the probability feature map F p By setting a threshold Hr, a binary map Bm (Binary Map) of the document image is obtained. The text positions on this map are called text box positions. Figure 5 In another processing branch, the probability feature map F pFeature selection is performed by setting a threshold Ht. Since text has a high probability of being selected at this threshold, feature selection yields a set of selected features for all text present in the probability feature map Fp. The elements of these selected feature sets are the probability feature maps of the text regions corresponding to each line of text. Because image acquisition is standardized, these selected feature maps correspond one-to-one with the text boxes in the binary image. The selected feature sets are then processed to predict the location information containing the search keyword, also known as GLR prediction. This GLR-predicted location information of the selected text features is then mapped back to the text boxes in a binary image of the same size as the original image, thus obtaining the location of the search keyword in the original image.

[0058] Cooperate Figure 5 As shown, the probability feature map F p Upsample to the original document image size and apply thresholding. By setting the threshold Hr, the binary image Bm of the original document image is obtained.

[0059] In step three, the text boxes that may contain the search keywords are predicted, and the specific process is as follows:

[0060] The received probabilistic feature map Fp has dimensions Bx1x1 / 4Hx1 / 4W, where 1 / 4H and 1 / 4W represent that the height and width of the feature map are 1 / 4 of the original image size. Next, feature extraction is performed to obtain the feature regions of the text lines. Since the input capacitance test report image was taken under uniform lighting conditions, and the document content is neat and clearly printed, the probabilistic feature map has good contour regions of each text line. By setting an appropriate threshold Ht, the probabilistic feature map containing the text line region can be effectively obtained from the probabilistic feature map; this process is called the feature selection stage.

[0061] Specifically, in the feature extraction stage, threshold processing is applied, and a threshold Ht is set to distinguish and extract the feature regions of each text line. These regions are bounded by their maximum outer contour to construct a selected feature map, with dimensions T×1×MaxH×MaxW. Here, T represents the number of text entities detected in the image, and MaxH and MaxW are the maximum height and width of the feature regions, respectively. In this invention, MaxH is set to 32, a value that matches the average height of the feature map in the capacitance detection report document. For MaxW, the widest feature value is selected as MaxW; other features smaller than this size are padded using zero-padding technology to ensure consistency in feature dimensions.

[0062] Next, the features of the selected feature maps are further compressed and distilled by a pooling operation with respect to the height, reducing their dimension to T x 1 x 1 x MaxW. The pooling operation helps to reduce the redundant information of the features while preserving the key spatial structural features. Subsequently, these features are fed into a multi-layer perceptron (MLP) network. The MLP includes 1 fully connected layer and 1 nonlinear activation function. In the present invention, the MLP network is responsible for mapping the pooled features to a higher dimensional feature space in order to better capture the complex patterns of the text regions. Through the processing of the MLP network, we obtain a feature representation with dimension Tx 1 x 1 x 512.

[0063] Finally, these features are fed into a fully connected layer, which is responsible for mapping the high-dimensional features to the target output space. In the present invention, the linear layer generates three prediction values for each text line region probability feature map: G, L, and R, which are referred to as GLR predictions in the present invention. The range of these three values is [0, 1] in the prediction. Among them, G represents whether the text box is similar to the text (usually implemented by a binary classifier, which is logistic regression in the present invention). When G is close to 1, it means that the text box is likely to contain the search keyword; when G is close to 0, it means that the text box is unlikely to contain the search keyword. L and R are the left and right boundaries of the region where the search text is located in the text box, so L must be less than R. These ratios allow the positioning information to be mapped back to the corresponding text box on the binary map Bm of the original image in the subsequent steps, thereby accurately locating the specific position of the text.

[0064] The text regions on the binary map Bm are represented by the maximum peripheral frame diagram of the segmented regions, which is obtained by setting a segmentation threshold. The threshold is set to a fixed value Hr, because the image acquisition is standardized by a uniform light source, so a fixed threshold is used for division, which can effectively separate the text from the background in the image. The main change in the numerical value of the binary map relative to the probability feature map is the resolution improvement brought about by the upsampling operation, but the value of each original pixel will remain consistent within a 4x4 pixel region. Therefore, the new threshold Hr for dividing the binary map is still theoretically used relative to the division threshold Ht of the probability feature map, and in practice the same value can be used for both, so in practice the text regions obtained by the binary map and the probability feature map are one-to-one corresponding, and the GLR values obtained from the feature probability map can be mapped to the binary map, thereby locating the position of the search text in the original image.

[0065] The network training of the present application includes two stages of training, namely the text line segmentation detection training stage and the GLR training stage, which correspond to the learning of the binary graph segmentation network and the training of the three prediction values of the probability feature graph respectively. In order to enable the model to converge quickly, we first train a stable text line detection network based on the binary graph segmentation model, and then perform the GLR training stage, so that the GLR prediction based on the probability feature graph is added to the training. Moreover, in the GLR prediction, the present application first marks the text box position in the document image through a detailed data labeling process, and accurately labels the coordinates of each single character in the image. This labeling not only covers the overall area of the text, but also goes deep into the single character level. Using these labeled data, the present application further calculates the specific position information of the keyword to be searched in the text line. In this process, through the combination logic of the single character position, the position of the keyword is accurately inferred and converted into a proportional value relative to the width of the text line. Specifically, for each text line to be searched, the present application determines three key parameters: G, L and R. Among them, G represents the binary identification of whether there is a keyword in the text line; L represents the distance between the left boundary of the keyword and the left boundary of the text line, which accounts for the proportion of the width of the text line; R corresponds to the distance between the right boundary of the keyword and the right boundary of the text line, which accounts for the proportion of the width of the text line. The introduction of these three parameters enables the network to more accurately understand and locate the text information in the image. In the GLR network training stage, the present application uses the L1 loss function to optimize the learning process of the three parameters G, L and R. The L1 loss function is known for its sensitivity to outliers, which helps to reduce prediction errors and improve the robustness of the model in network training. In order to further improve the accuracy of text line detection, in the text line segmentation detection training stage, the present application introduces the Dice loss function to complete the training process of image segmentation. The Dice loss function has a wide range of applications in the field of image segmentation processing, and it can effectively handle the imbalance between foreground and background, which is particularly suitable for challenging tasks such as text line detection.

[0066] In summary, the present application proposes a capacitive detection report document retrieval method based on text image alignment, which combines text positioning technology and text image self-attention alignment algorithm, aiming to learn more discriminative and expressive document feature representation, to locate the text key field from the document image, and to realize efficient and accurate retrieval and positioning of the key search words in the capacitive detection report. Specifically, in cooperation Figure 1The application firstly constructs a feature extraction model of a document image, and pre-processes and extracts features of the document image in a document library. The feature extraction focuses on extracting key text line image features from the document image. Then, the application extracts text semantic features of a search keyword input by a user, and fuses the features with text line features of each document image to generate a feature mapping diagram after self-attention alignment and correction. This step ensures consistency of the search keyword and the document content in the feature space, and improves the search accuracy. Finally, according to the query keyword input by the user, the position of the relevant text of the keyword in the document image is predicted, and a list of the most relevant capacitive detection report documents of the search keyword and the position of the search keyword in the detection report are returned. This step realizes fast and accurate retrieval of the required capacitive detection report from a large number of documents. The method can learn the semantic similarity between documents, strengthen the discriminability and expression ability of features, and is suitable for processing large-scale document set text retrieval and positioning, and provides strong support for management and application of capacitive detection reports.

[0067] Those skilled in the art will appreciate that embodiments of the application can be provided as methods, systems, or computer program products. Accordingly, the application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage media, etc.) having computer-usable program code embodied in the medium. The solutions in the embodiments of the application can be implemented in various computer languages, such as object-oriented programming languages Java and interpreted scripting language JavaScript.

[0068] The application is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in a flow or multiple flows and / or blocks Figure 1 The functions specified in a flow or multiple flows and / or blocks

[0069] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0070] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the Figure 1 function specified in the flow or flows and / or blocks ​ of the block or blocks.

[0071] Although preferred embodiments of the application have been described herein, substitutions and alterations can be made to these embodiments by those skilled in the art without departing from the spirit and scope of the application. Accordingly, it is intended that the appended claims be interpreted as including all such alternatives and modifications as fall within the spirit and scope of the present application.

[0072] It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application cover modifications and variations of this application provided they come within the scope of the appended claims and their equivalents.

Claims

1. A method for retrieving a document of a report of a capacitance detection based on alignment of a text image, characterized by Comprising the following steps: Step 1, extracting semantic features F of the input document image i and the semantic features F of the keyword to be searched s ; Step 2, the image semantic feature F i and the text semantic feature F s are fused to obtain a probability feature map F p after self-attention alignment correction; Step 3, based on the probability feature map F p A binary map Bm of the document image is obtained by setting a threshold Hr, and a text box position is acquired; based on the probability feature map F p Feature selection is performed by setting a threshold Ht, and a feature region of each text line is extracted, which is bounded by its largest peripheral contour, and a probability feature map F is constructed p All selected features with text exist; GLR prediction is performed based on the selected feature set, position information containing a keyword to be searched is obtained, the position information is mapped to a text box of the binary map Bm, and thus a position of the keyword to be searched in the input document image is obtained; In step 1, the semantic features F of the document image are extracted i The specific process is that, First, extract 1 / 2, 1 / 4, 1 / 8, 1 / 16 feature layers of the document image size respectively, and combine the four feature layers into four scale consistent feature maps; Then, the four scale-consistent feature maps are merged in the channel dimension to obtain the semantic feature F of the document image i ; In the step 1, the semantic feature F of the keyword to be searched is extracted s The specific process is that, First, the text to be searched keyword semantic analysis, get 1 x 1024 dimension of the text semantic features F t ; Then, the feature F t is subjected to a nonlinear transformation to obtain a 1x512-dimensional semantic feature F s ; The specific process of step 2 is, First, the attention weight alpha is generated by using the attention calculation process, alpha = softmax(Q * K T ) wherein * denotes matrix multiplication, T denotes the transpose operation; Q is a characteristic F s of the matrix K is a characteristic F i ; Then, the attention weight alpha is applied to the image feature K to obtain the alignment feature F after attention focus a , F a = alpha * K Where * represents element-wise multiplication; Finally, the number of channels of F a is reduced from 512 to 1, resulting in a probability feature map F p .

2. The method of claim 1, wherein: In step 3, the probability feature map F p Up-sampling to the same size as the input document image and applying threshold processing to obtain a binary image Bm by setting a threshold Hr.

3. The method of claim 1, wherein: In step 3, GLR prediction is performed based on the selected feature set to obtain the position information of the search keyword, and the specific process is as follows: performing pooling operation on the selected feature set, reducing the dimension from T*1*MaxH*MaxW to T*1*1*MaxW, wherein T is the number of detected text entities in the document image, MaxH and MaxW are the maximum height and width in the feature region respectively; then input into the multi-layer perception MLP network, map the pooled features to a feature space with a dimension of T*1*1*512; finally, use the fully connected layer to obtain the prediction values G, L and R, G represents whether the text box is similar to the text, and L and R represent the left boundary and right boundary of the region where the corresponding search text in the text box is located respectively.

Images