A self-attention test paper layout analysis method based on the maskrcnn backbone

Abstract

The present invention discloses a self-attention test paper layout analysis method based on the Mask R-CNN backbone, which is used to analyze and extract the content of the test paper according to the layout. The steps are as follows: Input the vector representation of the image data into the Mask R-CNN backbone to extract image features, obtaining the Region feature vector (local feature vector) and ROI (region of interest); Input the Region feature vector and ROI into the self-attention module to obtain the final vector representation for classification and regression; Then input this vector representation into the classification fully connected layer and the regression fully connected layer respectively to obtain the final classification result and regression coordinates. This method is designed based on computer vision, so it is completely not limited by the type of the input image (background brightness and darkness, resolution, layout format, font color, etc.), and is applicable to multiple subjects.

Description

Technical Field

[0001] The present invention relates to the technical field of instance segmentation in computer vision, and particularly to a self-attention test paper layout analysis method based on a maskrcnn backbone. Background Art

[0002] In recent years, electronic test papers have been widely used in teaching scenarios due to the convenience of storage and reconstruction. How to efficiently obtain the scope of each question in an electronic test paper and extract effective information is an important prerequisite for the development of teaching-related applications. Electronic test papers are mostly stored in picture format, lacking structured features and being difficult to directly analyze and statistically process. Layout analysis technology can divide the picture into regions, further dividing the test paper in picture format into illustration regions, text regions, etc., enabling the test paper pictures to be stored, analyzed, and statistically processed in a structured manner.

[0003] Traditional methods in this field rely too much on manual participation, such as manually cutting different regions of test paper questions, or manually presetting templates to set thresholds for pixel clustering into specific regions for different layout designs, etc. These methods are extremely dependent on manual labor, with too high time and economic costs. With the development of machine learning and deep learning, a series of methods have emerged, such as manually designing layout features to train machine learning classifiers, neural network models using multiple modal information, etc. However, most of these methods use overly simple neural network models and often cannot adapt to other complex layout situations. A small number of methods use large-scale pre-trained models and fine-tune them on downstream tasks, but the pre-trained models require relatively fine-grained text annotation information during the pre-training process, with too high annotation costs and data processing costs.

[0004] Considering the limitations and deficiencies of current methods, there is an urgent need to propose a test paper layout analysis method with low annotation costs and less manual participation. Summary of the Invention

[0005] The purpose of the present invention is to solve the above-mentioned defects in the prior art, and provide a self-attention test paper layout analysis method based on a maskrcnn backbone. This method first obtains ROI candidate regions and region feature vectors through the maskrcnn backbone, then uses the attention mechanism to obtain a comprehensive feature vector that integrates layout position information, and finally inputs this comprehensive feature vector into a classifier to obtain the final classification and bounding box regression results.

[0006] The purpose of the present invention can be achieved by adopting the following technical solutions:

[0007] A self-attention test paper layout analysis method based on a maskrcnn backbone, the test paper layout analysis method comprising the following steps:

[0008] S1. Input the RGB test paper image with size H×W×3 into ResNet-50 to obtain 5 feature maps with different sizes, denoted as C1, C2, C3, C4, C5 respectively. The sizes of the 5 feature maps are as follows in sequence Among them, H represents the height of the test paper image, and W represents the width of the test paper image;

[0009] S2. Use the FPN algorithm to fuse the four feature maps C2, C3, C4, C5 to obtain feature maps with the same number of channels but different sizes. Among them, the FPN algorithm process is as follows: First, perform nearest neighbor upsampling on C5, C4, C3, C2 from top to bottom to obtain the feature maps M5, M4, M3, M2. Then, perform a 1×1 convolution on each of the feature maps C5, C4, C3, C2, and then perform corresponding addition with the upsampled feature maps M5, M4, M3, M2 to obtain D5, D4, D3, D2. Finally, perform a 3×3 convolution on the added feature maps D5, D4, D3, D2 to finally obtain the fused feature maps P5, P4, P3, P2;

[0010] S3. Perform max pooling with a stride of 2 on the upsampled feature map M5 to obtain the fused feature map P6, which is used to obtain subsequent candidate protocol boxes;

[0011] S4. Generate 9 prior regions with 3 different lengths and aspect ratios of 1∶1, 1∶2, and 2∶1 respectively centered on each pixel point in the fused feature maps P6, P5, P4, P3, P2. Use the Region Proposal Network (RPN) to perform binary classification and bounding box regression on the above prior regions, and filter out the prior regions whose confidence of belonging to the target is less than γ, whose size is too small to completely contain the target, or whose size is too large to exceed the range of the feature map. Among them, γ is a predefined confidence threshold for filtering prior regions with low confidence. Then, use non-maximum suppression to filter out overlapping prior regions to obtain the final candidate regions ROI. Among them, each rectangular candidate region has four vertices, and ROI ∈ R N×4 , R represents the real number field, and N is the number of candidate boxes for each feature map;

[0012] S5. Perform ROI Align candidate region alignment on the candidate regions ROI to obtain several feature maps with the same size;

[0013] S6. Use a multi-layer neural network to perform boundary recognition and mask prediction on the feature maps obtained in step S5. Among them, boundary recognition includes region classification and boundary regression;

[0014] S7. Input the candidate region ROI obtained in step S4 into the position encoding layer Position_Embeddings to obtain the position embedding feature Position_Embeddings(ROI) of each candidate region, where Position_Embeddings(ROI) ∈ R N×D ;

[0015] S8. Add the region feature vector P and the position embedding feature Position_Embeddings(ROI) to obtain the fusion vector G, G = P + Position_Embeddings(ROI), and perform layer normalization on the fusion vector G to obtain the feature vector F, F = Layer_Norm(G), where G, F ∈ R N×D , and Layer_Norm() represents the layer normalization operation function;

[0016] S9. Input the feature vector F obtained in step S8 into the attention layer to obtain the attention feature A;

[0017] S10. Add the attention feature A and the original region feature vector P and perform layer normalization to obtain the final comprehensive feature vector P′, where P′ ∈ R N×D ;

[0018] S11. Input the comprehensive feature vector P′ into two independent fully connected layers: the first fully connected layer FC1 and the second fully connected layer FC2, which are used for classification and bounding box regression respectively, to obtain the final classification and regression results.

[0019] Furthermore, the ResNet-50 is a residual network sequentially spliced by 49 convolutional layers and 1 pooling layer, and is a picture feature extractor that can extract 5 different-sized feature maps from the original picture. In particular, ResNet-50 uses the way of residual connection to solve the problem of gradient disappearance caused by the increase of network depth, so that the network depth is no longer a constraint restricting the final performance.

[0020] Furthermore, in the process of using the FPN algorithm to fuse the four feature maps C2, C3, C4, and C5 in step S2, a 1×1 convolution is performed on each of the feature maps C5, C4, C3, and C2, and the output channels are all set to the same 256 channels. The FPN algorithm, that is, the Feature Pyramid Network algorithm, is a top-down feature fusion method that can perform multi-scale object detection and obtain multiple feature maps. The FPN algorithm enables the final prediction to be performed on the fusion feature maps of multiple different scales, achieving the maximization of detection accuracy.

[0021] Further, in step S4, the Region Proposal Network (RPN) is used to perform binary classification of the above prior regions as target or background. Based on the fused feature map obtained in the previous steps, RPN correspondingly divides the original image into several regions. The center of each region in the original image is represented by the coordinates of a pixel point on this feature map (assuming the size of the feature map is m×n, the original image is divided into m×n regions). Through the anchor mechanism, 9 candidate boxes where a target may exist can be generated for the region corresponding to each pixel point in the original image. Then, RPN determines whether the 9 regions corresponding to each pixel point contain a target, performs binary classification, and filters out all the above prior regions that do not meet the requirements.

[0022] Further, the specific process of non-maximum suppression in step S4 is as follows: Sort the confidence levels of all prior regions belonging to the target, select the prior region with the highest confidence level and its corresponding boundary, and filter out other prior regions whose IoU(A, V) value with this prior region is greater than δ; this process can filter out regions with a large amount of overlap to obtain the final candidate regions. A and V represent two different graphs, A∩V represents the overlapping area of regions A and V, A∪V represents the total area of regions A and V, IoU(A, V) is the ratio of the overlapping area of regions A and B to the total area of regions A and V, and δ is a predefined threshold used to limit the filtering process.

[0023] Further, the process of using a multi-layer neural network to perform boundary recognition and mask prediction on the feature map obtained in step S5 in step S6 is completed by two independent neural networks MLP1 and MLP2 respectively. Among them, MLP1 compresses the feature map obtained in step S5 into a region feature vector P, where P ∈ R N×D , where D is the dimension of the preset local feature vector, and MLP2 is used to obtain the mask of the feature map, that is, to classify each pixel point in the feature map as belonging to the target or the background.

[0024] Further, the operation process of the layer normalization operation function Layer_Norm() in step S8 is as follows: First, calculate the means E1, E2,..., E of each row in the N rows of the vector G N and variances σ1, σ2,..., σ N , the vector G is a vector with a size of N×D, then the element F at the i-th row and j-th column in F ij can be calculated by the following formula: where G ijDenote the element at the \(i\)-th row and \(j\)-th column in vector \(G\), where \(1\leq i\leq N\) and \(1\leq j\leq D\). \(\in\) is a predefined parameter used to prevent the variance of the data distribution from being 0, \(\gamma\) is a predefined parameter used to adjust the variance of the data distribution, and \(\beta\) is a predefined parameter used to adjust the mean of the data distribution. Layer normalization converts the input features into data with a mean of 1 and a variance of \(O\), smoothing out the size relationships between different samples while preserving the size relationships between different features, ensuring the stability of the data feature distribution and accelerating the convergence speed of the training process.

[0025] Further, the process of step S9 is as follows: Randomly initialize three weight matrices \(Q\), \(K\), and \(V\) of size \(D\times D\). Multiply \(F\) by \(Q\), \(K\), and \(V\) respectively to obtain matrices \(q\), \(k\), and \(v\) of size \(N\times D\). Calculate the attention weight matrix \(A\) based on \(q\), \(k\), and \(v\) and perform normalization using the softmax function. The calculation formula is as follows: The self-attention mechanism effectively captures the relationships between layout information, has a more sensitive perception for this special type of picture of test papers, and further improves the performance of layout analysis.

[0026] Further, the RGB test paper pictures include, but are not limited to, subjects such as Chinese, mathematics, politics, history, geography, physics, chemistry, and biology.

[0027] The present invention has the following advantages and effects compared with the prior art:

[0028] 1. The present invention uses a neural network model, eliminating the need for manual cutting of test papers and manual separation of content such as illustrations and texts, greatly reducing the time cost and economic cost of the task.

[0029] 2. The model proposed by the present invention is based on the pre-trained model maskrcnn. Therefore, it does not require retraining with a large amount of data sets. It can effectively play its role by fine-tuning on a limited data set. It has low requirements for the amount of data, and the annotation is simple (only the position and category of each region need to be annotated), greatly reducing the annotation cost and data processing cost.

[0030] 3. The present invention replaces the traditional object detection task with an instance segmentation task. The comprehensive feature vector \(P'\) is input into two independent fully connected layers. It is not only necessary to select the text regions, illustration regions, table regions, etc. of the test questions, but also classify them, treating different regions of the same category as different objects. It is more fine-grained than the traditional object detection task and is more accurate in localizing regions.

[0031] 4. The present invention adopts a self-attention mechanism, effectively capturing the relationships between layout information, making the model have a more sensitive perception for this special type of picture of test papers.

[0032] 5. The self-attention test paper layout analysis method based on the maskrcnn framework proposed by the present invention is designed based on computer vision, so it is completely unrestricted by the type of input pictures (background brightness, resolution, layout format, font color, etc.), has a wide range of applications, and performs well in multiple subjects such as Chinese, mathematics, politics, history, geography, physics, chemistry, and biology. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The drawings described herein are used to provide a further understanding of the present invention, form a part of this application, and the schematic embodiments and descriptions of the present invention are used to explain the present invention and do not constitute an improper limitation to the present invention. In the drawings:

[0034] Figure 1 is a flowchart of the self-attention test paper layout analysis method based on the maskrcnn framework disclosed in the present invention;

[0035] Figure 2 is a schematic diagram of the self-attention test paper layout analysis method based on the maskrcnn framework disclosed in the present invention. From Figure 2 it can be seen the principle of this test paper layout analysis method;

[0036] Figure 3 is an experimental example diagram of the self-attention test paper layout analysis method based on the maskrcnn framework disclosed in the present invention in comprehensive liberal arts;

[0037] Figure 4 is an experimental example diagram of the self-attention test paper layout analysis method based on the maskrcnn framework disclosed in the present invention in the chemistry subject. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0038] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0039] Embodiment 1

[0040] Figure 1 is a flowchart of the network text sentiment analysis method based on the perturbed and improved self-attention mechanism disclosed in the present invention. As Figure 1 shown, this network text sentiment analysis method includes the following steps:

[0041] S1. Input the RGB comprehensive liberal arts test paper image with a size of 1600×800×3 into ResNet-50 to obtain 5 feature maps with different sizes, denoted as C1, C2, C3, C4, and C5 respectively. The sizes of the 5 feature maps are 800×400×256, 400×200×256, 200×100×512, 100×50×1024, and 50×25×2048 in sequence;

[0042] S2. Use the FPN algorithm to fuse the four feature maps C2, C3, C4, and C5 to obtain feature maps with the same number of channels but different sizes. Among them, the FPN algorithm process is as follows: First, perform nearest neighbor upsampling on C5, C4, C3, and C2 from top to bottom to obtain the feature maps M5, M4, M3, and M2. Then, perform a 1×1 convolution on each of the feature maps C5, C4, C3, and C2, and then perform corresponding addition with the upsampled feature maps M5, M4, M3, and M2 to obtain D5, D4, D3, and D2. Finally, perform a 3×3 convolution on the added feature maps D5, D4, D3, and D2 to finally obtain the fused feature maps P5, P4, P3, and P2;

[0043] S3. Perform max pooling with a stride of 2 on the upsampled feature map M5 to obtain the fused feature map P6, which is used to obtain the subsequent candidate protocol boxes;

[0044] S4. Generate 9 prior regions with 3 different lengths and aspect ratios of 1∶1, 1∶2, and 2∶1 respectively centered on each pixel point in the fused feature maps P6, P5, P4, P3, and P2. Use the Region Proposal Network (RPN) to perform binary classification (object or background) and bounding box regression on the above prior regions, and filter out the prior regions whose confidence of belonging to the object is less than 0.7, whose size is too small to completely contain the object, or whose size is too large to exceed the range of the feature map. Then, use non-maximum suppression to filter out overlapping prior regions to obtain the final candidate regions ROI. Among them, each rectangular candidate region has four vertices, so ROI∈R N×4 , where R represents the real number field and N is the number of candidate boxes for each feature map;

[0045] S5. Align the candidate regions ROI to obtain several feature maps with the same size;

[0046] S6. Use a multi-layer neural network to perform boundary recognition and mask prediction on the feature maps obtained in S5. Among them, boundary recognition includes region classification and boundary regression. In particular, this step is completed by two independent neural networks MLP1 and MLP2 respectively. Among them, MLP1 compresses the feature maps obtained in S5 into region feature vectors P, where P∈R N×D, where D is the dimension of the preset local feature vector. MLP2 is used to obtain the mask of the feature map, that is, to classify whether each pixel point in the feature map belongs to the target or the background;

[0047] S7. Input the candidate region ROI obtained in step S4 into the position encoding layer Position_Embeddings to obtain the position embedding feature Position_Embeddings(ROI) of each candidate region. Among them, Position_Embeddings(ROI) ∈ R N×D .

[0048] S8. Add the region feature vector P and the position embedding feature Position_Embeddings(ROI) to obtain the fusion vector G, G = P + Position_Embeddings(ROI), and perform a layer normalization operation on the fusion vector G to obtain the feature vector F, F = Layer_Norm(G), where G, F ∈ R N×D , and Layer_Norm() represents the layer normalization operation function;

[0049] S9. Input the feature vector F obtained in step S8 into the attention layer to obtain the attention feature A;

[0050] S10. Add the attention feature A and the original region feature vector P and perform a layer normalization operation to obtain the final x-th feature vector P', where P' ∈ R N×D ; [[ID=***]]

[0051] S11. Input the comprehensive feature vector P' into two independent fully connected layers: the first fully connected layer FC1 and the second fully connected layer FC2, which are used for classification and bounding box regression respectively, to obtain the final classification and regression results.

[0052] Example 2

[0053] This embodiment adopts the self-attention test paper layout analysis method MRSA based on the maskrcnn framework, that is, the self-attention layout analysis model based on maskrcnn. The experimental results of this method on the test paper picture datasets of multiple subjects (Politics, Geography, Physics, Chemistry, Biology) with different iou_threshold (iou threshold) are shown in Table 1, Table 2, and Table 3.

[0054] Table 1. Experimental result table with iou_threshold = 0.80

[0055]

[0056] Table 2. Experimental result table with iou_threshold = 0.85

[0057]

[0058] Table 3. Experimental results table with iou_threshold = 0.9

[0059]

[0060]

[0061] In this experiment, IoU is used as the evaluation index. This index can reflect the overlapping degree between the predicted bounding box and the ground truth bounding box, representing the accuracy of the prediction. The closer it is to 1, the more accurate the prediction is.

[0062] In this paper, IoU = 0.80, 0.85, and 0.90 are used as thresholds respectively. For each category of each subject, the IoU of the predicted bounding boxes with IoU greater than the threshold in all images is calculated and averaged, which is used as the standard to measure the performance of the model.

[0063] As can be seen from the table, compared with other models, the self-attention test paper layout analysis method based on the maskrcnn backbone proposed in the present invention reaches the optimal result the most times in the text area and illustration area of each subject, with a total of 17 times.

[0064] Generally speaking, the self-attention test paper layout analysis method based on the maskrcnn backbone proposed in the present invention performs well in the four subjects of biology, chemistry, physics, and politics, and although it has good performance in geography, it is not the best. This phenomenon is closely related to the layout and annotation methods of test papers in different subjects: in the test papers of other subjects, a question (which may have several sub-questions) often corresponds to one or more images; however, in the geography subject, one illustration often corresponds to multiple questions, and these questions are all labeled as an entire text area, that is, the self-attention test paper layout analysis method is required to regard these questions as one entity. Therefore, it is difficult to achieve the best performance under this annotation mode.

[0065] By comparing the indexes of the self-attention test paper layout analysis method based on the maskrcnn backbone proposed in the present invention and MRSA w / oattn (that is, the test paper layout analysis method without self-attention, which is the original Mask-RCNN), it can be seen that if the self-attention module designed in this paper is removed and only Mask-RCNN is used for prediction, the performance will decrease significantly.

[0066] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications made without departing from the spirit and principle of the present invention shall be equivalent replacement methods and are all included in the protection scope of the present invention.

Claims

1. A self-attention test paper layout analysis method based on the maskrcnn backbone, characterized in that The test paper layout analysis method includes the following steps: S1. Input the RGB test paper image with size H×W×3 into ResNet-50 to obtain 5 feature maps with different sizes, denoted as C1, C2, C3, C4, and C5 respectively. The sizes of the 5 feature maps are as follows where H represents the height of the test paper image and W represents the width of the test paper image; S2. Use the FPN algorithm to fuse four feature maps C2, C3, C4, and C5 to obtain feature maps with the same number of channels but different sizes. Among them, the FPN algorithm process is as follows: First, perform nearest neighbor upsampling on C5, C4, C3, and C2 from top to bottom to obtain feature maps M5, M4, M3, and M2. Then, perform a 1×1 convolution on each of the feature maps C5, C4, C3, and C2. Next, perform corresponding addition with the upsampled feature maps M5, M4, M3, and M2 to obtain D5, D4, D3, and D2. Finally, perform a 3×3 convolution on the added feature maps D5, D4, D3, and D2 to finally obtain the fused feature maps P5, P4, P3, and P2; S3. Perform max pooling with a stride of 2 on the upsampled feature map M5 to obtain the fused feature map P6, which is used to obtain subsequent candidate protocol boxes; S4. Generate 9 prior regions with 3 different lengths and aspect ratios of 1:1, 1:2, and 2:1 respectively, centered on each pixel point in the fused feature maps P6, P5, P4, P3, and P2. Use the Region Proposal Network (RPN) to perform binary classification and bounding box regression on the above prior regions, and filter out prior regions whose confidence scores belonging to the target are less than γ, whose sizes are too small to completely contain the target, or whose sizes are too large to exceed the range of the feature map. Here, γ is a predefined confidence threshold for filtering prior regions with low confidence scores. Then, use non-maximum suppression to filter out overlapping prior regions to obtain the final candidate regions of interest (ROI). Each rectangular candidate region has four vertices, and ROI ∈ R N×4 , where R represents the real number field and N is the number of candidate boxes for each feature map; S5. Perform ROI Align candidate region alignment on the candidate region ROI to obtain several feature maps of the same size; S6. Use a multi-layer neural network to perform boundary recognition and mask prediction on the feature map obtained in step S5, where boundary recognition includes region classification and boundary regression; S7. Input the candidate region ROI obtained in step S4 into the position encoding layer Position_Embeddings to obtain the position embedding feature Position_Embeddings(ROI) of each candidate region, where Position_Embeddings(ROI) ∈ R N×D , and D is the dimension of the preset local feature vector; S8. Add the regional feature vector P and the position embedding feature Position_Embeddings(ROI) to obtain the fusion vector G, G = P + Position_Embeddings(ROI), and perform layer normalization on the fusion vector G to obtain the feature vector F, F = Layer_Norm(G), where G,F ∈ R N×D , and Layer_Norm() represents the layer normalization operation function; S9. Input the feature vector F obtained in step S8 into the attention layer to obtain the attention feature A; S10. Add the attention feature A and the original regional feature vector P and perform layer normalization operation to obtain the final comprehensive feature vector P′, where P′ ∈ R N×D ; S11. Input the comprehensive feature vector P′ into two independent fully connected layers: the first fully connected layer FC1 and the second fully connected layer FC2, which are used for classification and bounding box regression respectively, to obtain the final classification and regression results.

2. The self-attention test paper layout analysis method based on the maskrcnn backbone according to claim 1, wherein The ResNet-50 is a residual network sequentially spliced by 49 convolutional layers and 1 pooling layer, and is a picture feature extractor that extracts 5 different sizes of feature maps from the original picture.

3. A self-attention test paper layout analysis method based on the maskrcnn framework according to claim 1, characterized in that, In the process of using the FPN algorithm to fuse the four feature maps C2, C3, C4, and C5 in step S2, a 1×1 convolution is performed on each of the feature maps C5, C4, C3, and C2, and all the output channels are set to the same 256 channels.

4. The self-attention test paper layout analysis method based on the maskrcnn framework according to claim 1, characterized in that In step S4, the region proposal network RPN is used to perform binary classification of the prior region as the target or the background.

5. A self-attention test paper layout analysis method based on the maskrcnn framework according to claim 1, characterized in that, The specific process of non-maximum suppression in step S4 is as follows: Sort the confidence levels of all prior regions belonging to the target, select the prior region with the highest confidence level and its corresponding boundary, and filter out other prior regions with an IoU(A,B) value greater than δ with respect to this prior region; A and B represent two different graphs, A∩B represents the overlapping area of regions A and B, A∪B represents the total area of regions A and B, IoU(A,B) is the ratio of the overlapping area of regions A and B to the total area of regions A and B, and δ is a predefined threshold used to limit the filtering process.

6. The self-attention test paper layout analysis method based on the maskrcnn backbone according to claim 1, characterized in that, In the step S6, the process of using a multi-layer neural network to perform boundary recognition and mask prediction on the feature map obtained in step S5 is completed by two independent neural networks MLP1 and MLP2 respectively. Among them, MLP1 compresses the feature map obtained in step S5 into a regional feature vector P, where P ∈ R N×D , where D is the dimension of the preset local feature vector, and MLP2 is used to obtain the mask of the feature map, that is, to classify each pixel point in the feature map as belonging to the target or the background.

7. A self-attention test paper layout analysis method based on the maskrcnn backbone according to claim 1, characterized in that The operation process of the layer normalization operation function Layer_Norm() in step S8 is as follows: First, calculate the means E1, E2, …, E of each row in the N rows of vector G N and the variances σ1, σ2, …, σ N . If vector G is a vector with a size of N×D, then the element F at the i-th row and j-th column in F ij can be calculated by the following formula: where G ij represents the element at the i-th row and j-th column in vector G, 1 ≤ i ≤ N, 1 ≤ j ≤ D, ∈ is a predefined parameter used to prevent the variance of the data distribution from being 0, γ is a predefined parameter used to adjust the variance of the data distribution, and β is a predefined parameter used to adjust the mean of the data distribution.

8. The self-attention test paper layout analysis method based on the maskrcnn backbone according to claim 1, characterized in that, The process of step S9 is as follows: Randomly initialize three weight matrices Q, K, and V of size D×D. Multiply F by Q, K, and V respectively to obtain matrices q, k, and v of size N×D. Calculate the attention weight matrix A based on q, j, and v and perform normalization processing using the softmax function. The calculation formula is as follows:

9. A self-attention test paper layout analysis method based on the maskrcnn backbone according to any one of claims 1 to 8, characterized in that The RGB test paper pictures are selected from Chinese, mathematics, politics, history, geography, physics, chemistry, or biology.

Images