A Vietnamese Scene Text Detection Method Based on Edge Attention Guidance

Abstract

The present invention relates to the field of text detection, and particularly to a Vietnamese scene text detection method guided by edge attention, including: using ResNet to extract the feature information of the target, and using the receptive field residual block RFRB in ResNet to generate a rich receptive field; using the multi-path fusion feature pyramid network MF-FPN to fuse the feature information to obtain the feature information of different levels of the target; inputting the feature information into RPN to obtain a certain number of candidate boxes; inputting the candidate boxes and the feature information into the classification branch and the mask branch after RoI Align to predict the category information, bounding box information and mask information of the target, and using the Re-Score mechanism in the classification branch to suppress non-text targets and using the edge attention mechanism EAM in the classification branch and the mask branch to highlight the edges of the target. This method can effectively detect Vietnamese scene text targets of different scales and eliminate some non-text targets.

Description

Technical Field

[0001] The present invention relates to the field of text detection, and in particular to a Vietnamese scene text detection method based on edge attention guidance. Background Art

[0002] Natural scene text detection is a technology for automatically detecting text targets in natural scene images, which is widely applied to autonomous driving, sign recognition, scene understanding, etc. Moreover, natural scene text detection has also attracted the attention and research of numerous researchers. However, most of the existing methods are based on the research of non-tonal languages such as English, and there is little research on scene text detection of tonal languages such as Vietnamese.

[0003] Vietnamese is a tonal language that uses accent marks or diacritical marks to represent vowels and tones. Among them, there are three symbols used to add vowels, and five symbols represent the tones of Vietnamese. These five tone symbols determine the meaning of each word. The unique composition of Vietnamese characters makes the detection of Vietnamese text in natural scenes have the following difficulties compared with the existing detection technologies mainly for English:

[0004] 1. More abundant and robust feature information is required to detect Vietnamese scene text targets as much as possible and extract the features of diacritical marks.

[0005] 2. The existence of diacritical marks and the interference of background information make some text-like targets more likely to be misdetected as text targets, that is, there will be some false positive targets.

[0006] 3. The diacritical marks of Vietnamese text in natural scenes are smaller in shape compared with Latin letters and are easily ignored during detection, thus unable to completely express Vietnamese text targets (incomplete detection of diacritical marks and incomplete detection of text targets). At the same time, there will be two diacritical marks on the upper part of some characters.

[0007] 4. In natural scenes, the scale of Vietnamese scene text targets varies greatly. Summary of the Invention

[0008] The purpose of the present invention is to provide a Vietnamese scene text detection method based on edge attention guidance, aiming to more accurately detect Vietnamese scene text targets of different scales, especially diacritical mark information, and effectively eliminate non-text targets.

[0009] To achieve the above purpose, the present invention provides a Vietnamese scene text detection method based on edge attention guidance, including: using ResNet to extract the feature information of the target, and using the receptive field residual block RFRB in ResNet to generate a rich receptive field to adapt to Vietnamese scene text targets of different scales.

[0010] The multi-path fusion feature pyramid network MF-FPN is used to fuse feature information to obtain feature information of different levels of the target, such as: target spatial position information, diacritical mark detail information, etc.;

[0011] The feature information is input into the RPN to obtain a certain number of candidate boxes;

[0012] The candidate boxes and the feature information are input into the classification branch and the mask branch after RoI Align to predict the category information, bounding box information and mask information of the target, and the Re-Score mechanism is used to suppress non-text targets, and at the same time, the edge attention mechanism EAM is used to highlight the edges of the target.

[0013] In the Vietnamese scene text detection method of the present invention, the specific way of using the receptive field residual block RFRB to generate a rich receptive field is: first, use a 1×1 convolution to adjust the number of channels of the feature; then, concat and fuse the output features of three 3×3 dilated convolutions with dilation rates of 1, 2, and 3 respectively; then use a 1×1 convolution to adjust the number of channels for information blending, thereby generating a rich receptive field.

[0014] In the Vietnamese scene text detection method of the present invention, the multi-path fusion feature pyramid network MF-FPN is a network for feature fusion to generate feature maps of different levels containing different information. The specific way for the multi-path fusion feature pyramid network MF-FPN to extract feature information of different levels of the target is: the output obtained by inputting the feature of the current level obtained by ResNet into a 1×1 convolution with 256 channels, the output obtained by performing 2×2 average pooling on the feature of the previous level obtained by ResNet, and the output of the upsampling from top to bottom of the feature output by ResNet. After the three are fused, they are input into a 3×3 convolution with 256 channels, thereby obtaining feature information of different levels.

[0015] In the Vietnamese scene text detection method of the present invention, the candidate boxes and the feature information are input into the classification branch and the mask branch after RoI Align to predict the category information, bounding box information and mask information of the target. The specific way is:

[0016] The candidate boxes and the feature information are input into RoI Align to map the feature map of the target to a fixed size;

[0017] The feature map of the fixed size is input into the classification branch, and accurate category information is obtained through the Re-Score mechanism. At the same time, the edge branch is used to predict the edge contour probability map of the target, and the map is multiplied by the intermediate feature information of the category and the bounding box prediction branch to form the edge attention EAM, which guides the model to predict accurate bounding box information;

[0018] Input the feature map with a fixed size into the mask branch to obtain the target mask map. At the same time, use the edge branch to predict the edge contour probability map of the target, and multiply this map with the intermediate feature information of the mask prediction branch to form the edge attention EAM, which guides the model to predict accurate mask information.

[0019] Further, the specific steps for obtaining accurate class information by the Re-Score mechanism are as follows:

[0020] Input the feature information of the candidate box into the convolutional network to obtain the visual class confidence of the target;

[0021] Input the feature information of the candidate box into the sequence scoring branch to obtain the sequence confidence of the target;

[0022] Multiply each of them by 0.5 and then add them together to obtain the final class confidence, and select the class with the highest confidence as the class information of the target.

[0023] Further, the sequence scoring branch is composed of a 1×1 convolutional layer with 1 channel, a Bi-LSTM layer, and two fully connected layers, and sequence modeling is performed using Bi-LSTM in the width dimension of the feature information.

[0024] In the Vietnamese scene text detection method of the present invention, the specific way for the Re-Score mechanism to suppress non-text targets is as follows: use 1×1 convolution to adjust the number of channels of the feature information of the candidate box with a fixed size; use Bi-LSTM to extract sequence features; use two fully connected layers to predict the sequence score of the target; use a convolutional network to predict the visual classification score of the target; multiply each of the two scores by 0.5 and then add them together to obtain the final class confidence; use 0.7 as the threshold to eliminate the targets with confidence lower than the threshold.

[0025] In the Vietnamese scene text detection method of the present invention, the specific way for the edge attention mechanism EAM to highlight the edge of the target is as follows: use the edge branch composed of several convolutional layers to predict the edge contour probability map of the target, where the pixel values belonging to the edge are greater than or equal to 0.5, and the non-edge pixels are less than 0.5; regard the edge probability map as the attention weight and multiply it with the feature information to form the edge attention mechanism EAM.

[0026] A method for detecting Vietnamese scene text based on edge attention guidance in the present invention uses receptive field residual blocks (RFRBs) to generate different receptive fields, effectively adapting to Vietnamese scene text targets of different scales; designs a multi-path fusion feature pyramid network (MF-FPN) to fuse rich low-level features (relatively low-level), highlighting the positions of Vietnamese scene text targets and the detailed information of diacritics; the Re-Score mechanism effectively eliminates non-text targets; the EAM enhances the model's sensitivity to the edges of Vietnamese scene text targets, thereby completely detecting Vietnamese scene text targets, including diacritics, and eliminating some non-text targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0028] Figure 1 It is a flowchart of the method for detecting Vietnamese scene text in the embodiment.

[0029] Figure 2 It is a schematic diagram of the algorithm structure of the method for detecting Vietnamese scene text in the embodiment.

[0030] Figure 3 It is a structural diagram of the receptive field residual block (RFRB) provided in the embodiment.

[0031] Figure 4 It is a structural diagram of the multi-path fusion feature pyramid network (MF-FPN) provided in the embodiment.

[0032] Figure 5 It is a structural diagram of the improved stage1 provided in the embodiment.

[0033] Figure 6 It is a structural diagram of the Re-Score mechanism provided in the embodiment.

[0034] Figure 7 It is a structural diagram of the classification branch provided in the embodiment.

[0035] Figure 8 It is a structural diagram of the mask branch provided in the embodiment.

[0036] Figure 9 It is an example diagram of the experimental data for detecting Vietnamese scene text in the embodiment;

[0037] Among them, (a) is a sample of a Vietnamese scene text image with bounding box annotations; (b) is a binary text segmentation map corresponding to the image sample (a); (c) is a text edge contour map corresponding to the image sample (a).

[0038] Figure 10 is an example of a Vietnamese scene text picture required in the experiment of the embodiment.

[0039] Figure 11 is a comparison example of feature maps generated by the multi-path fusion feature pyramid network MF-FPN provided by the embodiment and other methods.

[0040] Figure 12 is an F-measure comparison graph at different IoU thresholds after combining the edge attention mechanism EAM provided by the embodiment.

[0041] Figure 13 is a comparison graph of detection result examples between the Vietnamese scene text detection method guided by edge attention provided by the embodiment and other people's methods;

[0042] Among them, (a) is the original image of Vietnamese scene text; (b) is the effect diagram of detecting the original image using the baseline algorithm; (c) is the effect diagram of detecting the original image using the improved Mask R-CNN; (d) is the effect diagram of detecting the original image using the detection method of the present invention. Detailed implementation manners

[0043] The embodiments of the present invention will be described in detail below. The examples of the embodiments are shown in the accompanying drawings. The embodiments described by referring to the accompanying drawings are exemplary and are intended to explain the present invention, and should not be construed as limiting the present invention.

[0044] Embodiment

[0045] Please refer to Figures 1 to 13 for a Vietnamese scene text detection method guided by edge attention, which includes the following steps, specifically referring to Figures 1 - 2 :

[0046] S101 Input a Vietnamese scene text picture, and use ResNet to extract the feature information C1 to C5 of the target;

[0047] Among them, the ResNet is a general feature extraction network in the art. The network includes 5 stages. Stages 2 to 5 are composed of a certain number of residual blocks. ResNet outputs a set of features at each stage, and a total of 5 features C1 to C5 are obtained.

[0048] To enable the present invention to effectively adapt to Vietnamese scene text targets of different scales, all residual blocks in ResNet are replaced with the receptive field residual block RFRB provided in the embodiment.

[0049] The specific structure of RFRB is as Figure 3 shown. First, in one branch, the feature is input into a 1×1 convolution, and then simultaneously input into dilated convolutions with dilation rates of 1, 2, and 3 respectively. Then, the outputs of the three are fused in the channel dimension by concat (concatenation), and then the 1×1 convolution is used to adjust the number of channels and perform information blending to obtain the output information; the other branch is an identity mapping shortcut, that is, the input information is directly output or output after adjusting the number of channels by a 1×1 convolution; finally, the outputs of the two branches are added by add, and then the final output is obtained through the ReLU activation function.

[0050] S102 fuses the 5 feature information output by ResNet using the multi-path fusion feature pyramid network MF-FPN to obtain feature information P1 to P6 at different levels;

[0051] MF-FPN is improved from FPN, and FPN is a general feature fusion network in the art.

[0052] The specific structure of MF-FPN is as Figure 4 shown, Figure 4 The dotted arrow in k is the improvement made by the present invention, that is, when each level of feature map P k is formed, it will not only fuse C k-1 and the features after upsampling, but also fuse the features of the lower level C

[0053] Considering that C1 needs to be fused when obtaining P2 features, and the C1 information extracted from stage1 in ResNet is limited, the present invention improves stage1 in ResNet so that the extracted C1 feature information is richer.

[0054] Figure 5The improved stage1 structure diagram is shown as follows. It contains two branches. One branch first performs Zero padding on the input information, then inputs a depthwise separable convolution with a size of 7×7, a stride of 2, and 32 channels, and then uses Max pooling to obtain the output information. The other branch first inputs a convolution with a size of 1×1, a stride of 1, and 32 channels, and then inputs two layers of depthwise separable convolutions with a size of 3×3, a stride of 2, and 32 channels to obtain the output information. Then, the information output by the two branches is fused by concat. Then, a convolution with a size of 1×1, a stride of 1, and 64 channels is input to obtain the final feature information C1.

[0055] S103 inputs the feature information P2 - P6 into the RPN to obtain a certain number of Region Proposals.

[0056] RPN is a region proposal network proposed by the well-known algorithm Faster R-CNN in the field. This network will generate a certain number of Region Proposals, representing the positions in the image that the model believes may be text targets.

[0057] S104 inputs the Region Proposals and the feature information P2 - P5 after RoI Align into the classification branch and the mask branch to predict the class information S, the bounding box information bbox, and the mask information mask of each Region Proposal (target).

[0058] RoI Align is proposed in the algorithm Mask R-CNN in the field, which is used to crop the feature information corresponding to the Region Proposals according to the position information and size of the Region Proposals and map it to a fixed size.

[0059] The structure diagram of the classification branch is as Figure 7 shown. The Region Proposals are mapped to a feature map of size 7×7 after RoI Align and are respectively input into the edge branch and the class and bounding box prediction branch. Among them, the edge branch consists of a 3×3 convolutional layer with 256 channels and a 1×1 convolutional layer with 1 channel, and then is activated by the Sigmoid function to obtain the edge contour probability map edge_map cls, and after binarization, an edge contour binary map is obtained. In the class and bounding box prediction branches, 4 convolutional layers (2 3×3 convolutions with 256 channels, 1 7×7 convolution with 1024 channels, and 1 1×1 convolution with 1024 channels) and 1 fully connected layer are used to predict the bounding box bbox and the visual classification confidence S v , and the predicted edge contour probability map edge_map cls is multiplied by the feature f output by the first convolutional layer cls at the corresponding positions to form an edge attention EAM, and the specific calculation is shown in Equation (1). At the same time, the Re-Score mechanism is incorporated into the class and bounding box prediction branches to obtain the final predicted class confidence S;

[0060]

[0061] The specific steps of incorporating the Re-Score mechanism into the class and bounding box prediction branches to obtain the final predicted class confidence S are as follows:

[0062] The schematic diagram of the Re-Score mechanism is as Figure 6 shown;

[0063] The visual classification score of the i-th candidate box is obtained using the aforementioned class and bounding box prediction branches The semantic sequence score of the candidate box is obtained using the sequence scoring branch The sequence scoring branch first adjusts the number of channels using convolution, then uses Bi-LSTM for sequence modeling with the w (width) dimension of the feature as the time step, and then obtains the sequence score of the target after passing through two fully connected layers and the Softmax function The class confidence S of the i-th candidate box predicted finally i is calculated by Equation (2);

[0064]

[0065] The structural diagram of the mask branch is as Figure 8 shown. The candidate box Region Proposals are mapped to a 14×14 feature map after RoI Align and are respectively input into the edge branch and the mask prediction branch. The edge branch consists of 4 convolutional layers (3 3×3 convolutions with 256 channels and 1 1×1 convolution with 1 channel), and then the edge contour probability map edge_map is obtained after activation by the Sigmoid function msk, and after binarization, the edge contour binary map is obtained. The mask prediction branch has the same network structure as the mask branch in Mask R-CNN, consisting of 4 layers of 3×3 convolution layers with 256 channels, 1 layer of 2×2 transposed convolution layer with 256 channels, and 1 layer of 1×1 convolution layer. The activation function is Sigmoid, and the target mask segmentation map is finally obtained. At the same time, the predicted edge contour probability map edge_map is used. msk The feature f output by the first convolution layer msk Multiply the corresponding position elements to form an edge attention EAM, which is calculated as shown in formula (3);

[0066]

[0067] The effects of the method of the present invention are further described below with reference to the embodiments.

[0068] The Vietnamese scene text detection method based on edge attention guidance proposed in this invention has the following test environment and experimental results:

[0069] 1) Test environment:

[0070] System environment: Ubuntu 16.04;

[0071] Hardware environment: 256GB memory, GPU: Tesla V100×4, CPU: 1.70GHz Intel(R) Xeon(R) E5-2609, hard disk: 8TB;

[0072] 2) Experimental data:

[0073] To verify the effectiveness of the present invention, the Multilingual Text Detection in Natural Scenes dataset (MLT 2017) was used, of which only 7,200 images containing only Latin text types and 200 images taken from real scenes in Vietnam were used.

[0074] Data annotation such as Figure 9 As shown, the coordinates of the bounding box position of the Vietnamese scene text target are marked ( Figure 9 (a)), binary mask map of text target in Vietnamese scene ( Figure 9 (b)), edge contour map of text target in Vietnamese scene ( Figure 9 (c)).

[0075] 3) Implementation details:

[0076] The method is trained and tested using the said dataset. When training the model, the experiment first pre-trains using the MLT2017 dataset (80 epochs), and then fine-tunes the entire model using the Vietnamese character dataset (20 epochs). Among them, the batch-size is set to 8, the optimizer is selected as SGD, the initial learning rate is 0.001, and the momentum is set to 0.9.

[0077] The experiment conducted is a five-fold cross-validation experiment. For the Vietnamese scene character dataset, the training set consists of 160 images (during the training process, each image undergoes five data augmentations), and the test set consists of 40 images. Precision, Recall, and F-measure are used as evaluation metrics. The calculation method of IoU is to calculate the intersection over union ratio of the mask segmentation matrix obtained from the mask branch and the mask matrix of the real target (binary mask segmentation map), rather than calculating the intersection over union ratio in the form of a traditional square box. The IoU threshold is set to 0.7.

[0078] In this experiment, ablation studies are conducted on the receptive field residual block RFRB, multi-path fusion feature pyramid network MF-FPN, Re-Score mechanism, and edge attention mechanism EAM, and comparisons are made between the present invention and other existing methods. The basic algorithm (baseline) is Mask R-CNN, and the present invention is an improvement based on the baseline.

[0079] 4) Experimental results:

[0080] a) Experimental results of the receptive field residual block RFRB

[0081] As shown in Table 1, after adding the RFRB module, all evaluation metrics are improved by nearly 2%. The improvement in its performance is precisely because RFRB can fuse different receptive fields and more flexibly adapt to Vietnamese scene character targets of different scales.

[0082] To highlight the adaptability of RFRB to Vietnamese scene character targets of different scales, the experiment evaluates the baseline and the combination with RFRB according to the definitions of small, medium, and large targets in COCO, and compares the number of real Vietnamese scene character targets of different scales detected by the two (true positive targets). Among them, the area of small targets (S) is less than 32 2 , the area of medium targets (M) is greater than 32 2 and less than 96 2 , the area of large targets (L) is greater than 96 2。The experimental results are shown in Table 2. It can be seen from the table that the number of true positive targets of different scales detected by adding the RFRB module is more than that detected by the baseline algorithm. Among them, the number of small targets detected is increased by 3.6% compared with the baseline algorithm, the medium targets are increased by 1.8%, and the large targets are increased by 5.5%, further indicating the adaptability of RFRB to text targets in Vietnamese scenes of different scales.

[0083] b) Experimental results of the multi-path fusion feature pyramid network MF-FPN

[0084] As shown in Table 1, compared with the baseline, after replacing the FPN in the baseline with MF-FPN(+modified stage1), the Recall index is increased by 2.8%, the Precision is increased by 3.8%, and the F-measure is increased by 3.2%. If combined with RFRB, each index is slightly improved.

[0085] In this experiment, the Figure 10 shown image was input into the algorithm, and the feature maps output by FPN and MF-FPN were visualized respectively. As Figure 11 shown, the feature map obtained by MF-FPN has more obvious target position information and contains more detailed information than FPN, which is more helpful for detecting text targets in Vietnamese scenes. For example, it can be seen that the feature information of the diacritical marks contained in the P2 feature map extracted by MF-FPN ( Figure 11 indicated by the arrow).

[0086] To further verify the necessity of improving stage1 and the effectiveness of MF-FPN, the following ablation experiments were conducted. In the experiment, stage1 of ResNet in the baseline was replaced with the improved stage1 network, and FPN was replaced with MF-FPN respectively. The experimental results are shown in Table 3. It can be seen from the table that only replacing stage1 of ResNet in the baseline with the improved stage1, the F-measure is slightly increased by 0.4%. After only replacing FPN with MF-FPN, the Precision, Recall, and F-measure are all increased by about 2%, further indicating the effectiveness of MF-FPN. As shown in the last row of Table 3, after combining the improved stage1 and MF-FPN (MF-FPN+modified stage1), the performance is further improved.

[0087] c) Experimental results of the Re-Score mechanism

[0088] The Re-Score mechanism adds sequence scoring to the candidate bounding boxes, making the class scoring of candidate targets more accurate, thus effectively eliminating non-text targets. As can be seen from Table 1, compared with the baseline algorithm after combining Re-Score, the Precision value has increased by nearly 8%. If RFRB and MF-FPN are combined at the same time, the Precision increases by 5.7%. This fully demonstrates the ability of the Re-Score mechanism to suppress false positive targets.

[0089] d) Experimental results of the Edge Attention Mechanism EAM

[0090] As shown in Table 1, under the condition that the IoU threshold is 0.7, in each fold of the cross-validation experiment, combining EAM has a greater improvement in various evaluation indicators compared with the baseline. After combining the previously mentioned modules, the F-measure has increased by 4.3%.

[0091] To further explore the detection ability of EAM for diacritics, the experiment uses higher IoU thresholds of 0.7, 0.8, and 0.9 to test EAM. As Figure 12 shown, at higher IoU thresholds, compared with the baseline after combining EAM, it still has good performance, which is sufficient to prove that it can effectively detect diacritics and further accurately segment Vietnamese scene text targets.

[0092] e) Comparison with other methods

[0093] As can be seen from Table 4, the present invention shows good performance. Compared with other methods, the Precision has increased by 10.6%, the Recall has increased by 0.6%, and the F-measure has increased by 5.8%. Figure 13 Shown are the detection results of several methods. From these results, it can be seen that the detection results of the present invention have finer and more accurate boundaries and can effectively eliminate some false positive targets.

[0094] Table 1 Ablation study (the experimental results of MF-FPN in the table are the results of combining the improved stage1)

[0095]

[0096]

[0097] Table 2 Number of true Vietnamese scene text targets (true positive targets) detected at different scales

[0098]

[0099]

[0100] Table 3 Experimental results of ablation study on improving stage1 and MF-FPN

[0101]

[0102] Table 4 Comparison results of different methods

[0103]

[0104]

[0105] Reference for the baseline algorithm in Table 4: He K, Gkioxari G, Dollár P, et al. Mask R-CNN[C] / / International Conference on Computer Vision (ICCV). Piscataway, NJ: IEEE, 2017: 2961-2969;

[0106] Reference for other people's algorithms: Feng Yate, Wen Yimin. Vietnamese scene text detection based on improved Mask R-CNN[J]. Journal of Computer Applications, 2021, 41(12): 7.

[0107] The above-disclosed is only a preferred embodiment of the present invention. Of course, the scope of rights of the present invention cannot be limited thereby. Those of ordinary skill in the art can understand all or part of the processes of implementing the above embodiments, and the equivalent changes made according to the claims of the present invention still fall within the scope covered by the invention.

Claims

1. A Vietnamese scene text detection method based on edge attention guidance, characterized in that: It includes: Using ResNet to extract the feature information of the target, and using the receptive field residual block RFRB in ResNet to generate a rich receptive field; Using a multi-path fusion feature pyramid network MF-FPN to fuse the feature information to obtain feature information of different levels of the target; Inputting the feature information into RPN to obtain a certain number of candidate boxes; Inputting the candidate boxes and feature information into RoI Align, and after RoI Align, inputting them into the classification branch and the mask branch to predict the category information, bounding box information and mask information of the target, using the Re-Score mechanism to suppress non-text targets, and at the same time using the edge attention mechanism EAM to highlight the edge of the target; The candidate boxes and feature information are input into the classification branch and the mask branch after RoI Align to predict the category information, bounding box information and mask information of the target. The specific method is: Inputting the candidate boxes and feature information into RoI Align to map the feature map of the target to a fixed size; Inputting the feature map of the fixed size into the classification branch, obtaining accurate category information through the Re-Score mechanism, and at the same time using the edge branch to predict the edge contour probability map of the target, and multiplying the predicted edge contour probability map with the intermediate feature information of the category and bounding box prediction branch to form edge attention EAM to guide the model to predict accurate bounding box information; Inputting the feature map of the fixed size into the mask branch to obtain the mask map of the target, and at the same time using the edge branch to predict the edge contour probability map of the target, and multiplying the predicted edge contour probability map with the intermediate feature information of the mask prediction branch to form edge attention EAM to guide the model to predict accurate mask information.

2. The Vietnamese scene text detection method according to claim 1, characterized in that: The specific method for using the receptive field residual block RFRB to generate a rich receptive field is: first using a 1×1 convolution to adjust the number of channels of the feature; then concatenating and fusing the output features of 3 3×3 dilated convolutions with dilation rates of 1, 2, and 3 respectively; then using a 1×1 convolution to adjust the number of channels and perform information blending, thereby generating a rich receptive field.

3. The Vietnamese scene text detection method according to claim 1, characterized in that: The specific method for the multi-path fusion feature pyramid network MF-FPN to extract feature information of different levels of the target is: fusing the output obtained by inputting the feature of the current level obtained by ResNet into a 1×1 convolution, the output obtained by performing 2×2 average pooling on the feature of the previous level obtained by ResNet, and the output of the upsampling from top to bottom of the feature output by ResNet, and then inputting the fused result into a 3×3 convolution to obtain feature information of different levels.

4. The Vietnamese scene text detection method according to claim 1, characterized in that: The specific steps for obtaining accurate category information by the Re-Score mechanism are: Inputting the feature information of the candidate box into the convolutional network to obtain the visual category confidence of the target; Input the feature information of the candidate bounding box into the sequence scoring branch to obtain the sequence confidence of the target; Multiply the two by 0.5 respectively and then add them to obtain the final class confidence, and select the class with the highest confidence as the class information of the target.

5. The Vietnamese scene text detection method according to claim 4, characterized in that The sequence scoring branch is composed of a 1×1 convolutional layer, a Bi-LSTM layer, and two fully connected layers, and sequence modeling is performed using Bi-LSTM in the width dimension of the feature information.

6. The Vietnamese scene text detection method according to claim 1, characterized in that The specific way of the Re-Score mechanism to suppress non-text targets is as follows: use 1×1 convolution to adjust the number of channels of the candidate bounding box feature information of a fixed size; use Bi-LSTM to extract sequence features; use two fully connected layers to predict the sequence score of the target; use a convolutional network to predict the visual classification score of the target; multiply the two scores by 0.5 respectively and then add them to obtain the final class confidence; use 0.7 as the threshold to eliminate the targets with confidence lower than the threshold.

7. The Vietnamese scene text detection method according to claim 1, characterized in that The specific way of using the edge attention mechanism EAM to highlight the edge of the target is as follows: input the features of a fixed size into a fully convolutional network composed of several convolutional layers, and obtain the edge information of the target after activation by the Sigmoid function. In this edge information, the pixel values belonging to the edge are greater than or equal to 0.5, and the non-edge pixels are less than 0.5.

Images