Scene text detection method, system and device based on deep learning

Abstract

The application discloses a scene text detection method, system and device based on deep learning, which comprises the following steps: S1, obtaining a text detection training data set; S2, performing data enhancement on the training data set pictures; S3, performing random center cropping on the data-enhanced training data set images to obtain random center cropped training data set images; S4, establishing a scene text detection model; S5, inputting the random center cropped training data set images into the scene text detection model for training to obtain a trained scene text detection model; S6, inputting a test set into the trained scene text detection model for scene text detection; and S7, evaluating the text detection result. The application can realize scene text detection with high efficiency.

Description

Deep learning-based scene text detection methods, systems, and devices Technical Field

[0001] This invention relates to the field of scene text detection based on deep learning, and in particular to a scene text detection method, system and apparatus based on deep learning. Background Technology

[0002] Traditional optical character recognition (OCR) has been used in industrial applications since the beginning of the century. Its application is common in areas such as the inspection of express delivery documents, paper characters, and workpiece markings. However, traditional OCR is only suitable for relatively simple inspection tasks and cannot adapt to extreme lighting and contrast conditions, or the inspection of multi-directional and multi-scale text blocks. For such scenarios, significant additional manpower and resources are required to calibrate text regions and manually input information, resulting in repetitive tasks and low efficiency. Summary of the Invention

[0003] The purpose of this invention is to provide a scene text detection method based on deep learning, which aims to solve the problem of scene text detection.

[0004] This invention provides a deep learning-based method for scene text detection, comprising:

[0005] S1. Obtain the text detection training dataset;

[0006] S2. Perform data augmentation on the images in the training dataset;

[0007] S3. Perform random center cropping on the data augmented training dataset images to obtain randomly center cropped training dataset images.

[0008] S4. Establish a scene text detection model;

[0009] S5. Input the images from the random center cropped training dataset into the scene text detection model for training to obtain the trained scene text detection model.

[0010] S6. Input the test set into the trained scene text detection model for scene text detection;

[0011] S7. Evaluate the text detection results.

[0012] This invention also provides a scene text detection system based on deep learning, comprising:

[0013] Acquisition module: Used to acquire the text detection training dataset;

[0014] Data augmentation module: used to augment images in the training dataset;

[0015] The cropping module is used to perform random center cropping on the data augmented training dataset images to obtain randomly center cropped training dataset images;

[0016] Establishment module: Used to build a scene text detection model;

[0017] Training module: Used to input images from the training dataset with random center cropping into the scene text detection model for training, and obtain the trained scene text detection model;

[0018] Detection module: Used to input the test set into the trained scene text detection model for scene text detection;

[0019] Evaluation module: Used to evaluate the text detection results.

[0020] This invention also provides a scene text detection device based on deep learning, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the above method.

[0021] This invention also provides a computer-readable storage medium storing an information transmission implementation program, which, when executed by a processor, implements the steps of the above-described method.

[0022] Using the embodiments of the present invention, scene text detection can be achieved with high efficiency.

[0023] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0024] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0025] Figure 1 is a flowchart of the deep learning-based scene text detection method according to an embodiment of the present invention;

[0026] Figure 2 is a schematic diagram of the convolutional neural network of the scene text detection method based on deep learning according to an embodiment of the present invention;

[0027] Figure 3 is a schematic diagram of the AGM and FGM networks of the deep learning-based scene text detection method according to an embodiment of the present invention;

[0028] Figure 4 is a schematic diagram of the Head network of the deep learning-based scene text detection method according to an embodiment of the present invention;

[0029] Figure 5 is a schematic diagram of a scene text detection system based on deep learning according to an embodiment of the present invention;

[0030] Figure 6 is a schematic diagram of a scene text detection device based on deep learning according to an embodiment of the present invention. Detailed Implementation

[0031] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] Method Implementation Examples

[0033] According to an embodiment of the present invention, a scene text detection method based on deep learning is provided. Figure 1 is a flowchart of the scene text detection method based on deep learning according to an embodiment of the present invention. As shown in Figure 1, it specifically includes:

[0034] S1. Obtain the text detection training dataset;

[0035] S2. Perform data augmentation on the images in the training dataset;

[0036] S2 specifically includes: randomly rotating and randomly mirroring the training set images.

[0037] S3. Perform random center cropping on the data augmented training dataset images to obtain randomly center cropped training dataset images.

[0038] S3 specifically includes: cropping an image to a size of 640x640 using a random pixel from the training set image as the center point.

[0039] S4. Establish a scene text detection model;

[0040] S4 specifically includes: establishing a scene text detection model, which consists of three parts: backbone, neck, and head;

[0041] Backbone uses ResNet18 as the feature extraction network to extract features from the test set and then inputs the features into Neck. The output features include C1, C2, C3, C4, and C5.

[0042] Neck, using NGNET as a feature fusion network, is used to fuse features of the test set. NGNET consists of two structures: an Axial Guide Module (AGM) and two Regular Guide Modules (FGMs).

[0043] Max pooling is performed on C4 along the row and column directions respectively. Then, the contour information is output through a pair of asymmetric convolutions. The contour information is multiplied point by point with the C5 feature after dimensionality reduction and upsampling. The result is then weighted into C4 in the form of an attention mechanism to obtain the first feature map.

[0044] C2 is input into the convolution kernel for feature extraction to obtain C2 features. C2 features are multiplied pointwise and then superimposed with C2. After superposition, the second feature map is obtained by feature extraction and dimensionality reduction through two layers of convolution kernels.

[0045] C3 is input into the convolution kernel for feature extraction to obtain C3 features. C3 features are multiplied pointwise and then superimposed with C3. After superposition, the third feature map is obtained by feature extraction and dimensionality reduction through two layers of convolution kernels.

[0046] The first feature map, the second feature map, and the third feature map are resized to 80x80 and then stitched together to obtain the fused feature. The fused feature is then input into the Head.

[0047] The head inputs the fused features into the prediction branch ingress points of the probability map and the contour map, respectively. After inputting into the probability map prediction branch ingress point, the first dimensionality-reduced feature is obtained through dimensionality reduction. This first dimensionality-reduced feature is then upsampled using a first deconvolutional network to obtain the probability map branch feature. This second dimensionality-reduced feature is then input into the contour map prediction branch ingress point and downsampled to obtain the contour map branch feature. A non-sampled discrete wavelet transform is then performed on the contour map branch feature to obtain the high-frequency and low-frequency feature components. The low-frequency feature component is then combined with the probability map branch feature... The dot product is obtained by multiplying the dot product features. The dot product features are then superimposed with the high-frequency feature components to obtain the fused features of the two branches. The fused features of the two branches and the probabilistic map branch features are concatenated and then input into the third deconvolutional network for dimensionality reduction and upsampling. After dimensionality reduction and upsampling, the features are input into the first sigmoid network for normalization. The normalized fused features are then post-processed to obtain the probabilistic map. The contour map branch features are input into the fourth deconvolutional network for dimensionality reduction and upsampling. After dimensionality reduction and upsampling, the features are input into the second sigmoid network for normalization. The normalized contour map features are then post-processed to obtain the contour map, thus completing the detection.

[0048] S5. Input the images from the random center cropped training dataset into the scene text detection model for training to obtain the trained scene text detection model.

[0049] S6. Input the test set into the trained scene text detection model for scene text detection;

[0050] S7. Evaluate the text detection results.

[0051] The specific implementation method is as follows:

[0052] First, data augmentation is performed on the ICDAR15 scene text dataset by randomly rotating and flipping it to increase the randomness of the dataset, thereby improving the generalization ability and robustness of the training model. Then, the preprocessed data is fed into the model for training. After training, the model is used to detect text instances in the scene.

[0053] To achieve the above objectives, the present invention includes the following steps:

[0054] Step S1: Obtain the ICDAR15 text detection dataset;

[0055] Step S2: Perform data augmentation on the dataset images to improve the network's generalization ability;

[0056] Step S3: Perform random center cropping on the image;

[0057] Step S4: Using the PyTorch deep learning framework, a feature fusion network based on neighbor-guided layers and a prediction module based on probability-contour image fusion are established. The convolutional neural network structure proposed in this invention is inspired by DBNET, adopting a segmentation-based scene text detection architecture of feature extraction network (backbone) - feature fusion network (neck) - prediction head (head). For the feature pyramid (FPN) used for feature fusion in the network, this invention replaces it with NGNET (Neighbor Guided Network), allowing features at adjacent levels to receive better supplementary low-level and semantic information, generating features with better representation capabilities. NGNET includes two types: an axial guided module (AGM) and a formal guided module (FGM), to adapt to features at different levels. Then, in the probabilistic map prediction branch and the contour map prediction branch connected by the fusion features, an Undecimated-discrete-wavelet-transform Fusion Head (UFH) is established. The contour information of the contour map prediction branch is extracted by the undecimated-discrete-wavelet transform and fused into the probabilistic map prediction branch to produce more accurate detection results that can better avoid the erroneous adhesion and separation of text instances.

[0058] Step S5: Test the model and comprehensively evaluate the scene text detection results based on the evaluation metrics.

[0059] The steps are explained in detail:

[0060] Step S1: Obtain a scene text dataset with images and annotations;

[0061] Step S2: Perform data augmentation on the dataset images to ensure the generalization performance of the model and obtain better detection results.

[0062] Step S2.1: Randomly flip the training set images within the range of [-10°, 10°].

[0063] Step S2.2: Perform random mirror reversal.

[0064] Step S3: After completing the image enhancement operation, perform random center cropping on the training set images. Use a random pixel in the image as the center point to crop the image to a size of 640x640, which will be used as the original input data to train the text detection model.

[0065] Figure 2 is a schematic diagram of the convolutional neural network of the scene text detection method based on deep learning according to an embodiment of the present invention;

[0066] Step S4: Build this network using the PyTorch deep learning framework. The convolutional neural network proposed in this invention, as shown in Figure 2, is based on the DBNET network and uses NGNET instead of FPN as the model framework. It is a convolutional neural network specifically designed for scene text detection and consists of three parts: backbone, neck, and head.

[0067] In the backbone section, the network designed in this invention uses ResNet18 as the basic feature extraction network. ResNet18 is lightweight and suitable for efficient and real-time text detection tasks. The features output by each layer in ResNet18 are referred to as C1-C5. Further feature extraction and feature fusion will then be performed in the neck section.

[0068] In the Neck section, the network designed in this invention uses NGNET as the feature fusion network. Considering the differences in semantic, positional, and scale information at different levels, NGNET includes two structures: an Axial Guided Module (AGM) and a Regular Guided Module (FGM). The structures of the two modules are shown in Figure 3, respectively acting on (C5,C4) and (C4,C3), (C3,C2) to handle the processing of features at different levels. Furthermore, to balance the rich semantic information of high-level layers with low-level information such as color contrast, we removed the lateral connections in layer C5, retaining only three layers of output features for fusion. Both AGM and FGM have two input features: a relatively low-level feature and a relatively high-level feature. The relatively high-level features are all subjected to dimensionality reduction using a 1x1 convolutional kernel with a stride of 1, and the image scale is upsampled to the same size as the relatively low-level image using nearest-neighbor interpolation.

[0069] Figure 3 is a schematic diagram of the AGM and FGM networks of the deep learning-based scene text detection method according to an embodiment of the present invention;

[0070] In terms of lower-level paths, AGM differs from FGM. Because higher-level features like C4 and C5 are more global and have a larger receptive field, they are closer to what the human eye observes than lower-level information like color, texture, and boundaries such as C2 and C3. Therefore, the AGM module performs max pooling on the lower-level feature (C4) along both row and column directions, and further extracts contour information of C4 along different directions through a pair of asymmetric convolutions. This makes the detection network more robust to text direction and extreme aspect ratio instances. It then performs pointwise multiplication with the C5 features from the higher-level channels (after dimensionality reduction and upsampling), providing positional and contour information for the higher-level features. The result of the pointwise multiplication is then weighted back to the original lower-level feature C4 using an attention mechanism (sigmoid processing followed by stacking). The working principle of AGM can be expressed by the following formula:

[0071] P4 = Conv 1x1 (C5′⊙C4 h ⊙C4 w )

[0072] C5′=Upsample(Conv 1x1 (C5))

[0073] C4 h =Conv 3x1 (Pool h (C4))

[0074] C4 w =Conv 1x3 (Pool w (C4))

[0075] C4 h (C, H, 1), C4 w (C, 1, W)

[0076] Where C is the number of channels in the low-level feature, H is the height of the low-level feature, W is the width of the low-level feature, ⊙ represents pointwise multiplication, and C4 h and C4 w These represent the characteristics of max pooling along the row and column directions, respectively.

[0077] Because the C2 and C3 features guided by FGM contain rich positional information but lack semantic information, FGM does not use the attention mechanism of weighted dot-multiplication of sigmoid images. Instead, it extracts features using only 3x3 convolutional kernels, multiplies them point-by-point, and then superimposes them with the original C2 or C3 image. This method utilizes relatively high-level features to activate the semantic information of relatively low-level features. Further feature extraction and dimensionality reduction are then performed using two layers of 3x3 convolutional kernels. The working principle of FGM can be expressed by the following formula (taking C4 and C3 as examples):

[0078] P3 = Conv 3x3 (Conv 3x3 (C3+C4′⊙C3′))

[0079] C4′=Upsample(Conv 1x1 (C4))

[0080] C3′=Conv 3x3 (C3)

[0081] The three sets of feature maps processed by AGM and FGM are resized to 1 / 4 of the original size and then stitched together. The features obtained from this feature fusion network are then fed into the prediction head to predict the corresponding probability map and contour map.

[0082] Figure 4 is a schematic diagram of the Head network of the deep learning-based scene text detection method according to an embodiment of the present invention;

[0083] In the Head section, to enable the probabilistic graph prediction branch and the contour graph prediction branch to generate more accurate prediction results and reduce errors such as detection result sticking and separation, the network designed in this invention uses UFH as the prediction head to replace the conventional prediction module composed only of deconvolution and convolution modules. The main idea of ​​UFH is to use the high-frequency and low-frequency components separated from the contour graph branch to guide the probabilistic graph branch, activate the pixels in the probabilistic graph branch that are consistent with the contour graph branch, thereby suppressing text sticking and separation.

[0084] First, the feature images processed by the feature fusion network are used as inputs to the probability map and contour map prediction branches, respectively. They are then subjected to dimensionality reduction by convolution kernels of size 1x1 and 3x3, respectively. Finally, the deconvolution module is used to upsample the images in the two prediction branches to half the size of the original image.

[0085] Then, non-sampling discrete wavelet transform is performed on the contour map branch features to separate its high-frequency and low-frequency features, which respectively represent the edge components and basic features in the contour map features. Compared with convolution, it can better preserve the overall features and image contour of the feature image. The low-frequency component LL is multiplied point by point with the probability map branch features to suppress the edge pixel weights in the probability map components that do not match the contour map branches.

[0086] Meanwhile, this invention uses pixel weights as detection thresholds to determine whether the current pixel belongs to a text instance. The features obtained by multiplying the probability map branch with the low-frequency component LL need to be superimposed with the three high-frequency component maps LH, HL, and HH to compensate for the loss of edge details in the result of the multiplication of the probability map branch with the low-frequency component. This further enhances the guidance of the contour map branch on the probability map branch while avoiding the situation where the image cannot be completely detected due to insufficient weight of edge pixels. The fusion process here can be expressed as follows:

[0087] ll,lh,hl,hh=udwt(contour)

[0088] fusion=0.3*(ll*probability+lh+hl+hh)+0.7*probability

[0089] Where probability is the probability graph branch, contour is the contour graph branch, fusion is the fusion feature of the two branches, and ll, lh, hl, hh are the four feature components generated by the non-sampling discrete wavelet transform of the contour graph branch features. ll corresponds to the low-frequency component of the contour graph, which represents the basic features in the image, while lh, hl, hh correspond to the high-frequency component, which represents the contour information extracted along the height, width, and diagonal directions, respectively. Adding lh, hl, hh to the dot product result yields a relatively complete image contour.

[0090] The fused features are weighted and superimposed onto the probability map branch. The two branches then undergo dimensionality reduction and deconvolution again, serving as the data source for the binarized image. After post-processing, the final predicted image is generated, completing the detection task. This can be expressed as follows:

[0091] probability'=ConvTranspose(fusion*probability)

[0092] contour'=ConvTranspose(contour)

[0093]

[0094] k is a preset hyperparameter, which is set to 50 in this invention, and Binarize is a differentiable binary image.

[0095] The loss function of the deep learning framework of this invention is as follows:

[0096] L = L s +a*L b +b*L t

[0097]

[0098]

[0099] Where L s For probabilistic graphical loss, L t For contour map loss, L b The loss is for the differentiable binarized function and the feature map. R d y is the set of pixel indices within the polygon. * i Labels for the outline drawing.

[0100] Step S5: After the scene text detection model is trained, the text detection results are evaluated according to evaluation metrics. Evaluation metrics include precision, recall, and the weighted harmonic mean (F-Measure). The formulas and relationships are as follows:

[0101]

[0102]

[0103] TP represents pixels whose labels are positive and whose detection results are positive; FP represents pixels whose labels are negative and whose detection results are positive; and precision indicates the accuracy among the results that are detected as positive.

[0104]

[0105] TP is the pixel that correctly predicts the target object in the image, FN is the pixel with a positive label and a negative detection result, and recall indicates how many positive objects are accurately detected.

[0106]

[0107] The F-measure is a comprehensive score based on precision and recall. It is generally used as a primary metric for comparing model performance.

[0108] This invention uses DBNET as a benchmark, replacing its Feature Pyramid (FPN) with the Neighboring Layer Guided Network (NGNET) proposed in this invention, and reducing one layer of lateral links to minimize the loss of small-sized text and reduce aliasing caused by differences in size and semantic information during the fusion of different levels. The feature fusion module includes an Axial Guided Module (AGM) and a Regular Guided Module (FGM) to process the semantic information represented by different levels accordingly.

[0109] Based on DBNET, this invention integrates the probabilistic graph prediction branch and the contour graph prediction branch in the prediction head module. It extracts edge information from the contour graph through non-sampling discrete wavelet transform and weights it into the probabilistic graph prediction branch, guiding the probabilistic graph prediction branch to generate more accurate text instances and suppressing text sticking and separation phenomena.

[0110] System Implementation Examples

[0111] According to an embodiment of the present invention, a scene text detection system based on deep learning is provided. Figure 3 is a schematic diagram of the scene text detection system based on deep learning according to an embodiment of the present invention. As shown in Figure 3, it specifically includes:

[0112] Acquisition module: Used to acquire the text detection training dataset;

[0113] Data augmentation module: used to augment images in the training dataset;

[0114] The data augmentation module is specifically used to: randomly rotate and randomly mirror the training set images.

[0115] The cropping module is used to perform random center cropping on the data augmented training dataset images to obtain randomly center cropped training dataset images;

[0116] The cropping module is specifically used to crop an image to a size of 640x640, using a random pixel from the training set image as the center point.

[0117] Establishment module: Used to build a scene text detection model;

[0118] The module is specifically used to: build a scene text detection model, which includes three parts: backbone, neck, and head;

[0119] Backbone uses ResNet18 as the feature extraction network to extract features from the test set and then inputs the features into Neck. The output features include C1, C2, C3, C4, and C5.

[0120] Neck, using NGNET as a feature fusion network, is used to fuse features of the test set. NGNET consists of two structures: an Axial Guide Module (AGM) and two Regular Guide Modules (FGMs).

[0121] Max pooling is performed on C4 along the row and column directions respectively. Then, the contour information is output through a pair of asymmetric convolutions. The contour information is multiplied point by point with the C5 feature after dimensionality reduction and upsampling. The result is then weighted into C4 in the form of an attention mechanism to obtain the first feature map.

[0122] C2 is input into the convolution kernel for feature extraction to obtain C2 features. C2 features are multiplied pointwise and then superimposed with C2. After superposition, the second feature map is obtained by feature extraction and dimensionality reduction through two layers of convolution kernels.

[0123] C3 is input into the convolution kernel for feature extraction to obtain C3 features. C3 features are multiplied pointwise and then superimposed with C3. After superposition, the third feature map is obtained by feature extraction and dimensionality reduction through two layers of convolution kernels.

[0124] The first feature map, the second feature map, and the third feature map are resized to 80x80 and then stitched together to obtain the fused feature. The fused feature is then input into the Head.

[0125] The head inputs the fused features into the prediction branch ingress points of the probability map and the contour map, respectively. After inputting into the probability map prediction branch ingress point, the first dimensionality-reduced feature is obtained through dimensionality reduction. This first dimensionality-reduced feature is then upsampled using a first deconvolutional network to obtain the probability map branch feature. This second dimensionality-reduced feature is then input into the contour map prediction branch ingress point and downsampled to obtain the contour map branch feature. A non-sampled discrete wavelet transform is then performed on the contour map branch feature to obtain the high-frequency and low-frequency feature components. The low-frequency feature component is then combined with the probability map branch feature... The dot product is obtained by multiplying the dot product features. The dot product features are then superimposed with the high-frequency feature components to obtain the fused features of the two branches. The fused features of the two branches and the probabilistic map branch features are concatenated and then input into the third deconvolutional network for dimensionality reduction and upsampling. After dimensionality reduction and upsampling, the features are input into the first sigmoid network for normalization. The normalized fused features are then post-processed to obtain the probabilistic map. The contour map branch features are input into the fourth deconvolutional network for dimensionality reduction and upsampling. After dimensionality reduction and upsampling, the features are input into the second sigmoid network for normalization. The normalized contour map features are then post-processed to obtain the contour map, thus completing the detection.

[0126] Training module: Used to input images from the training dataset with random center cropping into the scene text detection model for training, and obtain the trained scene text detection model;

[0127] Detection module: Used to input the test set into the trained scene text detection model for scene text detection;

[0128] Evaluation module: Used to evaluate the text detection results.

[0129] The embodiments of the present invention are system embodiments corresponding to the above method embodiments. The specific operation of each module can be understood by referring to the description of the method embodiments, and will not be repeated here.

[0130] Device Example 1

[0131] This invention provides a scene text detection device based on deep learning, as shown in FIG6, including: a memory 60, a processor 62, and a computer program stored in the memory 60 and executable on the processor 62. When the computer program is executed by the processor, it implements the steps in the above method embodiment.

[0132] Device Example 2

[0133] This invention provides a computer-readable storage medium storing an information transmission implementation program, which, when executed by a processor 62, implements the steps described in the above method embodiments.

[0134] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions to the technical solutions of the embodiments of the present invention do not cause the essence of the corresponding technical solutions to deviate from the scope of the present solution.

Claims

1. A scene text detection method based on deep learning, characterized in that, include: S1. Obtain the text detection training dataset; S2. Perform data augmentation on the images in the training dataset; S3. Perform random center cropping on the data augmented training dataset images to obtain randomly center cropped training dataset images. S4. Establish a scene text detection model; S4 specifically includes: establishing a scene text detection model, comprising three parts: backbone, neck, and head; the backbone uses ResNet18 as the feature extraction network to extract features from the test set and input these features into the neck, outputting features including C1, C2, C3, C4, and C5; the neck uses NGNET as the feature fusion network to fuse features from the test set, NGNET comprising one axial guided module (AGM) and two regular guided modules (FGM); C4 is max-pooled along both the row and column directions, and then processed through a pair of... Asymmetric convolution outputs contour information. This contour information is multiplied point-by-point with the C5 feature (after dimensionality reduction and upsampling) to obtain the result. This result is weighted into C4 using an attention mechanism to obtain the first feature map. C2 is input into the convolution kernel for feature extraction to obtain the C2 feature. The C2 feature is multiplied point-by-point and then superimposed with C2. After superposition, feature extraction and dimensionality reduction are performed through two layers of convolution kernels to obtain the second feature map. C3 is input into the convolution kernel for feature extraction to obtain the C3 feature. The C3 feature is multiplied point-by-point and then superimposed with C3. After superposition, feature extraction and dimensionality reduction are performed through two layers of convolution kernels to obtain the third feature map. The dimensions of the first, second, and third feature maps are then adjusted. Adjust the resolution to 80x80 and stitch the images together to obtain fused features. Input these fused features into the Header. In the Header, input the fused features into the prediction branch ingress points of the probability map and the contour map respectively. After inputting into the probability map prediction branch ingress point, perform dimensionality reduction to obtain the first dimensionality-reduced feature. Use a first deconvolutional network to upsample the first dimensionality-reduced feature to obtain the probability map branch feature. Input this feature into the contour map prediction branch ingress point and perform dimensionality reduction to obtain the second dimensionality-reduced feature. Use a second deconvolutional network to upsample the second dimensionality-reduced feature to obtain the contour map branch feature. Perform a non-sampling discrete wavelet transform on the contour map branch feature to obtain the high-frequency and low-frequency feature components. The detection process involves several steps: first, multiplying the low-frequency feature components with the probability map branch features point by point to obtain the multiplicative features; then, superimposing the multiplicative features with the high-frequency feature components to obtain the fused features of the two branches; finally, concatenating the fused features of the two branches with the probability map branch features and inputting them into the third deconvolutional network for dimensionality reduction and upsampling; then, inputting the dimensionality reduction and upsampling features into the first sigmoid network for normalization; finally, post-processing the normalized fused features to obtain the probability map; and then inputting the contour map branch features into the fourth deconvolutional network for dimensionality reduction and upsampling; finally, inputting the dimensionality reduction and upsampling features into the second sigmoid network for normalization; and finally, post-processing the normalized contour map features to obtain the contour map, thus completing the detection. S5. Input the images from the training dataset with random center cropping into the scene text detection model for training, and obtain the trained scene text detection model; S6. Input the test set into the trained scene text detection model for scene text detection; S7. Evaluate the text detection results.

2. The method according to claim 1, characterized in that, S2 specifically includes: randomly rotating and randomly mirroring the training set images.

3. The method according to claim 2, characterized in that, S3 specifically includes: cropping an image to a size of 640x640 using a random pixel from the training set image as the center point.

4. A scene text detection system based on deep learning, characterized in that, Includes: Acquisition module: used to acquire the text detection training dataset; Data augmentation module: used to augment images in the training dataset; The cropping module performs random center cropping on the augmented training dataset images to obtain randomly center-cropped training dataset images. The model building module is used to build a scene text detection model. Specifically, the model building module is used to build a scene text detection model, comprising three parts: backbone, neck, and head. The backbone uses ResNet18 as the feature extraction network to extract features from the test set and input these features into the neck, outputting features C1, C2, C3, C4, and C5. The neck uses NGNET as the feature fusion network to fuse features from the test set. NGNET includes an Axis Guided Module (AGM) and two regular... The FGM (Feature Generation Master) module has two structures: C4 is max-pooled along both row and column directions, then a pair of asymmetric convolutions outputs contour information. This contour information is multiplied point-by-point with the reduced and upsampled features of C5 to obtain the result. This result is weighted into C4 using an attention mechanism to obtain the first feature map. C2 is input into the convolution kernel for feature extraction to obtain C2 features. C2 features are multiplied point-by-point and then superimposed with C2. The superimposed feature map is then processed through two convolution kernels for feature extraction and dimensionality reduction to obtain the second feature map. C3 is input into the convolution kernel for feature extraction to obtain C3 features. C3 features are multiplied point-by-point and then superimposed with C3. The superimposed feature map is then processed through two convolution kernels for feature extraction and dimensionality reduction to obtain the second feature map. Three feature maps: The first, second, and third feature maps are resized to 80x80 and stitched together to obtain a fused feature. This fused feature is then input into the Header. In the Header, the fused feature is input into the prediction branch ingress points of the probability map and the contour map, respectively. After inputting into the probability map prediction branch ingress point, it undergoes dimensionality reduction to obtain the first dimensionality-reduced feature. This first dimensionality-reduced feature is then upsampled using a first deconvolutional network to obtain the probability map branch feature. This second dimensionality-reduced feature is then input into the contour map prediction branch ingress point and undergoes dimensionality reduction to obtain the second dimensionality-reduced feature. This second dimensionality-reduced feature is then upsampled using a second deconvolutional network to obtain the contour map branch feature. Finally, a non-sampling discrete wavelet transform is performed on the contour map branch feature to obtain the middle contour map branch feature. The high-frequency and low-frequency feature components are obtained. The low-frequency feature components are multiplied pointwise with the probability map branch features to obtain the dot product features. The dot product features are superimposed with the high-frequency feature components to obtain the fusion features of the two branches. The fusion features of the two branches and the probability map branch features are concatenated and then input into the third deconvolution network for dimensionality reduction and upsampling. After dimensionality reduction and upsampling, they are input into the first sigmoid for normalization. The normalized fusion features are post-processed to obtain the probability map. The contour map branch features are input into the fourth deconvolution network for dimensionality reduction and upsampling. After dimensionality reduction and upsampling, they are input into the second sigmoid for normalization. The normalized contour map features are post-processed to obtain the contour map, thus completing the detection. Training module: Used to input images from the training dataset with random center cropping into the scene text detection model for training, resulting in the trained scene text detection model; Detection module: Used to input the test set into the trained scene text detection model for scene text detection; Evaluation module: Used to evaluate the text detection results.

5. The system according to claim 4, characterized in that, The data augmentation module is specifically used to: randomly rotate and randomly mirror the training set images.

6. The system according to claim 5, characterized in that, The cropping module is specifically used to crop an image of size 640x640, using a random pixel from the training set image as the center point.

7. A scene text detection device based on deep learning, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the deep learning-based scene text detection method as described in any one of claims 1 to 3.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an implementation program for information transmission, which, when executed by a processor, implements the steps of the deep learning-based scene text detection method as described in any one of claims 1 to 3.

Images