Natural scene text recognition method based on GRUs network

Abstract

The invention discloses a natural scene text recognition method based on a GRUs network. The method comprises the following steps: 1) obtaining a text image in a natural scene; 2) segmenting the image, and determining the recognition number of characters in the text; 3) extracting high-dimensional features of the image and a label corresponding to the image; 4) establishing a text recognition model; the text recognition model comprises a GRUs network and an attention mechanism; and 5) training the text recognition model, and performing text recognition on the text image in the natural scene by using the trained model. The invention provides a robust recognition method for a natural scene text based on a GRUs network. The robust recognition method can be used for solving the problems of fuzzy text features, changeable environment, confused time sequence and the like under complex sea conditions.

Description

Technical Field

[0001] The present invention relates to computer vision technology, and in particular to a natural scene text recognition method based on a GRUs network. Background Art

[0002] The problem of text recognition in natural scenes is widely used in unmanned driving, label detection, scene semantic understanding, and semantic map construction. How to recognize the corresponding text in real natural environment pictures or video streams is a current research hotspot in computer vision and machine vision.

[0003] Text in natural scenes also faces challenges such as diverse text types, changing recognition scenes, especially backgrounds, complex scene noise, and uncertain text sizes and arrangements.

[0004] A common solution for text recognition in natural scenes is to combine a deep neural network with LSTM or BiLSTM and CTC. This solution has the characteristics of high accuracy and good robustness. At the same time, the introduction of the attention mechanism can improve the recognition accuracy and training convergence of the network. However, it has a large amount of calculation, high processing power requirements, and insufficient real-time performance. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a natural scene text recognition method based on a GRUs network in view of the defects in the prior art.

[0006] The technical solution adopted by the present invention to solve the technical problem is: a natural scene text recognition method based on a GRUs network, comprising the following steps:

[0007] 1) Obtain text images in natural scenes;

[0008] 2) Segment the image and determine the number of characters to be recognized in the text;

[0009] 3) Extract high-dimensional features of the image and the labels corresponding to the image;

[0010] The label status and high-level feature vectors constitute the data feature description of the image;

[0011] 4) Establish a text recognition model;

[0012] The text recognition model includes a GRUs network and an attention mechanism;

[0013] 5) Train the text recognition model and use the trained model to perform text recognition on text images in natural scenes.

[0014] According to the above scheme, in step 4), in the GRUs network, the gate output R is reset t, update gate output z t and candidate hidden states The design is as follows:

[0015] R t =Sigmoid(x t ⊙W xr +Ey t ⊙W yr +h t-1 ⊙W hr +b r )

[0016] z t =Sigmoid(x t ⊙W xz +Ey t ⊙W yz +h t-1 ⊙W hz +b z )

[0017]

[0018] The hidden state of the t-th unit corresponding to the t-1-th unit is calculated as follows:

[0019]

[0020] Among them, ⊙ refers to the element-wise product operation, W xr , W yr , W hr , W xz , W yz , W hz , W xh , W yh , W hh is the corresponding network weight hyperparameter, b r 、b z 、b h is the corresponding offset; Ey t is the embedding matrix covering the recognized objects of the t-th target text.

[0021] According to the above scheme, in step 4), the text recognition model also includes multi-layer perceptrons MLPs arranged between the GRUs network and the attention mechanism.

[0022] According to the above scheme, the output of the multi-layer perceptron MLPs is the hidden layer h t The compressed output h end,t :

[0023] h end,t =MLPs(h t ).

[0024] According to the above scheme, in step 4), the attention mechanism weight a t The build is as follows:

[0025]

[0026]

[0027] Among them, W e , W v is the introduced hyperparameter.

[0028] According to the above scheme, the loss function used in step 5) is as follows:

[0029] L=-λlog(p(y|(c t ,…,c L )))+(1-λ)∑ i ∑ t a ti ;

[0030] Among them, c t The text vector is calculated as follows:

[0031]

[0032] Among them, λ∈[0,1] is an introduced hyperparameter that can adjust the convergence speed.

[0033] The beneficial effects produced by the present invention are:

[0034] 1. The present invention provides a robust recognition method for natural scene text based on the GRUs network, which can be used to solve problems such as fuzzy text features, changeable environment, and time sequence confusion under complex sea conditions.

[0035] 2. Compared with LSTM, since there are only update gates and reset gates, its network structure is simpler, with fewer parameters and less calculation, which can effectively identify speed

[0036] 3. The present invention improves the existing GRU structure. The GRUs network unit design based on element product improves the problem caused by the loss of spatial information during temporal feature extraction and improves the recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0038] Figure 1 is a method flow chart of an embodiment of the present invention;

[0039] Figure 2is a schematic diagram of a text recognition model according to an embodiment of the present invention;

[0040] Figure 3 It is a schematic diagram of a GRUs network unit based on element product according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0042] like Figure 1 As shown, a natural scene text recognition method based on a GRUs network includes the following steps:

[0043] 1) Obtain text images in natural scenes;

[0044] 2) Segment the image and determine the number of characters to be recognized in the text;

[0045] 3) Extract high-dimensional features of the image and the labels corresponding to the image;

[0046] The label status and high-level feature vectors constitute the data feature description of the image;

[0047] 4) Establish a text recognition model; such as Figure 2 ;

[0048] The text recognition model includes a GRUs network and an attention mechanism;

[0049] like Figure 3 , reset gate output R t , update gate output z t and candidate hidden states The calculation is as follows:

[0050] R t =Sigmoid(x t ⊙W xr +h t-1 ⊙W hr +b r )

[0051] z t =Sigmoid(x t ⊙W xz +h t-1 ⊙W hz +b z )

[0052]

[0053] The hidden state of the t-th unit corresponding to the t-1-th unit is calculated as follows:

[0054]

[0055] Among them, ⊙ refers to the element-wise product operation, W xr , W yr , W hr , W xz , W yz , W hz , W xh , W yh , W hh is the corresponding network weight, b r 、b z 、b h is the corresponding offset;

[0056] Through the above design, we obtain a GRUs network unit based on element product. In the specific implementation of the GRUs network of element product, the GRUs network structure remains unchanged, and the input is the state x t and the hidden layer h t-1 , the output is h t , which consists of a reset gate, an update gate, and a candidate hidden state. Using element product instead of traditional vector multiplication retains the two-dimensional features of the extracted features and improves the network's spatial information representation capability. The initial hidden layer h0 is obtained as follows:

[0057] h0=H0V

[0058] Here H0 is the linear transformation matrix, V is the image feature vector extracted by the CNN network, and the transformed h0 has the same dimension as the image feature vector before the transformation, which can usually be taken as the unit matrix.

[0059] The embedding matrix E covering the recognition object is introduced, which includes Chinese, English and numerical characters. The specific scale can refer to the existing dataset, for example, the dimension of E is 5453×256. The embedding matrix of the t-th target text is represented by Ey t And the corresponding expectation matrix is ​​expressed as Both have the same dimensions.

[0060] Finally, in the GRUs network, the reset gate outputs R t , update gate output z t and candidate hidden states The design is as follows:

[0061] R t =Sigmoid(x t ⊙W xr +Ey t ⊙W yr+h t-1 ⊙W hr +b r )

[0062] z t =Sigmoid(x t ⊙W xz +Ey t ⊙W yz +h t-1 ⊙W hz +b z )

[0063]

[0064] The hidden state of the t-th unit corresponding to the t-1-th unit is calculated as follows:

[0065]

[0066] The model also introduces multi-layer perceptrons (MLPs) to obtain the hidden layer h t The compressed output h end,t :

[0067] h end,t =MLPs(h t ).

[0068] Introducing the attention mechanism weights a of multi-layer perceptrons MLPs t The build is as follows:

[0069]

[0070]

[0071] Among them, W e , W v is the introduced hyperparameter.

[0072] 5) Train the text recognition model and use the trained model to perform text recognition on text images in natural scenes.

[0073] The loss function used to train the text recognition model is as follows:

[0074] L=-λlog(p(y|(c t ,…,c L )))+(1-λ)∑ i ∑ t a ti ;

[0075] Among them, c t The text vector is calculated as follows:

[0076]

[0077] Among them, λ∈[0,1] is an introduced hyperparameter that can adjust the convergence speed.

[0078] One embodiment

[0079] Initially perform image segmentation, feature encoding and extraction to determine the number of recognition L and the corresponding initial label states x1, x2, x3, x4 and high-level features. Figure 2 For the input image in , L = 4, the label state consists of 4 components, denoted as x1, x2, x3, x4, etc. Common segmentation networks include Faster-CNN, Mask R-CNN, and Cascade R-CNN.

[0080] The label status, text quantity and high-level features obtained by the segmentation network are sent to a simple CNN network to further propose high-dimensional features of the image and obtain a data feature description consisting of label status and high-level feature vectors. Figure 2 CNN feature extraction network in.

[0081] use Figure 3 The element-by-element product GRUs network shown in the figure establishes a basic hidden state calculation unit. Here, the reset gate, update gate, and hidden state update in the GRUs network are as follows:

[0082] R t =Sigmoid(x t ⊙W xr +Ey t ⊙W yr +h t-1 ⊙W hr +b r )

[0083] z t =Sigmoid(x t ⊙W xz +Ey t ⊙W yz +h t-1 ⊙W hz +b z )

[0084]

[0085]

[0086] The embedding matrix E of the recognized objects during network training and hidden state update, which includes numbers, letters and Chinese characters.

[0087] After the trained GRUs module, we can calculate the hidden states h1, h2, h3, h4 corresponding to the label states x1, x2, x3, x4. When calculating the hidden states h1, h2, h3, h4, the corresponding initial hidden state h0 is taken as V. This part corresponds to Figure 3 The element-wise product of the GRUs network part.

[0088] Send the hidden states h1, h2, h3, h4 into the trained multi-layer perceptron MLPs and calculate the corresponding hidden layer compressed output h end,1 ,h end,2 ,h end,3 ,h end,4 Combined with the initial extraction V = [V1, V2, V3, V4], calculate the corresponding attention weights and the corresponding feature text vectors c1, c2, c3, c4, and decode and output the corresponding text recognition results y1, y2, y3, y4. This part corresponds to Figure 2 Attention mechanism network in .

[0089] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all these improvements and changes should fall within the scope of protection of the appended claims of the present invention.

Claims

1. A natural scene text recognition method based on GRUs network, characterized in that: The following steps are involved: 1) Obtain text images in natural scenes; 2) Segment the image and determine the number of characters to be recognized in the text; 3) Extract high-dimensional features of the image and the labels corresponding to the image; 4) Establish a text recognition model; The text recognition model includes a GRUs network and an attention mechanism; 5) Train the text recognition model and use the trained model to perform text recognition on text images in natural scenes.

2. The natural scene text recognition method based on the GRUs network according to claim 1 is characterized in that: In step 4), in the GRUs network, reset the gate output R t , update gate output z t and candidate hidden states The design is as follows: R t =Sigmoid(x t ⊙W xr +Ey t ⊙W yr +h t-1 ⊙W hr +b r ) z t =Sigmoid(x t ⊙W xz +Ey t ⊙W yz +h t-1 ⊙W hz +b z ) The hidden state of the t-th unit corresponding to the t-1-th unit is calculated as follows: Among them, ⊙ refers to the element-wise product operation, W xr , W yr , W hr , W xz , W yx , W hz , W xh , W yh , W hh is the corresponding network weight hyperparameter, b r , b z , b h is the corresponding offset, Ey t is the embedding matrix covering the recognized objects of the t-th target text.

3. The natural scene text recognition method based on GRUs network according to claim 1 is characterized in that: In the step 4), the text recognition model also includes multi-layer perceptrons (MLPs) arranged between the GRUs network and the attention mechanism.

4. The natural scene text recognition method based on the GRUs network according to claim 3 is characterized in that: The output of the multi-layer perceptron MLPs is the hidden layer h t The compressed output h end,t : h end,t =MLPs(h t )。 5. The natural scene text recognition method based on the GRUs network according to claim 4 is characterized in that: In step 4), the attention mechanism weight a t The build is as follows: Among them, W e , W v is the introduced hyperparameter.

6. The natural scene text recognition method based on GRUs network according to claim 1, characterized in that: The loss function used in step 5) is as follows: L=-λlog(p(y|(c t ,…,c L )))+(1-λ)∑ i ∑ t a ti ; Among them, c t The text vector is calculated as follows: Among them, λ∈[0,1] is the introduced hyperparameter.

7. An electronic device, characterized in that: include: one or more processors; as well as a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are caused to execute the method according to any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images