An end-to-end communication big data journey card identification method
By combining Bi-GRU recurrent neural networks and residual networks of deep neural networks, the problem of insufficient speed and accuracy of text recognition under large data volumes is solved, achieving faster and more accurate text recognition results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CLOUD DATALINK (GUIZHOU) INFORMATION TECH CO LTD
- Filing Date
- 2023-08-01
- Publication Date
- 2026-08-04
AI Technical Summary
Existing deep learning algorithms are insufficient in terms of speed and accuracy for text recognition with large amounts of data, and the image blurring caused by shooting behavior and lighting conditions affects the recognition results.
A Bi-GRU recurrent neural network is combined with a deep neural network and a residual network. The size and font of the text box are corrected, and features are extracted by separating convolutional blocks and residual blocks. The parameters of Bi-GRU are optimized by using unaligned CTC, and optimization is carried out by linking temporal classification.
It achieves faster and more accurate text recognition under large data volumes, improves recognition speed and accuracy, solves the problem that deep convolutional neural networks have difficulty in handling text recognition sequences, and reduces the parameters for training and prediction.
Smart Images
Figure CN117237959B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method for identifying travel cards based on end-to-end communication big data. Background Technology
[0002] In recent years, deep learning has made significant progress in text recognition, with performance far exceeding traditional methods. Common algorithms for recognizing text in text boxes include CRNN and CRNN+CTC. However, for text recognition with large datasets, recognition speed must be considered, and deep convolutional neural networks struggle to directly handle sequential text recognition. Furthermore, the recognition speed of existing algorithms needs improvement with large datasets. Additionally, image blurring and lighting conditions can cause glare, hindering accurate text extraction and affecting algorithm accuracy. To address these issues, we propose an end-to-end communication big data travel card recognition method. Summary of the Invention
[0003] The purpose of this invention is to provide an end-to-end communication big data travel card recognition method, which effectively integrates the separation network and residual network of a deep neural network, enabling faster and more accurate recognition of text in text boxes. In the text recognition process, the Bi-GRU used in this solution has fewer parameters than the traditional Bi-LSTM recognition method while maintaining the same performance, thus resulting in faster training and prediction speeds. Furthermore, compared with the method of using softmax for parameter optimization, this invention utilizes CTC, which does not require alignment, to optimize the network parameters of Bi-GRU, making the optimization more reasonable and the text recognition effect better.
[0004] The embodiments of the present invention are achieved through the following technical solutions:
[0005] A method for identifying travel cards based on end-to-end communication big data, the method comprising the following steps:
[0006] Adjust and correct the text box according to its size information so that the text box size meets the required angle and proportion for adjustment;
[0007] The corrected text box is separated and its features are extracted by a pre-set neural network to obtain the font feature data. The text data corresponding to the feature data is obtained by using a Bi-GRU recurrent neural network combined with an activation function.
[0008] By effectively integrating the separation network and residual network of deep neural networks, the text in the text box can be recognized faster and more accurately. In the text recognition process, the Bi-GRU adopted in this solution has fewer parameters than the traditional Bi-LSTM recognition method while maintaining the same performance, thus resulting in faster training and prediction speed.
[0009] Optionally, the deep convolutional neural network is a DCNN structure that includes separate convolutional blocks and residual blocks, and the specific steps for extracting the feature data of the font in the text box are as follows;
[0010] The corrected text box is subjected to feature extraction through a split convolution block. Each channel in the split convolution block uses a different convolution kernel. The results of the convolution of each channel are concatenated and then pooled.
[0011] The residual block is used to perform a convolution operation on the input of the deconvolution block, and the output of the residual block is added to the result of the deconvolution block as the input of the next deconvolution block.
[0012] The last output is processed using flint, and features are extracted using fully connected layer convolutions as the feature data of the font in the text box.
[0013] Using split convolutional blocks significantly reduces the number of training parameters. Using residual blocks allows for continuous increases in network depth to learn more and more useful features; each use of a residual structure does not necessarily worsen the performance compared to its input. This invention effectively integrates these two approaches, improving both training speed and performance of the entire model by adjusting network depth and parameters.
[0014] Optionally, the single-channel convolution kernel in the separated convolution is 3x1, 1x3, or 3x3, 1x1.
[0015] Optionally, the Bi-GRU recurrent neural network is specifically shown in the following formula:
[0016]
[0017] y t =σ(W o ·H t )
[0018] Among them, y t W represents the output of the model at time t. o h represents the weight. t For the final state, To hide parameters sequentially, σ represents the reverse-order hidden parameters, and σ is the activation function si gmo id.
[0019] The Gated Recurrent Unit (GRU) is a variant of LSTM, consisting of reset and update gates. GRU can selectively retain relevant information while discarding irrelevant information. GRU has a simple structure and fast training and prediction speeds.
[0020] Optionally, an activation function is used to prevent gradient vanishing or gradient exploding.
[0021] Preventing gradient vanishing or gradient exploding can avoid biased results.
[0022] Optionally, the Bi-GRU recurrent neural network is also optimized using linked temporal classification, and the parameters of the Bi-GRU recurrent neural network are updated using batch gradient descent during the optimization process.
[0023] Softmax is generally used to optimize the loss for fixed classification. However, for non-fixed long sequences, Softmax is difficult to handle or the optimization effect is not obvious. Batch gradient descent is more effective in optimizing these sequences.
[0024] Optionally, the adjustment and correction process for the text box specifically includes the following steps;
[0025] Without tilting the text box, font correction is performed on the text box to address the impact of text design.
[0026] When the text box is tilted, the size of the text box is corrected to account for the impact of the shooting behavior, and then the font of the text box is corrected.
[0027] Due to the influence of shooting angle or the design of the text itself, the text in the image may be tilted at a certain angle. After the text box is detected by the detection algorithm, the angle of the text will not change. In order to more accurately identify the text, the text box is corrected to the maximum extent.
[0028] Optionally, the size correction process includes rotation processing, erosion processing, contrast adjustment, compression adjustment, and expansion adjustment; the font correction process includes erosion processing, contrast adjustment, compression adjustment, and expansion adjustment.
[0029] Rotation processing: Since a typical text box only has four vertices, namely the top left, top right, bottom left, and bottom right, rotation technology can be used to correct tilted text.
[0030] The technical solutions of the embodiments of the present invention have at least the following advantages and beneficial effects:
[0031] This invention effectively integrates the separation network and residual network of a deep neural network. Compared with traditional statistical learning methods and convolutional neural networks, it can recognize the text in the text box faster and more accurately. In the text recognition process, the Bi-GRU used in this embodiment has fewer parameters than the traditional Bi-LSTM recognition method while maintaining the same performance, thus resulting in faster training and prediction speeds. Moreover, compared with the method of using softmax for parameter optimization, this invention utilizes CTC, which does not require alignment, to optimize the network parameters of Bi-GRU, making the optimization more reasonable and the text recognition effect better. Attached Figure Description
[0032] Figure 1 A schematic diagram illustrating the principle of an end-to-end communication big data travel card identification method provided in an embodiment of the present invention;
[0033] Figure 2 A schematic diagram of the overall process of an end-to-end communication big data travel card identification method provided in an embodiment of the present invention;
[0034] Figure 3 This is a schematic diagram of a convolutional network provided in an embodiment of the present invention. Detailed Implementation
[0035] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.
[0036] like Figure 1 , Figure 2 As shown, Figure 1 A schematic diagram illustrating the principle of an end-to-end communication big data travel card identification method provided in an embodiment of the present invention; Figure 2 This invention provides an overall flowchart of a method for identifying travel cards based on end-to-end communication big data, as provided in an embodiment of the invention. One embodiment of the invention includes: a method for identifying travel cards based on end-to-end communication big data, comprising the following steps:
[0037] Adjust and correct the text box according to its size information so that the text box size meets the required angle and proportion for adjustment;
[0038] The corrected text box is separated and its features are extracted by a pre-set neural network to obtain the font feature data. The text data corresponding to the feature data is obtained by a Bi-GRU recurrent neural network combined with an activation function.
[0039] In this embodiment, the deep convolutional neural network is a DCNN structure that includes separate convolutional blocks and residual blocks. The specific steps for extracting the feature data of the font in the text box are as follows;
[0040] The corrected text box is subjected to feature extraction through a split convolution block. Each channel in the split convolution block uses a different convolution kernel. The results of the convolution of each channel are concatenated and then pooled.
[0041] The residual block is used to perform a convolution operation on the input of the deconvolution block, and the output of the residual block is added to the result of the deconvolution block as the input of the next deconvolution block.
[0042] The last output is processed using flint, and features are extracted using fully connected layer convolutions as the feature data of the font in the text box.
[0043] like Figure 1 , Figure 3 As shown, the DCNN structure is... Figure 1 The DNCC backbone in the model is based on the basic structure of the Inception network in Google Net and the residual idea of ResNet. That is, each input channel uses a separate convolution kernel with different strides and kernel sizes, and a residual network is used.
[0044] Separate convolutional blocks: In traditional deep convolutional neural networks, each convolution uses a single-dimensional kernel to extract features. Directly improving accuracy involves increasing the network's width and depth, which increases computational resources without necessarily improving performance with increased depth. Separate convolution uses different kernels for each channel and employs smaller convolutions to reduce computational cost.
[0045] Residual network blocks: such as Figure 3 The input of the deconvolution block is convolved using a kernel function of size 1 by 1 to achieve non-linear processing. The output is added to the result of the deconvolution, and the sum is used as the input for the next deconvolution.
[0046] Furthermore, in the separate convolution, the single-channel convolution kernel is 3x1, 1x3, or 3x3, 1x1. Compared with 5x1, 1x5, and 5x5 or larger convolution kernels, it has fewer computational parameters, is faster, and does not cause any loss in accuracy.
[0047] Furthermore, the Bi-GRU recurrent neural network is specifically shown in the following formula:
[0048]
[0049] y t =σ(W o ·H t )
[0050] Among them, y t W represents the output of the model at time t. o h represents the weight. t For the final state, To hide parameters sequentially, σ represents the reverse-order hidden parameters, and σ is the activation function si gmo id.
[0051] The Gated Recurrent Unit (GRU) is a variant of LSTM, consisting of reset and update gates. GRU selectively retains relevant information while discarding irrelevant information. GRU has a simple structure and fast training and prediction speeds. The GRU is illustrated in the following equation:
[0052] z t =σ(W z h t-1 +V z x t )
[0053] Update Gate Z t The input is the input x at the current time t. t h t-1 This is the output of GRU at the previous moment, W. z and V z σ represents the weight, and σ is the activation function sigmoid. The activation function is used to prevent gradient vanishing or gradient exploding.
[0054] The reset portal is similar to the update portal:
[0055] r t =σ(W r h t-1 +V r x t )
[0056] W r and V r Indicates weight;
[0057]
[0058] Use the reset gate r t Creating memory units Where ⊙ represents convolution, and Indicates weight;
[0059]
[0060] Final state h t This is accomplished by updating the memory units that were previously output and are now reset.
[0061] Hidden parameters in sequence With reverse hidden parameters The splicing process forms the Bi-GRU recurrent neural network.
[0062] Furthermore, the activation function is used to prevent gradient vanishing or gradient exploding.
[0063] Furthermore, the Bi-GRU recurrent neural network is optimized using linked temporal classification, and the parameters of the Bi-GRU recurrent neural network are updated using batch gradient descent during the optimization process.
[0064] Softmax is generally used to optimize the loss for fixed-length classifications. However, for non-fixed-length sequences, Softmax is difficult to handle or its optimization effect is not significant.
[0065] Linked-Temporal Classification (CTC) loss is primarily used for training models whose output is a sequence of variable length (i.e., an unaligned sequence). Let the output sequence of Bi-GRU be y = y1, ..., y2. T T is the length of the sequence, y i ∈R L′ R represents the real number field, L' contains L and blank (indicating it is not text, but a blank space), and L is the set of Chinese characters, English letters, numbers, and Chinese and English punctuation marks. CTC uses maximum likelihood for training. Given input y, the conditional probability of the output sequence π is:
[0066]
[0067] in, It outputs π. t The log-likelihood probability at time t;
[0068] Let function F be L' T To L T The mapping function F is used to remove duplicate characters and whitespace characters.
[0069]
[0070] The above formula represents the probability that the input is y and the output is the value l of the given label; F -1 () represents an inverse function;
[0071] Construct an objective function to maximize the label:
[0072]
[0073] Among them, L ≤T L represents the set of output sequences. Because F is used for deduplication and whitespace removal, the length of set L is less than or equal to T. Therefore, L is used. ≤T express.
[0074] In this invention, the purpose of model training is to achieve gradient... Adjust the weight parameters ω of the deep convolutional neural network and Bi-GRU so that for input samples π∈F -1 p(l|y) reaches its maximum value when (l).
[0075] Furthermore, the adjustment and correction process for the text box specifically includes the following steps;
[0076] Without tilting the text box, font correction is performed on the text box to address the impact of text design.
[0077] When the text box is tilted, the size of the text box is corrected to account for the impact of the shooting behavior, and then the font of the text box is corrected.
[0078] Due to the influence of shooting angle or the design of the text itself, the text in the image may be tilted at a certain angle. After the text box is detected by the detection algorithm, the angle of the text will not change. In order to more accurately identify the text, the text box is corrected to the maximum extent.
[0079] Furthermore, the size correction process includes rotation processing, erosion processing, contrast adjustment, compression adjustment, and expansion adjustment; the font correction process includes erosion processing, contrast adjustment, compression adjustment, and expansion adjustment.
[0080] Rotation processing: Since a typical text box only has four vertices, namely the top left, top right, bottom left, and bottom right, rotation technology can be used to correct tilted text; the rotation angle range is between [-90°, 90°];
[0081] Erosion processing: When shooting with a mobile phone, objects appear larger when closer and smaller when farther away. This is usually because mobile phone manufacturers design their cameras with medium-short or short-short telephoto lenses. Therefore, in the captured image, text / strokes closer to the lens may be too thick, while strokes / fonts farther away may be thinner. In addition, the user's font style settings may also cause an excessively large ratio of stroke thickness. In this case, erosion image processing technology can be used to slim down the thicker text or strokes, making the text in the text box more uniform.
[0082] Contrast Adjustment: Doubles and halves the contrast and brightness respectively. This simulates shooting / screenshot effects in low-light (or high-light) environments by adjusting contrast and brightness.
[0083] Compression adjustment: Reduces or enlarges the text box;
[0084] Dilation adjustment: Appropriately enlarge and thicken broken or thin strokes in the text box. These breaks are usually caused by slight reflections in the image or camera shake.
[0085] Text boxes are processed using techniques such as erosion, contrast adjustment, compression, and dilation. This enhances the stability and generalization performance of the algorithm, while also balancing data categories and increasing the amount of training data. During the training phase, the input text boxes are processed using a combination of techniques, a single technique, or no processing at all. During the inference and testing phases, no preprocessing methods are used to process the text boxes.
[0086] This invention uses text boxes detected from communication big data travel cards as the dataset. The dataset contains 100,000 text boxes, which are divided into a test set and a training set. 30% of the text boxes in the dataset are used as the test set, and the other 70% are used as the training set.
[0087] Evaluation metrics include accuracy and FPS. Accuracy refers to the proportion of text boxes that can be correctly identified out of the total number of text boxes. Correct identification means that the entire line of text is correctly identified. Accuracy measures the performance of the model. FPS refers to the average number of text boxes identified per second. FPS measures the prediction rate. More complex models may have better performance, but the speed may be reduced due to the complexity of the model. In production environments that perform large-scale data recognition, a balance between performance and speed often brings greater benefits.
[0088] The testing environment used in this invention is as follows: CentOS 7 system, Intel i7 4-core 8-thread processor, NVIDIA GeForce-GTX 1080 graphics card, two memory sticks, each with 8GB.
[0089] In this invention, the impact of different backbones on model performance and the impact of different time series networks on model performance were compared.
[0090] Table 1
[0091] backbone accuracy fps ResNet18 86.5 54.2 ResNet34 92.2 29.9 ResNet50 95 3.5 Inception-v3 96.4 5.1 Ours 97.1 52.1
[0092] The models in Table 1 all use Bi-GRU. As shown in Table 1, if the backbone of the model is set to ResNet, ResNet18 has a shallow network depth and fast recognition speed, but its recognition accuracy is insufficient due to its relatively simple network structure. ResNet34 has 16 more residual structures than ResNet18, making its structure more complex; its recognition speed is average, but its accuracy is better. ResNet50, due to its large number of residual structures and complex structure, has high accuracy but slow recognition speed. I-ception-v3 mainly uses depthwise separable convolutions, resulting in a complex network structure with many parameters. If the backbone of the model is set to I-ception-v3, as shown in Table 1, the algorithm performance is good, but the prediction speed is slow. This invention redesigns the depthwise separable convolutions and residual structures, compressing network parameters, and significantly improves the prediction speed while maintaining performance.
[0093] Table 2
[0094] Temporal Networks accuracy fps LSTM 93.8 44.7 Bi-LSTM 97.2 41.5 GRU 94.4 53.6 Bi-GRU 97.1 52.1
[0095] As shown in Table 2, when using GRU and LSTM, the accuracy of the two is not much different, but GRU is faster. Comparing Bi-GRU and Bi-LSTM, it can be found that the two contribute similarly to the accuracy of the model, but Bi-GRU is faster because Bi-GRU only has two gates and fewer parameters. Bi-GRU not only uses the historical information of the text sequence, but also infers the previous information from the information after the text sequence, so its performance is better than GRU.
[0096] DCNNs are primarily used for image processing, understanding features such as texture and structure. However, they lack the ability to understand the semantics and context of text, making them unsuitable for processing text content. Bi-GRUs, on the other hand, are not suitable for extracting image features but perform well in understanding the context and semantics of sentence text. Travel card recognition in telecommunications big data is a special type of text recognition, containing different colors such as green, red, yellow, white, and black, different font sizes, different fonts such as Song, Li, and Xing, occasionally including Martian text, and sometimes even Pinyin. Therefore, building an effective DCNN model to extract these complex features is difficult. This invention, by adjusting the number of Bi-GRUs, vector dimensions, and their output, enables the Bi-GRU model to handle text with special logical relationships, such as phone numbers, locations, and dates, found in travel card data. Therefore, this invention organically combines a DCNN with a Bi-GRU, effectively improving the algorithm's accuracy, stability, generalization performance, and prediction speed.
[0097] In summary, this invention employs correction techniques to handle slanted text within text boxes and image preprocessing techniques for text boxes. This enables the model to recognize text within normal text boxes, as well as text boxes with fonts under certain distortion conditions, such as those caused by shooting methods, lighting intensity, reflections due to light angle, and text blurring. The DCNN backbone and Bi-GRU network jointly constructed in this invention address the poor performance of traditional methods for text recognition; solve the problem of deep convolutional neural networks being unable to directly handle text recognition sequences; and resolve the issues of image blurring due to shooting behavior and image reflections due to lighting conditions, which prevent accurate recognition of corresponding text. Even with large datasets, the recognition speed of the algorithm proposed in this invention meets the requirements.
[0098] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for identifying travel cards based on end-to-end communication big data, characterized in that, The steps of this method include: The text boxes are detected from the image and their angles are corrected. A pre-trained recognition model is used to recognize the text in the corrected text boxes and output the recognized text content. The recognition model includes a deep convolutional neural network and a Bi-GRU recurrent neural network. The deep convolutional neural network is a DCNN structure that includes separate convolutional blocks and residual blocks. The deep convolutional neural network is used to extract the feature data of the text boxes, and the Bi-GRU recurrent neural network obtains the text content corresponding to the feature data. When adjusting and correcting a text box, the specific steps include: Without tilting the text box, font correction is performed on the text box to address the impact of text design. When the text box is tilted, the size of the text box is corrected to account for the impact of the shooting behavior, and then the font of the text box is corrected. The size correction process includes rotation, erosion, contrast adjustment, compression adjustment, and expansion adjustment; the font correction process includes erosion, contrast adjustment, compression adjustment, and expansion adjustment. The Bi-GRU recurrent neural network is specifically shown in the following formula: ； ； in, This represents the output of the model at time t. Indicates weight, For the final state, To hide parameters sequentially, To hide parameters in reverse order, The activation function is sigmoid. For Bi-GRU output; The Bi-GRU recurrent neural network is optimized using a linked temporal classification method, and the parameters of the Bi-GRU recurrent neural network are updated using the batch gradient descent method during the optimization process. The specific steps for extracting the feature data of the text box are as follows: In the separate convolutional block, each channel uses a different convolutional kernel to extract features from the corrected text, and the results of the convolution of each channel are concatenated and pooled. The residual block is used to perform a convolution operation on the input of the split convolution block, and the output of the residual block is added to the result of the pooling process as the input of the next split convolution block. When performing convolution operations without residual blocks, the last output is converted into a one-dimensional vector for processing, and features are extracted using fully connected layer convolution as the feature data of the text box. The single-channel convolution kernel in the separated convolution is specifically 3x1, 1x3, or 3x3, 1x1.
2. The end-to-end communication big data travel card identification method according to claim 1, characterized in that, The method of correcting the angle of the text box specifically includes the following steps: rotating the text box, with the rotation angle ranging from -90° to 90°.