A liquid crystal interface character recognition method based on deep learning

Abstract

The application discloses a liquid crystal interface character recognition method based on deep learning, which comprises the following steps: acquiring a public available or self-built liquid crystal interface character image dataset, and pre-processing the liquid crystal interface character image data; constructing a lightweight visual feature extraction backbone network, extracting the character features after the preprocessing, and outputting a character feature map; inputting the character feature map into a bidirectional long short-term memory network, capturing the context dependency of the character sequence through the bidirectional long short-term memory network, and obtaining a character feature sequence containing timing information; and adopting a connection timing classification method to decode and process the character feature sequence containing the timing information, and outputting a recognition result; the application can adapt to the deployment requirements of a low computing resource platform, is especially suitable for character recognition tasks in industrial scenes such as liquid crystal display devices and instruments and meters, and has wide practical application value and popularization prospect.

Description

Technical Field

[0001] This invention relates to the field of character recognition technology, and more specifically to a deep learning-based method for character recognition on a liquid crystal interface. Background Technology

[0002] Liquid Crystal Displays (LCDs) are widely used in industrial control, medical equipment, transportation, and home appliances due to their low power consumption, stable display, and strong adaptability. They are used to display equipment operating status, parameter information, and operation prompts in real time. The characters on LCDs are typically characterized by high density, diverse font styles, and frequent font size changes, and the displayed background is often quite complex. In practical applications, factors such as changes in lighting conditions, screen reflections, low-resolution imaging, and shooting angle deviations can easily cause quality problems such as blurring, ghosting, distortion, and missing characters in the acquired LCD character images.

[0003] Existing Optical Character Recognition (OCR) technology can achieve high recognition accuracy in ideal environments with regular character shapes and clear images. However, in real-world scenarios involving LCD displays, characters often exhibit non-standard fonts, dynamic refresh rates, and low-contrast displays, and are frequently accompanied by complex backgrounds and noise interference. This leads to problems such as misidentification, missed identification, and positioning errors in the character detection and recognition stages of traditional OCR methods, significantly reducing recognition accuracy and robustness. Especially in applications such as industrial production and medical monitoring where high recognition accuracy is required, this technological deficiency directly affects the stable operation of equipment and the reliability of information reading.

[0004] Therefore, there is an urgent need for a character recognition method that can balance high robustness and high accuracy for complex liquid crystal display interfaces, in order to solve the problems of decreased recognition performance and insufficient adaptability of existing technologies in complex environments, thereby meeting the needs of practical applications. Summary of the Invention

[0005] The purpose of this invention is to propose a character recognition method for liquid crystal interfaces based on deep learning, comprising the following steps: S1. Obtain publicly available or self-built LCD interface character image datasets and preprocess the LCD interface character image data. S2, construct a lightweight visual feature extraction backbone network, extract preprocessed character features, and output character feature maps; S3, Input the character feature map into the bidirectional long short-term memory network, and capture the contextual dependencies of the character sequence through the bidirectional long short-term memory network to obtain the character feature sequence containing temporal information; S4 uses a connection-based temporal classification method to decode the character feature sequence containing temporal information and outputs the recognition result.

[0006] Further, in step S1, the character image data of the liquid crystal interface undergoes preprocessing, specifically including: Obtain the LCD interface character image dataset to get the original input image; The original input image is uniformly converted into a three-channel format, and the image is standardized. Image pixels are processed by subtracting the mean from each pixel and then dividing by the standard deviation; Set the standard size of image pixels and fill pixels that are deformed or have structural distortion. The data after padding is normalized and resized to obtain preprocessed image data.

[0007] Furthermore, a bidirectional long short-term memory network is constructed as a temporal coding module to map the two-dimensional visual features output by the backbone network into a one-dimensional temporal feature sequence, thereby capturing the contextual dependencies of the character sequence in both directions. A connection-based temporal classification decoding module is constructed, and a SoftMax function is established to perform unaligned supervised character prediction on the encoded feature sequence. The calculation formula of the SoftMax function is as follows:

[0008] in, Let C represent the output value of the i-th node, and C be the number of output nodes, i.e. the number of categories. The SoftMax function is used to convert the output values ​​of the multi-class classification into a probability distribution in the range [0, 1] and with a value of 1.

[0009] Furthermore, the images and their corresponding character labels in the training set are input into the constructed character recognition model, and the model is optimized using an end-to-end joint training method; During training, the connection-time classification loss function is used as the optimization objective. Through multiple rounds of iterative optimization, the network parameters are gradually adjusted to map the character sequences in the input image to the corresponding text labels. The formula for calculating the connection-time classification loss function is as follows:

[0010] Where i represents the i-th sample in the batch, and T is the sequence step size. Let i be the target label sequence of the i-th sample. This represents the model's predicted probability for class k at time step t. For all sequences mapped to the target sequence after compression The set of paths, where B represents the compression mapping function used to remove whitespace symbols and consecutively repeated symbols.

[0011] Furthermore, the bidirectional long short-term memory network includes a forward LSTM layer and a backward LSTM layer. The forward LSTM layer extracts temporal features from the character feature map from left to right, and the backward LSTM layer extracts temporal features from the character feature map from right to left. The output features of the forward LSTM layer and the backward LSTM layer are concatenated to obtain a character feature sequence containing complete contextual dependency information.

[0012] Furthermore, step S2 also includes: Character features are obtained, and horizontal global average pooling is performed along the width direction to compress the width dimension of each channel to 1, resulting in a feature description vector that aggregates the height direction information. Perform vertical global average pooling along the height direction to compress the height dimension to 1, and obtain a feature description vector that aggregates the width direction information; The feature description vectors that aggregate height direction information and the feature description vectors that aggregate width direction information are concatenated along the channel dimension, and the number of channels is compressed to obtain the joint spatial representation features. The joint features are then processed through a shared 1×1 convolutional layer, batch normalization, and non-linear activation function to fuse contextual information from two spatial directions, generating a low-dimensional joint representation. The low-dimensional joint representation is divided into two independent branches, which are used to recover attention information in the height and width directions, respectively. One branch generates attention weights in the height direction through a Sigmoid activation function, while the other branch generates attention weights in the width direction through a Sigmoid activation function. The two attention weights are multiplied element-wise with the original input feature map to complete the spatially perceptual weighting operation of the channel features, and the weighted character feature map is output.

[0013] Furthermore, the expression for calculating the Sigmoid activation function is as follows:

[0014] The range of the Sigmoid function is [0,1], and x is [-∞, +∞].

[0015] Furthermore, the lightweight visual feature extraction backbone network adopts an improved MobileViT-XS structure, which includes a backbone MV2-block, a backbone MV2-block fused with a PConv module group, a synthesized MV2P module, a CA channel attention mechanism module, and a MobileViT-block module.

[0016] Furthermore, the MV2P module is based on the basic unit of MobileNetV2 and improves the original depthwise separable convolution structure by introducing Pixel Conditioned Convolution to enhance the expressive power of convolution. The CA attention mechanism module is embedded in the shallow feature extraction stage and is set before the Conv3×3 convolution operation of the backbone network. By jointly modeling spatial and channel information, it guides the network to focus on the salient features of the character region. The MobileViT-block module combines local convolution and global Transformer feature modeling capabilities to enhance global context awareness.

[0017] Due to the application of the above technical solution, the present invention has the following advantages compared with the prior art: This invention employs an improved MobileViT network as the backbone for feature extraction, and introduces partial convolutional fusion (PConv) and coordinate attention (CA) mechanisms to enhance the model's ability to model character structure information. It combines a bidirectional long short-term memory network (BiLSTM) as a temporal coding module for temporal context modeling, and utilizes a connection-time classification (CTC) decoding module to achieve character sequence prediction under unaligned supervision.

[0018] The character recognition network constructed in this invention has the advantages of lightweight model, high accuracy and fast inference speed. It can adapt to the deployment requirements of low computing resource platforms and is especially suitable for character recognition tasks in industrial scenarios such as LCD display devices and instruments. It has broad practical application value and promotion prospects. Attached Figure Description

[0019] Figure 1 The diagram shows a flowchart of a deep learning-based character recognition method for a liquid crystal interface provided by an embodiment of the present invention. Figure 2 This embodiment shows the improved MobileViT network architecture diagram. Figure 3 A schematic diagram of the CA attention mechanism provided in this embodiment is shown; Figure 4 This embodiment shows a schematic diagram of the MV2P structure. Figure 5 This embodiment shows a schematic diagram of the MobileViT-Block structure. Figure 6 This embodiment shows a schematic diagram of the sub-attention mechanism structure. Figure 7A schematic diagram of the BiLSTM structure provided in this embodiment is shown. Detailed Implementation

[0020] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0021] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0022] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0023] like Figures 1-7 As shown, this embodiment of the invention provides a method for character recognition on a liquid crystal interface based on deep learning, including the following steps: S1. Obtain publicly available or self-built LCD interface character image datasets and preprocess the LCD interface character image data. S2, construct a lightweight visual feature extraction backbone network, extract preprocessed character features, and output character feature maps; S3, Input the character feature map into the bidirectional long short-term memory network, and capture the contextual dependencies of the character sequence through the bidirectional long short-term memory network to obtain the character feature sequence containing temporal information; S4 uses a connection-based temporal classification method to decode the character feature sequence containing temporal information and outputs the recognition result.

[0024] According to an embodiment of the present invention, the preprocessing of the liquid crystal interface character image data in step S1 specifically includes: Obtain the LCD interface character image dataset to get the original input image; The original input image is uniformly converted into a three-channel format, and the image is standardized. Image pixels are processed by subtracting the mean from each pixel and then dividing by the standard deviation; Set the standard size of image pixels and fill pixels that are deformed or have structural distortion. The data after padding is normalized and resized to obtain preprocessed image data.

[0025] Specifically, the process begins by collecting and organizing LCD interface character image data from the internet, and then labeling the character regions in the images using a standardized annotation format. The labeled dataset is then divided into training and testing sets, serving as the foundational data for model training and evaluation.

[0026] The dataset is the ICDAR open dataset provided by Baidu AI Studio platform in China, which contains various LCD interface character images, including 19,912 training set images and 4,099 test set images. The standard annotation tool PaddleLabel was used to annotate the LCD character area, generating corresponding annotation files to ensure data quality and facilitate supervised learning during the training and testing phases.

[0027] To standardize the network input size, all images need to be resized to a fixed size of 3×32×100. To avoid character distortion or structural loss during image scaling, an image padding strategy that maintains the original character aspect ratio is adopted. This involves padding the image edges with pixels to fit the target size while preserving the integrity of the character structure. According to an embodiment of the present invention, a bidirectional long short-term memory network is constructed as a temporal coding module to map the two-dimensional visual features output by the backbone network into a one-dimensional temporal feature sequence, thereby capturing the contextual dependencies of the character sequence in both directions. A connection-based temporal classification decoding module is constructed, and a SoftMax function is established to perform unaligned supervised character prediction on the encoded feature sequence. The calculation formula of the SoftMax function is as follows:

[0028] in, Let C represent the output value of the i-th node, and C be the number of output nodes, i.e. the number of categories. The SoftMax function is used to convert the output values ​​of the multi-class classification into a probability distribution in the range [0, 1] and with a value of 1.

[0029] According to an embodiment of the present invention, the images in the training set and their corresponding character labels are input into the constructed character recognition model, and the model is optimized by an end-to-end joint training method; During training, the connection-time classification loss function is used as the optimization objective. Through multiple rounds of iterative optimization, the network parameters are gradually adjusted to map the character sequences in the input image to the corresponding text labels. The formula for calculating the connection-time classification loss function is as follows:

[0030] Where i represents the i-th sample in the batch, and T is the sequence step size. Let i be the target label sequence of the i-th sample. This represents the model's predicted probability for class k at time step t. For all sequences mapped to the target sequence after compression The set of paths, where B represents the compression mapping function used to remove whitespace symbols and consecutively repeated symbols.

[0031] It should be noted that an end-to-end joint training method is adopted, in which training set images and their corresponding character labels are input into the network for optimization training, so that the network parameters converge collaboratively in feature extraction, temporal modeling and decoding tasks; in the prediction stage, the test set or the actual collected LCD interface images are input into the trained character recognition model, and feature extraction, temporal modeling and decoding are completed in sequence, and the final character recognition result is output.

[0032] In the temporal modeling stage, a recurrent neural network module based on a bidirectional long short-term memory (BiLSTM) network is constructed to model the contextual information in the visual feature sequence. This module consists of two stacked bidirectional LSTM layers, which can simultaneously learn the forward and backward dependencies of the character sequence.

[0033] A decoding module based on Connectionist Temporal Classification (CTC) is constructed. This module receives the feature sequence output from the bidirectional LSTM encoder. First, the features at each time step are projected into the class space through a fully connected layer (Linear). Then, the output at each time step is normalized using the SoftMax function to obtain the probability distribution of each class, thus achieving unaligned supervised character sequence prediction.

[0034] According to an embodiment of the present invention, the bidirectional long short-term memory network includes a forward LSTM layer and a backward LSTM layer. The forward LSTM layer extracts temporal features from the character feature map from left to right, and the backward LSTM layer extracts temporal features from the character feature map from right to left. The output features of the forward LSTM layer and the backward LSTM layer are concatenated to obtain a character feature sequence containing complete contextual dependency information.

[0035] According to an embodiment of the present invention, step S2 further includes: Character features are obtained, and horizontal global average pooling is performed along the width direction to compress the width dimension of each channel to 1, resulting in a feature description vector that aggregates the height direction information. Perform vertical global average pooling along the height direction to compress the height dimension to 1, and obtain a feature description vector that aggregates the width direction information; The feature description vectors that aggregate height direction information and the feature description vectors that aggregate width direction information are concatenated along the channel dimension, and the number of channels is compressed to obtain the joint spatial representation features. The joint features are then processed through a shared 1×1 convolutional layer, batch normalization, and non-linear activation function to fuse contextual information from two spatial directions, generating a low-dimensional joint representation. The low-dimensional joint representation is divided into two independent branches, which are used to recover attention information in the height and width directions, respectively. One branch generates attention weights in the height direction through a Sigmoid activation function, while the other branch generates attention weights in the width direction through a Sigmoid activation function. The two attention weights are multiplied element-wise with the original input feature map to complete the spatially perceptual weighting operation of the channel features, and the weighted character feature map is output.

[0036] It should be noted that, as Figure 3 As shown, the input feature map is first subjected to a Conv1×1 convolution for preliminary channel adjustment. This operation linearly transforms the channel dimension of the feature map without changing the spatial size, while realizing the fusion and compression of local features. Furthermore, a Coordinate Attention (CA) channel attention mechanism is introduced after Conv1×1 convolution to effectively improve the model's ability to focus on salient regions; Furthermore, a Conv3×3 convolution is used to further extract local neighborhood features in the spatial dimension to supplement fine-grained spatial structure information; then, the MV2P module is used to further compress and nonlinearly express the features, and the stacking is repeated 4 times in this structure to enhance the feature extraction capability while maintaining computational efficiency. Furthermore, in the middle of the network, deep feature modeling is performed through a combination module of (MV2P→MobileViT block→MV2P). In the final stage, the feature map is subjected to the rightmost Conv1×1 convolution for final adjustment and refinement in the channel dimension. Then, the spatial dimension is compressed through global average pooling, and the spatial features of the entire image are aggregated into a single global feature vector, which serves as the input for the subsequent recognition task module, thereby completing the efficient transformation process from the original image features to high-order semantic representation.

[0037] Furthermore, the specific calculation expression for average pooling is as follows:

[0038] in, This indicates the region in the matrix associated with the k-th feature map. Average pooling output value, Represents the matrix region The element located at (p, q) in the middle. Represents the matrix region The number of elements in the middle.

[0039] Furthermore, the CA module, such as Figure 3 As shown, global average pooling is performed on the feature map in two spatial dimensions. First, horizontal global average pooling (X AvgPool) is performed along the width direction, compressing the width dimension of each channel to 1, resulting in a feature description vector that aggregates the height direction information, with dimensions [C, H, 1]. Then, vertical global average pooling (Y AvgPool) is performed along the height direction, compressing the height dimension to 1, resulting in a feature description vector that aggregates the width direction information, with dimensions [C, 1, W]. These two vectors represent the global spatial features of the feature map in the height and width directions, respectively. Subsequently, these two description vectors are concatenated along the channel dimension, and the number of channels is compressed to obtain a joint spatial representation feature, with the concatenated dimension being [C / r, H + W], where r is the channel compression ratio used to reduce computation. The joint features are then processed through a shared 1×1 convolutional layer, batch normalization, and a non-linear activation function to further fuse contextual information from both spatial directions, generating a low-dimensional joint representation. This low-dimensional joint representation is then divided into two independent branches to recover attention information in the height and width directions, respectively. The first branch maps the features back to dimension [C, H, 1] through a 1×1 convolution operation and generates attention weights in the height direction using a sigmoid activation function. The second branch similarly recovers features of dimension [C, 1, W] through a 1×1 convolution and generates attention weights in the width direction using a sigmoid activation. The attention weights in both directions are then expanded to the same dimensions [C, H, W] as the original feature map through a broadcast mechanism. Finally, these two attention weights are multiplied element-wise with the original input feature map to complete the spatially aware weighting of the channel features, outputting a weighted feature map.

[0040] According to an embodiment of the present invention, the calculation expression of the Sigmoid activation function is as follows:

[0041] The range of the Sigmoid function is [0,1], and x is [-∞, +∞].

[0042] According to an embodiment of the present invention, the lightweight visual feature extraction backbone network adopts an improved MobileViT-XS structure. The improved MobileViT-XS structure includes a backbone MV2-block, a backbone MV2-block fused with a PConv module group, a synthesized MV2P module, a CA channel attention mechanism module, and a MobileViT-block module.

[0043] It should be noted that the improved MobileVit network structure incorporates an attention mechanism module (CA), an improved MV2P module, and replaces the depthwise separable convolutions with the efficient PConv module, along with a MobileVit-block module containing a visual transformer. The CA attention mechanism module is embedded in the shallow feature extraction stage, preceding the Conv3×3 convolutions in the backbone network. By modeling spatial and channel information, it guides the network to focus on salient features of character regions. Replacing depthwise separable convolutions with PConv enhances the model's ability to perceive local character structures while maintaining network lightweightness, thereby improving the accuracy and robustness of character recognition tasks. The proposed improved MobileVit module is also mentioned.

[0044] like Figure 2 As shown in Table 1, the specific parameter settings for each module are as follows; Table 1 layer Output size Step length Number of output channels Image [32,100] - 3 Conv 1×1 [32,100] 1 3 CA [32,100] 1 3 Conv 3×3 [16, 50] 2 16 MV2P×4 [16, 50] 1 48 MV2P [8, 25] 2 64 Mobile Vit-Block [8, 25] 1 64 MV2P [8, 25] 1 80 Mobile Vit-Block [8, 25] 1 80 MV2P [8, 25] 1 96 Mobile Vit-Block [8, 25] 1 96 Conv 1×1 [8, 25] 1 384 Avg Pool [1, 25] 1 384 According to an embodiment of the present invention, the MV2P module is based on the basic unit of MobileNetV2 and improves the original depth-separable convolution structure by introducing Pixel Conditioned Convolution to enhance the expressive power of convolution. The CA attention mechanism module is embedded in the shallow feature extraction stage and is set before the Conv3×3 convolution operation of the backbone network. By jointly modeling spatial and channel information, it guides the network to focus on the salient features of the character region. The MobileViT-block module combines local convolution and global Transformer feature modeling capabilities to enhance global context awareness.

[0045] According to embodiments of the present invention, such as Figure 4 As shown, the input is processed by 1×1 pointwise convolution (Conv2d 1×1) to adjust the number of input feature channels. Then, batch normalization (BN) is used to stabilize the data distribution. The ReLU6 activation function introduces non-linearity and limits the output range to [0,6]. Furthermore, the output of ReLU6 is then spatially convolved channel by channel by PConv, followed by BN and ReLU6 enhancement of the feature transformation; then, 1×1 pointwise convolution is used to restore the number of channels, BN is used to normalize the features, and finally, the initial input is added to the above-transformed features through residual connection to output the result.

[0046] Furthermore, the MobileViT block takes the input feature map [C, H, W] and sequentially undergoes three stages: local feature extraction, global correlation mining, and feature fusion reconstruction. First, it extracts local spatial information using a 3×3 convolution while preserving the original size, and then compresses the channels to [D, H, W] using a 1×1 convolution to adapt to subsequent processing. Then, the features are reshaped into a sequence [D, N, P] and input into the Transformer module, where a self-attention mechanism is used to capture long-distance global dependencies, and the feature map is inversely reshaped back to [D, H, W]. Finally, a 1×1 convolution adjusts the channels back to [C, H, W], which is then concatenated and fused with the initial branch features. Finally, a 3×3 convolution compresses the output to [C, H, W], achieving synergistic enhancement and reconstruction of local and global features.

[0047] Furthermore, the Transformer module is a neural network based on an attention mechanism. Its core principle is a self-attention mechanism. As shown in the figure, it uses scaled dot product attention to calculate the attention value of the feature matrix. First, the query matrix (Q) and the key matrix (K) are scaled by dot product and normalized by SoftMax to calculate the weight coefficients. Then, the value matrix (V) is weighted and summed according to the weight coefficients.

[0048] Furthermore, the scaling dot product attention formula is as follows:

[0049] Where Q, K, V are linear mapping schemes, defined , , ; is the projection size; SoftMax is a function to obtain the weights of the values.

[0050] Furthermore, in order to comprehensively consider the importance of the information contained in the input sequence, a multi-head attention mechanism is needed to capture information from different subspaces at different locations.

[0051] Furthermore, the formula for the multi-head attention mechanism is as follows:

[0052] This module involves building a temporal coding neural network, specifically a variant of RNN called BiLSTM, which is used for encoding. The network consists of an input layer, an LSTM forward layer, an LSTM backward layer, and an output layer.

[0053] Furthermore, the input layer is used to receive timing data. And gradually input it into the network; Furthermore, the LSTM forward layer along the forward time direction Information is transmitted by introducing a gating mechanism to detect the hidden state at the current moment. Determined by the hidden state of the previous time step and the current input, it is used to extract historical dependency features in the sequence; Furthermore, the LSTM backward layer moves in the reverse time direction. The transmission of information, its hidden state The future hidden state is determined by the current input, thus extracting future-dependent features from the sequence; Furthermore, at each time t, the forward hidden state and the backward hidden state are concatenated or weighted and fused in the bidirectional fusion layer to obtain a comprehensive feature that simultaneously contains historical and future information. This feature, expressed as follows, simultaneously incorporates both historical and future information.

[0054]

[0055] Furthermore, the output layer is based on comprehensive features. The output result is obtained through output layer mapping. This enables the prediction of time-series targets.

[0056] The CTC decoding module is used to transform the character probability distribution arranged by time step output by the BiLSTM temporal coding network into the final character sequence. By introducing whitespace characters into the label set, it avoids the erroneous merging of consecutive identical characters. In the inference stage, it first uses a greedy or bundle search method to select the optimal path, and then performs adjacent repeated character compression and whitespace character deletion in sequence. Finally, it obtains a recognition result with variable length that corresponds to the target text, realizing automatic alignment and decoding of the input sequence and output labels.

[0057] According to an embodiment of the present invention, character image data is first collected and preprocessed; secondly, a feature extraction network based on an improved MobileViT is constructed to obtain high-quality feature representations of the input image; subsequently, the feature sequence is input into an RNN temporal coding network for bidirectional contextual information modeling; then, a CTC sequence decoding network is used to map the feature sequence to the character sequence; next, the constructed character recognition model is trained based on the preprocessed data; finally, the trained model is used to perform character recognition preprocessing on LCD screen interface images, thus forming a complete end-to-end recognition process from data input to character output. To evaluate the contribution of each module to the model, the present invention conducted two ablation experiments on the ICDAR dataset.

[0058] Specifically, the ablation experiment was conducted in two aspects: on the one hand, to verify the improvement of the network structure, and on the other hand, to analyze the hyperparameter configuration.

[0059] Furthermore, regarding network structure improvement, five ablation models were constructed based on the same optimization conditions (using the Adam optimizer and fixing the learning rate to 0.0005, with 100 training epochs). The specific composition and differences are shown in Table 2.

[0060]

[0061] Furthermore, regarding hyperparameter settings, to verify the impact of different configurations on model performance, four ablation experimental models were constructed, and their specific structures and parameter configurations are shown in Table 3.

[0062]

[0063] The specific expression for calculating the learning rate (Cosine) in the task is as follows:

[0064] in, This represents the learning rate in the t-th training round. This represents the initial learning rate. This represents the final learning rate, and T represents the total number of training steps.

[0065] The task is used to evaluate the overall performance of the model, and it is defined as:

[0066] Where T is the number of positive classes predicted as positive, FN is the number of positive classes predicted as negative, FP is the number of negative classes predicted as positive, and TNw is the number of negative classes predicted as negative.

[0067] The results show that the proposed model has good accuracy. Compared with the MobileNetV3+RNN baseline, the accuracy (ACC) of the MobileVit+RNN model under different configurations of CA attention mechanism and PConv is compared. The results show that the MobileVit+RNN model achieves an ACC of 75.78% when both CA attention mechanism and PConv are enabled, verifying that reasonable module combination can improve model performance. For the improved MobileVit+RNN model, the impact of hyperparameter configuration is explored. Two optimizers, AdamW and Adam, are set with a fixed learning rate of 0.0005 and a Cosine dynamic learning rate. The experiment shows that when the Adam optimizer is used and combined with the Cosine learning rate, the model ACC can reach 77.61%, highlighting the performance gain effect of hyperparameter optimization.

[0068] In summary, this invention employs an improved MobileViT network as the backbone for feature extraction, introduces partial convolutional fusion (PConv) and coordinate attention (CA) mechanisms to enhance the model's ability to model character structure information, combines a bidirectional long short-term memory network (BiLSTM) as a temporal encoding module for temporal context modeling, and utilizes a connection-time classification (CTC) decoding module to achieve character sequence prediction under unaligned supervision.

[0069] The character recognition network constructed in this invention has the advantages of lightweight model, high accuracy and fast inference speed. It can adapt to the deployment requirements of low computing resource platforms and is especially suitable for character recognition tasks in industrial scenarios such as LCD display devices and instruments. It has broad practical application value and promotion prospects.

[0070] The present invention also provides a computer-readable storage medium storing a computer program that causes a computer to execute in order to implement any of the above-described deep learning-based liquid crystal interface character recognition methods.

[0071] Those skilled in the art will understand that, for ease of explanation, the example is provided with one memory and one processor. In actual terminals or servers, multiple processors and memories may exist. Memory can also be referred to as storage medium or storage device, etc., and the embodiments of this application do not limit this.

[0072] It should be understood that in the embodiments of this application, the processor may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor may also be a general-purpose microprocessor, graphics processing unit (GPU), or one or more integrated circuits to execute relevant programs to achieve the functions required by the embodiments of this application.

[0073] The processor can also be an integrated circuit chip with signal processing capabilities. In implementation, each step of this application can be completed through integrated logic circuits in the processor hardware or instructions in software form. The aforementioned processor can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The steps of the methods disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the functions required by the units included in the methods, devices, and storage media of the embodiments of this application.

[0074] It should also be understood that the memory mentioned in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache.

[0075] By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0076] The memory can also be a Compact Disc Read-Only Memory (CD-ROM) or other optical disc storage, optical disk storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. The memory can exist independently and be connected to the processor via a bus. The memory can also be integrated with the processor. The memory can store programs, and when the program stored in the memory is executed by the processor, the processor performs the various steps of the method determined in the above embodiments of this application.

[0077] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, the memory (storage module) is integrated into the processor. It should be noted that the memory described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0078] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0079] In implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software. The steps of the method disclosed in the embodiments of this application can be directly implemented by a hardware processor, or by a combination of hardware and software modules within the processor. The software modules can reside in mature storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. Since this storage medium is located in memory, the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method; to avoid repetition, these will not be described in detail here.

[0080] Those skilled in the art will recognize that the various illustrative logical blocks (ILBs) and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0081] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer-programmed program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a processor, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a computer network, or other programmable device.

[0082] This embodiment also provides a computer-readable storage medium storing a computer program that causes a computer to execute in order to implement the above-described deep learning-based liquid crystal interface character recognition method.

[0083] It should be noted that computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic) or wireless (e.g., infrared, wireless, microwave, etc.) means, or from one website, computer, server, or data center to a mobile phone processor via a wired means. A computer-readable storage medium can be any usable medium that a computer can access, or a data storage device such as a server or data center that integrates one or more usable media. Usable media can be magnetic media (e.g., floppy disks, hard disks), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives), etc.

[0084] Finally, it should be noted that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for character recognition on a liquid crystal interface based on deep learning, characterized in that, Includes the following steps: S1. Obtain publicly available or self-built LCD interface character image datasets and preprocess the LCD interface character image data. S2, construct a lightweight visual feature extraction backbone network, extract preprocessed character features, and output character feature maps; S3, Input the character feature map into the bidirectional long short-term memory network, and capture the contextual dependencies of the character sequence through the bidirectional long short-term memory network to obtain the character feature sequence containing temporal information; S4 uses a connection-based temporal classification method to decode the character feature sequence containing temporal information and outputs the recognition result.

2. The method for character recognition on a liquid crystal interface based on deep learning as described in claim 1, characterized in that, In step S1, the character image data of the LCD interface is preprocessed, specifically including: Obtain the LCD interface character image dataset to get the original input image; The original input image is uniformly converted into a three-channel format, and the image is standardized. Image pixels are processed by subtracting the mean from each pixel and then dividing by the standard deviation; Set the standard size of image pixels and fill pixels that are deformed or have structural distortion. The data after padding is normalized and resized to obtain preprocessed image data.

3. The method for character recognition on a liquid crystal interface based on deep learning as described in claim 2, characterized in that, A bidirectional long short-term memory network was constructed as a temporal coding module to map the two-dimensional visual features output by the backbone network into a one-dimensional temporal feature sequence, thereby capturing the contextual dependencies of the character sequence in both directions. A connection-based temporal classification decoding module is constructed, and a SoftMax function is established to perform unaligned supervised character prediction on the encoded feature sequence. The calculation formula of the SoftMax function is as follows: ； in, Let C represent the output value of the i-th node, and C be the number of output nodes, i.e. the number of categories. The SoftMax function is used to convert the output values ​​of the multi-class classification into a probability distribution in the range [0, 1] and with a value of 1.

4. The method for character recognition on a liquid crystal interface based on deep learning as described in claim 3, characterized in that, The images and their corresponding character labels in the training set are input into the constructed character recognition model, and the model is optimized by end-to-end joint training. During training, the connection-time classification loss function is used as the optimization objective. Through multiple rounds of iterative optimization, the network parameters are gradually adjusted to map the character sequences in the input image to the corresponding text labels. The formula for calculating the connection-time classification loss function is as follows: ； Where i represents the i-th sample in the batch, and T is the sequence step size. Let i be the target label sequence of the i-th sample. This represents the model's predicted probability for class k at time step t. For all sequences mapped to the target sequence after compression The set of paths, where B represents the compression mapping function used to remove whitespace symbols and consecutively repeated symbols.

5. The method for character recognition on a liquid crystal interface based on deep learning as described in claim 4, characterized in that, The bidirectional long short-term memory network includes a forward LSTM layer and a backward LSTM layer. The forward LSTM layer extracts temporal features from the character feature map from left to right, and the backward LSTM layer extracts temporal features from the character feature map from right to left. The output features of the forward LSTM layer and the backward LSTM layer are concatenated to obtain a character feature sequence containing complete contextual dependency information.

6. The method for character recognition on a liquid crystal interface based on deep learning as described in claim 5, characterized in that, Step S2 also includes: Character features are obtained, and horizontal global average pooling is performed along the width direction to compress the width dimension of each channel to 1, resulting in a feature description vector that aggregates the height direction information. Perform vertical global average pooling along the height direction to compress the height dimension to 1, and obtain a feature description vector that aggregates the width direction information; The feature description vectors that aggregate height direction information and the feature description vectors that aggregate width direction information are concatenated along the channel dimension, and the number of channels is compressed to obtain the joint spatial representation features. The joint features are then processed through a shared 1×1 convolutional layer, batch normalization, and non-linear activation function to fuse contextual information from two spatial directions, generating a low-dimensional joint representation. The low-dimensional joint representation is divided into two independent branches, which are used to recover attention information in the height and width directions, respectively. One branch generates attention weights in the height direction through a Sigmoid activation function, while the other branch generates attention weights in the width direction through a Sigmoid activation function. The two attention weights are multiplied element-wise with the original input feature map to complete the spatially perceptual weighting operation of the channel features, and the weighted character feature map is output.

7. The method for character recognition on a liquid crystal interface based on deep learning as described in claim 4, characterized in that, The Sigmoid activation function is calculated as follows: ； The range of the Sigmoid function is [0,1], and x is [-∞, +∞].

8. The method for character recognition on a liquid crystal interface based on deep learning as described in claim 1, characterized in that, The lightweight visual feature extraction backbone network adopts an improved MobileViT-XS structure, which includes a backbone MV2-block, a backbone MV2-block fused with a PConv module group, a synthesized MV2P module, a CA channel attention mechanism module, and a MobileViT-block module.

9. The method for character recognition on a liquid crystal interface based on deep learning as described in claim 8, characterized in that, The MV2P module is based on the basic unit of MobileNetV2 and improves the original depthwise separable convolution structure by introducing PixelConditioned Convolution to enhance the expressive power of convolution. The CA attention mechanism module is embedded in the shallow feature extraction stage and is set before the Conv3×3 convolution operation of the backbone network. By jointly modeling spatial and channel information, it guides the network to focus on the salient features of the character region. The MobileViT-block module combines local convolution and global Transformer feature modeling capabilities to enhance global context awareness.

Images