Digital display meter identification method and system based on E-Net model and LeNet-5 network

Abstract

The invention discloses a digital display meter identification method and system based on an E-Net model and a LeNet-5 network. The method comprises the following steps: applying a bottleneck convolution structure to an ENet network model, and carrying out digital region segmentation on a to-be-identified image based on the E-Net model; carrying out numeric character extraction on the segmented numeric region by utilizing a binarization method of an adaptive threshold; and carrying out denoising and tilt correction on the extracted characters, and carrying out character recognition by using an attention mechanism-based LeNet-5 network model to obtain a digital display meter recognition result. According to the scheme of the invention, high-precision identification of the reading of the digital electric meter in various complex environments is realized.

Description

Technical Field

[0001] The invention belongs to the field of digital display meter recognition, and in particular relates to a digital display meter recognition method and system based on an E-Net model and a LeNet-5 network. Background Art

[0002] Digital meters are a new type of electrical measuring instruments, which are different from analog meters in terms of measurement principle, instrument structure and operation method. In recent years, digital meters have been widely used in various industrial measurement and control fields due to their advantages such as high accuracy, high sensitivity, easy reading, easy setting and complete functions. Although the current high-end digital meters have wireless communication interfaces, in a considerable number of application environments, such as scientific experiments, measurement control, meter billing, etc., it is still necessary to manually record the meter readings and then transcribe them into the computer for processing. This method is not only time-consuming and labor-intensive, but also has low work efficiency and is prone to reading errors. Summary of the invention

[0003] In order to solve the deficiencies in the prior art, the present invention provides a digital meter recognition method and system based on an E-Net model and a LeNet-5 network to solve the problem of high-precision recognition of digital meter readings in a variety of complex environments.

[0004] In order to solve the above technical problems, the present invention adopts the following technical solutions.

[0005] The present invention first discloses a digital display meter recognition method based on an E-Net model and a LeNet-5 network, the method comprising the following steps: Step 1: Apply the bottleneck convolution structure to the E-Net network model, and perform digital region segmentation on the image to be recognized based on the E-Net model; Step 2: Use the adaptive threshold binarization method to extract digital characters from the segmented digital area; Step 3: De-noise and tilt-correct the extracted characters, and use the LeNet-5 network model based on the attention mechanism for character recognition to obtain the digital display recognition results.

[0006] The present invention further includes the following preferred embodiments: After applying the bottleneck convolution structure to the ENet network model, the structure of the ENet network model is three convolution layers; In the first convolutional layer, for the downsampling module, a convolution of size 2×2 and stride 2 is used; for the non-downsampling module, the first convolutional layer uses a 1×1 convolution with a stride of 1 for feature dimensionality reduction; The second convolutional layer contains a normal convolution with a kernel size of 3×3 and a stride of 1, a dilated convolution with dilation rates of 2, 4, 8, and 16, and an asymmetric convolution with a combination of 1×5 and 5×1; The third convolutional layer uses 1×1 convolution to achieve feature enhancement; Each convolutional layer contains BatchNorm and activation function PReLu layer.

[0007] The calculation formula of the activation function PReLu is: , in are learnable parameters.

[0008] if , then the PReLu activation function degenerates into the ReLu activation function; if is a fixed constant less than the preset value, the PReLu activation function degenerates into the Leaky ReLu activation function.

[0009] The method of using the adaptive threshold binarization method to extract digital characters from the segmented digital area further includes: Divide the grayscale into 5 categories, each containing 50 pixels, and define is the i-th cluster, is the minimum gray value of the initial image; Calculate the peak value corresponding to each cluster and define is the first peak, i.e., the background image. is the second peak, i.e. the foreground image; like ,but ,

[0010] like ,but , , ,

[0011] and are the ranges of the foreground area and the background area respectively, and is the maximum value of the gray levels of the two regions in this method; and yes , , Scope limitations; like , ,

[0012] otherwise , ,

[0013] like , , ,

[0014] in, It is a sign to enter the next gray level. is the number of pixels in cluster i; otherwise ,

[0015] like , , ,

[0016] otherwise ,

[0017] Thus, the adaptive threshold value is obtained; By using the adaptive threshold as the threshold for binarization processing, a binarized image can be obtained.

[0018] Preferably, the step of performing denoising and tilt correction on the extracted characters further comprises: The eight-neighborhood filtering method is used to filter the binary image to eliminate the salt and pepper noise. The characters are tilted and corrected using the rotational projection method.

[0019] Preferably, the character recognition using the LeNet-5 network model based on the attention mechanism further includes: Use global average pooling GAP to convert the data output of the dense layer from Zoom out to ; The specific formula is as follows: ,

[0020] in represents the global average pooling function; Perform two full connection operations in the network: ,

[0021] in, and Represent sigmoid and ReLu activation functions respectively; C is the dimension reduction coefficient; Then perform a scaling operation to modify the data to : .

[0022] The present invention also discloses a digital display meter recognition system based on an E-Net model and a LeNet-5 network using the aforementioned digital display meter recognition method based on an E-Net model and a LeNet-5 network, comprising: A digital region segmentation module, used for applying a bottleneck convolution structure to an ENet network model, and performing digital region segmentation on the image to be identified based on the E-Net model; A digital character extraction module is used to extract digital characters from the segmented digital area using a binarization method with an adaptive threshold; The character recognition module is used to denoise and tilt correct the extracted characters, and use the LeNet-5 network model based on the attention mechanism to perform character recognition to obtain the digital display recognition results.

[0023] Accordingly, the present application also discloses a terminal, including a processor and a storage medium; The storage medium is used to store instructions; The processor is used to operate according to the instructions to execute the steps of the digital display meter recognition method based on the aforementioned E-Net model and LeNet-5 network.

[0024] Accordingly, the present application also discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the aforementioned digital display meter recognition method based on the E-Net model and the LeNet-5 network.

[0025] The beneficial effect of the present invention is that, compared with the prior art, the present invention provides a method and system for digital display recognition based on the E-Net model and the LeNet-5 network, applies the inverted residual structure to the E-Net model for digital region segmentation, avoids the gradient vanishing problem, and extracts richer feature information. The adaptive threshold binarization method is used to extract digital characters, and the K-Means clustering technology is combined to determine the foreground and background areas of the image, which can effectively extract digital characters under various complex conditions such as complex factory conditions, uneven background images, and extremely poor quality. Combining the self-attention mechanism with the LeNet-5 network can improve the performance of the model, help the model better and more accurately capture important information in the input sequence, realize parallel computing, enhance context sensitivity, and achieve more accurate character and digital recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1It is a structural diagram of a network unit based on an E-Net model in the present invention.

[0027] Figure 2 This is the ENet network structure diagram based on the bottleneck convolution structure in the present invention.

[0028] Figure 3 It is a schematic diagram of the attention mechanism in the present invention.

[0029] Figure 4 It is a schematic diagram of the network adjustment of the spatial pyramid pooling structure in the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present invention more clear, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0031] The embodiments described in this application are only some embodiments of the present invention, not all embodiments. Based on the spirit of the present invention, other embodiments obtained by ordinary technicians in this field without creative work are all within the protection scope of the present invention.

[0032] In view of the shortcomings of the prior art, the present invention proposes a digital display meter recognition method and system based on an E-Net model and a LeNet-5 network, introduces a bottleneck convolution structure into the ENet network, performs digital region segmentation based on the improved lightweight E-Net model; performs digital character extraction according to adaptive threshold binarization; uses an eight-neighborhood filtering method for denoising, and performs tilt correction by a rotation projection method; combines a self-attention mechanism with a LeNet-5 network, and performs high-precision digital recognition based on the LeNet-5 network combined with the self-attention mechanism to extract richer information from the image, thereby increasing the accuracy of the recognition result while ensuring the recognition speed, and ultimately achieving high-precision recognition of digital meter readings in a variety of complex environments.

[0033] The digital display meter recognition method based on the E-Net model and the LeNet-5 network disclosed in the present invention comprises the following steps: Step 1: Apply the bottleneck convolution structure to the E-Net network model, and perform digital region segmentation on the image to be recognized based on the E-Net model.

[0034] ENet uses the current common encoder-decoder network structure and is a lightweight semantic segmentation network. Its network unit structure is as follows: Figure 1 As shown in the figure, through the convolution operation with a step size of 2 and the maximum pooling method, the input dimension is reduced in the initial stage to remove the redundant information in the image, and then the different image features are merged according to the specific dimension through the merging layer.

[0035] The present invention introduces a bottleneck convolution structure into the ENet network. The essence of this module is an improved residual structure based on ResNet, which is used for encoder and decoder. The specific structure is as follows: Figure 2 shown.

[0036] The main line of each structural unit contains three convolutional layers. The first convolutional layer is divided into two cases: for the downsampling module, such as Figure 2 As shown on the left, a convolution of size 2×2 and stride 2 is used. For the non-downsampling module, such as Figure 2 As shown on the right, a 1×1 convolution with a stride of 1 is used for feature dimensionality reduction. The second layer of convolution uses different convolution methods at different stages. It contains a normal convolution with a kernel size of 3×3 and a stride of 1, a dilated convolution with dilation rates of 2, 4, 8, and 16, and an asymmetric convolution with a combination of 1×5 and 5×1. The third layer uses a 1×1 convolution to achieve feature enhancement.

[0037] Each convolutional layer contains BatchNorm and activation function PReLu layer. There are also two cases for the auxiliary route. For the downsampling module, such as Figure 2 As shown on the left, first, the maximum pooling method is used to extract image features, and then 1×1 convolution is used to fill the channel to meet the size of residual fusion. For non-downsampling modules, such as Figure 2 As shown on the right, the residual is directly mapped identically. "+" means that the elements corresponding to two different input units are merged according to the summation strategy, and the fusion is connected to PReLu for activation. The negative part of PReLu has a fixed slope a, which can solve the problem of ReLu's gradient disappearance in the negative interval. The calculation formula of the activation function PReLu is as follows: ,

[0038] in is a learnable parameter that changes as the network is optimized. , then the PReLu activation function degenerates into the ReLu activation function. is a fixed constant less than the preset value, such as 0.01, then the PReLu activation function degenerates into the Leaky ReLu activation function.

[0039] Step 2: Use the adaptive threshold binarization method to extract digital characters from the segmented digital area.

[0040] Binarization is the process of converting a digital grayscale image into a binary image with grayscale values ​​of 0 and 1, and dividing the image into two parts: the target and the background. In order to solve the problems of unclear photos of the meter display bottle and reflection in the complex environment of the factory, the present invention adopts an adaptive threshold binarization method. The specific steps are as follows: The histogram of the image is calculated and a Gaussian window is used to get a smooth line graph before calculating the two peaks. Then, the background and foreground areas of the image are determined using K-Means clustering technique. The classification of K-Means clustering is to determine the location of the pixels in the two areas, which also helps to eliminate the shadows around the text. The image is then subjected to higher image processing based on the two-layer histogram to obtain a better and cleaner binary image. The specific steps are as follows: First, the grayscale is divided into 5 categories, each containing 50 pixels, and the definition is the i-th cluster, is the minimum gray value of the initial image.

[0041] Then, calculate the peak value corresponding to each cluster and define is the first peak, i.e., the background image. is the second peak, i.e. the foreground image.

[0042] like ,but ,

[0043] like ,but , (Indicates that the value range is ~ ), ,

[0044] and are the ranges of the foreground area and the background area respectively, and is the maximum value of the gray levels of the two regions in this method. and yes , , range limitation.

[0045] like , ,

[0046] otherwise , ,

[0047] like , , ,

[0048] in, It is a sign to enter the next gray level. is the number of pixels in cluster i.

[0049] otherwise ,

[0050] like , , ,

[0051] otherwise ,

[0052] Thus, the adaptive threshold value is obtained, and this adaptive threshold value is used as the threshold value for binary processing. ,

[0053] in is the original image pixel, is the value of the output image pixel.

[0054] Through the above steps, the bottleneck convolution structure is introduced into the E-Net model for digital area segmentation, and adaptive threshold binarization is used for digital character extraction. For images with complex factory conditions, uneven background images and extremely poor quality, accurate digital character extraction can be achieved.

[0055] Step 3: De-noise and tilt-correct the extracted characters, and use the LeNet-5 network model based on the attention mechanism for character recognition to obtain the digital display recognition results.

[0056] Specifically, the eight-neighborhood filtering method is used to filter the binary image to eliminate the salt and pepper noise, and then the rotation projection method is used to correct the tilt of the characters to prevent the recognition results from being affected.

[0057] Based on the traditional CNN model LeNet-5 structure, in order to maintain the running speed while reducing the loss of accuracy, an attention module is added to the network to make more accurate judgments on character recognition.

[0058] The attention mechanism helps to integrate different information to better understand the image model. Combining the characteristics of the LeNet-5 network, the attention module is added to put the pixel information into the grid. The grid resolution is improved by scaling the attention coefficient, and information of different scales is aggregated to complete the splicing of up-sampled and down-sampled information. Therefore, during the model training process, more attention information is mined and background information is suppressed. Figure 3 This is a schematic diagram of the attention mechanism.

[0059] like Figure 3 As shown, firstly, global average pooling (GAP) is used to transform the data output of the dense layer from Zoom out to The specific formula is as follows: ,

[0060] Among them, W is the image width, H is the image height, and K is the number of feature maps. represents the global average pooling function, is the input image of the Cth layer.

[0061] Then, two fully connected operations are performed in the network: ,

[0062] in, is the cth layer in z, Represents the first full connection operation function, Represents the second full connection operation function, and Represent the sigmoid and ReLu activation functions respectively. C is the dimension reduction coefficient, which can be adjusted according to the specific network.

[0063] Then perform a scaling operation to modify the data to : ,

[0064] is the cth layer in s. In terms of pooling, the present invention adds spatial pyramid pooling (ASPP) with extended convolutions of different expansion rates on the basis of spatial pyramid pooling (SSP). By combining global average pooling with image features, the network adjustment diagram is shown in Figure 4 shown.

[0065] Through the above steps, the attention mechanism is combined with the LeNet-5 network, which can improve the model performance, help the model better and more accurately capture the important information in the input sequence, improve the model's performance, restore the extracted digital characters to the greatest extent, and achieve high-precision character recognition.

[0066] The beneficial effect of the present invention is that, compared with the prior art, the present invention provides a method and system for digital display recognition based on the E-Net model and the LeNet-5 network, applies the inverted residual structure to the E-Net model for digital region segmentation, avoids the gradient vanishing problem, and extracts richer feature information. The adaptive threshold binarization method is used to extract digital characters, and the K-Means clustering technology is combined to determine the foreground and background areas of the image, which can effectively extract digital characters under various complex conditions such as complex factory conditions, uneven background images, and extremely poor quality. Combining the self-attention mechanism with the LeNet-5 network can improve the performance of the model, help the model better and more accurately capture important information in the input sequence, realize parallel computing, enhance context sensitivity, and achieve more accurate character and digital recognition.

[0067] The present invention may be a system, a method and / or a computer program product. The present invention also discloses a digital display meter recognition system based on an E-Net model and a LeNet-5 network based on the aforementioned digital display meter recognition method based on an E-Net model and a LeNet-5 network, comprising: A digital region segmentation module, used for applying a bottleneck convolution structure to an ENet network model, and performing digital region segmentation on the image to be identified based on the E-Net model; A digital character extraction module is used to extract digital characters from the segmented digital area using a binarization method with an adaptive threshold; The character recognition module is used to denoise and tilt correct the extracted characters, and use the LeNet-5 network model based on the attention mechanism to perform character recognition to obtain the digital display recognition results.

[0068] Based on the spirit of the present invention, those skilled in the art can easily think that a computer program product can be obtained based on the aforementioned digital display meter identification method based on the E-Net model and the LeNet-5 network. The computer program product may include a computer-readable storage medium, which carries computer-readable program instructions for enabling a processor to implement various aspects of the present disclosure. That is, the present application also includes a terminal, including a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of the aforementioned digital display meter identification method based on the E-Net model and the LeNet-5 network.

[0069] Computer readable storage medium can be a tangible device that can keep and store the instructions used by the instruction execution device. Computer readable storage medium can be, for example, - but not limited to - electrical storage device, magnetic storage device, optical storage device, electromagnetic storage device, semiconductor storage device or any suitable combination of the above. More specific examples (non-exhaustive list) of computer readable storage medium include: portable computer disk, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device, such as punch card or groove protrusion structure with instructions stored thereon and any suitable combination of the above. Computer readable storage medium used here is not interpreted as instantaneous signal itself, such as radio wave or other free propagating electromagnetic wave, electromagnetic wave propagated by waveguide or other transmission medium (for example, light pulse by optical fiber cable) or electrical signal transmitted by wire.

[0070] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0071] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect through the Internet). In some embodiments, by using the state information of the computer-readable program instructions to personalize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA) or a programmable logic array (PLA), the electronic circuit may execute the computer-readable program instructions, thereby implementing various aspects of the present disclosure.

[0072] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A digital display meter recognition method based on E-Net model and LeNet-5 network, characterized in that: The following steps are involved: Step 1: Apply the bottleneck convolution structure to the E-Net network model, and perform digital region segmentation on the image to be recognized based on the E-Net model; Step 2: Use the adaptive threshold binarization method to extract digital characters from the segmented digital area; Step 3: De-noise and tilt-correct the extracted characters, and use the LeNet-5 network model based on the attention mechanism for character recognition to obtain the digital display recognition results.

2. The digital display meter recognition method based on the E-Net model and the LeNet-5 network according to claim 1, characterized in that: After applying the bottleneck convolution structure to the ENet network model, the structure of the ENet network model is three convolution layers; In the first convolutional layer, for the downsampling module, a convolution of size 2×2 and stride 2 is used; for the non-downsampling module, the first convolutional layer uses a 1×1 convolution with a stride of 1 for feature dimensionality reduction; The second convolutional layer contains a normal convolution with a kernel size of 3×3 and a stride of 1, a dilated convolution with dilation rates of 2, 4, 8, and 16, and an asymmetric convolution with a combination of 1×5 and 5×1; The third convolutional layer uses 1×1 convolution to achieve feature enhancement; Each convolutional layer contains BatchNorm and activation function PReLu layer.

3. The digital display meter recognition method based on the E-Net model and the LeNet-5 network according to claim 2, characterized in that: The calculation formula of the activation function PReLu is: ， in are learnable parameters.

4. The digital display meter recognition method based on the E-Net model and the LeNet-5 network according to claim 3, characterized in that: if , then the PReLu activation function degenerates into the ReLu activation function; if is a fixed constant less than the preset value, the PReLu activation function degenerates into the Leaky ReLu activation function.

5. The digital display meter recognition method based on the E-Net model and the LeNet-5 network according to claim 4, characterized in that: The method of using the adaptive threshold binarization method to extract digital characters from the segmented digital area further includes: Divide the grayscale into 5 categories, each containing 50 pixels, and define is the i-th cluster, is the minimum gray value of the initial image; Calculate the peak value corresponding to each cluster and define is the first peak, i.e., the background image. is the second peak, i.e. the foreground image; like ,but , like ,but ， ， , and are the ranges of the foreground area and the background area respectively, and is the maximum value of the gray levels of the two regions in this method; and yes , , Scope limitations; like , , otherwise ， , like , ， in, It is a sign to enter the next gray level. is the number of pixels in cluster i; otherwise ， like , ， otherwise ， Thus, the adaptive threshold value is obtained; By using the adaptive threshold as the threshold for binarization processing, a binarized image can be obtained.

6. The digital display meter recognition method based on the E-Net model and the LeNet-5 network according to claim 5, characterized in that: The step of performing denoising and tilt correction on the extracted characters further comprises: The eight-neighborhood filtering method is used to filter the binary image to eliminate the salt and pepper noise. The characters are tilted and corrected using the rotational projection method.

7. The digital display meter recognition method based on the E-Net model and the LeNet-5 network according to claim 6, characterized in that: The character recognition using the LeNet-5 network model based on the attention mechanism further includes: Use global average pooling GAP to convert the data output of the dense layer from Zoom out to ; The specific formula is as follows: ， in represents the global average pooling function; Perform two full connection operations in the network: ， in, and Represent sigmoid and ReLu activation functions respectively; C is the dimension reduction coefficient; Then perform a scaling operation to modify the data to : 。 8. A digital display meter recognition system based on E-Net model and LeNet-5 network, characterized in that: include: A digital region segmentation module, used for applying a bottleneck convolution structure to an ENet network model, and performing digital region segmentation on the image to be identified based on the E-Net model; A digital character extraction module is used to extract digital characters from the segmented digital area using a binarization method with an adaptive threshold; The character recognition module is used to denoise and tilt correct the extracted characters, and use the LeNet-5 network model based on the attention mechanism to perform character recognition to obtain the digital display recognition results.

9. A terminal comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is used to operate according to the instructions to execute the steps of the digital display meter recognition method based on the E-Net model and the LeNet-5 network according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the digital display meter recognition method based on the E-Net model and the LeNet-5 network described in any one of claims 1 to 7 are implemented.

Images