Image recognition method for electric energy meter recognition

Abstract

The invention discloses an image recognition method for electric energy meter recognition. The method comprises the steps that an image with a metering electric energy meter display screen is acquired; preprocessing the image for correcting light reflection distortion of the image; performing text detection, and identifying and positioning a text region of the preprocessed image; character information identification of the concerned text area is carried out; performing layout analysis, and identifying digital information corresponding to different elements in the layout; performing NER detection filtering to obtain entity data in the image; and displaying the entity data. According to the method, the image is preprocessed for image reflection distortion correction, so that the image recognition result is more accurate. According to the method, the Fast R-CNN method is adopted for layout analysis, and different elements such as characters, charts and pictures in the layout can be quickly and accurately positioned and classified.

Description

Technical Field

[0001] The invention belongs to the technical field of image recognition, and in particular relates to an image recognition method for electric energy meter recognition. Background Art

[0002] Traditional meter recognition methods mainly rely on manual meter reading and simple digital image processing technology. Although these methods have been widely used, they have problems such as low efficiency, low accuracy and high labor costs.

[0003] Manual meter reading involves a meter reader who regularly visits each customer's home or business to manually read the numbers on the meter and record them. The meter reader goes to the customer's meter location and observes the reading on the meter. Mechanical meters typically have a rotating needle or scrolling digital display. The meter reader manually records the observed reading in a meter book or on a mobile device. This step is susceptible to human error, such as reading the wrong number or missing a reading. The manually recorded data is entered into the company's billing system. This often requires converting paper records to an electronic format, further increasing the potential for error.

[0004] With the development of science and technology, image recognition technology has begun to be applied to the automated processing of meter readings. Traditional meter image recognition mainly includes the following steps: Use a camera device (such as a mobile phone camera, a dedicated camera device) to capture an image of the meter. Image quality directly affects the accuracy of subsequent recognition, so it is necessary to ensure that the image is clear, without reflections and occlusions. Image preprocessing: Preprocess the acquired image, including grayscale, binarization, denoising, and edge detection. Grayscale converts color images into grayscale images, simplifying the processing difficulty; binarization converts grayscale images into images with only black and white colors, thereby highlighting the numbers on the meter. Digital region segmentation: Use projection analysis, connected domain analysis, and other methods to separate the digital area on the meter from the background. Projection analysis finds the boundaries of the digital area by analyzing the horizontal and vertical projections of the image; connected domain analysis identifies the connected pixel areas in the image. Summary of the invention

[0005] In order to solve the problems existing in the prior art, the present invention proposes an image recognition method for electric energy meter recognition.

[0006] The technical solution of the present invention is as follows: The technical solution of the present invention is as follows: An image recognition method for electric energy meter recognition, comprising: Step 1) acquiring an image with a display screen of a metering electric energy meter; Step 2) preprocessing the image for image reflection distortion correction; Step 3) Perform text detection to identify and locate the text area of ​​the preprocessed image; Step 4) recognizing text information in the text area; Step 5) Perform layout analysis to identify digital information corresponding to different elements in the layout; Step 6) Perform NER detection and filtering to obtain entity data in the image; Step 7) Displaying the entity data.

[0007] Furthermore, the specific method of step 2) includes: 2-1) Standardize the image to a standardized image size; 2-2) grayscale processing is performed on the image after the standardization process to convert the color image into a grayscale image; 2-3) Binarization is performed on the grayscale image to convert the grayscale image into a black and white image; 2-4) Performing noise removal on the binarized image to eliminate interference information in the image; 2-5) Performing edge detection on the image after noise removal to identify contours and key areas in the image; 2-6) Perform image reflection distortion correction on the image after edge detection.

[0008] Furthermore, the specific method of step 3) is as follows: Use the pre-trained CNN convolutional neural network combined with the Canny edge detection algorithm to perform text detection and identify and locate the text area of ​​the pre-processed image; Furthermore, the CNN convolutional neural network automatically adjusts the network weights through back propagation and optimization algorithms to minimize the gap between the predicted results and the true labels. The specific training method is as follows: A large number of images with electric energy meter display screens are obtained as samples to form a sample data set; the sample data set includes various types of electric energy meters and electric energy meter measurement texts, so that the CNN convolutional neural network can learn different writing styles, fonts, lighting conditions, background complexity, etc.; The samples are preprocessed for image reflection distortion correction and data augmentation is performed on the samples to increase the generalization ability of the network model, making it suitable for subsequent analysis steps; Label the text regions of the samples in the sample data set and obtain the labels corresponding to each sample in the sample data set; The CNN convolutional neural network is trained with the label corresponding to each sample in the sample data set until the set requirements are met.

[0009] Furthermore, the Canny edge detection algorithm includes: Gaussian filtering: First apply a Gaussian filter to smooth the image; Edge detection: Calculate the gradient strength and direction of the preprocessed image in both the x and y directions; Non-maximum suppression: Apply a local maximum test around each possible edge point, retain the local maximum, remove the local minimum and local minimum to eliminate redundant edge responses; Dual threshold detection: Based on the calculated gradient strength, two preset thresholds are used to retain strong edges and weak edges respectively; Chain structure connection: Edge chains are formed by connecting adjacent edge points, removing isolated points and short chains, and retaining long chains to determine the final edge.

[0010] Furthermore, the specific method of step 4) includes: The SAN spatial attention mechanism is used to weight the text area and extract text information; the STAR-Net network architecture is used to process the spatial deformation of the text area.

[0011] Furthermore, the specific method of step 5) includes: Use the Fast R-CNN network model to locate and classify different elements in the layout; First, the board features are extracted: a pre-trained convolutional neural network is used as the basic network to extract the board features of the entire input board image; Then perform region proposal: Generate possible object bounding boxes through the Region Proposal Network sub-network; Then, feature vector extraction and classification are performed: for each region, a fixed-size convolution pooling operation is performed on it to extract local features, namely feature vectors; the extracted local features are then sent to the fully connected layer for classification and bounding box adjustment; through the softmax function, the model predicts the probability of each region belonging to a different category, and at the same time fine-tunes the position of the bounding box through the regression network; and converts different regions into feature vectors; Finally, digital recognition is performed: based on the extraction and classification of feature vectors, the pattern recognition algorithm is used to classify the extracted feature vectors to identify the numbers.

[0012] Furthermore, the entity data includes power consumption, number, power consumption period and barcode.

[0013] Compared with the prior art, the present invention has the following beneficial effects: The present invention proposes an image recognition method for electric energy meter identification. The method solves the problems of a large number of overexposure and reflection problems in images obtained by existing methods by preprocessing the images for image reflection distortion correction, thereby making the image recognition results of the present invention more accurate.

[0014] The method of the present invention adopts the Fast R-CNN method to perform layout analysis, which can quickly and accurately locate and classify different elements in the layout, such as text, charts, pictures, etc., improve the efficiency of technicians and make image recognition results more accurate.

[0015] The method of the present invention uses artificial intelligence OCR (Optical Character Recognition) technology to realize intelligent recognition, intelligent extraction, and intelligent maintenance of metering information in various metering business application scenarios of the power system, which can effectively reduce the work pressure of business personnel, improve human resource efficiency, and free business personnel from tedious data entry and verification work, so that they can devote themselves to more meaningful work. The method provides users with intelligent collection of unstructured data, improves the collection, management, analysis and utilization mechanism of business data, and further promotes the advancement of business management from "accurate reflection" and "lean management" to "digital management". BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 The figure is a flow chart of an image recognition method for electric energy meter recognition. DETAILED DESCRIPTION

[0017] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent modifications of the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0018] Example: An image recognition method for electric energy meter recognition of the present invention is as follows: Figure 1 As shown, including: Step 1) acquiring an image with a display screen of a metering electric energy meter; Step 2) preprocessing the image for image reflection distortion correction; Step 3) Perform text detection on the preprocessed image to identify and locate the text area of ​​the preprocessed image; Step 4) Recognize text information in the text area of ​​the preprocessed image; Step 5) Perform layout analysis to identify digital information corresponding to different elements in the layout; Step 6) Perform NER detection and filtering to obtain entity data in the image; Step 7) Display entity data.

[0019] Furthermore, the specific method of step 2) includes: 2-1) Standardize the image to a standardized image size; 2-2) Converting the color image into a grayscale image after the standardization process to simplify the subsequent processing steps; 2-3) Binarization is performed on the grayscale image to convert the grayscale image into a black and white image, thereby highlighting the digital area; 2-4) Performing noise removal on the binarized image to eliminate interference information in the image; 2-5) Performing edge detection on the image after noise removal to identify contours and key areas in the image; 2-6) Perform image reflection distortion correction on the image after edge detection.

[0020] In the process of collecting electricity meter images, common shooting methods and images will have common collection and shooting problems such as overexposure and reflection. If image preprocessing is not performed, the accuracy of characters will be affected in the field recognition stage.

[0021] The following is a description of the code for the distortion processing: # Define camera calibration parameters (example) # Note: In actual use, you need to calibrate the camera according to the specific situation and obtain appropriate parameters # The following parameters are for example only. The actual parameters need to be adjusted according to the actual situation. K = np.array([[1000, 0, 320], [0, 1000, 240], [0, 0, 1]]) # internal parameter matrix D = np.array([0.01, -0.02, 0.005, -0.001, 0.01]) # Distortion coefficient # Image size height, width = image.shape[:2] # Correct distortion new_camera_matrix, roi = cv2.getOptimalNewCameraMatrix(K, D, (width,height), 1, (width, height)) undistorted_image = cv2.undistort(image, K, D, None, new_camera_matrix) # Display the original image and the rectified image cv2.imshow('Original Image', image) cv2.imshow('Undistorted Image', undistorted_image) cv2.waitKey(0) cv2.destroyAllWindows() After preliminary image preprocessing, the direction of the text is corrected.

[0022] Furthermore, the specific method of step 3) is as follows: Use the pre-trained CNN convolutional neural network combined with the Canny edge detection algorithm to perform text detection and identify and locate the text area of ​​the pre-processed image; Further, CNN (Convolutional Neural Network): CNN is a commonly used model in deep learning, especially good at processing image data. In text detection, CNN is usually used as a feature extractor. It learns abstract features related to text detection, such as edges, corners, textures, and shapes, from raw pixel data through a series of convolutional layers, pooling layers, and fully connected layers.

[0023] CNN convolutional neural network automatically adjusts network weights through back propagation and optimization algorithms (such as gradient descent) to minimize the gap between the predicted results and the true labels. The specific training method is as follows: A large number of images with electric energy meter display screens are obtained as samples to form a sample data set; the sample data set contains various types of electric energy meters and electric energy meter metering text, so that the CNN convolutional neural network can learn different writing styles, fonts, lighting conditions, and background complexity; Preprocess the samples for image reflection distortion correction and perform data augmentation (such as rotation, scaling, translation, etc.) on the samples to increase the generalization ability of the network model and make it suitable for subsequent analysis steps; Label the text regions of the samples in the sample data set and obtain the labels corresponding to each sample in the sample data set; The CNN convolutional neural network is trained with the label corresponding to each sample in the sample data set until the set requirements are met.

[0024] The trained CNN model is able to quickly and accurately extract features from the input image, which provides key information for subsequent text region localization.

[0025] Furthermore, Canny edge detection is a classic edge detection algorithm, which is used to extract obvious edges from an image, and these edges usually correspond to the edges of objects or the outlines of text in the image.

[0026] The Canny edge detection algorithm includes: Gaussian filtering: First, a Gaussian filter is applied to smooth the image, reduce noise interference, and retain edge information; Edge detection: Calculate the gradient strength and direction of the preprocessed image in both the x and y directions. This step usually uses the Sobel or Laplacian operator; Non-maximum suppression: In order to eliminate redundant edge responses, the algorithm applies a local maximum test around each possible edge point, retains the local maximum, removes the local minimum and local minimum, to eliminate redundant edge responses; Dual threshold detection: Based on the calculated gradient strength, two preset thresholds are used to retain strong edges and weak edges respectively; Chain structure connection: Edge chains are formed by connecting adjacent edge points, removing isolated points and short chains, and retaining long chains to determine the final edge.

[0027] In text detection, the Canny edge detection algorithm is applied on the output of CNN to locate the text area. After that, further steps such as connected component analysis and text line or word segmentation can be performed to accurately segment the text from the edge information.

[0028] Furthermore, the specific method of step 4) includes: The SAN (Spatial Attention) spatial attention mechanism is used to weight text areas and extract text information to improve recognition accuracy. For example, the model will learn to focus on text areas while ignoring background noise or other non-text areas. This is usually achieved through attention modules, such as self-attention mechanisms or position-based attention, to help the network focus more when processing information; the STAR-Net (Spatial Transformation-Aware Representation Network) network architecture is used to handle spatial deformation of text areas. The shape of text in an image may change due to changes in camera angle, lighting conditions, font size, or image resolution. The goal of STAR-Net design is to learn a representation that can robustly handle these spatial transformations and enhance the model's recognition capabilities under different text shapes and sizes.

[0029] Furthermore, the specific method of step 5) includes: Use the Fast R-CNN network model to locate and classify different elements in the layout, such as text, charts, pictures, etc. Fast R-CNN is a deep learning method for object detection that combines the advantages of region proposal and convolutional neural network. In layout analysis, the task of Fast R-CNN is mainly to locate and classify different elements in the layout, such as text, graphics, pictures, etc. The specific steps include: First, the board features are extracted: a pre-trained convolutional neural network (such as VGG-Net, ResNet, etc.) is used as the basic network to extract the board features of the entire input board image; the basic network can recognize various low-level to high-level features in the image, such as edges, colors, textures, and shapes.

[0030] Then make region proposals: Unlike its predecessor R-CNN, which needs to generate candidate boxes at every possible location, Fast R-CNN generates possible object bounding boxes through a sub-network called Region Proposal Network (RPN), which greatly improves efficiency; Then, feature vector extraction and classification are performed: for each region, Fast R-CNN performs a fixed-size convolution pooling operation on it to extract local features, namely feature vectors; the extracted local features are sent to the fully connected layer for classification and bounding box adjustment; through the softmax function, the model predicts the probability of each region belonging to different categories, and fine-tunes the position of the bounding box through the regression network; the different regions are converted into feature vectors; feature extraction is to convert each segmented digital region into a feature vector that can be recognized by the computer. These features can be geometric features, such as the height, width, area, etc. of the number, or shape features, such as contours, corners, etc. Common feature extraction methods include HOG (Histogram of Oriented Gradients) and SIFT (Scale-Invariant Feature Transform). Geometric feature extraction focuses on the basic geometric properties of characters, including size, position, and scale. These features are simple and effective, and are commonly used by early OCR systems. Height and width: The height and width of a character are the most basic geometric features. These features can help distinguish the size and shape of characters. For example, the character "1" is usually much narrower than the character "0", and the character "W" is wider than the character "V". Area: The area of ​​a character is the total number of pixels in the character. This feature is helpful for identifying the thickness of the character and the size of the space it occupies. Bounding Box: The minimum enclosing rectangle of a character, that is, the minimum rectangular area covering the character. By analyzing the aspect ratio of the bounding box, certain characters can be effectively distinguished. For example, "L" usually has a larger aspect ratio, while "O" is close to 1. Center Point: The center of gravity or centroid of a character, which is the average position of all the pixels of the character. This feature helps to understand the position distribution and symmetry of characters in the image.

[0031] Finally, digital recognition is performed: based on feature vector extraction and classification, the extracted feature vectors are classified using pattern recognition algorithms to identify numbers; commonly used algorithms include support vector machines (SVM), neural networks, and convolutional neural networks (CNN). These algorithms can learn digital patterns through training on a large number of samples, thereby accurately identifying numbers in new images.

[0032] Furthermore, in the NER detection and filtering in step 6), in the captured meter image, the recognized text contains multiple entities, such as numbers, time, organization, product, special terminology, unit, etc. The content to be extracted includes the meter's electricity consumption, barcode, number, and electricity consumption period. The acquired text data is subjected to named entity detection, and the extracted content is summarized uniformly. Unused named entities are excluded, and valid entities such as units, barcodes, and time are selected as the correctness check of the extraction results. The accuracy of meter recognition is improved; entity data includes electricity consumption, number, electricity consumption period, and barcode.

[0033] The above description is only a specific implementation of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a technician familiar with the technical field within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention.

Claims

1. An image recognition method for electric energy meter recognition, characterized in that: include: Step 1) acquiring an image with a display screen of a metering electric energy meter; Step 2) preprocessing the image for image reflection distortion correction; Step 3) Perform text detection to identify and locate the text area of ​​the preprocessed image; Step 4) recognizing text information in the text area; Step 5) Perform layout analysis to identify digital information corresponding to different elements in the layout; Step 6) Perform NER detection and filtering to obtain entity data in the image; Step 7) Displaying the entity data.

2. The image recognition method for electric energy meter recognition according to claim 1, characterized in that: The specific methods of step 2) include: 2-1) Standardize the image to a standardized image size; 2-2) grayscale processing is performed on the image after the standardization process to convert the color image into a grayscale image; 2-3) Binarization is performed on the grayscale image to convert the grayscale image into a black and white image; 2-4) Performing noise removal on the binarized image to eliminate interference information in the image; 2-5) Performing edge detection on the image after noise removal to identify contours and key areas in the image; 2-6) Perform image reflection distortion correction on the image after edge detection.

3. The image recognition method for electric energy meter recognition according to claim 2, characterized in that: The specific method of step 3) is as follows: The pre-trained CNN convolutional neural network is combined with the Canny edge detection algorithm to perform text detection and identify and locate the text area of ​​the preprocessed image.

4. The image recognition method for electric energy meter recognition according to claim 3, characterized in that: The CNN convolutional neural network automatically adjusts the network weights through back propagation and optimization algorithms to minimize the gap between the predicted results and the true labels. The specific training method is as follows: A large number of images with electric energy meter display screens are obtained as samples to form a sample data set; the sample data set includes various types of electric energy meters and electric energy meter measurement texts, so that the CNN convolutional neural network can learn different writing styles, fonts, lighting conditions, background complexity, etc.; The samples are preprocessed for image reflection distortion correction and data augmentation is performed on the samples to increase the generalization ability of the network model, making it suitable for subsequent analysis steps; Label the text regions of the samples in the sample data set and obtain the labels corresponding to each sample in the sample data set; The CNN convolutional neural network is trained with the label corresponding to each sample in the sample data set until the set requirements are met.

5. The image recognition method for electric energy meter recognition according to claim 4, characterized in that: The Canny edge detection algorithm includes: Gaussian filtering: First apply a Gaussian filter to smooth the image; Edge detection: Calculate the gradient strength and direction of the preprocessed image in both the x and y directions; Non-maximum suppression: Apply a local maximum test around each possible edge point, retain the local maximum, remove the local minimum and local minimum to eliminate redundant edge responses; Dual threshold detection: Based on the calculated gradient strength, two preset thresholds are used to retain strong edges and weak edges respectively; Chain structure connection: Edge chains are formed by connecting adjacent edge points, removing isolated points and short chains, and retaining long chains to determine the final edge.

6. The image recognition method for electric energy meter recognition according to claim 5, characterized in that: The specific methods of step 4) include: The SAN spatial attention mechanism is used to weight the text region and extract text information; the STAR-Net network architecture is used to process the spatial deformation of the text region.

7. The image recognition method for electric energy meter recognition according to claim 6, characterized in that: The specific methods of step 5) include: Use the Fast R-CNN network model to locate and classify different elements in the layout; First, the board features are extracted: a pre-trained convolutional neural network is used as the basic network to extract the board features of the entire input board image; Then perform region proposal: Generate possible object bounding boxes through the Region Proposal Network sub-network; Then, feature vector extraction and classification are performed: for each region, a fixed-size convolution pooling operation is performed on it to extract local features, namely feature vectors; the extracted local features are then sent to the fully connected layer for classification and bounding box adjustment; through the softmax function, the model predicts the probability of each region belonging to a different category, and at the same time fine-tunes the position of the bounding box through the regression network; and converts different regions into feature vectors; Finally, digital recognition is performed: based on the extraction and classification of feature vectors, the pattern recognition algorithm is used to classify the extracted feature vectors to identify the numbers.

8. The image recognition method for electric energy meter recognition according to claim 7, characterized in that: The entity data includes power consumption, number, power consumption period and barcode.

Images