A method and device for identifying information during meter rotation process
By using technologies such as YOLO object detection and convolutional neural circulation network during the meter rotation process, the problem of low accuracy of meter information recognition in complex environments is solved, and fast and accurate information recognition and efficient meter rotation operation are achieved.
Patent Information
- Application Number
- CN202111595568.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-23
- Publication Date
- 2025-04-15
- Estimated Expiration
- 2041-12-23
AI Technical Summary
The prior art has low accuracy and slow speed for identifying meter information in complex environments, which cannot effectively solve the problem of errors in manual reading of information during meter rotation.
The information recognition method for the meter rotation process is adopted, and a specific information area is located through the YOLO object detection positioning algorithm, image preprocessing and text information positioning are performed, and the information is converted into horizontal single-line text, and the improved convolutional neural recurrent network is used for identification.
Quickly and accurately position specific information areas in the meter image in complex environments, improving the recognition speed and accuracy, reducing the probability of manual errors, and improving the efficiency of meter rotation operations.
Smart Images
Figure CN114241469B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of power system and computer image processing, and more specifically, to an information recognition method and device for an electric meter rotation process. Background Art
[0002] With the advancement of science and technology and the development of the power industry, every household uses smart meters to measure electricity consumption. In order to ensure the accuracy of the energy meter measurement, the power company will regularly replace the energy meter according to national regulations, that is, the meter rotation. The traditional practice is that the construction personnel take the energy meter image and manually enter the energy meter's power consumption, asset number, meter number and other information into the system by naked eye observation. Due to the influence of environmental factors, the energy meter is located in different positions, some energy meters are too high and too biased, and the meter screen has stains and scratches, which makes this meter information entry method not only consume a lot of manpower and time, but also cannot guarantee the accuracy of the data, and there are a certain degree of missed detection and reading errors. The existing method uses infrared transmitting and receiving tubes, Bluetooth or USB communication, but to a certain extent, it needs to rely on too much hardware equipment support and has certain operating specifications. Therefore, the knowledge of computer image processing related fields can be combined to adopt the method of software algorithm design to realize the reading of information. In the recognition of meter readings based on traditional image processing methods, information areas are often obtained through image preprocessing methods such as binarization, edge detection, Hough line detection and morphological operations, and then characters are segmented through methods such as projection and minimum enclosing rectangles. Finally, template matching or threading methods are used for character recognition. These methods are not robust enough and have insufficient generalization capabilities in complex environments, and the recognition accuracy and speed are not ideal. In image processing methods based on deep learning, the accuracy of classic target detection networks is acceptable when the shooting angle is relatively correct and there is no excessive interference. When these algorithms are directly applied to the detection of meter reading areas, the effect is not ideal, because compared with conventional objects, the length and width ratio of the meter reading area varies greatly and has directionality; at the same time, when uneven lighting, exposure shadows, and shooting problems cause the LCD screen area image to tilt and deform, the horizontal text box generated by these algorithms cannot accurately surround the target reading area, which has a serious impact on subsequent character segmentation and recognition, and the accuracy will be greatly reduced.
[0003] The prior art discloses a method for identifying the value of an intelligent meter based on R-CNN, which is offline training, collects a large number of meter images taken by a camera, and preprocesses these meter image samples by an image preprocessing method, and then adds labels to the samples for network learning, wherein the labels are set to the reading values of the dial, and a network for identifying the dial is obtained; online prediction, which includes extracting the meter dial position area, firstly generating a series of candidate areas by a selective search algorithm; the recognition process is to input the scaled image of each candidate area into the R-CNN network for feature extraction, and then identify whether the candidate area image is the target area, i.e., the meter dial area, through the SVM network, identify the dial information through a multi-layer R-CNN network and a fully connected network, and output the final recognized meter reading. When the method is offline trained, a large number of meter images taken by a camera need to be collected, which is inefficient; when predicting, the dial position area extraction cannot accurately surround the target reading area, which has a serious impact on the subsequent character segmentation and recognition, and the accuracy rate will be greatly reduced. Summary of the invention
[0004] In order to overcome the defects of the above-mentioned prior art in low recognition accuracy and slow recognition speed in complex environments, the present invention provides an information recognition method and device for the meter rotation process, which can quickly and accurately locate the specific information area in the meter image in a complex environment and recognize the corresponding text to obtain specific information, reduce the probability of errors in manual reading of information during the meter rotation operation, and improve the operation efficiency.
[0005] In order to solve the above technical problems, the technical solution of the present invention is as follows:
[0006] The present invention provides an information identification method for an electric meter rotation process, comprising:
[0007] S1: Acquire the electric meter image;
[0008] S2: Locate a specific information area in the meter image;
[0009] S3: performing image preprocessing on the specific information region to obtain a preprocessed image of the specific information region;
[0010] S4: preprocessing the image based on the specific information region, locating the text information in the specific information region, and obtaining the text information region;
[0011] S5: Processing the text information area to obtain a horizontal single line of text;
[0012] S6: Recognize a horizontal single line of text to obtain specific information.
[0013] The present invention first obtains an image of an old meter that needs to be replaced, and locates the required specific information area; performs image preprocessing on the specific information area to eliminate the influence of complex environmental factors; then locates the text information in the specific information area to obtain the text information area, and then processes the area to convert it into a horizontal single-line text; finally, recognition is performed based on the horizontal single-line text, with a fast recognition speed and high accuracy of the recognized specific information.
[0014] Preferably, in step S2, a YOLO target detection and positioning algorithm is used to locate a specific information area in the meter image; the specific process is as follows:
[0015] S2.1: adjust the size of the meter image;
[0016] S2.2: Based on the resized electricity meter image, upsampling is performed using the Darknet-53 network to obtain three feature maps with different resolutions, namely the first feature map, the second feature map, and the third feature map;
[0017] S2.3: Perform feature fusion on the first feature map, the second feature map, and the third feature map to obtain a first fused feature map;
[0018] S2.4: Generate multiple anchor boxes on the first fused feature map;
[0019] The anchor box includes confidence and 4 predicted values, which are respectively the predicted value of the center coordinate of the anchor box, the horizontal coordinate, and the predicted value t x , vertical coordinate prediction value of anchor box center coordinate t y , anchor box width prediction value t w And the anchor box height prediction value t h ;
[0020] S2.5: Obtain specific information area;
[0021] Calculate the confidence score of the anchor box, which is equal to the category conditional probability multiplied by the confidence; sort the confidence scores of all anchor boxes according to the preset category of specific information, the category of the anchor box with the highest confidence score corresponds to the category of specific information, and the anchor box area is the specific information area.
[0022] The YOLO target detection and positioning algorithm is used to locate the specific information area. First, the meter image is adjusted to the same size. The Darknet-53 network is used as the backbone feature extraction part to upsample the meter image to obtain three feature maps with different resolutions. The resolution of the first feature map is 1 / 32 of the meter image, the resolution of the second feature map is 1 / 16 of the meter image, and the resolution of the third feature map is 1 / 8 of the meter image. Then, the three feature maps are feature fused to obtain the first fused feature map, in order to improve the detection effect of multi-scale targets. Multiple anchor frames are generated on the first fused feature map, and the number of anchor frames is equal to the number of required specific information regions. Each anchor frame includes confidence and 4 predicted values, and the predicted values are the center coordinates, horizontal coordinates, predicted values t, and t of the anchor frame. x , center coordinate vertical coordinate predicted value t y , width prediction value t w and the predicted height t h When predicting the category of the anchor box, the confidence score of a category is equal to the category conditional probability multiplied by the confidence score. The category with the largest confidence score among all categories is taken as the predicted category, which corresponds to the category of specific information. At this time, the anchor box area composed of the predicted values is the specific information area.
[0023] Preferably, the specific information area includes a power consumption area, an asset number area and a meter serial number area; the specific information includes power consumption, asset number and meter serial number.
[0024] The information that needs to be recorded during the meter rotation operation includes power consumption, asset number and meter serial number. The above information is set as specific information, and the area where the above information is located is the specific information area.
[0025] Preferably, in step S3, the image preprocessing performed on the specific information area includes grayscale, Gaussian blur and histogram equalization operations.
[0026] Gaussian blur is used to remove image noise, and histogram equalization is used to enhance image contrast. Image preprocessing can effectively remove the interference of environmental factors.
[0027] Preferably, in step S4, the text information in the specific information area is located using an improved differentiable binarization network, and the specific method for obtaining the text information area is:
[0028] S4.1: extracting features from the preprocessed image in the specific information region to obtain feature maps of four different scales, namely, a fourth feature map, a fifth feature map, a sixth feature map, and a seventh feature map;
[0029] S4.2: upsampling the fourth feature map, the fifth feature map, the sixth feature map and the seventh feature map, and then concatenating them to achieve multi-scale information fusion, thereby obtaining a second fused feature map;
[0030] S4.3: Adaptively binarize each pixel on the second fused feature map to obtain a probability map and a threshold map, and calculate a binarized map based on the probability map and the threshold map. The formula is:
[0031]
[0032] In the formula, Represents the pixel point in the i-th row and j-th column of the binary image, P ij represents the pixel in the i-th row and j-th column on the probability map, T ij represents the pixel at the i-th row and j-th column on the threshold map, and k represents the coefficient factor;
[0033] S4.4: Based on the binary image, the coordinates of the upper left point, the upper right point, the lower right point and the lower left point in the image are obtained. The area formed by the four vertices is the text information area.
[0034] The improved differentiable binarization network includes an EfficientNet backbone feature extraction module, a BiFPN feature fusion module and a post-processing module connected in sequence; the EfficientNet backbone feature extraction module consists of 5 MBConv convolution blocks, each of which includes a 1×1 convolution layer, a deep convolution layer and a point convolution layer connected in sequence, and an SE attention mechanism block is added to the deep convolution layer and the point convolution layer; the 1×1 convolution layer is used for dimensionality increase operation, and the feature map is output after a deep separable convolution composed of a deep convolution layer and a point convolution layer. The added SE attention mechanism block can increase the learning proportion of important channel features, reduce the proportion of unimportant features, and improve the accuracy of training results; the EfficientNet backbone feature extraction module outputs a total of 4 feature maps of different scales, and the scales of the fourth feature map, the fifth feature map, the sixth feature map and the seventh feature map are respectively It is 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the preprocessed image of the specific information area; the feature maps of 4 different scales are input into the BiFPN feature fusion module, and the second fused feature map obtained after upsampling and splicing the output fully integrates the features of each feature map, realizes the fusion of multi-scale information, and effectively improves the detection effect of text information areas of different scales; the post-processing module adaptively binarizes each pixel on the second fused feature map, predicts the probability map and the threshold map, and then uses the sigmoid function with coefficients to calculate the approximate binarization map, and determines the coordinates of the upper left point, the upper right point, the lower right point and the lower left point on the binarization map. The area formed by the four vertices is the text information area; the improved differentiable binarization network has good robustness, and the network structure can maintain high detection and positioning accuracy of the text information area while reducing a large number of parameters and calculations, thereby improving the detection accuracy.
[0035] Preferably, in step S5, the specific method of processing the text information area to obtain a horizontal single-line text is:
[0036] S5.1: Input the text information area into the getPerspectiveTransform function, perform perspective transformation, and obtain the perspective transformation matrix;
[0037] S5.2: Input the deformation matrix into the warpPerspective function to obtain a horizontal single line of text.
[0038] An improved differentiable binarization network is used to locate the text information in a specific information area, and the coordinate values of the upper left point, upper right point, lower right point, and lower left point are predicted. Since the text information area formed by these four vertices is not a regular rectangle, the getPerspectiveTransform function in OPENCV is used to obtain the perspective transformation matrix, and the warpPerspective function is used to obtain a horizontal single line of text to facilitate subsequent text recognition.
[0039] Preferably, in step S6, the specific method for identifying the horizontal single-line text and obtaining the specific information is:
[0040] S6.1: Based on a horizontal single line of text, obtain a convolutional feature map;
[0041] S6.2: extract feature sequences from the convolution feature graph and obtain a probability distribution matrix after processing;
[0042] S6.3: Analyze the probability distribution matrix to obtain specific information.
[0043] 8. The information recognition method for the meter rotation process according to claim 7 is characterized in that an improved convolutional neural recurrent network is constructed to recognize horizontal single-line text to obtain specific information; the improved convolutional neural recurrent network includes a convolutional neural network module, a recurrent neural network module and an information transcription module connected in sequence;
[0044] The existing convolutional neural recurrent network includes a VGG-16 network, an LSTM network, and a Softmax cross entropy loss function layer connected in sequence; the MobileNetV3 network is used to replace the structure of the VGG-16 network except the first convolution layer to form a convolutional neural network module; the GRU network is used to replace the LSTM network to form a recurrent neural network module; the CTC loss function layer is used to replace the Softmax cross entropy loss function layer to form an information transcription module;
[0045] A horizontal single line of text is input into the convolutional neural network module to obtain a convolutional feature map; the recurrent neural network module extracts the feature sequence of the convolutional feature map and obtains a probability distribution matrix after processing; the information transcription module parses the probability distribution matrix to obtain specific information.
[0046] In the MobileNetV3 network, a linear bottleneck layer is first used to increase the dimension, and then a depth-separable convolutional layer is passed through. Finally, the flags of each layer are used to determine whether the SE attention mechanism block needs to be added. At the same time, the maximum pooling operation is added after the feature maps output by the 1st, 2nd, 4th, and 6th layers of the network. The first two maximum pooling layers compress the width and height of the feature map to half of the original. The last two maximum pooling operations only scale the height in half, while the width remains unchanged, so that the height of the convolutional feature map output by the last layer is equal to 1, which greatly reduces the number of network parameters while maintaining a high recognition accuracy. The convolutional feature map is input into a deep bidirectional GRU network for feature sequence extraction. Compared with traditional RNN and LSTM networks, the GRU network has fewer parameters and is easier to converge. By performing feature sequence extraction, the feature map is extracted. Column extraction strengthens the connection between contexts; since the shape categories, lengths and character intervals of single-line texts at different levels are different, if the common Softmax cross entropy loss function is used after the recurrent neural network, it is necessary to ensure that each column, that is, each small block, corresponds to a character element, and before training, it is necessary to mark the specific position of each character in the image, and then align the receptive field of the convolutional neural network to each column of the feature map to obtain the corresponding label. This will bring particularly great difficulties to the annotation of the data set. Therefore, the CTC loss function layer is finally used to parse the probability distribution matrix to solve the problem that characters cannot be aligned. By introducing blank characters, the problem of no characters in some positions is solved, and then the gradient is quickly calculated by recursion to adjust the weight parameters of the GRU, and finally accurate specific information is obtained.
[0047] Preferably, the method further comprises:
[0048] S7: Upload the electric meter image and corresponding specific information to the server for storage.
[0049] The present invention also provides an information identification device for an electric meter rotation process, comprising:
[0050] An image acquisition module, used for acquiring an electric meter image;
[0051] A first detection module is used to locate a specific information area in the electric meter image;
[0052] A preprocessing module is used to perform image preprocessing on a specific information area to obtain a preprocessed image of the specific information area;
[0053] The second detection module pre-processes the image based on the specific information area, locates the text information in the specific information area, and obtains the text information area;
[0054] A processing module, used for processing the text information area to obtain a horizontal single line of text;
[0055] The recognition module is used to recognize horizontal single-line text and obtain specific information.
[0056] Preferably, the device further comprises:
[0057] The communication module is used to upload the meter image and corresponding specific information to the server for storage.
[0058] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:
[0059] The present invention aims at the meter rotation operation process, firstly obtains the meter image of the old meter that needs to be replaced, and locates the required specific information area; performs image preprocessing on the specific information area to eliminate the influence of complex environmental factors; then locates the text information in the specific information area to obtain an accurate text information area; then processes the area and converts it into a horizontal single-line text to facilitate subsequent text recognition; finally, recognition is performed based on the horizontal single-line text, with a fast recognition speed and high accuracy of the recognized specific information. BRIEF DESCRIPTION OF THE DRAWINGS
[0060] Figure 1 This is a flow chart of an information identification method for an electric meter rotation process described in Example 1;
[0061] Figure 2 This is a structural diagram of the improved differentiable binary network described in Example 2;
[0062] Figure 3 This is a structural diagram of the MBConv convolution block described in Example 2;
[0063] Figure 4 This is a structural diagram of the improved convolutional neural recurrent network described in Example 2.
[0064] Figure 5 This is a structural diagram of an information identification device for an electric meter rotation process described in Example 3. DETAILED DESCRIPTION
[0065] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;
[0066] In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;
[0067] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.
[0068] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0069] Example 1
[0070] This embodiment provides an information identification method for the meter rotation process, such as Figure 1 As shown, including:
[0071] S1: Acquire the electric meter image;
[0072] S2: Locate a specific information area in the meter image;
[0073] S3: performing image preprocessing on the specific information region to obtain a preprocessed image of the specific information region;
[0074] S4: preprocessing the image based on the specific information region, locating the text information in the specific information region, and obtaining the text information region;
[0075] S5: Processing the text information area to obtain a horizontal single line of text;
[0076] S6: Recognize a horizontal single line of text to obtain specific information.
[0077] In the specific implementation process, this embodiment first obtains the meter image of the old meter that needs to be replaced, and locates the required specific information area; performs image preprocessing on the specific information area to eliminate the influence of complex environmental factors; then locates the text information in the specific information area to obtain the text information area, and then processes the area and converts it into a horizontal single-line text; finally, recognition is performed based on the horizontal single-line text, with a fast recognition speed and high accuracy of the recognized specific information.
[0078] Example 2
[0079] The present invention provides an information identification method for an electric meter rotation process, comprising:
[0080] S1: Acquire the electric meter image;
[0081] S2: Locate a specific information area in the meter image; the specific information area includes a power consumption area, an asset number area, and a meter serial number area;
[0082] Specifically, the YOLO target detection and positioning algorithm is used to locate specific information areas in the meter image:
[0083] S2.1: adjust the size of the meter image;
[0084] S2.2: Based on the resized meter image, upsampling is performed using the Darknet-53 network to obtain three feature maps with different resolutions, namely, the first feature map, the second feature map, and the third feature map; the resolution of the first feature map is 1 / 32 of the meter image, the resolution of the second feature map is 1 / 16 of the meter image, and the resolution of the third feature map is 1 / 8 of the meter image;
[0085] S2.3: Perform feature fusion on the first feature map, the second feature map, and the third feature map to obtain a first fused feature map;
[0086] S2.4: Generate multiple anchor boxes on the first fused feature map;
[0087] The anchor box includes confidence and 4 predicted values, which are respectively the predicted value of the center coordinate of the anchor box, the horizontal coordinate, and the predicted value t x , vertical coordinate prediction value of anchor box center coordinate t y , anchor box width prediction value t w And the anchor box height prediction value t h ;
[0088] S2.5: Obtain specific information area;
[0089] Calculate the confidence score of the anchor box, which is equal to the category conditional probability multiplied by the confidence score; sort the confidence scores of all anchor boxes according to the preset category of specific information, the category of the anchor box with the highest confidence score corresponds to the category of specific information, and the anchor box area is the specific information area;
[0090] S3: performing image preprocessing on the specific information region to obtain a preprocessed image of the specific information region;
[0091] The image preprocessing includes grayscale, Gaussian blur and histogram equalization operations;
[0092] Gaussian blur is used to remove image noise, and histogram equalization is used to enhance image contrast. Image preprocessing can effectively remove the interference of environmental factors;
[0093] S4: Preprocessing the image based on the specific information area, locating the text information in the specific information area, and obtaining the text information area; specifically:
[0094] S4.1: extracting features from the preprocessed image in the specific information region to obtain feature maps of four different scales, namely, a fourth feature map, a fifth feature map, a sixth feature map, and a seventh feature map;
[0095] S4.2: upsampling the fourth feature map, the fifth feature map, the sixth feature map and the seventh feature map, and then concatenating them to achieve multi-scale information fusion, thereby obtaining a second fused feature map;
[0096] S4.3: Adaptively binarize each pixel on the second fused feature map to obtain a probability map and a threshold map, and calculate a binarized map based on the probability map and the threshold map. The formula is:
[0097]
[0098] In the formula, Represents the pixel point in the i-th row and j-th column of the binary image, P ij represents the pixel in the i-th row and j-th column on the probability map, T ij represents the pixel at the i-th row and j-th column on the threshold map, and K represents the coefficient factor;
[0099] S4.4: Based on the binary image, the coordinates of the upper left point, the upper right point, the lower right point and the lower left point in the image are obtained, and the area formed by the four vertices is the text information area;
[0100] like Figure 2 As shown in FIG, the improved differentiable binarization network includes sequentially connecting the EfficientNet backbone feature extraction module, the BiFPN feature fusion module and the post-processing module; the EfficientNet backbone feature extraction module is composed of 5 MBConv convolution blocks, such as Figure 3 As shown in the figure, each MBConv convolution block includes a 1×1 convolution layer, a deep convolution layer and a point convolution layer connected in sequence, and an SE attention mechanism block is added to the deep convolution layer and the point convolution layer; the 1×1 convolution layer is used to perform dimensionality increase operation, and after the deep separable convolution composed of the deep convolution layer and the point convolution layer, the feature map is output. The added SE attention mechanism block can increase the learning proportion of important channel features, reduce the proportion of unimportant features, and improve the accuracy of training results; the EfficientNet backbone feature extraction module outputs a total of 4 feature maps of different scales. The scales of the fourth feature map, the fifth feature map, the sixth feature map and the seventh feature map are 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the preprocessed image of the specific information area respectively; the 4 feature maps of different scales are input into B The iFPN feature fusion module fully integrates the features of each feature map into the second fused feature map obtained after output upsampling and splicing, realizes the fusion of multi-scale information, and effectively improves the detection effect of text information areas of different scales; the post-processing module adaptively binarizes each pixel on the second fused feature map, predicts the probability map and threshold map, and then uses the sigmoid function with coefficients to calculate the approximate binarization map, determines the coordinates of the upper left point, upper right point, lower right point and lower left point on the binarization map, and the area formed by the four vertices is the text information area; the improved differentiable binarization network has good robustness, and the network structure can maintain high detection and positioning accuracy of the text information area while reducing a large number of parameters and calculations, thereby improving the detection accuracy;
[0101] S5: Process the text information area to obtain a horizontal single-line text; specifically:
[0102] S5.1: Input the text information area into the getPerspectiveTransform function, perform perspective transformation, and obtain the perspective transformation matrix;
[0103] S5.2: Input the deformation matrix into the warpPerspective function to obtain a horizontal single line of text.
[0104] The improved differentiable binarization network is used to locate the text information in the specific information area, and the coordinate values of the upper left point, upper right point, lower right point and lower left point are predicted. Since the text information area composed of these four vertices is not a regular rectangle, the getPerspectiveTransform function in OPENCV is used to obtain the perspective transformation matrix, and the warpPerspective function is used to obtain the horizontal single-line text, which is convenient for subsequent text recognition;
[0105] S6: Recognize the horizontal single-line text to obtain specific information; the specific information includes power consumption, asset number and meter serial number; specifically:
[0106] S6.1: Based on a horizontal single line of text, obtain a convolutional feature map;
[0107] S6.2: extract feature sequences from the convolution feature graph and obtain a probability distribution matrix after processing;
[0108] S6.3: Analyze the probability distribution matrix to obtain specific information.
[0109] An improved convolutional neural recurrent network is constructed to recognize horizontal single-line text and obtain specific information; the improved convolutional neural recurrent network includes a convolutional neural network module, a recurrent neural network module and an information transcription module connected in sequence;
[0110] The existing convolutional neural recurrent network includes a VGG-16 network, an LSTM network, and a Softmax cross entropy loss function layer connected in sequence; the MobileNetV3 network is used to replace the structure of the VGG-16 network except the first convolution layer to form a convolutional neural network module; the GRU network is used to replace the LSTM network to form a recurrent neural network module; the CTC loss function layer is used to replace the Softmax cross entropy loss function layer to form an information transcription module;
[0111] A horizontal single line of text is input into the convolutional neural network module to obtain a convolutional feature map; the recurrent neural network module extracts the feature sequence of the convolutional feature map and obtains a probability distribution matrix after processing; the information transcription module parses the probability distribution matrix to obtain specific information.
[0112] In the MobileNetV3 network, a linear bottleneck layer is first used to increase the dimension, and then a depth-separable convolutional layer is passed through. Finally, the flags of each layer are used to determine whether the SE attention mechanism block needs to be added. At the same time, the maximum pooling operation is added after the feature maps output by the 1st, 2nd, 4th, and 6th layers of the network. The first two maximum pooling layers compress the width and height of the feature map to half of the original. The last two maximum pooling operations only scale the height in half, while the width remains unchanged, so that the height of the convolutional feature map output by the last layer is equal to 1, which greatly reduces the number of network parameters while maintaining a high recognition accuracy. The convolutional feature map is input into a deep bidirectional GRU network for feature sequence extraction. Compared with traditional RNN and LSTM networks, the GRU network has fewer parameters and is easier to converge. By performing feature sequence extraction, the feature map is extracted. Column extraction strengthens the connection between contexts. Since the shape categories, lengths, and character intervals of single-line texts at different levels are different, if the common Softmax cross entropy loss function is used after the recurrent neural network, it is necessary to ensure that each column, that is, each small block, corresponds to a character element. In addition, the specific position of each character in the image needs to be marked before training, and then the receptive field of the convolutional neural network is aligned to each column of the feature map to obtain the corresponding label. This will bring great difficulties to the annotation of the data set. Therefore, the CTC loss function layer is finally used to parse the probability distribution matrix to solve the problem that characters cannot be aligned. By introducing blank characters, the problem of no characters in some positions is solved. Then, the gradient is quickly calculated by recursion to adjust the weight parameters of the GRU, and finally accurate specific information is obtained.
[0113] S7: Upload the electric meter image and corresponding specific information to the server for storage.
[0114] In the specific implementation process, this embodiment obtains the meter image of the old meter that needs to be replaced, and uses the YOLO target detection and positioning algorithm to locate the power consumption area, asset number area and meter serial number area in the meter image; first, the meter image size is adjusted to a dimension of 256×256×3, and Darknet-53 is used as the backbone feature extraction part for upsampling to obtain the first feature map, the second feature map and the third feature map with the resolution rates of 1 / 32, 1 / 16 and 1 / 8 of the meter image respectively; feature maps of different sizes are upsampled and spliced to improve the detection efficiency of multi-scale targets. As a result, a first fused feature map is obtained, with a dimension of N×N×[3×(4+1+80)], where N is the size of the first fused feature map; three anchor frames are generated on the first fused feature map, each of which includes a confidence score and four predicted values; in this embodiment, there are 80 object categories. When calculating the predicted category, the confidence score of the anchor frame for a certain category is equal to the conditional probability of the category corresponding to the current predicted frame multiplied by the confidence of the current predicted frame, and the category corresponding to the maximum confidence score is taken as the predicted category, and the center point coordinates of the anchor frame and the numerical information such as the frame height and frame width are output to form the power area, the asset number area and the meter serial number area; The image in the above area is grayed, Gaussian blurred and histogram equalized to obtain a preprocessed image of the specific information area; the text information in the specific information area is located using an improved differentiable binarization network; the improved differentiable binarization network includes an EfficientNet backbone feature extraction module, a BiFPN feature fusion module and a post-processing module connected in sequence; first, the EfficientNet backbone feature extraction module includes 5 MBConv convolution blocks connected in sequence, each MBConv convolution block includes a 1×1 convolution layer, a depth convolution layer and a point convolution layer connected in sequence, and the depth convolution layer is connected in sequence. An SE attention mechanism block is also added to the convolution layer and the point convolution layer, and the scale factor in the SE module is set to 0.25; the EfficientNet backbone feature extraction module outputs four feature maps of different scales, with shapes of [1, 16, 56, 56], [1, 24, 28, 28], [1, 56, 14, 14] and [1, 480, 7, 7], respectively, and the scales are 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the preprocessed image of the specific information area; the BiFPN feature fusion module is input to adjust the number of channels of the four feature maps of different scales to 24, and the number of channels after fusion is 96; Figure 2As shown in the figure, the BiFPN feature fusion module is different from the traditional FPN in that the output of the BiFPN feature fusion module except the first and last layers is composed of three parts, namely the original input, the output of the previous layer after downsampling, and the output of the original input and the upsampling of the next layer. Taking P3_out as an example, it is composed of the upsampled results of P3_in and P2_out and P3_up multiplied by a weight coefficient respectively, where P3_up is obtained by multiplying the upsampled results of P3_in and P4_up by a coefficient and then upsampling. The purpose of this is to fully integrate the features of each feature map in each output, so as to achieve the fusion of multi-scale information, which can effectively improve the detection effect of text lines of different scales; after feature fusion, the four outputs are upsampled and spliced to obtain the second fused feature map with a shape of [1,96,56,56]; each pixel on the second fused feature map is adaptively binarized to obtain a probability map and a threshold map, and the binarized map is calculated according to the probability map and the threshold map to obtain the upper left corner of the figure. The coordinates of the upper right point, the lower right point and the lower left point, the area formed by the four vertices is the text information area; since the text information area formed by these four vertices is not a regular rectangle, it is necessary to use the getPerspectiveTransform function in OPENCV to obtain the perspective transformation matrix. The specific method is to set two arrays to store the rectangular areas before and after correction, where maxWidth represents the maximum width pixel value, maxHeight represents the maximum height pixel value, the coordinates of the upper left point before correction correspond to the corrected origin (0,0), the coordinates of the upper right point before correction correspond to the corrected (maxWidth-1,0), the coordinates of the lower right point before correction correspond to the corrected (maxWidth-1,maxHeight-1), and the coordinates of the lower left point before correction correspond to the corrected (0,maxHeight-1). After obtaining the perspective transformation matrix, use the warpPerspective function to obtain the horizontal single-line text after perspective transformation; finally, use the improved convolutional neural recurrent network to recognize the horizontal single-line text; such as Figure 4As shown in the figure, the improved convolutional neural network includes a convolutional neural network module, a recurrent neural network module and an information transcription module connected in sequence; the existing convolutional neural network includes a VGG-16 network, an LSTM network and a Softmax cross entropy loss function layer connected in sequence; the MobileNetV3 network is used to replace the structure except the first convolution layer in the VGG-16 network to form a convolutional neural network module. In the MobileNetV3 network, a linear bottleneck layer is first used to increase the dimension, and then a depth-separable convolution layer is passed through. Finally, the SE attention mechanism block needs to be added according to the flags of each layer; at the same time, the maximum pooling operation is added to the feature maps output by the 1st, 2nd, 4th and 6th layers of the network. The first two maximum pooling layers compress the width and height of the feature maps to half of the original, and the last two maximum pooling operations only scale the height in half, while the width remains unchanged, so that the height of the convolutional feature map output by the last layer is equal to 1, and the dimension changes from (32, 100, 3) to (1, 25, 128) of the last layer; the GRU network is used to replace LSTM network constitutes the recurrent neural network module. The performance of GRU network in completing text recognition tasks is similar to LSTM and the number of parameters is smaller than LSTM, so GRU network is used to extract text context sequence features. The number of hidden layer neurons is set to 96. Since the dimension of the image input to the GRU network is (32, 100, 3), the height of the output feature map is 25. Let T = 25, as the maximum time length input of the recurrent neural network, which is equivalent to dividing the entire image into 25 small blocks, so as to predict the probability distribution of all digital characters in each small block area, and finally obtain 25 probability vectors with a length of the number of character categories. After processing, the probability distribution matrix is obtained; the CTC loss function layer is used instead of the Softmax cross entropy loss function layer to form the information transcription module, and the CTC loss function layer is used to parse the probability distribution matrix to solve the problem that characters cannot be aligned. By introducing blank characters, the problem of no characters in some positions is solved, and then the recursive gradient is quickly calculated to adjust the weight parameters of GRU, and finally the accurate power consumption, asset number and meter serial number are obtained. The meter image of the old meter is matched with the identified power consumption, asset number and meter serial number, and then packaged and sent to the server for storage. The method provided in this embodiment can achieve a recognition accuracy rate of about 98% on meter images taken under actual complex backgrounds, simplifying the meter stop recognition and recording work during the meter rotation operation, eliminating tedious and laborious manual processing steps, thereby improving actual work efficiency.
[0115] Example 3
[0116] This embodiment provides an information identification device for the meter rotation process, such as Figure 5 As shown, including:
[0117] An image acquisition module, used for acquiring an electric meter image;
[0118] A first detection module is used to locate a specific information area in the electric meter image;
[0119] A preprocessing module is used to perform image preprocessing on a specific information area to obtain a preprocessed image of the specific information area;
[0120] The second detection module pre-processes the image based on the specific information area, locates the text information in the specific information area, and obtains the text information area;
[0121] A processing module, used for processing the text information area to obtain a horizontal single line of text;
[0122] A recognition module is used to recognize horizontal single-line text and obtain specific information;
[0123] The communication module is used to upload the meter image and corresponding specific information to the server for storage.
[0124] The same or similar reference numerals correspond to the same or similar components;
[0125] The terms used in the drawings to describe positional relationships are only used for illustrative purposes and should not be construed as limiting this patent;
[0126] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.
Claims
1. An information identification method for an electric meter rotation process, characterized in that: include: S1: Acquire the electric meter image; S2: Locate a specific information area in the meter image; S3: performing image preprocessing on the specific information region to obtain a preprocessed image of the specific information region; S4: Preprocess the image based on the specific information area, use the improved differentiable binarization network to locate the text information in the specific information area, and obtain the text information area, including: S4.1: extracting features from the preprocessed image in the specific information region to obtain feature maps of four different scales, namely, a fourth feature map, a fifth feature map, a sixth feature map, and a seventh feature map; S4.2: upsampling the fourth feature map, the fifth feature map, the sixth feature map and the seventh feature map, and then concatenating them to achieve multi-scale information fusion, thereby obtaining a second fused feature map; S4.3: Adaptively binarize each pixel on the second fused feature map to obtain a probability map and a threshold map, and calculate a binarized map based on the probability map and the threshold map. The formula is: In the formula, Represents the pixel point in the i-th row and j-th column of the binary image, P ij represents the pixel in the i-th row and j-th column on the probability map, T ij represents the pixel at the i-th row and j-th column on the threshold map, and k represents the coefficient factor; S4.4: Based on the binary image, the coordinates of the upper left point, the upper right point, the lower right point and the lower left point in the image are obtained, and the area formed by the four vertices is the text information area; S5: Processing the text information area to obtain a horizontal single line of text; S6: Recognize a horizontal single line of text to obtain specific information.
2. The information identification method for the meter rotation process according to claim 1 is characterized in that: In step S2, a specific information area is located in the meter image using the YOLO target detection and positioning algorithm; the specific process is as follows: S2.1: adjust the size of the meter image; S2.2: Based on the resized electricity meter image, upsampling is performed using the Darknet-53 network to obtain three feature maps with different resolutions, namely the first feature map, the second feature map, and the third feature map; S2.3: Perform feature fusion on the first feature map, the second feature map, and the third feature map to obtain a first fused feature map; S2.4: Generate multiple anchor boxes on the first fused feature map; The anchor box includes confidence and 4 predicted values, which are respectively the predicted value of the center coordinate of the anchor box, the horizontal coordinate, and the predicted value t x , vertical coordinate prediction value of anchor box center coordinate t y , anchor box width prediction value t w And the anchor box height prediction value t h ; S2.5: Obtain specific information area; Calculate the confidence score of the anchor box, which is equal to the category conditional probability multiplied by the confidence; sort the confidence scores of all anchor boxes according to the preset category of specific information, the category of the anchor box with the highest confidence score corresponds to the category of specific information, and the anchor box area is the specific information area.
3. The information identification method for the meter rotation process according to claim 2 is characterized in that: The specific information area includes a power consumption area, an asset number area and a meter serial number area; the specific information includes power consumption, asset number and meter serial number.
4. The information identification method for the meter rotation process according to claim 1 is characterized in that: In step S3, the image preprocessing performed on the specific information area includes grayscale, Gaussian blur and histogram equalization operations.
5. The information identification method for the meter rotation process according to claim 1 is characterized in that: In step S5, the specific method of processing the text information area to obtain a horizontal single-line text is: S5.1: Input the text information area into the getPerspectiveTransform function, perform perspective transformation, and obtain the perspective transformation matrix; S5.2: Input the deformation matrix into the warpPerspective function to obtain a horizontal single line of text.
6. The information identification method for the meter rotation process according to claim 1 is characterized in that: In step S6, the specific method of identifying the horizontal single-line text and obtaining the specific information is as follows: S6.1: Based on a horizontal single line of text, obtain a convolutional feature map; S6.2: extract feature sequences from the convolution feature graph and obtain a probability distribution matrix after processing; S6.3: Analyze the probability distribution matrix to obtain specific information.
7. The information identification method for the meter rotation process according to claim 6 is characterized in that: An improved convolutional neural recurrent network is constructed to recognize horizontal single-line text and obtain specific information; the improved convolutional neural recurrent network includes a convolutional neural network module, a recurrent neural network module and an information transcription module connected in sequence; The existing convolutional neural recurrent network includes a VGG-16 network, an LSTM network, and a Softmax cross entropy loss function layer connected in sequence; the MobileNetV3 network is used to replace the structure of the VGG-16 network except the first convolution layer to form a convolutional neural network module; the GRU network is used to replace the LSTM network to form a recurrent neural network module; the CTC loss function layer is used to replace the Softmax cross entropy loss function layer to form an information transcription module; A horizontal single line of text is input into the convolutional neural network module to obtain a convolutional feature map; the recurrent neural network module extracts the feature sequence of the convolutional feature map and obtains a probability distribution matrix after processing; the information transcription module parses the probability distribution matrix to obtain specific information.
8. The information identification method for the meter rotation process according to claim 1 is characterized in that: The method further comprises: S7: Upload the electric meter image and corresponding specific information to the server for storage.
9. An information identification device for an electric meter rotation process, characterized in that: include: An image acquisition module, used for acquiring an electric meter image; A first detection module is used to locate a specific information area in the electric meter image; A preprocessing module is used to perform image preprocessing on a specific information area to obtain a preprocessed image of the specific information area; The second detection module pre-processes the image based on the specific information area, uses the improved differentiable binarization network to locate the text information in the specific information area, and obtains the text information area, including: Perform feature extraction on the preprocessed image of the specific information area to obtain feature maps of four different scales, namely, a fourth feature map, a fifth feature map, a sixth feature map and a seventh feature map; The fourth feature map, the fifth feature map, the sixth feature map, and the seventh feature map are all upsampled and then spliced to achieve multi-scale information fusion to obtain a second fused feature map; Adaptively binarize each pixel on the second fusion feature map to obtain a probability map and a threshold map. The binarization map is calculated based on the probability map and the threshold map. The formula is: In the formula, Represents the pixel point in the i-th row and j-th column of the binary image, P ij represents the pixel in the i-th row and j-th column on the probability map, T ij represents the pixel at the i-th row and j-th column on the threshold map, and k represents the coefficient factor; Based on the binary image, the coordinates of the upper left point, upper right point, lower right point and lower left point in the image are obtained, and the area formed by the four vertices is the text information area; A processing module, used for processing the text information area to obtain a horizontal single line of text; The recognition module is used to recognize horizontal single-line text and obtain specific information.