A method, device and medium for identifying substation instrument readings in an adaptive environment

Abstract

The invention relates to an adaptive environment transformer substation meter reading recognition method, comprising the following steps: determining N types of environments that affect imaging quality, collecting original data according to the N types of environments, and establishing an environment recognition model for identifying the type of environment in which an image is located, an image enhancement model for enhancing images in each type of environment respectively, and an instrument positioning recognition model for identifying the location of an instrument dial and identifying the instrument reading based on the original data; inputting an original image of a transformer substation meter into the environment recognition model to identify the environment type in the original image; inputting the original image into the image enhancement model to enhance the original image according to the identified environment type to obtain an enhanced image; inputting the enhanced image into the instrument positioning recognition model to locate the instrument area and identify the instrument reading.

Description

Technical Field

[0001] The invention relates to a method, equipment and medium for identifying instrument readings in a transformer substation in an adaptive environment, and belongs to the technical field of electric power instrument identification. Background Art

[0002] Due to the large number of instruments and their scattered locations, the current collection of instrument readings is mainly done through automatic collection of instrument images by inspection robots, drones and other inspection equipment, and then automatic identification of instrument readings in the inspection images through image recognition technology.

[0003] For example, the invention patent with application number "201910319860.4" discloses a method for identifying a single-pointer sulfur hexafluoride instrument, which uses a YOLO model based on deep learning for detection and uses simple enhancement, distortion processing and contour features to identify the pointer instrument;

[0004] The invention patent with application number "201710882172.X" discloses a digital recognition method for substation digital instruments, which uses HSV color features for digital segmentation and combines SVM classifier for digital recognition;

[0005] In the above-mentioned prior art, the existing methods can be used normally only if the image brightness is uniform, the contrast is good enough, and the dial is clearly photographed. They cannot work normally at night, in rain, snow, fog, and other bad weather conditions. In addition, the positioning of the instrument panel using the yolo model or color features is not accurate enough. The yolo model may cause the feature frame to shift when rotating, and the color feature is easy to fail when the brightness and contrast change. Summary of the invention

[0006] In order to solve the problems existing in the above-mentioned prior art, the present invention proposes an adaptive environment substation instrument reading recognition method, establishes an environmental recognition model and an image enhancement model, automatically identifies the environment type of the instrument image, and enhances the image according to the environment type. It can be used in harsh environments to improve the recognition effect.

[0007] The technical solution of the present invention is as follows:

[0008] A method for identifying substation instrument readings in an adaptive environment comprises the following steps:

[0009] Determine N types of environments that affect imaging quality, collect raw data according to the N types of environments, and establish an environment recognition model for identifying the type of environment in which the image is located, an image enhancement model for enhancing the image in each type of environment, and an instrument positioning recognition model for identifying the location of an instrument dial and identifying the instrument reading based on the raw data;

[0010] Input the original image of the substation instrument into the environment recognition model to identify the environment type in the original image;

[0011] The original image is input into the image enhancement model, and the original image is enhanced according to the identified environment type to obtain an enhanced image;

[0012] The enhanced image is input into the instrument location recognition model to locate the instrument area and recognize the instrument reading.

[0013] As a preferred implementation, the steps of establishing the environment recognition model are specifically as follows:

[0014] Collect images in N types of environments respectively, and add environment type labels to the collected images to obtain a training sample set;

[0015] A neural network is constructed, and images in a training sample set are used as inputs of the neural network. A predicted value of the environment type is output, and a loss function is established based on the deviation between the predicted value of the output environment type and the label. The value of the loss function is calculated and the parameters of the neural network are updated, and iterative training is performed; until the value of the loss function is less than a set threshold, the iteration ends, and the parameters of the neural network are saved to obtain the environment recognition model.

[0016] As a preferred implementation, the steps of establishing the image enhancement model are specifically as follows:

[0017] Collect several instrument images in an environment that does not affect the imaging quality as a standard image set, process the standard images according to the influence of different environmental types on the standard images, and obtain N groups of control image sets corresponding to each environmental type;

[0018] N image enhancement networks are constructed, and the control image sets of each environment type are used as the input of each image enhancement network. The enhanced image is output, and a loss function is constructed based on the deviation between the enhanced image and the standard image. The value of the loss function is calculated and the parameters of each image enhancement network are updated, and iterative training is performed; until the value of the loss function is less than the set threshold, the iteration ends, and the parameters of each image enhancement network are saved to obtain the image enhancement model corresponding to each environment type.

[0019] As a preferred implementation, the image enhancement network includes:

[0020] Several convolutional attention modules connected in sequence are used to extract features from the input image and obtain several initial feature maps respectively;

[0021] Several feature map re-encoding modules corresponding to each convolutional attention module encode and decode each initial feature map to obtain several advanced feature maps;

[0022] The attention mechanism module is used to evaluate the importance of each advanced feature map, assign different channel weights to each advanced feature map based on the importance, and combine the channels of each advanced feature map according to the channel weights of each advanced feature map;

[0023] Convolution modules and activation functions are used to output enhanced images based on the advanced feature maps after channel combination.

[0024] As a preferred implementation, the loss function of the image enhancement network is:

[0025] Loss = α(x, y)L2 + β(x, y)L ssim +γ(x,y)L perceptual ;

[0026] Among them, L2 is the MSE loss of pixel value; L ssim is the structural similarity loss; L perceptual is the perceptual loss; α(x, y), β(x, y), and γ(x, y) are used to set different weights for different losses of each pixel;

[0027] Structural similarity loss L ssim Defined as:

[0028]

[0029] Among them, μ p1 and μ p2 Respectively represent the mean of the two images p1 and p2, σ p1 and σ p2 is the variance of the two images, σ p1p2 is the covariance, C1 and C2 are constants;

[0030] Perceptual loss L perceptual Defined as:

[0031]

[0032] Among them, W, H, and C are the width, height, and number of channels of the feature map respectively. is the feature map.

[0033] As a preferred implementation, the instrument positioning and recognition model includes an instrument positioning module, a deviation correction module, and a reading recognition module;

[0034] The method for establishing the instrument positioning module is specifically as follows:

[0035] Collect multiple instrument images, annotate the position data of the instrument dial for each instrument image, obtain a training sample set including the position data of each instrument image and the corresponding instrument dial, train the target detection network through the training sample set, and obtain an instrument positioning model for predicting the area where the instrument dial is located;

[0036] The correction module cuts out a dial image from the image according to the area where the instrument dial is located output by the instrument positioning module, and performs transformation processing on the dial image to transform the dial image into a standard dial image;

[0037] The reading recognition module recognizes the meter reading according to the standard dial image.

[0038] On the other hand, the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the meter reading identification method as described in any embodiment of the present invention is implemented.

[0039] On the other hand, the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the meter reading identification method as described in any embodiment of the present invention is implemented.

[0040] The present invention has the following beneficial effects:

[0041] 1. The present invention provides an adaptive environment substation meter reading recognition method, which establishes an environment recognition model and an image enhancement model to automatically recognize the environment type of the meter image, so that the meter recognition can obtain good recognition effect regardless of the weather conditions such as daytime (good lighting conditions and high imaging quality) or night, rainy, snowy, foggy and hazy days (poor lighting conditions and very low visibility).

[0042] 2. The present invention provides an adaptive environment substation meter reading recognition method, establishes a suitable image enhancement model for substation meters, designs a unique network structure and loss function, trains different enhancement models for each environment type, can effectively enhance the instrument area and suppress the background area, and is beneficial to the subsequent meter reading recognition.

[0043] 3. The present invention provides an adaptive environment substation meter reading recognition method, which proposes an algorithm for accurate positioning and transformation correction of the meter area in the meter positioning recognition model, and also exceeds the existing methods in terms of stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a flow chart of Embodiment 1 of the present invention;

[0045] Figure 2A schematic diagram of the network structure of an image enhancement model in an embodiment of the present invention;

[0046] Figure 3 Schematic diagram of the structure of the convolutional attention module in an embodiment of the present invention;

[0047] Figure 4 The present invention is a flowchart of an embodiment of the present invention for identifying the reading of a pointer-type instrument. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] It should be understood that the step numbers used in this document are only for convenience of description and are not intended to limit the order in which the steps are executed.

[0050] It should be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0051] The terms “include” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0052] The term "and / or" means and includes any and all possible combinations of one or more of the associated listed items.

[0053] Embodiment 1:

[0054] See also Figure 1 , a method for identifying substation instrument readings in an adaptive environment, comprising the following steps:

[0055] Determine N types of environments that affect the imaging quality. In this embodiment, four types of environments that affect the imaging quality, namely night, rain, snow, and haze, are determined, and the daytime is regarded as a normal environment type.

[0056] In the substation, several instrument images are taken in five environmental types: daytime, night, rain, snow, and haze as raw data. The number of images in the five environmental types needs to be kept close. An environmental recognition model for identifying the type of environment the image is in is established based on the raw data.

[0057] Establish an image enhancement model that enhances images in different types of environments and an instrument location recognition model that recognizes the location of instrument dials and instrument readings;

[0058] Input the original image of the substation instrument into the environment recognition model to identify the environment type in the original image; if the environment type is identified as daytime, no processing is performed;

[0059] If the environment type is identified as one of the four types of image quality, the original image is input into the image enhancement model, and the original image is enhanced according to the identified environment type to obtain an enhanced image;

[0060] The enhanced image is input into the instrument location recognition model to locate the instrument area and recognize the instrument reading.

[0061] As a preferred implementation of this embodiment, the steps of establishing the environment recognition model are specifically:

[0062] Add environmental type labels to several instrument images taken in five environmental types: daytime, night, rain, snow, and haze, and obtain five sets of training sample sets;

[0063] Select the densenet picture classification network pre-trained on imagenet, and replace the last layer output of the densenet picture classification network with the above five categories of daytime, night, rain, snow, and haze. The input size of the network is 224*224, the convolution layer of the picture classification network is frozen, and the fully connected layer is fine-tuned; the images in each training sample set are used as the input of the picture classification network, and the predicted value of the environment type is output. A loss function is established based on the deviation between the predicted value of the output environment type and the label. The loss function adopts cross entropy, calculates the value of the loss function, and adopts the adam optimization function to update the parameters of the neural network, and performs iterative training; until the value of the loss function is less than the set threshold, the iteration ends, and the parameters of the picture classification network are saved to obtain the environment recognition model.

[0064] When identifying the environment type of the input image, the collected image is scaled to 224*224 and input into the trained image classification network, and the output result is the corresponding environment type.

[0065] As a preferred implementation of this embodiment, the steps of establishing the image enhancement model are specifically as follows:

[0066] Collect several instrument images in a daytime environment as a standard image set; reduce the brightness and contrast of the images in the standard image set to generate a control image set for nighttime environment types; use the images in the standard image set as the background, superimpose the rain, snow, and haze foreground, and synthesize the control image set under rain, snow, and haze;

[0067] Four image enhancement networks are constructed, and the control image sets of each environment type are used as the input of each image enhancement network. The enhanced image is output, and the loss function is constructed based on the deviation between the enhanced image and the corresponding standard image. The value of the loss function is calculated and the parameters of each image enhancement network are updated for iterative training. The iteration ends when the value of the loss function is less than the set threshold. The parameters of each image enhancement network are saved to obtain the night image enhancement model, rainy day image enhancement model, snowy day image enhancement model and haze day image enhancement model corresponding to each environment type.

[0068] As a preferred implementation of this embodiment, the image enhancement network includes:

[0069] Several convolutional attention modules ResCBAM connected in sequence are used to extract features from the input image and obtain several initial feature maps respectively; see Figure 2 , the input image is a three-channel image of size w*h*3, where w and h are the width and height of the image respectively. i (i=1, ..., N) is the initial feature map extracted after different layers of ResCBAM, and the size of the initial feature map is w*h*C i Here, the feature maps of different layers are made of the same size to avoid feature loss after deep convolution.

[0070] Several feature map re-encoding modules UnetBlock corresponding to each convolutional attention module encode and decode each initial feature map to obtain several advanced feature maps; the UNet structure can ensure the feature connectivity between different layers of the encoder and decoder, and further maintain the detail information. All unetblocks are trained simultaneously but do not share parameters. The advanced feature maps after UNetBlock are map1, map2, ..., mapN, and the size of the advanced feature map is w*h.

[0071] The attention mechanism module SEBlock is used to evaluate the importance of each advanced feature map, and assign a larger channel weight to the more important advanced feature map based on the importance, and combine the channels of each advanced feature map according to the channel weight of each advanced feature map;

[0072] Finally, after a layer of convolution and activation function, the enhanced image is output. In this embodiment, for the three environment types of night, rain, and snow, the activation function uses relu, and the network output is the enhanced image; for the haze environment type, the activation function uses bilateral rectified linear unit, and the network output is the atmospheric transmittance; the image is enhanced by atmospheric transmittance:

[0073] Assuming that the atmospheric transmittance output by the network is t, the atmospheric light intensity is A (the atmospheric light intensity is the brightest point in the image), and the original haze image is I, the formula for enhancing the image using atmospheric transmittance is:

[0074] J = [IA*(1-t)] / t;

[0075] Among them, J is the enhanced image.

[0076] The formula for the bilateral rectified linear unit is:

[0077] F=min(t max ,max(t min , W*F in +B))

[0078] Among them, tmin and tmax are the minimum and maximum values ​​of bilateral truncation, respectively. in is the input of the activation function, W and B are the weight and offset of the activation function respectively, and F is the output of the activation function.

[0079] The structure of the convolutional attention module ResCBAM is as follows Figure 3 As shown in the figure, the introduction of spatial and channel attention mechanisms, through cooperation with the loss function, can make the weights of feature maps of different layers concentrated in more important areas (instrument areas), while suppressing the interference of unimportant background around them.

[0080] As a preferred implementation of this embodiment, the loss function of the image enhancement network is:

[0081] Loss = α(x, y)L2 + β(x, y)L ssim +γ(x,y)L perceptual ;

[0082] Among them, L2 is the MSE loss of pixel value, which is used to calculate the difference in pixel value between the enhanced image and the standard image to ensure network convergence; L ssim is the structural similarity loss, used to suppress blur and artifacts; L perceptualFor perceptual loss, the VGG19 network pre-trained on imagenet is used to calculate the difference and at different feature layers. This loss function term can ensure that the generated image has good visual characteristics. α(x, y), β(x, y), and γ(x, y) are used to set different weights for different losses of each pixel; in this embodiment, in the instrument area, α(x, y), β(x, y), and γ(x, y) take larger values, while outside the instrument area, α(x, y), β(x, y), and γ(x, y) take smaller values, so as to focus on enhancing the instrument area and suppress the background area, so as to achieve the purpose of removing interference factors.

[0083] Structural similarity loss L ssim Defined as:

[0084]

[0085] Among them, μ p1 and μ p2 Respectively represent the mean of the two images p1 and p2, σ p1 and σ p2 is the variance of the two images, σ p1p2 is the covariance, C1 and C2 are constants;

[0086] Perceptual loss L perceptual Defined as:

[0087]

[0088] Among them, W, H, and C represent the width, height, and number of channels of the feature map of this layer. Represents the feature map of this layer in the VGG network.

[0089] In this embodiment, four different image enhancement models are trained for four environmental conditions that affect imaging quality, namely night, rain, snow, and haze. Similarly, other adverse environmental conditions can also be trained and enhanced according to the above steps. The present invention takes night, rain, snow, and haze as examples, but is not limited to this.

[0090] As a preferred implementation of this embodiment, this embodiment provides an instrument positioning and recognition model for a pointer-type instrument, including an instrument positioning module, a deviation correction module, and a reading recognition module;

[0091] The method for establishing the instrument positioning module is specifically as follows:

[0092] Collect multiple instrument images {I1, I2, ..., I M}, for each image I i (i=1, 2, ..., M), a mask R marking the visible area of ​​the dashboard i, so we get {(I1, R1), (I2, R2), ..., (I M , R M )}, the target segmentation network is trained by the training sample set. In this embodiment, the target segmentation network adopts the maskrcnn network to obtain the instrument positioning model for predicting the area where the instrument dial is located; at the same time, a front instrument image is taken as a template image, and the four vertices of the quadrilateral of the template image are {(t 1x , t 1y ), (t 2x , t 2y ), (t 3x , t 3y ), (t 4x , t 4y )}, the pointer center of the template image is (c x , c y ), when the scale is 0, the angle between the pointer and the horizontal line is When the scale is full, the angle between the pointer and the horizontal line is The range is S.

[0093] For the enhanced image I e , use the maskrcnn network to predict the mask R of the dial area e , according to the mask R e From Mask I e Cut out the dial image I p Since the above image enhancement model focuses on enhancing the dial area and suppresses the background area, the extracted mask will be more accurate; at the same time, it ensures that even if the dial is offset and misaligned, the area can be accurately located. Therefore, it is suitable for inspection robots that are easily affected by external interference factors (such as bumps and positioning deviations).

[0094] The correction module corrects the mask R e The 3*3 Sobel operator is used to extract the edge, and then the Hough transform is used to extract the four intersecting straight lines. The intersection points of the four straight lines are {(p1x, p1y), (p2x, p2y), (p3x, p3y), (p4x, p4y)} in counterclockwise order. The Sobel operator is defined as:

[0095]

[0096] The transformation matrix M is calculated based on {(t1x, t1y), (t2x, t2y), (t3x, t3y), (t4x, t4y)} and {(p1x, p1y), (p2x, p2y), (p3x, p3y), (p4x, p4y)}, and the dial image I is transformed according to the transformation matrix M. pStretch to the front to get the standard dial image I w , so that the four corners of the dial correspond to the four corners of the template image;

[0097] For the standard dial image I w Use the Sobel operator and Hough transform to extract the straight line again and determine whether the straight line passes through the center of the template pointer (c x , c y ). If the straight line passes through the center of the pointer, it is the pointer straight line. Assume that the angle between the straight line and the horizontal line is , then the current reading d is:

[0098]

[0099] Figure 4 The process of identifying the reading of a pointer-type instrument through this embodiment is demonstrated.

[0100] Embodiment 2:

[0101] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the meter reading recognition method as described in any embodiment of the present invention is implemented.

[0102] Embodiment three:

[0103] This embodiment provides a computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the meter reading recognition method as described in any embodiment of the present invention is implemented.

[0104] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for identifying substation instrument readings in an adaptive environment, characterized in that: The following steps are involved: Determine N types of environments that affect imaging quality, collect raw data according to the N types of environments, and establish an environment recognition model for identifying the type of environment in which the image is located, an image enhancement model for enhancing the image in each type of environment, and an instrument positioning recognition model for identifying the location of an instrument dial and identifying the instrument reading based on the raw data; Input the original image of the substation instrument into the environment recognition model to identify the environment type in the original image; The original image is input into the image enhancement model, and the original image is enhanced according to the identified environment type to obtain an enhanced image; Input the enhanced image into the instrument location recognition model to locate the instrument area and recognize the instrument reading; The steps of establishing the image enhancement model are specifically as follows: Collect several instrument images in an environment that does not affect the imaging quality as a standard image set, process the standard images according to the influence of different environmental types on the standard images, and obtain N groups of control image sets corresponding to each environmental type; Construct N image enhancement networks, use the control image sets of each environment type as the input of each image enhancement network, output the enhanced image, construct a loss function based on the deviation between the enhanced image and the standard image, calculate the value of the loss function and update the parameters of each image enhancement network, and perform iterative training; until the value of the loss function is less than the set threshold, the iteration ends, and the parameters of each image enhancement network are saved to obtain the image enhancement model corresponding to each environment type; Wherein, the image enhancement network comprises: Several convolutional attention modules connected in sequence are used to extract features from the input image and obtain several initial feature maps respectively; Several feature map re-encoding modules corresponding to each convolutional attention module encode and decode each initial feature map to obtain several advanced feature maps; The attention mechanism module is used to evaluate the importance of each advanced feature map, assign different channel weights to each advanced feature map based on the importance, and combine the channels of each advanced feature map according to the channel weights of each advanced feature map; Convolution modules and activation functions are used to output enhanced images based on the advanced feature maps after channel combination; Among them, the loss function of the image enhancement network is: ； in, is the MSE loss of pixel value; is the structural similarity loss; For perceived loss; , , Used to set different weights for different losses of each pixel; Structural Similarity Loss Defined as: ; in, and Represent the mean of the two images p1 and p2 respectively, and is the variance of the two images, is the covariance, C1 and C2 are constants; Perceived loss Defined as: Among them, W, H, and C are the width, height, and number of channels of the feature map respectively. is the feature map.

2. The method for identifying substation meter readings in an adaptive environment according to claim 1, characterized in that: The steps of establishing the environment recognition model are specifically as follows: Collect images in N types of environments respectively, and add environment type labels to the collected images to obtain a training sample set; A neural network is constructed, and images in a training sample set are used as inputs of the neural network. A predicted value of the environment type is output, and a loss function is established based on the deviation between the predicted value of the output environment type and the label. The value of the loss function is calculated and the parameters of the neural network are updated, and iterative training is performed; until the value of the loss function is less than a set threshold, the iteration ends, and the parameters of the neural network are saved to obtain the environment recognition model.

3. The method for identifying substation meter readings in an adaptive environment according to claim 1, characterized in that: The instrument positioning and recognition model includes an instrument positioning module, a deviation correction module and a reading recognition module; The method for establishing the instrument positioning module is specifically as follows: Collect multiple instrument images, annotate the position data of the instrument dial for each instrument image, obtain a training sample set including the position data of each instrument image and the corresponding instrument dial, train the target detection network through the training sample set, and obtain an instrument positioning model for predicting the area where the instrument dial is located; The correction module cuts out a dial image from the image according to the area where the instrument dial is located output by the instrument positioning module, and performs transformation processing on the dial image to transform the dial image into a standard dial image; The reading recognition module recognizes the meter reading according to the standard dial image.

4. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the meter reading recognition method according to any one of claims 1 to 3 is implemented.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the meter reading recognition method according to any one of claims 1 to 3 is implemented.

Images