Electric energy meter seal integrity detection method and system based on U-Net semantic segmentation

Abstract

The invention discloses an electric energy meter seal integrity detection method and system based on U-Net semantic segmentation, and the method comprises the steps: obtaining original electric energy meter image data in batches, carrying out the feature extraction of the original electric energy meter image data, and obtaining a seal region, an electric energy meter region and a background region; the extracted seal area, the extracted electric energy meter area and the extracted background area are marked in a partitioned mode in sequence, and an image data set with marks is obtained; constructing and training a U-Net semantic segmentation model based on the image data set, and constructing an electric energy meter seal integrity detection model based on the trained U-Net semantic segmentation model; inputting to-be-detected electric energy meter image data into the electric energy meter seal integrity detection model to obtain an abnormal detection result, namely the detection result of the electric energy meter seal integrity. According to the invention, batch electric energy meter seal integrity detection can be realized, and the detection accuracy is high.

Description

Technical Field

[0001] The present invention relates to the field of fault identification of electric energy metering devices, and in particular to an electric energy meter seal integrity detection method and system based on U-Net semantic segmentation. Background Art

[0002] The electric energy meter seal is a device that ensures that the internal structure and data of the electric energy meter are not illegally modified. It plays an important role in ensuring the fairness and reliability of the electric energy meter. Its integrity is an important item in the factory inspection of the metering device.

[0003] The existing technology usually uses image position recognition and contour recognition technology to detect the seal integrity of the energy meter. It usually requires the assembly line camera to take pictures of the energy meter at a set angle. The quality requirements for the original image data are high, and due to the influence of the photo angle, the seal integrity of the energy meter can only be detected one by one. However, with the increase in the number of energy meters, the existing technology can no longer fully meet the needs of energy meter seal integrity detection, and cannot realize batch energy meter seal integrity detection, and the detection accuracy is not high.

[0004] In view of this, this application is hereby filed. Summary of the invention

[0005] The purpose of the present invention is to provide an electric energy meter seal integrity detection method and system based on U-Net semantic segmentation. The constructed model has a simple structure, with only a small number of convolutional layers and pooling layers, and no full connection layer is required. The input image can be of any size and is suitable for integrity detection of seals of various types of electric energy meters. It has good adaptability to pipeline detection and can realize batch electric energy meter seal integrity detection with high detection accuracy.

[0006] The present invention is achieved through the following technical solutions:

[0007] In a first aspect, the present invention provides a method for detecting the seal integrity of an electric energy meter based on U-Net semantic segmentation, the method comprising:

[0008] Batch acquire original electric energy meter image data, perform feature extraction on the original electric energy meter image data, and obtain the seal area, electric energy meter area and background area;

[0009] The extracted seal area, electric energy meter area and background area are sequentially partitioned and labeled to obtain an image dataset with annotations; the image dataset is the dataset D containing semantics;

[0010] A U-Net semantic segmentation model is built and trained based on the image dataset, and an electric energy meter seal integrity detection model is built based on the trained U-Net semantic segmentation model.

[0011] The image data of the electric energy meter to be detected is input into the electric energy meter seal integrity detection model to obtain an abnormal detection result, which is the detection result of the electric energy meter seal integrity.

[0012] Furthermore, feature extraction is performed on the original electric energy meter image data, including:

[0013] An electric energy meter seal image extraction method based on modified discrete Gaussian-Hermitte moment is used to extract features from the original electric energy meter image data; among them, the electric energy meter seal image extraction method based on modified discrete Gaussian-Hermitte moment is a matching algorithm based on MIFT (mirror flip invariant feature transform), which realizes feature extraction of the original electric energy meter image data through seal scale space construction, seal image key point positioning, direction matching, descriptor generation and matching, and area positioning, and extracts the seal part of the electric energy meter.

[0014] Furthermore, the specific steps of the electric energy meter seal image extraction method based on the modified discrete Gauss-Hermite moment are as follows:

[0015] 1) Construction of seal scale space: The template image of the area to be extracted is obtained by manual cropping and screening, and the multi-scale feature space is generated using the modified discrete Gauss-Hermitian moment, and the scale space moment of the template image and the scale space moment of the electric energy meter image are calculated;

[0016] 2) Positioning the key points of the seal image: According to the scale space moment of the template image and the scale space moment of the electric energy meter image, the key points of the template image and the key points of the electric energy meter image are calculated respectively;

[0017] 3) Direction matching: For each key point in the template image and the electric energy meter image, the MDGHM amplitude and direction are calculated;

[0018] 4) Descriptor generation and matching: For each key point in the template image and the electric energy meter image, the Euclidean distance of each feature between all key points is calculated;

[0019] 5) Area positioning: Find the four pairs of key points with the smallest Euclidean distance from each Euclidean distance, where the closed rectangular area formed by the four pairs of key points corresponding to the electric energy meter area image is the area to be extracted.

[0020] The above steps 1) to 5) are used to sequentially complete the image extraction of the electric energy meter seal area, the electric energy meter area, and the background area.

[0021] Furthermore, the Euclidean distance calculation formula of each feature between all key points is:

[0022]

[0023] Among them, d(i,j) is the Euclidean distance between the calculated template image and the key points of the electric energy meter image. is the g-th feature of the i-th key point of the template image, is the g-th feature of the j-th key point of the electric energy meter image; N1 is the number of key points of the template image, and N2 is the number of key points of the electric energy meter image.

[0024] Furthermore, the key point calculation formula of the template image is:

[0025]

[0026] in: is the key point of the template image Tep; is the differential feature D of the template image Tep T The gradient of (x,y,s); H T D T The Hessian matrix of (x,y,s);

[0027] D T The calculation formula for (x, y, s) is: T (x,y,s)=L T (x,y,s)-L T (x,y,1.26×s);

[0028] The key point calculation formula of the electric energy meter image is:

[0029]

[0030] in: is the key point of the energy meter image Em; is the differential feature D of the energy meter image Em E The gradient of (x′,y′,s); H E D E The Hessian matrix of (x′,y′,s);

[0031] D E The calculation formula for (x′, y′, s) is: E (x′,y′,s)=L E (x′,y′,s)-L E (x′,y′,1.26×s).

[0032] Further, the structure of the U-Net semantic segmentation model includes a first convolutional layer, a first downsampling layer, a second convolutional layer, a second downsampling layer, a third convolutional layer, a third downsampling layer, a fourth convolutional layer, a first upsampling layer, a first connection layer, a first deconvolutional layer, a second upsampling layer, a second connection layer, a second deconvolutional layer, a third upsampling layer, a third connection layer and a third deconvolutional layer, which are connected in sequence;

[0033] The first convolutional layer is also connected to the third connection layer, the second convolutional layer is also connected to the second connection layer, and the third convolutional layer is also connected to the first connection layer;

[0034] Input the image X in the labeled image dataset into the first convolutional layer, Where H is the height of the image, W is the width of the image, and C is the number of channels of the image; the result is output from the third deconvolution layer.

[0035] Furthermore, the loss function of the U-Net semantic segmentation model is:

[0036]

[0037] Where m is a pixel; is the predicted probability; m is the target label; C class It is the type of classification, here it is 3, representing abnormal seal, normal seal and background respectively.

[0038] In a second aspect, the present invention further provides an electric energy meter seal integrity detection system based on U-Net semantic segmentation, the system comprising:

[0039] An acquisition unit, used for batch acquisition of original electric energy meter image data;

[0040] A feature extraction unit is used to extract features from the original electric energy meter image data to obtain a seal area, an electric energy meter area and a background area;

[0041] The annotation unit is used to sequentially perform partition annotation on the extracted seal area, electric energy meter area and background area to obtain an image data set with annotations; the image data set is the data set D containing semantics;

[0042] A model building and training unit, used to build and train a U-Net semantic segmentation model based on an image data set, and to build an electric energy meter seal integrity detection model based on the trained U-Net semantic segmentation model;

[0043] The detection unit is used to input the image data of the electric energy meter to be detected into the electric energy meter seal integrity detection model to obtain an abnormal detection result, which is the detection result of the electric energy meter seal integrity.

[0044] Furthermore, the feature extraction unit uses an electric energy meter seal image extraction method based on modified discrete Gaussian-Hermite moments to extract features from the original electric energy meter image data; wherein, the electric energy meter seal image extraction method based on modified discrete Gaussian-Hermite moments is a matching algorithm based on MIFT (mirror flip invariant feature transform), which realizes feature extraction of the original electric energy meter image data through seal scale space construction, seal image key point positioning, direction matching, descriptor generation and matching, and area positioning, and extracts the seal part of the electric energy meter.

[0045] In a third aspect, the present invention further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned electric energy meter seal integrity detection method based on U-Net semantic segmentation is implemented.

[0046] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0047] 1. The present invention provides an electric energy meter seal integrity detection method and system based on U-Net semantic segmentation. The U-Net semantic segmentation model constructed by the present invention has a simple structure, with only a small number of convolutional layers and pooling layers, and does not require a fully connected layer. The input image can be of any size and is suitable for integrity detection of seals of various types of electric energy meters. It has good adaptability to pipeline detection and can realize batch electric energy meter seal integrity detection with high detection accuracy.

[0048] 2. The present invention provides an electric energy meter seal integrity detection method and system based on U-Net semantic segmentation. The MIFT-based matching algorithm in the electric energy meter seal image extraction method based on modified discrete Gauss-Hermite moment has strong robustness to deformed electric energy meter seals and strong anti-noise ability, and can correctly identify seals in images with poor imaging effects; at the same time, the positioning of this method is more precise, and the seal position in the electric energy meter image can be found more accurately. This method does not use a neural network, does not require training data, and only needs to provide a template image to find the position of the seal in the electric energy meter image by itself. The computational complexity is low, the hardware requirements required for the algorithm are low, and it has a high computing speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0050] Figure 1 It is a flow chart of the electric energy meter seal integrity detection method based on U-Net semantic segmentation of the present invention;

[0051] Figure 2It is a detailed flow chart of the electric energy meter seal integrity detection method based on U-Net semantic segmentation of the present invention;

[0052] Figure 3 It is a structural diagram of the U-Net semantic segmentation model of the present invention;

[0053] Figure 4 This is a comparison diagram of the U-Net semantic segmentation model of the present invention and other models;

[0054] Figure 5 This is a structural block diagram of the electric energy meter seal integrity detection system based on U-Net semantic segmentation of the present invention. DETAILED DESCRIPTION

[0055] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.

[0056] Example 1

[0057] like Figure 1 and Figure 2 As shown, the electric energy meter seal integrity detection method based on U-Net semantic segmentation of the present invention comprises:

[0058] Step 1, batch-acquire original electric energy meter image data, perform feature extraction on the original electric energy meter image data, and obtain a seal area, an electric energy meter area, and a background area;

[0059] In this embodiment, the original electric energy meter image data is acquired in batches through the camera of the electric energy meter verification line, and the feature extraction is performed on the original electric energy meter image data to divide the electric energy meter image into a seal area, an electric energy meter area, and a background area;

[0060] Step 2: sequentially partition and annotate the extracted seal area, electric energy meter area, and background area to obtain an image dataset with annotations; the image dataset is the dataset D containing semantics;

[0061] In this embodiment, during the partition marking, the seal area labels are divided into three types: normal, abnormal and missing, and the electric energy meter area and the background area have no label classification.

[0062] In this embodiment, the feature extraction of the original electric energy meter image data is performed by using the electric energy meter seal image extraction method based on the modified discrete Gauss-Hermite moment; wherein, the electric energy meter seal image extraction method based on the modified discrete Gauss-Hermite moment is a matching algorithm based on MIFT (mirror flip invariant feature transform), which realizes feature extraction of the original electric energy meter image data through seal scale space construction, seal image key point positioning, direction matching, descriptor generation and matching and area positioning, and extracts the electric energy meter seal part. The above-mentioned method can realize the extraction of the deformed electric energy meter seal part, and can identify the seal part in the electric energy meter image data with poor imaging effect.

[0063] Specifically, the specific steps of the electric energy meter seal image extraction method based on the modified discrete Gauss-Hermitt moment are:

[0064] 1) Construction of seal scale space:

[0065] The template image Tep of the area to be extracted (seal area, electric energy meter area) is obtained by manual cropping and screening, and the modified discrete Gaussian-Hermitian moment (MDGHM) is used to generate a multi-scale feature space, that is, for each pixel position (x, y) in Tep and different scales s (the initial value of s is usually 1.4-1.6), the modified discrete Gaussian-Hermitian moment is calculated:

[0066]

[0067] In formula (1), η p,q (x, y, s) is the MDGHM local moment of pixel (x, y) in the template image at scale s; M, N are the mask sizes used for the local area, usually equal to 7; denote the horizontal and vertical parts of the Gauss-Hermitian polynomial respectively; σ is the Gaussian standard deviation. The calculation formula is:

[0068]

[0069] in:

[0070]

[0071] Calculate the scale space moment of the template image Tep:

[0072] L T (x,y,s)=η p,0 (x,y,s)+η 0,q (x,y,s)

[0073] The same operation is performed for the electric energy meter image Em that needs to be searched for seals: for each pixel (x′, y′) in Em, the modified discrete Gauss-Hermitian moment is calculated at different scales s:

[0074]

[0075] Then calculate the scale space moment of the energy meter image Em:

[0076] L E (x′,y′,s)=η p,0 (x′,y′,s)+η 0,q (x′,y′,s)

[0077] 2) Positioning of key points of seal image:

[0078] In order to accurately locate the image during image matching, it is necessary to calculate the key points of the image.

[0079] According to the scale space moment of the template image and the scale space moment of the electric energy meter image, the key points of the template image and the key points of the electric energy meter image are calculated respectively;

[0080] Specifically, the key point calculation formula of the template image Tep is:

[0081]

[0082] in: is the key point of the template image Tep; is the differential feature D of the template image Tep T The gradient of (x,y,s); H T D T The Hessian matrix of (x,y,s);

[0083] D T The calculation formula for (x, y, s) is: T (x,y,s)=L T (x,y,s)-L T (x,y,1.26×s);

[0084] In the same way, the key points of the electric energy meter image can be calculated as follows:

[0085]

[0086] in: is the key point of the energy meter image Em; is the differential feature D of the energy meter image Em E The gradient of (x′,y′,s); H E D E The Hessian matrix of (x′,y′,s);

[0087] D E The calculation formula for (x′, y′, s) is: E (x′,y′,s)=L E (x′,y′,s)-L E (x′,y′,1.26×s).

[0088] 3) Direction matching:

[0089] In order to ensure the invariance of the matching to rotation transformation, a direction needs to be assigned to each key point. For each key point in the template image and the electric energy meter image, a direction is assigned according to the local MDGHM magnitude and direction. For each key point in the template image Tep Calculate the magnitude and direction of its MDGHM:

[0090]

[0091] Among them, Am T is the magnitude of MDGHM of the template image Tep, Dr T is the direction of MDGHM of the template image Tep.

[0092] Similarly, calculate each key point of the energy meter image Em The magnitude and direction of the MDGHM:

[0093]

[0094] Among them, Am E is the magnitude of MDGHM of the energy meter image Em, Dr E is the direction of MDGHM of the electric energy meter image Em.

[0095] 4) Descriptor matching:

[0096] For each key point, a 16×16 pixel region centered on the key point is defined. This region is further divided into 4×4 sub-regions, each of which contains 4×4 pixels. In each sub-region, the gradient magnitude and gradient direction of each pixel are calculated using the following formula:

[0097]

[0098] G A is the gradient amplitude, G D is the gradient direction, and I is the pixel value.

[0099] Then the gradient information of the entire sub-region is attributed to a histogram with 8 directions. The specific operation is as follows:

[0100] Define 8 directions:

[0101] Direction 1: -π≤θ<-3π / 4 (approximately -180 degrees to -135 degrees);

[0102] Direction 2: -3π / 4≤θ<-π / 2 (approximately -135 degrees to -90 degrees);

[0103] Direction 3: -π / 2≤θ<-π / 4 (approximately -90 degrees to -45 degrees);

[0104] Direction 4: -π / 4≤θ<0 (approximately -45 degrees to 0 degrees);

[0105] Direction 5: 0≤θ<π / 4 (approximately 0 to 45 degrees);

[0106] Direction 6: π / 4≤θ<π / 2 (approximately 45 degrees to 90 degrees);

[0107] Direction 7: π / 2≤θ<3π / 4 (approximately 90 degrees to 135 degrees);

[0108] Direction 8: 3π / 4≤θ<π (approximately 135 degrees to 180 degrees);

[0109] The initial accumulated amplitude value in each direction is 0. The gradient direction calculated for each pixel is determined to be within the corresponding direction range. The gradient amplitude is added to the amplitude value of the corresponding direction. After traversing the 16 pixels in the sub-area, a histogram with 8 directions can be obtained, that is, an 8-dimensional array.

[0110] Each key point has 16 sub-regions, that is, each key point has 16×8=128 feature information, which can well describe the local structure of the image around the key point and is robust to transformations such as scale and rotation.

[0111] Each key point feature information of the template image Tep is Key Ti , i=1,2,…,N1, there are N1 key points in total, and the feature information of each key point of the energy meter image Em is Key Ej ,j=1,2,…,N2, there are N2 key points in total.

[0112] Then the Euclidean distance of 128 features between all key points is calculated using the following formula:

[0113]

[0114] Among them, d(i,j) is the Euclidean distance between the calculated template image and the key points of the electric energy meter image. is the g-th feature of the i-th key point of the template image, is the g-th feature of the j-th key point of the electric energy meter image; N1 is the number of key points of the template image, and N2 is the number of key points of the electric energy meter image.

[0115] 5) Region positioning: Find the four pairs of key points with the smallest Euclidean distance from each Euclidean distance in step 4), where the closed rectangular area formed by the four pairs of key points corresponding to the electric energy meter area image is the area to be extracted.

[0116] The above steps 1) to 5) are used to sequentially complete the image extraction of the electric energy meter seal area, the electric energy meter area, and the background area.

[0117] Step 3: construct and train a U-Net semantic segmentation model based on the image dataset, and construct an electric energy meter seal integrity detection model based on the trained U-Net semantic segmentation model;

[0118] In this embodiment, Figure 3 As shown, the structure of the U-Net semantic segmentation model includes a first convolutional layer, a first downsampling layer, a second convolutional layer, a second downsampling layer, a third convolutional layer, a third downsampling layer, a fourth convolutional layer, a first upsampling layer, a first connection layer, a first deconvolutional layer, a second upsampling layer, a second connection layer, a second deconvolutional layer, a third upsampling layer, a third connection layer and a third deconvolutional layer, which are connected in sequence; the first convolutional layer is also connected to the third connection layer, the second convolutional layer is also connected to the second connection layer, and the third convolutional layer is also connected to the first connection layer;

[0119] Input the image X in the labeled image dataset into the first convolutional layer, where X is the labeled image in the dataset D. Where H is the height of the image, W is the width of the image, and C is the number of channels of the image. Since all images are RGB images, C = 3; the results are output from the third deconvolution layer. The optimization method used by the U-Net semantic segmentation model is Adam optimization.

[0120] The U-Net semantic segmentation model has a symmetrical encoding-decoding structure that can extract feature information of different scales. Due to the full convolutional design of U-Net, it can adapt to a variety of input sizes and can easily improve the generalization ability of the model through data enhancement techniques (such as translation, rotation, scaling, etc.), thereby improving the accuracy of new data and having great advantages in small sample learning.

[0121] The structure of the U-Net semantic segmentation model is described as follows:

[0122] 1) Convolutional Layer

[0123] The convolution layer is a combination of a linear convolution operation and a nonlinear activation function, which is used to extract image features. First, set the convolution kernel where k h,k w are the height and width of the convolution kernel, both of which are 3, C in ,C out are the number of input and output channels, both of which are 3 here. The convolution operation can be expressed as:

[0124] X conv =ReLU(X*K+b)

[0125] Where X conv is the image after convolution, ReLU is the ReLU activation function, and b is the offset term in the training process.

[0126] 2) Downsampling layer

[0127] The downsampling layer uses the maximum pooling method to compress the image. The pooling window is 2×2, and the pooling operation can be expressed as:

[0128]

[0129] X pool This is the image after pooling. After each downsampling, the image will be reduced to 1 / 4 of its original size.

[0130] 3) Deconvolution layer

[0131] Deconvolution layer is a technique for upsampling through reverse convolution operation to increase the spatial resolution of feature maps. Its operation can be expressed as:

[0132]

[0133] Where (i ′ ,j′) is the coordinate of the output feature map, (i,j) is the coordinate of the input feature map, and s is the stride, which indicates the step length of the convolution kernel movement, which is 1 here.

[0134] 4) Upsampling layer

[0135] The upsampling layer is used to increase the spatial resolution of the feature map. Here, the bilinear difference method is used. The value of each output pixel is obtained by the difference of the four adjacent pixels in the input feature map. Its operation can be expressed as:

[0136]

[0137] in They respectively represent the displacement of the pixel relative to its original position.

[0138] Specifically, for semantic segmentation tasks, a pixel-level cross entropy loss function is usually used. For each pixel m, the target label is defined as y m , the predicted probability is The loss function of the U-Net semantic segmentation model is:

[0139]

[0140] Where m is a pixel; is the predicted probability; m is the target label; C class It is the type of classification, here it is 3, representing abnormal seal, normal seal and background respectively.

[0141] The above trained U-Net semantic segmentation model is used as the electricity meter seal integrity detection model.

[0142] Step 4: input the image data of the electric energy meter to be detected into the electric energy meter seal integrity detection model to obtain an abnormal prediction result, which is the detection result of the electric energy meter seal integrity.

[0143] In the specific implementation, the image data of electric energy meters collected on-site from a batch of single-phase electric energy meters and three-phase electric energy meters calibration lines are used as sample data.

[0144] S1: The feature extraction of image data is completed through seal scale space construction, seal image key point positioning, direction matching, descriptor generation and matching, and region positioning; then the extracted seal area, electric energy meter area, and background area are partitioned and labeled in turn, and a data set D for training the U-net semantic segmentation model is generated. The data set D contains positive samples: 1200 normal seals, negative samples: 300 abnormal seals, and 500 missing seals. The specific composition is shown in Table 1.

[0145] Table 1

[0146] Training set Test Set Qualified seal 2000 400 Abnormal Seal 400 200 Missing Seal 800 200 total 3200 800

[0147] S2: Figure 3 As shown in the figure, a U-net semantic segmentation model is constructed, and the dataset D is brought into the training model to start training. After the training is completed, the model performance is tested with the test set. The training was carried out for 9 iterations in total. The specific training and test results are shown in the figure. Figure 4 shown.

[0148] At the same time, in order to further demonstrate the electric energy meter seal abnormality recognition effect of the present invention, the method adopted by the present invention and the residual optimized convolutional neural network image recognition method (Res-CNN) are used to conduct a control experiment using the test set of data set D. The comparison effect is shown in Table 2.

[0149] Table 2

[0150]

[0151] S3: Input the image data of the electric energy meter to be detected into the trained U-net semantic segmentation model (i.e., the electric energy meter seal integrity detection model) to obtain the abnormality prediction result.

[0152] (1) The present invention proposes an electric energy meter seal image extraction method based on modified discrete Gaussian-Hermite moment. The method can automatically find the position of the seal in the electric energy meter image by only providing a template image, does not require a large number of training sets, and has good adaptability to application scenarios; (2) The present invention proposes a method for constructing an electric energy meter seal integrity detection model based on a U-Net semantic segmentation model. The constructed model has a simple structure, only a small number of convolutional layers and pooling layers, and does not require a fully connected layer. The input image can be of any size and is suitable for integrity detection of seals of various types of electric energy meters. It has good adaptability to pipeline detection.

[0153] Example 2

[0154] like Figure 5 As shown, the difference between this embodiment and embodiment 1 is that this embodiment provides an electric energy meter seal integrity detection system based on U-Net semantic segmentation, and the system corresponds to the electric energy meter seal integrity detection method based on U-Net semantic segmentation in embodiment 1 in a one-to-one manner; the system includes:

[0155] An acquisition unit, used for batch acquisition of original electric energy meter image data;

[0156] A feature extraction unit is used to extract features from the original electric energy meter image data to obtain a seal area, an electric energy meter area and a background area;

[0157] The annotation unit is used to sequentially perform partition annotation on the extracted seal area, electric energy meter area and background area to obtain an image data set with annotations; the image data set is the data set D containing semantics;

[0158] A model building and training unit, used to build and train a U-Net semantic segmentation model based on an image data set, and to build an electric energy meter seal integrity detection model based on the trained U-Net semantic segmentation model;

[0159] The detection unit is used to input the image data of the electric energy meter to be detected into the electric energy meter seal integrity detection model to obtain the detection result of the electric energy meter seal integrity.

[0160] As a further implementation, the feature extraction unit uses an electric energy meter seal image extraction method based on modified discrete Gaussian-Hermite moments to extract features from the original electric energy meter image data; wherein, the electric energy meter seal image extraction method based on modified discrete Gaussian-Hermite moments is a matching algorithm based on MIFT (mirror flip invariant feature transform), which realizes feature extraction of the original electric energy meter image data through seal scale space construction, seal image key point positioning, direction matching, descriptor generation and matching, and area positioning, and extracts the seal part of the electric energy meter.

[0161] Among them, the execution process of each unit can be executed according to the process steps of the electric energy meter seal integrity detection method based on U-Net semantic segmentation in Example 1, and will not be repeated one by one in this embodiment.

[0162] At the same time, the present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned electric energy meter seal integrity detection method based on U-Net semantic segmentation is implemented.

[0163] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0164] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0165] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0166] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0167] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. The electric energy meter seal integrity detection method based on U-Net semantic segmentation is characterized by: The method includes: Batch acquire original electric energy meter image data, perform feature extraction on the original electric energy meter image data, and obtain the seal area, electric energy meter area and background area; Sequentially partition and annotate the extracted seal area, electric energy meter area, and background area to obtain an annotated image dataset; Building and training a U-Net semantic segmentation model based on the image data set, and building an electric energy meter seal integrity detection model based on the trained U-Net semantic segmentation model; The image data of the electric energy meter to be detected is input into the electric energy meter seal integrity detection model to obtain an abnormal detection result, which is the detection result of the electric energy meter seal integrity.

2. The electric energy meter seal integrity detection method based on U-Net semantic segmentation according to claim 1 is characterized in that: Feature extraction of the original electric energy meter image data, including: An electric energy meter seal image extraction method based on modified discrete Gaussian-Hermite moment is used to extract features from the original electric energy meter image data; wherein, the electric energy meter seal image extraction method based on modified discrete Gaussian-Hermite moment is a matching algorithm based on MIFT, which realizes feature extraction of the original electric energy meter image data through seal scale space construction, seal image key point positioning, direction matching, descriptor generation and matching, and area positioning, and extracts the seal part of the electric energy meter.

3. The electric energy meter seal integrity detection method based on U-Net semantic segmentation according to claim 2 is characterized in that: The specific steps of the electric energy meter seal image extraction method based on modified discrete Gauss-Hermite moment are as follows: The template image of the area to be extracted is obtained by manual cropping and screening, and the multi-scale feature space is generated using the modified discrete Gauss-Hermitian moment, and the scale space moment of the template image and the scale space moment of the electric energy meter image are calculated; According to the scale space moment of the template image and the scale space moment of the electric energy meter image, respectively calculating the key points of the template image and the key points of the electric energy meter image; For each key point in the template image and the electric energy meter image, calculating the MDGHM magnitude and direction; For each key point in the template image and the electric energy meter image, calculating the Euclidean distance of each feature between all key points; The four pairs of key points with the smallest Euclidean distance are found from each Euclidean distance, and the closed rectangular area formed by the four pairs of key points corresponding to the electric energy meter area image is the area to be extracted.

4. The electric energy meter seal integrity detection method based on U-Net semantic segmentation according to claim 3 is characterized in that: The Euclidean distance calculation formula for each feature between all key points is: Among them, d(i,j) is the Euclidean distance between the calculated template image and the key points of the electric energy meter image. is the g-th feature of the i-th key point of the template image, is the g-th feature of the j-th key point of the electric energy meter image; N1 is the number of key points of the template image, and N2 is the number of key points of the electric energy meter image.

5. The electric energy meter seal integrity detection method based on U-Net semantic segmentation according to claim 1 is characterized in that: The key point calculation formula of the template image is: in: is the key point of the template image Tep; is the differential feature D of the template image Tep T The gradient of (x,y,s); H T D T The Hessian matrix of (x,y,s); D T The calculation formula for (x, y, s) is: T (x,y,s)=L T (x,y,s)-L T (x,y,1.26×s); The key point calculation formula of the electric energy meter image is: in: is the key point of the energy meter image Em; is the differential feature D of the energy meter image Em E The gradient of (x′,y′,s); H E D E The Hessian matrix of (x′,y′,s); D E The calculation formula for (x′, y′, s) is: E (x′,y′,s)=L E (x′,y′,s)-L E (x′,y′,1.26×s).

6. The electric energy meter seal integrity detection method based on U-Net semantic segmentation according to claim 1 is characterized in that: The structure of the U-Net semantic segmentation model includes a first convolutional layer, a first downsampling layer, a second convolutional layer, a second downsampling layer, a third convolutional layer, a third downsampling layer, a fourth convolutional layer, a first upsampling layer, a first connection layer, a first deconvolutional layer, a second upsampling layer, a second connection layer, a second deconvolutional layer, a third upsampling layer, a third connection layer and a third deconvolutional layer, which are connected in sequence; The first convolutional layer is further connected to the third connection layer, the second convolutional layer is further connected to the second connection layer, and the third convolutional layer is further connected to the first connection layer; Input the image X in the labeled image dataset into the first convolutional layer, Where H is the height of the image, W is the width of the image, and C is the number of channels of the image; the result is output from the third deconvolution layer.

7. The electric energy meter seal integrity detection method based on U-Net semantic segmentation according to claim 6 is characterized in that: The loss function of the U-Net semantic segmentation model is: Where m is a pixel; is the predicted probability; m is the target label; C class It is the type of classification, here it is 3, representing abnormal seal, normal seal and background respectively.

8. The electric energy meter seal integrity detection system based on U-Net semantic segmentation is characterized by: The system includes: An acquisition unit, used for batch acquisition of original electric energy meter image data; A feature extraction unit is used to extract features from the original electric energy meter image data to obtain a seal area, an electric energy meter area and a background area; A labeling unit, used to sequentially label the extracted seal area, electric energy meter area and background area to obtain a labeled image data set; A model building and training unit, used to build and train a U-Net semantic segmentation model based on the image data set, and to build an electric energy meter seal integrity detection model based on the trained U-Net semantic segmentation model; The detection unit is used to input the image data of the electric energy meter to be detected into the electric energy meter seal integrity detection model to obtain an abnormal detection result, which is the detection result of the electric energy meter seal integrity.

9. The electric energy meter seal integrity detection system based on U-Net semantic segmentation according to claim 8 is characterized in that: The feature extraction unit uses an electric energy meter seal image extraction method based on modified discrete Gauss-Hermitte moment to extract features from the original electric energy meter image data; wherein, the electric energy meter seal image extraction method based on modified discrete Gauss-Hermitte moment is a matching algorithm based on MIFT, which realizes feature extraction of the original electric energy meter image data through seal scale space construction, seal image key point positioning, direction matching, descriptor generation and matching, and area positioning, and extracts the electric energy meter seal part.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for detecting the seal integrity of an electric energy meter based on U-Net semantic segmentation is implemented.

Images