Substation monitoring image cleaning method and device, electronic equipment and storage medium

Abstract

The application provides a substation monitoring image cleaning method and device, electronic equipment and storage medium, wherein the normalized monitoring images of any two light rays are input into an image similarity evaluation model to obtain the similarity of the normalized monitoring images of any two light rays; the image similarity evaluation model comprises two image feature extraction layers with the same parameter structure, two hash conversion layers with the same parameter structure and one similarity prediction layer; the hash conversion layer comprises a fully connected layer with an output dimension of Mxb and a hyperbolic tangent activation layer, M is the output dimension of the feature extraction layer, and b>1; based on the similarity of the normalized monitoring images of any two light rays, the repeated labels of the normalized monitoring images of any two light rays are determined, and the image cleaning is performed based on the repeated labels of the normalized monitoring images of any two light rays, so that the recognition ability of the image cleaning method to slight differences is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of device monitoring, and in particular to a substation monitoring image cleaning method and device, electronic equipment and a storage medium. BACKGROUND

[0002] With the increasing scale of the power system, the operation of the equipment of the substation, especially the extra-high voltage substation, as an important hub of the power network, is directly related to the stable supply of electricity and the safety of the power system. Therefore, it is particularly important to monitor the substation equipment. At present, when monitoring the substation equipment, a calibration point is usually set for each device and a camera is installed at each calibration point to take pictures at regular intervals, for example, one monitoring image every 5 minutes. Such a scheme can provide relatively comprehensive device monitoring, but there are problems of redundant data and heavy data processing pressure. Among them, taking pictures at regular intervals at each calibration point will generate a large amount of picture data, especially for substations with a large number of devices, the data storage and transmission pressure is very large, and usually the device state does not change frequently, and regular photographing may generate a large number of repeated or invalid pictures, resulting in waste of storage resources and reduction of data processing efficiency. Therefore, it is necessary to clean the monitoring images of the substation to remove highly repetitive monitoring images and only keep the images that are helpful to determine whether the substation equipment is abnormal.

[0003] However, the abnormality of the substation equipment is usually reflected in subtle temperature, color or deformation changes, which are different from ordinary repetitive images. Even in approximately repetitive images, these subtle differences may indicate early signs of equipment failure. However, it is difficult to accurately distinguish whether two monitoring images are substantially different (i.e., the target region indicating the device state is different, such as instrument indicator lights, sensor readings, etc.) in the image cleaning process, and false image cleaning may mistakenly delete images containing minor abnormalities, thereby causing the device abnormality to be missed and affecting the safety of the substation equipment. SUMMARY

[0004] The present application provides a substation monitoring image cleaning method, device, electronic equipment and storage medium to solve the problem of insufficient accuracy of monitoring image cleaning in the prior art affecting the safety of the substation equipment.

[0005] The present application provides a substation monitoring image cleaning method, comprising:

[0006] Obtaining monitoring images of any calibration point corresponding to the substation equipment within a preset time period and performing light standardization processing on each monitoring image;

[0007] input the two light normalized monitoring images to an image similarity evaluation model to obtain a similarity of the two light normalized monitoring images output by the image similarity evaluation model;

[0008] based on the similarity of the two light normalized monitoring images, determine a repeated label of the two light normalized monitoring images, and perform image cleaning based on the repeated label of the two light normalized monitoring images;

[0009] The image similarity evaluation model comprises two image feature extraction layers with the same parameter structure, two hash conversion layers with the same parameter structure, and a similarity prediction layer, the feature extraction layers, the hash conversion layers and the similarity prediction layer are sequentially connected, the hash conversion layer comprises a fully connected layer with an output dimension of M x b and a hyperbolic tangent activation layer, M is an output dimension of the feature extraction layer, and b > 1.

[0010] According to the substation monitoring image cleaning method provided by the application, the image similarity evaluation model is trained in the following manner:

[0011] input any sample image pair in a sample image pair set to the image similarity evaluation model to obtain a similarity of the sample image pair output by the similarity prediction layer and a hash vector of each sample image in the sample image pair output by the hash conversion layer;

[0012] construct a predicted similarity matrix based on the similarity of each sample image pair output by the image similarity evaluation model, construct a hash splicing vector based on the hash vector of each sample image, and construct a labeled similarity matrix based on the labeled similarity of each sample image pair;

[0013] determine a similarity loss based on the difference between the predicted similarity matrix and the labeled similarity matrix;

[0014] determine a binarization loss based on the product of the transpose of the hash splicing vector and the hash splicing vector and a unit matrix;

[0015] determine a diversification loss based on the hash splicing vector and an all-1 vector;

[0016] adjust the parameters of the image similarity evaluation model based on the similarity loss, the binarization loss and the diversification loss.

[0017] According to the substation monitoring image cleaning method provided by the application, the binarization loss is the Euclidean distance between the first normalized matrix obtained by normalizing the product of the transpose of the hash splicing vector and the hash splicing vector and a unit matrix.

[0018] According to the substation monitoring image cleaning method provided by the application, the diversity loss is the Frobenius norm of a second normalized matrix obtained by normalizing the product between the transpose of the hash splicing vector and a full one vector.

[0019] According to the substation monitoring image cleaning method provided by the application, the similarity of the arbitrary two light normalized monitoring images is determined based on the following manner:

[0020] The image features of the arbitrary two light normalized monitoring images are extracted based on the two image feature extraction layers respectively, to obtain the image features of the arbitrary two light normalized monitoring images.

[0021] The hash vectors of the arbitrary two light normalized monitoring images are obtained by performing hash conversion on the image features corresponding to the arbitrary two light normalized monitoring images based on the two hash conversion layers respectively.

[0022] The hash vectors of the arbitrary two light normalized monitoring images are divided into M b-bit sub-vectors respectively.

[0023] Based on the similarity prediction layer, the maximum difference of the corresponding sub-vector is obtained by calculating the difference of the sub-vectors of the arbitrary two light normalized monitoring images bit by bit, and the similarity of the arbitrary two light normalized monitoring images is determined based on the sum of the maximum differences of all the sub-vectors of the arbitrary two light normalized monitoring images.

[0024] According to the substation monitoring image cleaning method provided by the application, the light normalization processing of each monitoring image comprises:

[0025] Any monitoring image is decomposed into channel images under each channel.

[0026] The channel images under each channel are processed based on the light normalization vectors under each channel respectively, to obtain the normalized images of each channel image.

[0027] The normalized images of each channel image are fused.

[0028] According to the substation monitoring image cleaning method provided by the application, the light normalization vector under any channel is determined based on the following steps:

[0029] Based on the standard light image under the any channel and the corresponding non-standard light image, the curve second derivative prediction network corresponding to the any channel is trained to obtain the trained curve second derivative network.

[0030] inputting the channel image under any channel into the trained second derivative network of the curve to obtain a second derivative of a curve corresponding to the light normalization vector under the any channel;

[0031] restoring the second derivative of the curve corresponding to the light normalization vector under the any channel to obtain the light normalization vector under the any channel.

[0032] The application further provides a substation monitoring image cleaning device, comprising:

[0033] an image acquisition unit configured to acquire monitoring images of any calibration point of a substation device taken within a preset time period and perform light normalization processing on each monitoring image;

[0034] a similarity evaluation unit configured to input any two light normalized monitoring images into an image similarity evaluation model to obtain a similarity of the any two light normalized monitoring images output by the image similarity evaluation model;

[0035] an image cleaning unit configured to determine a duplicate label of the any two light normalized monitoring images based on the similarity of the any two light normalized monitoring images and perform image cleaning based on the duplicate label of the any two light normalized monitoring images.

[0036] The image similarity evaluation model comprises two image feature extraction layers having the same parameter structure, two hash conversion layers having the same parameter structure, and a similarity prediction layer, the feature extraction layers, the hash conversion layers, and the similarity prediction layer are sequentially connected, the hash conversion layer comprises a fully connected layer with an output dimension of M x b and a hyperbolic tangent activation layer, M is the output dimension of the feature extraction layer, and b>1.

[0037] The application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the substation monitoring image cleaning method as described above when executing the program.

[0038] The application further provides a non-transitory computer readable storage medium having a computer program stored thereon, and the computer program is executable on a processor to implement the substation monitoring image cleaning method as described above.

[0039] The application further provides a computer program product comprising a computer program, and the computer program is executable on a processor to implement the substation monitoring image cleaning method as described above.

[0040] The substation monitoring image cleaning method, apparatus, electronic device, and storage medium provided by this invention input any two light-standardized monitoring images into an image similarity evaluation model to obtain the similarity between the two light-standardized monitoring images output by the image similarity evaluation model. The image similarity evaluation model includes two image feature extraction layers with identical parameter structures, two hash conversion layers with identical parameter structures, and a similarity prediction layer, which are sequentially connected. The hash conversion layer includes a fully connected layer with an output dimension of M×b and a hyperbolic tangent activation layer, where M is the output dimension of the feature extraction layer and b>1. Based on the similarity between any two light-standardized monitoring images, duplicate labels of the two light-standardized monitoring images are determined, and image cleaning is performed based on these duplicate labels. This improves the image cleaning method's ability to identify subtle differences and enhances its accuracy. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0042] Figure 1 This is a flowchart illustrating the substation monitoring image cleaning method provided by the present invention.

[0043] Figure 2 This is a schematic diagram of the image similarity evaluation model provided by the present invention;

[0044] Figure 3 This is a schematic diagram of the operation flow of the image similarity evaluation model provided by the present invention;

[0045] Figure 4 This is a schematic diagram of the substation monitoring image cleaning device provided by the present invention;

[0046] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0047] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in conjunction with the drawings in the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work belong to the protection scope of the present application.

[0048] Figure 1 is a flowchart of a substation monitoring image cleaning method provided by the present application, as shown in Figure 1 , the method comprises:

[0049] Step 110, acquiring monitoring images taken by any calibration point corresponding to substation equipment within a preset time period and performing light standardization processing on each monitoring image;

[0050] Step 120, inputting any two light-standardized monitoring images into an image similarity evaluation model to obtain the similarity of the two light-standardized monitoring images output by the image similarity evaluation model;

[0051] Step 130, determining the repeated labels of the two light-standardized monitoring images based on the similarity of the two light-standardized monitoring images, and performing image cleaning based on the repeated labels of the two light-standardized monitoring images;

[0052] , the image similarity evaluation model comprises two image feature extraction layers with the same parameter structure, two hash conversion layers with the same parameter structure, and a similarity prediction layer, the feature extraction layer, the hash conversion layer and the similarity prediction layer are connected in sequence; the hash conversion layer comprises a fully connected layer with an output dimension of Mxb and a hyperbolic tangent activation layer, M is the output dimension of the feature extraction layer, and b>1.

[0053] Here, all monitoring images taken within a preset time period can be acquired from any monitoring calibration point of the substation equipment. Since the monitoring images will show light differences due to changes in time, weather and lighting conditions, each monitoring image can be subjected to light standardization processing in order to eliminate the influence of ambient light on image content, so that the images look more consistent under different light conditions and reduce unnecessary feature differences.

[0054] In some embodiments, considering that the monitoring image is generally an RGB image, in order to perform light normalization on any monitoring image, the monitoring image can be channel-decomposed to obtain channel images under each channel (R channel, G channel and B channel). Since the channel images under different channels have different characteristics, and the channel images corresponding to the same channel of different monitoring images also have different characteristics due to different light conditions. Therefore, for the current channel image, the light normalization vectors corresponding to each channel of the channel image can be obtained in advance for processing the channel image of the corresponding channel. Subsequently, the channel images under each channel can be processed based on the light normalization vectors under each channel respectively to obtain normalized images of each channel image, and then the normalized images of each channel image are fused to realize the light normalization of the monitoring image.

[0055] In other embodiments, in order to establish the light normalization vectors corresponding to each channel of the channel image, for any channel, the curve second derivative prediction network corresponding to the channel can be trained in advance based on the standard light image under the channel and the non-standard light image corresponding thereto to obtain the trained curve second derivative network. Wherein, the curve second derivative prediction network can be constructed based on a convolutional neural network, and the last layer thereof is a RELU layer to ensure that the output second derivative is less than 0, thereby ensuring that the curve corresponding to the subsequent restored light normalization vector is a monotonically increasing concave curve. Specifically, in training the curve second derivative prediction network, the non-standard light image under the channel can be input into the initial network to obtain the second derivative of the curve corresponding to the light normalization vector under the channel predicted by the initial network, the second derivative of the curve corresponding to the light normalization vector under the channel is restored to obtain the light normalization vector under the channel. The non-standard light image is converted based on the light normalization vector to obtain a predicted normalized image, and then the predicted normalized image and the corresponding standard light image are compared based on an image comparison network to obtain a comparison result, so that the parameters of the initial network are adjusted according to the comparison result to obtain the curve second derivative prediction network. After obtaining the curve second derivative prediction network corresponding to the channel, the channel image under the channel can be input into the curve second derivative network to obtain the second derivative of the curve corresponding to the light normalization vector under the corresponding channel, and then the second derivative of the curve corresponding to the light normalization vector under the channel is restored to obtain the light normalization vector under the channel.

[0056] Wherein, in restoring the second derivative output by the curve second derivative prediction network, a preset lower triangular matrix can be multiplied by the negative of the second derivative to obtain a first product, a preset upper triangular matrix can be multiplied by the first product to obtain a second product, and then the diagonal elements of the second product are combined into a vector form after normalization, that is, the corresponding light normalization vector is obtained.

[0057] The light normalized monitoring images are input into the image similarity evaluation model in pairs, and the similarity of each pair of monitoring images is calculated. The image similarity evaluation model can use a deep learning-based feature extraction technology, such as a convolutional neural network, to extract image features of the image pair for similarity evaluation based on the image features of the image pair. Considering that the state of the substation equipment does not change frequently, and the abnormality of the substation equipment is usually reflected in subtle temperature, color or deformation changes, more attention should be paid to the local similarity of the images when evaluating the similarity of the image pair to improve the accuracy of image cleaning of the substation monitoring images.

[0058] To achieve this purpose, as shown in Figure 2 The image similarity evaluation model can be constructed based on two image feature extraction layers with the same parameter structure, two hash conversion layers with the same parameter structure, and one similarity prediction layer, wherein the feature extraction layers, the hash conversion layers, and the similarity prediction layer are connected in sequence. In some embodiments, as shown in Figure 3 The specific operation process of the image similarity evaluation model can be as follows:

[0059] Step 310, respectively based on two said image feature extraction layers, the features of the said two arbitrary light normalized monitoring images are extracted, and the image features of the said two arbitrary light normalized monitoring images are obtained;

[0060] Step 320, respectively based on two said hash conversion layers, the image features corresponding to the said two arbitrary light normalized monitoring images are converted into hash vectors, and the hash vectors of the said two arbitrary light normalized monitoring images are obtained;

[0061] Step 330, respectively, the hash vectors of the said two arbitrary light normalized monitoring images are divided into M b-bit sub-vectors;

[0062] Step 340, based on the similarity prediction layer, the difference values of the sub-vectors of the said two arbitrary light normalized monitoring images are calculated, the maximum difference values of the corresponding sub-vectors are obtained, and based on the sum of the maximum difference values of all sub-vectors of the said two arbitrary light normalized monitoring images, the similarity of the said two arbitrary light normalized monitoring images is determined.

[0063] Specifically, assuming that the image pair input to the model is represented as (A, B), two image feature extraction layers are used to extract features of image A and image B, respectively, to obtain image features of image A and image B. In some embodiments, the image feature extraction layer can be constructed based on a convolutional neural network. The hash conversion layers connected to the two image feature extraction layers, respectively, perform hash conversion on the image features of image A and image B, respectively, to obtain hash vectors of image A and image B. Wherein, the hash vectors of different images are different, and the hash vectors of the same image are the same. Next, the hash vectors of image A and image B will enter the similarity prediction layer for similarity calculation to obtain the similarity of image A and image B.

[0064] It is worth noting that, unlike the way of generating hash vectors by using Sign function in the existing hash conversion method in the image field, in the embodiment of the present application, when constructing the hash conversion layer, considering the non-differentiable characteristics of the Sign function, if the hash vector is generated by using the Sign function, the problem of gradient disappearance will be caused, which will cause the image similarity evaluation model trained by using the gradient descent algorithm to fail to complete the training process. Therefore, in the embodiment of the present application, a fully connected layer with an output dimension of M x b and a hyperbolic tangent activation layer are used to construct the hash conversion layer to simulate the hash conversion to obtain a hash vector similar to a binary vector, and since the output result of the hyperbolic tangent activation layer is derivable, the above problem of gradient disappearance can be solved. Wherein, M is the output dimension of the feature extraction layer, and b > 1. More importantly, the hash vector is composed of M b-bit vectors, each b-bit vector corresponds to a point in the image feature (i.e. a local region in the original image), and therefore the hash vector contains the features of each local region of the corresponding image. The similarity prediction layer can compare and evaluate each local region of the image region by region by comparing the hash vectors of the two images, and therefore it is more sensitive to the slight differences in the local regions, and the accuracy of the image pair similarity evaluation is higher.

[0065] Further, in order to further improve the sensitivity of the image similarity evaluation model to the slight differences in the substation monitoring images, the hash vectors of image A and image B can be divided into M b-bit sub-vectors H1, H2,..., H M , wherein H i = (h1, h2,..., hb). Subsequently, the similarity prediction layer can calculate the difference between the sub-vectors of image A and image B bit by bit to obtain the maximum difference of the corresponding sub-vector. Taking the i-th sub-vector as an example, the difference between the positions in H i of image A and image B is calculated bit by bit, and the maximum difference is selected. Based on the sum of the above maximum differences of all sub-vectors of image A and image B, the similarity of image A and image B is determined. Wherein, the similarity of image A and image B can be calculated by using the following formula:

[0066] Sim(A, B) = 1 - ∑dif(i) / 2M

[0067] wherein dif(i) is the maximum difference value of the i-th sub-vector of image A and image B.

[0068] In some embodiments, the image similarity evaluation model is trained based on the following manner:

[0069] inputting any sample image pair in the set of sample image pairs into the image similarity evaluation model to obtain the similarity of the sample image pair output by the similarity prediction layer and the hash vectors of each sample image in the sample image pair output by the hash conversion layer. Wherein, the running mechanism of the image similarity evaluation model in the training stage is the same as the running mechanism given in the above embodiments, which will not be repeated here.

[0070] constructing a predicted similarity matrix based on the similarities of each sample image pair output by the image similarity evaluation model, constructing a hash splicing vector based on the hash vectors of each sample image, and constructing a labeled similarity matrix based on the labeled similarities of each sample image pair. Wherein, the labeled similarity of any sample image pair is 1 or 0.

[0071] On the one hand, the similarity loss can be determined based on the difference between the predicted similarity matrix and the labeled similarity matrix to guide the entire image similarity evaluation model to better learn the same points and local minor differences between the input image pairs, and to improve the evaluation accuracy of the model for the similarity of the image pairs. In some embodiments, the Euclidean distance between the predicted similarity matrix and the labeled similarity matrix can be calculated, and the similarity loss can be obtained by dividing the Euclidean distance by the total number of image pairs.

[0072] On the other hand, since the hash conversion layer uses a fully connected layer and a hyperbolic tangent activation layer to simulate hash conversion to generate hash vectors that are approximately binary vectors, in order to guide the hash conversion layer to generate vectors with elements closer to the extreme values of {-1, 1}, a binary loss can be determined based on the product of the transpose of the hash splicing vector and the hash splicing vector and the unit matrix. In some embodiments, the Euclidean distance between the first normalized matrix obtained by normalizing the product of the transpose of the hash splicing vector and the hash splicing vector and the unit matrix can be determined as the binary loss.

[0073] In addition, a diversification loss can also be determined based on the hash splicing vector and the all-1 vector to guide the hash conversion layer to generate more diversified hash vectors, thereby further reducing the probability of collision of hash vectors of different images. In some embodiments, the Frobenius norm of the second normalized matrix obtained by normalizing the product between the transpose of the hash splicing vector and the all-1 vector can be determined as the diversification loss.

[0074] Subsequently, based on the similarity loss, the binarization loss and the diversification loss, the parameters of the image similarity evaluation model are adjusted in reverse based on a gradient descent algorithm, and one round of training of the image similarity evaluation model is completed. Through repeated training, a trained image similarity evaluation model can be obtained.

[0075] After obtaining the similarity of any two light normalized monitoring images, the similarity threshold can be used to label each monitoring image pair as "repeated" or "non-repeated". Based on the repeated label of each monitoring image pair, a large number of similar images can be cleaned up to remove redundant data. For example, for multiple monitoring images that are repeated with each other, only the best quality or the latest time image can be retained, and the remaining repeated images can be deleted, thereby reducing the pressure of data transmission and image storage and the difficulty of subsequent image analysis.

[0076] In summary, the method provided by the embodiments of the present application inputs any two light normalized monitoring images into an image similarity evaluation model to obtain the similarity of the any two light normalized monitoring images output by the image similarity evaluation model. The image similarity evaluation model includes two image feature extraction layers with the same parameter structure, two hash conversion layers with the same parameter structure, and a similarity prediction layer. The feature extraction layer, the hash conversion layer, and the similarity prediction layer are connected in sequence. The hash conversion layer includes a fully connected layer with an output dimension of Mxb and a hyperbolic tangent activation layer, where M is the output dimension of the feature extraction layer, and b>1. Based on the similarity of any two light normalized monitoring images, the repeated label of any two light normalized monitoring images is determined, and image cleaning is performed based on the repeated label of any two light normalized monitoring images, thereby improving the recognition ability of the image cleaning method for small differences and improving the accuracy of the image cleaning method.

[0077] The substation monitoring image cleaning device provided by the present application is described below. The substation monitoring image cleaning device described below can be referred to in conjunction with the substation monitoring image cleaning method described above.

[0078] Based on any of the above embodiments, Figure 4 is a structural schematic diagram of the substation monitoring image cleaning device provided by the present application, as Figure 4 shown, the device comprises:

[0079] The image acquisition unit 410 is configured to acquire monitoring images of any calibration point corresponding to the substation equipment within a preset time period and perform light normalization on each monitoring image.

[0080] The similarity evaluation unit 420 is configured to input any two light ray normalized monitoring images into an image similarity evaluation model to obtain a similarity of the any two light ray normalized monitoring images output by the image similarity evaluation model.

[0081] The image cleaning unit 430 is configured to determine a repetition label of the any two light ray normalized monitoring images based on the similarity of the any two light ray normalized monitoring images, and perform image cleaning based on the repetition label of the any two light ray normalized monitoring images.

[0082] The image similarity evaluation model comprises two image feature extraction layers with the same parameter structure, two hash conversion layers with the same parameter structure, and a similarity prediction layer, the feature extraction layers, the hash conversion layers, and the similarity prediction layer are sequentially connected, the hash conversion layer comprises a fully connected layer with an output dimension of Mxb and a hyperbolic tangent activation layer, M is an output dimension of the feature extraction layer, and b>1.

[0083] The device provided by the embodiment of the application is configured to input any two light ray normalized monitoring images into an image similarity evaluation model to obtain a similarity of the any two light ray normalized monitoring images output by the image similarity evaluation model, wherein the image similarity evaluation model comprises two image feature extraction layers with the same parameter structure, two hash conversion layers with the same parameter structure, and a similarity prediction layer, the feature extraction layers, the hash conversion layers, and the similarity prediction layer are sequentially connected, the hash conversion layer comprises a fully connected layer with an output dimension of Mxb and a hyperbolic tangent activation layer, M is an output dimension of the feature extraction layer, and b>1, a repetition label of the any two light ray normalized monitoring images is determined based on the similarity of the any two light ray normalized monitoring images, and image cleaning is performed based on the repetition label of the any two light ray normalized monitoring images, thereby improving the recognition ability of the image cleaning method to slight differences and improving the accuracy of the image cleaning method.

[0084] Based on any one of the above embodiments, the image similarity evaluation model is trained based on the following manner:

[0085] Any sample image pair in a sample image pair set is input into the image similarity evaluation model to obtain a similarity of the any sample image pair output by the similarity prediction layer and a hash vector of each sample image in the any sample image pair output by the hash conversion layer.

[0086] construct a predicted similarity matrix based on the similarity of each sample image pair output by the image similarity evaluation model, construct a hash splicing vector based on the hash vector of each sample image, and construct a labeled similarity matrix based on the labeled similarity of each sample image pair;

[0087] determine a similarity loss based on the difference between the predicted similarity matrix and the labeled similarity matrix;

[0088] determine a binarization loss based on the product of the transpose of the hash splicing vector and the hash splicing vector and the unit matrix;

[0089] determine a diversification loss based on the hash splicing vector and the all-1 vector;

[0090] adjust the parameters of the image similarity evaluation model based on the similarity loss, the binarization loss, and the diversification loss.

[0091] Based on any of the above embodiments, the binarization loss is the Euclidean distance between the first normalized matrix obtained by normalizing the product of the transpose of the hash splicing vector and the hash splicing vector, and the unit matrix.

[0092] Based on any of the above embodiments, the diversification loss is the Frobenius norm of the second normalized matrix obtained by normalizing the product of the transpose of the hash splicing vector and the all-1 vector.

[0093] Based on any of the above embodiments, the similarity of the two arbitrary light normalized monitoring images is determined based on the following manner:

[0094] extract image features of the two arbitrary light normalized monitoring images based on the two image feature extraction layers respectively, to obtain the image features of the two arbitrary light normalized monitoring images;

[0095] perform hash conversion on the image features corresponding to the two arbitrary light normalized monitoring images based on the two hash conversion layers respectively, to obtain hash vectors of the two arbitrary light normalized monitoring images;

[0096] divide the hash vectors of the two arbitrary light normalized monitoring images into M b-bit sub-vectors respectively;

[0097] based on the similarity prediction layer, calculate the maximum difference value of the corresponding sub-vector by calculating the difference value of the sub-vectors of the two arbitrary light normalized monitoring images bit by bit, and determine the similarity of the two arbitrary light normalized monitoring images based on the sum of the maximum difference values of all sub-vectors of the two arbitrary light normalized monitoring images.

[0098] According to any of the above embodiments, the light normalization processing on each monitoring image comprises:

[0099] channel decomposition is performed on any monitoring image to obtain a channel image under each channel;

[0100] the channel image under each channel is processed based on the light normalization vector under the corresponding channel to obtain a normalized image of each channel image;

[0101] the normalized images of each channel image are fused.

[0102] According to any of the above embodiments, the light normalization vector under any channel is determined based on the following steps:

[0103] based on the standard light graph under the any channel and the corresponding non-standard light graph, a curve second derivative prediction network corresponding to the any channel is trained to obtain a trained curve second derivative network;

[0104] the channel image under the any channel is input into the trained curve second derivative network to obtain a second derivative of a curve corresponding to the light normalization vector under the corresponding channel;

[0105] the second derivative of the curve corresponding to the light normalization vector under the any channel is restored to obtain the light normalization vector under the any channel.

[0106] Figure 5 is a structural schematic diagram of an electronic device provided by the present application, such as Figure 5As shown, the electronic device can include a processor 510, a memory 520, a communications interface 530, and a communications bus 540, wherein the processor 510, the memory 520, and the communications interface 530 complete mutual communication through the communications bus 540. The processor 510 can invoke a logic instruction in the memory 520 to execute a substation monitoring image cleaning method, which includes: acquiring monitoring images of any calibration point corresponding to substation equipment within a preset time period and performing light standardization processing on each monitoring image; inputting any two light-standardized monitoring images into an image similarity evaluation model to obtain a similarity of the any two light-standardized monitoring images output by the image similarity evaluation model; determining a repetition label of the any two light-standardized monitoring images based on the similarity of the any two light-standardized monitoring images, and performing image cleaning based on the repetition label of the any two light-standardized monitoring images; wherein the image similarity evaluation model includes two image feature extraction layers with the same parameter structure, two hash conversion layers with the same parameter structure, and a similarity prediction layer, the feature extraction layer, the hash conversion layer, and the similarity prediction layer are connected in sequence; the hash conversion layer includes a fully connected layer with an output dimension of Mxb and a hyperbolic tangent activation layer, M is the output dimension of the feature extraction layer, and b>1.

[0107] In addition, the logic instruction in the memory 520 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0108] In another aspect, the present application also provides a computer program product, which comprises a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions that, when executed by a computer, enable the computer to perform the substation monitoring image cleaning method provided by any of the above methods, the method comprising: obtaining monitoring images of any calibration point corresponding to substation equipment taken within a preset time period and performing light standardization processing on each monitoring image; inputting any two light-standardized monitoring images into an image similarity evaluation model to obtain a similarity of the any two light-standardized monitoring images output by the image similarity evaluation model; determining a duplicate label of the any two light-standardized monitoring images based on the similarity of the any two light-standardized monitoring images, and performing image cleaning based on the duplicate label of the any two light-standardized monitoring images; wherein the image similarity evaluation model comprises two image feature extraction layers with the same parameter structure, two hash conversion layers with the same parameter structure, and a similarity prediction layer, the feature extraction layers, the hash conversion layers, and the similarity prediction layer are connected in sequence; the hash conversion layer comprises a fully connected layer with an output dimension of Mxb, and a hyperbolic tangent activation layer, M is the output dimension of the feature extraction layer, and b>1.

[0109] In yet another aspect, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a substation monitoring image cleaning method provided by any of the above methods, the method comprising: obtaining monitoring images of any calibration point corresponding to substation equipment taken within a preset time period and performing light standardization processing on each monitoring image; inputting any two light-standardized monitoring images into an image similarity evaluation model to obtain a similarity of the any two light-standardized monitoring images output by the image similarity evaluation model; determining a duplicate label of the any two light-standardized monitoring images based on the similarity of the any two light-standardized monitoring images, and performing image cleaning based on the duplicate label of the any two light-standardized monitoring images; wherein the image similarity evaluation model comprises two image feature extraction layers with the same parameter structure, two hash conversion layers with the same parameter structure, and a similarity prediction layer, the feature extraction layers, the hash conversion layers, and the similarity prediction layer are connected in sequence; the hash conversion layer comprises a fully connected layer with an output dimension of Mxb, and a hyperbolic tangent activation layer, M is the output dimension of the feature extraction layer, and b>1.

[0110] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purposes of the embodiments according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0111] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and the necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0112] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A substation monitoring image cleaning method characterized by, The method comprises: acquiring monitoring images of any calibration point corresponding to substation equipment within a preset time period and performing light standardization processing on each monitoring image; inputting any two light-standardized monitoring images into an image similarity evaluation model to obtain a similarity of the any two light-standardized monitoring images output by the image similarity evaluation model; based on the similarity of any two light-standardized monitoring images, determining a repeated label of the any two light-standardized monitoring images, and performing image cleaning based on the repeated label of any two light-standardized monitoring images; wherein the image similarity evaluation model comprises two image feature extraction layers with the same parameter structure, two hash conversion layers with the same parameter structure, and a similarity prediction layer, the feature extraction layers, the hash conversion layers, and the similarity prediction layer are connected in sequence; the hash conversion layer comprises a fully connected layer with an output dimension of Mxb and a hyperbolic tangent activation layer, M is the output dimension of the feature extraction layer, and b>1; the similarity of the any two light-standardized monitoring images is determined based on the following way: respectively based on two image feature extraction layers, the image features of the any two light-standardized monitoring images are extracted to obtain the image features of the any two light-standardized monitoring images; respectively based on two hash conversion layers, the image features corresponding to the any two light-standardized monitoring images are hashed to obtain hash vectors of the any two light-standardized monitoring images; respectively, the hash vectors of the any two light-standardized monitoring images are divided into M b-bit sub-vectors; based on the similarity prediction layer, the difference values of the sub-vectors of the any two light-standardized monitoring images are calculated bit by bit to obtain the maximum difference values of the corresponding sub-vectors, and based on the sum of the maximum difference values of all sub-vectors of the any two light-standardized monitoring images, the similarity of the any two light-standardized monitoring images is determined.

2. The substation monitoring image washing method of claim 1, wherein, The image similarity evaluation model is trained based on the following way: inputting any sample image pair in a sample image pair set into the image similarity evaluation model to obtain the similarity of the sample image pair output by the similarity prediction layer and the hash vector of each sample image in the sample image pair output by the hash conversion layer; based on the similarity of each sample image pair output by the image similarity evaluation model, a predicted similarity matrix is constructed, a hash splicing vector is constructed based on the hash vector of each sample image, and a labeled similarity matrix is constructed based on the labeled similarity of each sample image pair; based on the difference between the predicted similarity matrix and the labeled similarity matrix, a similarity loss is determined; based on the product of the transpose of the hash splicing vector and the hash splicing vector and the unit matrix, a binary loss is determined; based on the hash splicing vector and the all-1 vector, a diversification loss is determined; Adjust parameters of the image similarity evaluation model based on the similarity loss, the binarization loss, and the diversification loss.

3. The substation monitoring image washing method according to claim 2, characterized by, The binarization loss is a Euclidean distance between a first normalized matrix obtained by normalizing a product of a transpose of the hash concatenation vector and the hash concatenation vector and a unit matrix.

4. The substation monitoring image washing method of claim 2, wherein, The diversification loss is a Frobenius norm of a second normalized matrix obtained by normalizing a product of a transpose of the hash concatenation vector and an all-1 vector.

5. The substation monitoring image washing method of claim 2, wherein, The light normalization processing of each monitoring image comprises: channel decomposition of any monitoring image to obtain channel images under each channel; processing of the channel images under each channel based on the light normalization vectors under the corresponding channels to obtain normalized images of the channel images; fusion of the normalized images of the channel images.

6. The substation monitoring image washing method of claim 5, wherein, The light normalization vector under any channel is determined based on the following steps: training of a curve second derivative prediction network corresponding to the any channel based on the standard light image and the corresponding non-standard light image under the any channel to obtain a trained curve second derivative network; input of the channel image under the any channel into the trained curve second derivative network to obtain a second derivative of a curve corresponding to the light normalization vector under the corresponding channel; restoration of the second derivative of the curve corresponding to the light normalization vector under the any channel to obtain the light normalization vector under the any channel.

7. A substation monitoring image cleaning device, characterized in that, comprises: an image acquisition unit configured to acquire monitoring images of any calibration point of a power transformation station device captured within a preset time period and perform light normalization processing on each monitoring image; a similarity evaluation unit configured to input any two light normalized monitoring images into an image similarity evaluation model to obtain a similarity of the any two light normalized monitoring images output by the image similarity evaluation model; an image cleaning unit configured to determine a duplicate label of the any two light normalized monitoring images based on the similarity of the any two light normalized monitoring images and perform image cleaning based on the duplicate label of the any two light normalized monitoring images; The image similarity evaluation model comprises two image feature extraction layers with the same parameter structure, two hash conversion layers with the same parameter structure, and a similarity prediction layer, the feature extraction layers, the hash conversion layers, and the similarity prediction layer are connected in sequence, the hash conversion layer comprises a fully connected layer with an output dimension of Mxb and a hyperbolic tangent activation layer, M is an output dimension of the feature extraction layer, and b>1; The similarity of the any two light normalized monitoring images is determined based on the following manner: feature extraction of the any two light normalized monitoring images based on two image feature extraction layers to obtain image features of the any two light normalized monitoring images; respectively based on two hash conversion layers, image features corresponding to the two normalized monitoring images are subjected to hash conversion to obtain hash vectors of the two normalized monitoring images; respectively based on two hash conversion layers, image features corresponding to the two normalized monitoring images are subjected to hash conversion to obtain hash vectors of the two normalized monitoring images; respectively based on two hash conversion layers, image features corresponding to the two normalized monitoring images are subjected to hash conversion to obtain hash vectors of the two normalized monitoring images; 8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the substation monitoring image cleaning method according to any one of claims 1 to 6 when executing the program. 9.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program implements the substation monitoring image cleaning method according to any one of claims 1 to 6 when executed by the processor.

Images