Transformer substation monitoring image cleaning method and device, electronic equipment and storage medium

Abstract

The invention provides a transformer substation monitoring image cleaning method and device, electronic equipment and a storage medium, and the method comprises the steps: inputting any two monitoring images after light standardization processing into an image similarity evaluation model, and obtaining the similarity of any two monitoring images after light standardization processing; wherein the image similarity evaluation model comprises two image feature extraction layers with the same parameter structure, two hash conversion layers with the same parameter structure and a similarity prediction layer; the Hash conversion layer comprises a full connection layer with the output dimension being M * b and a hyperbolic tangent activation layer, M is the output dimension of the feature extraction layer, and bgt is included; 1; according to the image cleaning method, the repeated labels of any two monitoring images after light standardization processing are determined on the basis of the similarity of any two monitoring images after light standardization processing, and image cleaning is performed on the basis of the repeated labels of any two monitoring images after light standardization processing, so that the recognition capability of the image cleaning method on tiny differences is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of equipment monitoring, and in particular to a substation monitoring image cleaning method, device, electronic equipment and storage medium. Background Art

[0002] As the scale of the power system continues to expand, substations, especially ultra-high voltage substations, are important hubs of the power network. The operation of their equipment is directly related to the stable supply of electricity and the safety of the power system. Therefore, it is particularly important to monitor substation equipment. At present, when monitoring substation equipment, calibration points are usually set for each device and cameras are installed at each calibration point to take regular photos, for example, taking a monitoring image every 5 minutes. Although such a solution can provide more comprehensive equipment monitoring, there are problems of redundant data and high data processing pressure. Among them, taking regular photos at each calibration point will generate a large amount of image data, especially for substations with a large number of equipment. The pressure of data storage and transmission is very high, and the equipment status usually does not change frequently. Regular photography may generate a large number of duplicate or invalid images, resulting in a waste of storage resources and reduced data processing efficiency. Therefore, it is necessary to clean the monitoring images of the substation to remove highly repeated monitoring images and retain only images that are helpful in judging whether the substation equipment is abnormal.

[0003] However, substation equipment anomalies are usually reflected in slight changes such as temperature, color or deformation. Unlike ordinary repeated images, these subtle differences may mean early signs of equipment failure even in nearly repeated images. However, it is difficult to accurately distinguish whether two monitoring images are substantially different (i.e., the target areas that can indicate the status of the equipment are different, such as instrument indicators, sensor readings, etc.) during the image cleaning process. Incorrect image cleaning may mistakenly delete images containing minor anomalies as duplicate images, resulting in missed reports of equipment anomalies, affecting the safety of substation equipment. Summary of the invention

[0004] The present invention provides a substation monitoring image cleaning method, device, electronic equipment and storage medium, which are used to solve the defect in the prior art that the monitoring image cleaning accuracy is insufficient and affects the safety of substation equipment.

[0005] The present invention provides a substation monitoring image cleaning method, comprising:

[0006] Obtain monitoring images taken at any calibration point corresponding to the substation equipment within a preset time period and perform light standardization processing on each monitoring image;

[0007] Inputting any two light-standardized surveillance images into an image similarity evaluation model to obtain the similarity between the any two light-standardized surveillance images output by the image similarity evaluation model;

[0008] Based on the similarity of any two surveillance images processed by light standardization, determining repeated labels of the surveillance images processed by any two light standardization, and performing image cleaning based on the repeated labels of the surveillance images processed by any two light standardization;

[0009] Among them, the image similarity evaluation model includes two image feature extraction layers with the same parameter structure, two hash conversion layers with the same parameter structure and a similarity prediction layer, and the feature extraction layer, the hash conversion layer and the similarity prediction layer are connected in sequence; the hash conversion layer includes a fully connected layer with an output dimension of M×b and a hyperbolic tangent activation layer, M is the output dimension of the feature extraction layer, and b>1.

[0010] According to a substation monitoring image cleaning method provided by the present invention, the image similarity evaluation model is trained based on the following method:

[0011] Input any sample image pair in the sample image pair set into the image similarity evaluation model to obtain the similarity of any sample image pair output by the similarity prediction layer and the hash vector of each sample image in any sample image pair output by the hash conversion layer;

[0012] Constructing a prediction similarity matrix based on the similarities of each sample image pair output by the image similarity evaluation model, constructing a hash concatenation vector based on the hash vectors of each sample image, and constructing a labeling similarity matrix based on the labeling similarities of each sample image pair;

[0013] Determining a similarity loss based on a difference between the predicted similarity matrix and the annotated similarity matrix;

[0014] Determine a binarization loss based on a transpose of the hash concatenated vector, a product of the hash concatenated vector, and a unit matrix;

[0015] Determine a diversity loss based on the hash concatenation vector and the all-1 vector;

[0016] Parameters of the image similarity evaluation model are adjusted based on the similarity loss, the binarization loss, and the diversification loss.

[0017] According to a substation monitoring image cleaning method provided by the present invention, the binarization loss is the Euclidean distance between a first normalized matrix obtained by normalizing the transpose of the hash splicing vector and the product of the hash splicing vector and the unit matrix.

[0018] According to a substation monitoring image cleaning method provided by the present invention, the diversification loss is the Frobenius norm of the second normalized matrix obtained by normalizing the product between the transpose of the hash splicing vector and the all-1 vector.

[0019] According to a substation monitoring image cleaning method provided by the present invention, the similarity between any two monitoring images after light standardization processing is determined based on the following method:

[0020] Performing feature extraction on the arbitrary two light-standardized monitoring images based on the two image feature extraction layers respectively to obtain image features of the arbitrary two light-standardized monitoring images;

[0021] Performing hash conversion on the image features corresponding to the two monitoring images processed by light standardization based on the two hash conversion layers respectively, to obtain hash vectors of the two monitoring images processed by light standardization;

[0022] Splitting the hash vectors of the two monitoring images after light normalization processing into M b-bit sub-vectors respectively;

[0023] Based on the similarity prediction layer, the difference between the sub-vectors of any two monitoring images after light normalization is calculated bit by bit to obtain the maximum difference of the corresponding sub-vectors, and based on the sum of the maximum differences of all sub-vectors of the monitoring images after light normalization, the similarity of the any two monitoring images after light normalization is determined.

[0024] According to a substation monitoring image cleaning method provided by the present invention, the light standardization processing of each monitoring image includes:

[0025] Decompose any monitoring image into channels to obtain channel images under each channel;

[0026] The channel images under the corresponding channels are processed based on the light normalization vectors under each channel to obtain the normalized images of each channel image;

[0027] The standardized images of each channel image are fused.

[0028] According to a substation monitoring image cleaning method provided by the present invention, the light normalization vector under any channel is determined based on the following steps:

[0029] Based on the standard light graph under any channel and its corresponding non-standard light graph, a curve second-order derivative prediction network corresponding to any channel is trained to obtain a trained curve second-order derivative network;

[0030] Inputting the channel image under any channel into the trained curve second-order derivative network to obtain the second-order derivative of the curve corresponding to the light normalized vector under the corresponding channel;

[0031] Restoration is performed based on the second-order derivative of the curve corresponding to the light normalized vector under any channel to obtain the light normalized vector under any channel.

[0032] The present invention also provides a substation monitoring image cleaning device, comprising:

[0033] An image acquisition unit, used to acquire monitoring images taken at any calibration point corresponding to the substation equipment within a preset time period and perform light standardization processing on each monitoring image;

[0034] A similarity evaluation unit, used for inputting any two surveillance images processed by light standardization into an image similarity evaluation model to obtain the similarity of the any two surveillance images processed by light standardization output by the image similarity evaluation model;

[0035] An image cleaning unit, configured to determine, based on the similarity of any two light-standardized surveillance images, repeated labels of the surveillance images, and perform image cleaning based on the repeated labels of the surveillance images;

[0036] Among them, the image similarity evaluation model includes two image feature extraction layers with the same parameter structure, two hash conversion layers with the same parameter structure and a similarity prediction layer, and the feature extraction layer, the hash conversion layer and the similarity prediction layer are connected in sequence; the hash conversion layer includes a fully connected layer with an output dimension of M×b and a hyperbolic tangent activation layer, M is the output dimension of the feature extraction layer, and b>1.

[0037] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-mentioned substation monitoring image cleaning methods is implemented.

[0038] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the substation monitoring image cleaning method described in any one of the above methods is implemented.

[0039] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned substation monitoring image cleaning methods.

[0040] The substation monitoring image cleaning method, device, electronic device and storage medium provided by the present invention obtain the similarity of any two monitoring images after light standardization processed by the image similarity evaluation model by inputting any two monitoring images after light standardization processed into an image similarity evaluation model; wherein the image similarity evaluation model comprises two image feature extraction layers with the same parameter structure, two hash conversion layers with the same parameter structure and a similarity prediction layer, and the feature extraction layer, the hash conversion layer and the similarity prediction layer are connected in sequence; the hash conversion layer comprises a fully connected layer with an output dimension of M×b and a hyperbolic tangent activation layer, M is the output dimension of the feature extraction layer, and b>1; based on the similarity of any two monitoring images after light standardization processed, the repeated labels of any two monitoring images after light standardization processed are determined, and the image cleaning is performed based on the repeated labels of any two monitoring images after light standardization processed, thereby improving the recognition ability of the image cleaning method for small differences and improving the accuracy of the image cleaning method. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 It is a flow chart of the substation monitoring image cleaning method provided by the present invention;

[0043] Figure 2 It is a structural schematic diagram of the image similarity evaluation model provided by the present invention;

[0044] Figure 3 It is a schematic diagram of the operation process of the image similarity evaluation model provided by the present invention;

[0045] Figure 4 It is a structural schematic diagram of a substation monitoring image cleaning device provided by the present invention;

[0046] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] Figure 1 FIG. 1 is a flow chart of a substation monitoring image cleaning method provided by the present invention, such as Figure 1 As shown, the method includes:

[0049] Step 110, obtaining monitoring images taken at any calibration point corresponding to the substation equipment within a preset time period and performing light standardization processing on each monitoring image;

[0050] Step 120, inputting any two surveillance images processed by light standardization into an image similarity evaluation model, and obtaining the similarity between the any two surveillance images processed by light standardization output by the image similarity evaluation model;

[0051] Step 130, determining repeated labels of any two surveillance images processed after light standardization based on similarities between the surveillance images processed after light standardization, and performing image cleaning based on the repeated labels of any two surveillance images processed after light standardization;

[0052] Among them, the image similarity evaluation model includes two image feature extraction layers with the same parameter structure, two hash conversion layers with the same parameter structure and a similarity prediction layer, and the feature extraction layer, the hash conversion layer and the similarity prediction layer are connected in sequence; the hash conversion layer includes a fully connected layer with an output dimension of M×b and a hyperbolic tangent activation layer, M is the output dimension of the feature extraction layer, and b>1.

[0053] Here, all monitoring images taken within a preset time period can be obtained from any monitoring calibration point of the substation equipment. Since monitoring images will show light differences due to changes in time, weather and lighting conditions, each monitoring image can be subjected to light standardization processing to eliminate the impact of ambient light on the image content, making the image look more consistent under different lighting conditions and reducing unnecessary feature differences.

[0054] In some embodiments, considering that monitoring images are generally RGB images, in order to perform light standardization processing on any monitoring image, the monitoring image can be channel-decomposed to obtain channel images under each channel (R channel, G channel, and B channel). Since channel images under different channels have different characteristics, and different monitoring images have different light conditions, their channel images corresponding to the same channel also have different characteristics. Therefore, for the current channel image, the light standardization vectors of each channel corresponding to the channel image can be obtained in advance for processing the channel image of the corresponding channel. Subsequently, the channel images under the corresponding channels can be processed based on the light standardization vectors under each channel, respectively, to obtain standardized images of each channel image, and then the standardized images of each channel image can be fused to achieve light standardization of the monitoring image.

[0055] In other embodiments, in order to establish the light normalization vectors of each channel corresponding to the channel image, for any channel, the curve second-order derivative prediction network corresponding to the channel can be trained in advance based on the standard light diagram under the channel and its corresponding non-standard light diagram to obtain a trained curve second-order derivative network. Among them, the curve second-order derivative prediction network can be constructed based on a convolutional neural network, and its last layer is a RELU layer to ensure that the output second-order derivative is less than 0, thereby ensuring that the curve corresponding to the light normalization vector subsequently restored is a monotonically increasing concave curve. Specifically, when training the curve second-order derivative prediction network, the non-standard light diagram under the channel can be input into the initial network to obtain the second-order derivative of the curve corresponding to the light normalization vector under the channel predicted by the initial network, and the second-order derivative of the curve corresponding to the light normalization vector under the channel is restored to obtain the light normalization vector under the channel. The non-standard light map is transformed based on the light normalization vector to obtain a predicted normalized image, and then the predicted normalized image and the corresponding standard light map are compared based on an image comparison network to obtain a comparison result, thereby adjusting the parameters of the initial network according to the comparison result to obtain a curve second-order derivative prediction network. After obtaining the curve second-order derivative prediction network corresponding to the channel, the channel image under the channel can be input into the curve second-order derivative network to obtain the second-order derivative of the curve corresponding to the light normalization vector under the corresponding channel, and then restored based on the second-order derivative of the curve corresponding to the light normalization vector under the channel to obtain the light normalization vector under the channel.

[0056] Among them, when restoring the second-order derivative output by the curve second-order derivative prediction network, the preset lower triangular matrix can be multiplied by the negative of the second-order derivative to obtain a first product, and then the preset upper triangular matrix can be multiplied by the first product to obtain a second product. After the second product is normalized, the diagonal elements are combined into a vector form to obtain the corresponding light standardized vector.

[0057] The light-standardized monitoring images are input into the image similarity evaluation model in pairs to calculate the similarity of each pair of monitoring images. The image similarity evaluation model can use deep learning-based feature extraction technology, such as convolutional neural networks, to extract image features of image pairs, and perform similarity evaluation based on the image features of image pairs. Considering that the status of substation equipment does not change frequently, and the abnormality of substation equipment is usually reflected in slight changes such as temperature, color or deformation, when evaluating the similarity of image pairs, more attention needs to be paid to the local similarity of images to improve the image cleaning accuracy of substation monitoring images.

[0058] To achieve this goal, Figure 2 As shown, an image similarity assessment model can be constructed based on two image feature extraction layers with the same parameter structure, two hash conversion layers with the same parameter structure, and a similarity prediction layer, wherein the feature extraction layer, the hash conversion layer, and the similarity prediction layer are connected in sequence. In some embodiments, Figure 3 As shown, the specific operation process of the image similarity evaluation model can be as follows:

[0059] Step 310, performing feature extraction on the arbitrary two light-standardized monitoring images based on the two image feature extraction layers respectively, to obtain image features of the arbitrary two light-standardized monitoring images;

[0060] Step 320, performing hash conversion on the image features corresponding to the arbitrary two light-standardized monitoring images based on the two hash conversion layers respectively, to obtain hash vectors of the arbitrary two light-standardized monitoring images;

[0061] Step 330, dividing the hash vectors of the two monitoring images after the light normalization processing into M b-bit sub-vectors respectively;

[0062] Step 340, based on the similarity prediction layer, calculate the difference between the sub-vectors of any two light-normalized monitoring images bit by bit to obtain the maximum difference of the corresponding sub-vectors, and determine the similarity of any two light-normalized monitoring images based on the sum of the maximum differences of all sub-vectors of any two light-normalized monitoring images.

[0063] Specifically, assuming that the image pair input to the model is represented as (A, B), the two image feature extraction layers are used to extract features from image A and image B, respectively, to obtain image features of image A and image B. In some embodiments, the image feature extraction layer can be constructed based on a convolutional neural network. The hash conversion layers connected to the two image feature extraction layers respectively perform hash conversion on the image features of image A and image B, respectively, to obtain hash vectors of image A and image B. Among them, the hash vectors of different images are different, while the hash vectors of the same image are the same. Next, the hash vectors of image A and image B will enter the similarity prediction layer for similarity calculation to obtain the similarity between image A and image B.

[0064] It is worth noting that, unlike the existing hash conversion method in the image field, which uses the Sign function to generate a hash vector, the embodiment of the present invention takes into account the non-differentiable characteristics of the Sign function when constructing the hash conversion layer. If the Sign function is used to generate the hash vector, the gradient will disappear. This problem will cause the image similarity evaluation model trained by the gradient descent algorithm to fail to complete the training process. Therefore, the embodiment of the present invention uses a fully connected layer with an output dimension of M×b and a hyperbolic tangent activation layer to construct a hash conversion layer to simulate the hash conversion to obtain a hash vector that is similar to a binary vector, and because the output result of the hyperbolic tangent activation layer is differentiable, the above-mentioned gradient disappearance problem can be solved. Wherein, M is the output dimension of the feature extraction layer, and b>1. More importantly, the hash vector is composed of M b-bit vectors, each b-bit vector corresponds to a point in the image feature (i.e., a local area in the original image), so the hash vector contains the features of each local area of ​​the corresponding image. The subsequent similarity prediction layer can compare and evaluate each local area of ​​the image region by region by comparing the hash vectors of the two images, so it is more sensitive to the slight differences in the local area, and the accuracy of the image similarity evaluation is higher.

[0065] Furthermore, in order to further improve the sensitivity of the image similarity evaluation model to the slight differences in the substation monitoring images, the hash vectors of image A and image B can be divided into M b-bit sub-vectors H 1 , H 2 , ..., H M , where H i =(h1,h2,...,hb). Then the similarity prediction layer can calculate the difference between the subvectors of image A and image B bit by bit to obtain the maximum difference of the corresponding subvectors. Taking the i-th subvector as an example, the H difference between image A and image B is calculated bit by bit. iThe difference of each position in the image is calculated and the maximum difference is selected. Based on the sum of the maximum differences of all subvectors of image A and image B, the similarity between image A and image B is determined. The similarity between image A and image B can be calculated using the following formula:

[0066] Sim(A,B)=1-∑dif(i) / 2M

[0067] Where dif(i) is the maximum difference between the i-th subvector of image A and image B.

[0068] In some embodiments, the image similarity assessment model is trained based on:

[0069] Any sample image pair in the sample image pair set is input into the image similarity evaluation model to obtain the similarity of the sample image pair output by the similarity prediction layer and the hash vector of each sample image in the sample image pair output by the hash conversion layer. The operating mechanism of the image similarity evaluation model in the training phase is the same as the operating mechanism given in the above embodiment, which will not be repeated here.

[0070] A prediction similarity matrix is ​​constructed based on the similarity of each sample image pair output by the image similarity evaluation model, a hash concatenation vector is constructed based on the hash vector of each sample image, and a labeling similarity matrix is ​​constructed based on the labeling similarity of each sample image pair. The labeling similarity of any sample image pair is 1 or 0.

[0071] On the one hand, the similarity loss can be determined based on the difference between the predicted similarity matrix and the annotated similarity matrix to guide the entire image similarity assessment model to better learn the similarities and local minor differences between the input image pairs, thereby improving the model's assessment accuracy for the similarity of image pairs. In some embodiments, the Euclidean distance between the predicted similarity matrix and the annotated similarity matrix can be calculated, and the Euclidean distance can be divided by the total number of image pairs to obtain the similarity loss.

[0072] On the other hand, since the hash conversion layer uses the fully connected layer and the hyperbolic tangent activation layer to simulate the hash conversion to generate a hash vector that is close to a binary vector, in order to guide the hash conversion layer to generate a vector whose elements are closer to the extreme value {-1, 1}, the binarization loss can be determined based on the product of the transpose of the hash splicing vector and the hash splicing vector and the unit matrix. In some embodiments, the Euclidean distance between the first normalized matrix obtained by normalizing the product of the transpose of the hash splicing vector and the hash splicing vector and the unit matrix can be determined as the binarization loss.

[0073] In addition, a diversification loss can be determined based on the hash concatenation vector and the all-1 vector to guide the hash conversion layer to generate more diverse hash vectors, thereby further reducing the probability of hash vector collisions of different images. In some embodiments, the Frobenius norm of the second normalized matrix obtained by normalizing the product of the transpose of the hash concatenation vector and the all-1 vector can be determined as the diversification loss.

[0074] Subsequently, based on the above similarity loss, binarization loss and diversification loss, the parameters of the image similarity evaluation model are reversely adjusted based on the gradient descent algorithm to complete a round of training of the image similarity evaluation model. By repeating the above training process, a trained image similarity evaluation model can be obtained.

[0075] After obtaining the similarity of any two surveillance images after light standardization, each surveillance image pair can be labeled as "duplicate" or "non-duplicate" according to the similarity threshold. Based on the duplicate labels of each surveillance image pair, a large number of similar images can be cleaned up to remove redundant data. For example, for multiple surveillance images that are repeated, only the image with the best quality or the latest time can be retained, and the remaining duplicate images can be deleted, thereby reducing the pressure on data transmission and image storage and the difficulty of subsequent image analysis.

[0076] In summary, the method provided by the embodiment of the present invention obtains the similarity of any two light-standardized surveillance images output by the image similarity evaluation model by inputting any two light-standardized surveillance images into an image similarity evaluation model; wherein the image similarity evaluation model includes two image feature extraction layers with the same parameter structures, two hash conversion layers with the same parameter structures and a similarity prediction layer, and the feature extraction layer, the hash conversion layer and the similarity prediction layer are connected in sequence; the hash conversion layer includes a fully connected layer with an output dimension of M×b and a hyperbolic tangent activation layer, M is the output dimension of the feature extraction layer, and b>1; based on the similarity of any two light-standardized surveillance images, the repeated labels of any two light-standardized surveillance images are determined, and image cleaning is performed based on the repeated labels of any two light-standardized surveillance images, thereby improving the image cleaning method's ability to recognize subtle differences and improving the accuracy of the image cleaning method.

[0077] The substation monitoring image cleaning device provided by the present invention is described below. The substation monitoring image cleaning device described below and the substation monitoring image cleaning method described above can be referred to each other.

[0078] Based on any of the above embodiments, Figure 4 Schematic diagram of the structure of the substation monitoring image cleaning device provided by the present invention. Figure 4As shown, the device comprises:

[0079] The image acquisition unit 410 is used to acquire the monitoring images taken at any calibration point corresponding to the substation equipment within a preset time period and perform light standardization processing on each monitoring image;

[0080] A similarity evaluation unit 420 is used to input any two light-normalized surveillance images into an image similarity evaluation model to obtain the similarity between the any two light-normalized surveillance images output by the image similarity evaluation model;

[0081] An image cleaning unit 430, configured to determine, based on the similarity of any two light-standardized surveillance images, repeated labels of the surveillance images, and perform image cleaning based on the repeated labels of the surveillance images;

[0082] Among them, the image similarity evaluation model includes two image feature extraction layers with the same parameter structure, two hash conversion layers with the same parameter structure and a similarity prediction layer, and the feature extraction layer, the hash conversion layer and the similarity prediction layer are connected in sequence; the hash conversion layer includes a fully connected layer with an output dimension of M×b and a hyperbolic tangent activation layer, M is the output dimension of the feature extraction layer, and b>1.

[0083] The device provided by the embodiment of the present invention obtains the similarity of any two surveillance images processed by light standardization output by the image similarity evaluation model by inputting any two surveillance images processed by light standardization into an image similarity evaluation model; wherein the image similarity evaluation model includes two image feature extraction layers with the same parameter structure, two hash conversion layers with the same parameter structure and a similarity prediction layer, and the feature extraction layer, the hash conversion layer and the similarity prediction layer are connected in sequence; the hash conversion layer includes a fully connected layer with an output dimension of M×b and a hyperbolic tangent activation layer, M is the output dimension of the feature extraction layer, and b>1; based on the similarity of any two surveillance images processed by light standardization, the repeated labels of any two surveillance images processed by light standardization are determined, and image cleaning is performed based on the repeated labels of any two surveillance images processed by light standardization, thereby improving the recognition ability of the image cleaning method for slight differences and improving the accuracy of the image cleaning method.

[0084] Based on any of the above embodiments, the image similarity evaluation model is trained based on the following method:

[0085] Input any sample image pair in the sample image pair set into the image similarity evaluation model to obtain the similarity of any sample image pair output by the similarity prediction layer and the hash vector of each sample image in any sample image pair output by the hash conversion layer;

[0086] Constructing a prediction similarity matrix based on the similarities of each sample image pair output by the image similarity evaluation model, constructing a hash concatenation vector based on the hash vectors of each sample image, and constructing a labeling similarity matrix based on the labeling similarities of each sample image pair;

[0087] Determining a similarity loss based on a difference between the predicted similarity matrix and the annotated similarity matrix;

[0088] Determine a binarization loss based on a transpose of the hash concatenated vector, a product of the hash concatenated vector, and a unit matrix;

[0089] Determine a diversity loss based on the hash concatenation vector and the all-1 vector;

[0090] Parameters of the image similarity evaluation model are adjusted based on the similarity loss, the binarization loss, and the diversification loss.

[0091] Based on any of the above embodiments, the binarization loss is the Euclidean distance between a first normalized matrix obtained by normalizing the transpose of the hash concatenated vector and the product of the hash concatenated vector and the identity matrix.

[0092] Based on any of the above embodiments, the diversity loss is the Frobenius norm of a second normalized matrix obtained by normalizing the product of the transpose of the hash concatenated vector and the all-1 vector.

[0093] Based on any of the above embodiments, the similarity between any two surveillance images after light standardization is determined based on the following method:

[0094] Performing feature extraction on the arbitrary two light-standardized monitoring images based on the two image feature extraction layers respectively to obtain image features of the arbitrary two light-standardized monitoring images;

[0095] Performing hash conversion on the image features corresponding to the two monitoring images processed by light standardization based on the two hash conversion layers respectively, to obtain hash vectors of the two monitoring images processed by light standardization;

[0096] Splitting the hash vectors of the two monitoring images after light normalization processing into M b-bit sub-vectors respectively;

[0097] Based on the similarity prediction layer, the difference between the sub-vectors of any two monitoring images after light normalization is calculated bit by bit to obtain the maximum difference of the corresponding sub-vectors, and based on the sum of the maximum differences of all sub-vectors of the monitoring images after light normalization, the similarity of the any two monitoring images after light normalization is determined.

[0098] Based on any of the above embodiments, the step of performing light standardization processing on each monitoring image includes:

[0099] Decompose any monitoring image into channels to obtain channel images under each channel;

[0100] The channel images under the corresponding channels are processed based on the light normalization vectors under each channel to obtain the normalized images of each channel image;

[0101] The standardized images of each channel image are fused.

[0102] Based on any of the above embodiments, the normalized vector of light in any channel is determined based on the following steps:

[0103] Based on the standard light graph under any channel and its corresponding non-standard light graph, a curve second-order derivative prediction network corresponding to any channel is trained to obtain a trained curve second-order derivative network;

[0104] Inputting the channel image under any channel into the trained curve second-order derivative network to obtain the second-order derivative of the curve corresponding to the light normalized vector under the corresponding channel;

[0105] Restoration is performed based on the second-order derivative of the curve corresponding to the light normalized vector under any channel to obtain the light normalized vector under any channel.

[0106] Figure 5 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 5As shown, the electronic device may include: a processor (processor) 510, a memory (memory) 520, a communication interface (Communications Interface) 530 and a communication bus 540, wherein the processor 510, the memory 520, and the communication interface 530 communicate with each other through the communication bus 540. The processor 510 can call the logic instructions in the memory 520 to execute the substation monitoring image cleaning method, which includes: obtaining the monitoring images taken by any calibration point corresponding to the substation equipment within a preset time period and performing light standardization processing on each monitoring image; inputting any two monitoring images after light standardization processing into the image similarity evaluation model to obtain the similarity of the any two monitoring images after light standardization processing output by the image similarity evaluation model; based on the similarity of any two monitoring images after light standardization processing, determining the repeated labels of the any two monitoring images after light standardization processing, and performing image cleaning based on the repeated labels of the any two monitoring images after light standardization processing; wherein the image similarity evaluation model includes two image feature extraction layers with the same parameter structure, two hash conversion layers with the same parameter structure and a similarity prediction layer, and the feature extraction layer, the hash conversion layer and the similarity prediction layer are connected in sequence; the hash conversion layer includes a fully connected layer with an output dimension of M×b and a hyperbolic tangent activation layer, M is the output dimension of the feature extraction layer, and b>1.

[0107] In addition, the logic instructions in the above-mentioned memory 520 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0108] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the substation monitoring image cleaning method provided by the above methods, and the method includes: obtaining a monitoring image taken at any calibration point corresponding to the substation equipment within a preset time period and performing light standardization processing on each monitoring image; inputting any two monitoring images after light standardization processing into an image similarity evaluation model to obtain the monitoring images after the any two light standardization processing output by the image similarity evaluation model. similarity; based on the similarity of any two surveillance images after light standardization, determine the repeated labels of the surveillance images after any two light standardization, and perform image cleaning based on the repeated labels of the surveillance images after any two light standardization; wherein the image similarity evaluation model includes two image feature extraction layers with the same parameter structure, two hash conversion layers with the same parameter structure and a similarity prediction layer, and the feature extraction layer, the hash conversion layer and the similarity prediction layer are connected in sequence; the hash conversion layer includes a fully connected layer with an output dimension of M×b and a hyperbolic tangent activation layer, M is the output dimension of the feature extraction layer, and b>1.

[0109] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when executed by a processor to execute the above-mentioned substation monitoring image cleaning method, the method comprising: obtaining monitoring images taken at any calibration point corresponding to the substation equipment within a preset time period and performing light standardization processing on each monitoring image; inputting any two monitoring images after light standardization processing into an image similarity evaluation model to obtain the similarity of the any two monitoring images after light standardization processing output by the image similarity evaluation model; based on the similarity of any two monitoring images after light standardization processing, determining the repeated labels of the any two monitoring images after light standardization processing, and performing image cleaning based on the repeated labels of the any two monitoring images after light standardization processing; wherein the image similarity evaluation model includes two image feature extraction layers with the same parameter structure, two hash conversion layers with the same parameter structure and a similarity prediction layer, the feature extraction layer, the hash conversion layer and the similarity prediction layer are connected in sequence; the hash conversion layer includes a fully connected layer with an output dimension of M×b and a hyperbolic tangent activation layer, M is the output dimension of the feature extraction layer, and b>1.

[0110] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0111] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0112] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A substation monitoring image cleaning method, characterized in that: include: Obtain monitoring images taken at any calibration point corresponding to the substation equipment within a preset time period and perform light standardization processing on each monitoring image; Inputting any two light-standardized surveillance images into an image similarity evaluation model to obtain the similarity between the any two light-standardized surveillance images output by the image similarity evaluation model; Based on the similarity of any two surveillance images processed by light standardization, determining repeated labels of the surveillance images processed by any two light standardization, and performing image cleaning based on the repeated labels of the surveillance images processed by any two light standardization; Among them, the image similarity evaluation model includes two image feature extraction layers with the same parameter structure, two hash conversion layers with the same parameter structure and a similarity prediction layer, and the feature extraction layer, the hash conversion layer and the similarity prediction layer are connected in sequence; the hash conversion layer includes a fully connected layer with an output dimension of M×b and a hyperbolic tangent activation layer, M is the output dimension of the feature extraction layer, and b>1.

2. The substation monitoring image cleaning method according to claim 1 is characterized in that: The image similarity assessment model is trained based on the following method: Input any sample image pair in the sample image pair set into the image similarity evaluation model to obtain the similarity of any sample image pair output by the similarity prediction layer and the hash vector of each sample image in any sample image pair output by the hash conversion layer; Constructing a prediction similarity matrix based on the similarities of each sample image pair output by the image similarity evaluation model, constructing a hash concatenation vector based on the hash vectors of each sample image, and constructing a labeling similarity matrix based on the labeling similarities of each sample image pair; Determining a similarity loss based on a difference between the predicted similarity matrix and the annotated similarity matrix; Determine a binarization loss based on a transpose of the hash concatenated vector, a product of the hash concatenated vector, and a unit matrix; Determine a diversity loss based on the hash concatenation vector and the all-1 vector; Parameters of the image similarity evaluation model are adjusted based on the similarity loss, the binarization loss, and the diversification loss.

3. The substation monitoring image cleaning method according to claim 2 is characterized in that: The binarization loss is the Euclidean distance between a first normalized matrix obtained by normalizing the transpose of the hash concatenated vector and the product of the hash concatenated vector and the identity matrix.

4. The substation monitoring image cleaning method according to claim 2 is characterized in that: The diversification loss is the Frobenius norm of a second normalized matrix obtained by normalizing the product of the transpose of the hash concatenated vector and the all-1 vector.

5. The substation monitoring image cleaning method according to any one of claims 1 to 4, characterized in that: The similarity between any two light-normalized surveillance images is determined based on the following method: Performing feature extraction on the arbitrary two light-standardized monitoring images based on the two image feature extraction layers respectively to obtain image features of the arbitrary two light-standardized monitoring images; Performing hash conversion on the image features corresponding to the two monitoring images processed by light standardization based on the two hash conversion layers respectively, to obtain hash vectors of the two monitoring images processed by light standardization; Splitting the hash vectors of the two monitoring images after light normalization processing into M b-bit sub-vectors respectively; Based on the similarity prediction layer, the difference between the sub-vectors of any two monitoring images after light normalization is calculated bit by bit to obtain the maximum difference of the corresponding sub-vectors, and based on the sum of the maximum differences of all sub-vectors of the monitoring images after light normalization, the similarity of the any two monitoring images after light normalization is determined.

6. The substation monitoring image cleaning method according to any one of claims 1 to 4, characterized in that: The light standardization processing of each monitoring image includes: Decompose any monitoring image into channels to obtain channel images under each channel; The channel images under the corresponding channels are processed based on the light normalization vectors under each channel to obtain the normalized images of each channel image; The standardized images of each channel image are fused.

7. The substation monitoring image cleaning method according to claim 6 is characterized in that: The normalized vector of a ray in any channel is determined based on the following steps: Based on the standard light graph under any channel and its corresponding non-standard light graph, a curve second-order derivative prediction network corresponding to any channel is trained to obtain a trained curve second-order derivative network; Inputting the channel image under any channel into the trained curve second-order derivative network to obtain the second-order derivative of the curve corresponding to the light normalized vector under the corresponding channel; Restoration is performed based on the second-order derivative of the curve corresponding to the light normalized vector under any channel to obtain the light normalized vector under any channel.

8. A substation monitoring image cleaning device, characterized in that: include: An image acquisition unit, used to acquire monitoring images taken at any calibration point corresponding to the substation equipment within a preset time period and perform light standardization processing on each monitoring image; A similarity evaluation unit, used for inputting any two surveillance images processed by light standardization into an image similarity evaluation model to obtain the similarity of the any two surveillance images processed by light standardization output by the image similarity evaluation model; An image cleaning unit, configured to determine, based on the similarity of any two light-standardized surveillance images, repeated labels of the surveillance images, and perform image cleaning based on the repeated labels of the surveillance images; Among them, the image similarity evaluation model includes two image feature extraction layers with the same parameter structure, two hash conversion layers with the same parameter structure and a similarity prediction layer, and the feature extraction layer, the hash conversion layer and the similarity prediction layer are connected in sequence; the hash conversion layer includes a fully connected layer with an output dimension of M×b and a hyperbolic tangent activation layer, M is the output dimension of the feature extraction layer, and b>1.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the substation monitoring image cleaning method as described in any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the substation monitoring image cleaning method according to any one of claims 1 to 7 is implemented.

Images